Developers Guide
WattDepot: A system for collecting and storing data from
electricity meters for smart grid research and experimentation.
This page contains instructions on how to check out, build, and test the WattDepot code base.
WattDepot requires Java 1.6 or above. You can use any flavor of java.
To check that Java is installed.
% java -version
% javac -version
Find a package for your operating system at the GitHub install wiki. We recommend following the GitHub setup instructions at https://help.github.com/articles/set-up-git.
To check that Git is installed.
% git --version
To check that Maven is installed.
% mvn --version
The WattDepot code is managed with Git. To check out a local copy, use
git clone https://github.com/csdl/wattdepot.git
or fork the csdl/wattdepot repository to your own GitHub account. Then clone your WattDepot repository to your development machine.
WattDepot stores all its files in a server home directory. By default, this will be a .wattdepot
directory in your home directory. If you want to create the .wattdepot directory elsewhere, set
the wattdepot.user.home Java property to the desired parent directory. The name .wattdepot
directory name itself is not configurable.
Within the server home directory are subdirectories that store all the data for a particular server
instance. The default subdirectory is server, but this can be changed on the command line with
the -d argument. This allows multiple !WattDepot server instances to run in parallel, each
writing to a different subdirectory. Create this directory:
mkdir ~/.wattdepot
mkdir ~/.wattdepot/server
The server reads properties on launch from a file named wattdepot-server.properties in the server
subdirectory. A default properties file can be found in the wattdepot files you downloaded as
src/main/resources/etc/wattdepot-server.properties.default. Copy that file to your new server
subdirectory (~/.wattdepot/server if you are using the defaults), and rename it to
wattdepot-server.properties.
After the file is copied into place, open it in your favorite text editor and change or uncomment
any lines you want. You will want to change the default administrator username and password. Also
edit the wattdepot-server.db.username and wattdepot-server.db.password.
WattDepot supports several different database back-ends. If you are doing development you will need to install PostgreSQL. We recommend version 9.1.3.
After installing PostgreSQL create the Postgres role for the user you defined in the
wattdepot-server.properties file.
For example if your wattdepot-server.properties file looked like:
wattdepot-server.db.username=wattdepot
wattdepot-server.db.password=myPassword
the psql command to create the user is:
postgres=# CREATE USER wattdepot WITH PASSWORD 'myPassword';
Next create the two databases used by WattDepot:
postgres=# CREATE DATABASE wattdepot;
postgres=# GRANT ALL PRIVILEGES ON DATABASE wattdepot to wattdepot;
postgres=# CREATE DATABASE testwattdepot;
postgres=# GRANT ALL PRIVILEGES ON DATABASE testwattdepot to wattdepot;
WattDepot uses Maven 3 as a build system. Maven is configured
with the pom.xml file found at the root of the WattDepot source directory. Maven has a number of
lifecycle phases - the most useful in this project are clean, compile, test, package,
verify, and site.
-
mvn cleanwill delete the target directory. -
mvn compilewill compile the project. -
mvn testwill compile the project, then run JUnit and QA tests. Note: Until you set up the database implementations the test will not pass.- The POM runs the JUnit tests in multiple sections. First all non-database tests are executed with the main storage implementation (currently Derby). After those pass the database tests are executed separately for each storage implementation. Note: the postgres storage implementation requires you to setup the wattdepot-server.properties file in order to supply the wattdepot-server.db.username property.
- FindBugs
- Checkstyle
- PMD
- JavaDocs are created
-
mvn packagewill compile the project, run the tests, and then create .jar and .zip files for a distribution- The .jar files are created from anttasks.build.xml (this functionality hasn't been ported from Ant to Maven)
- The POM zips the source
- The POM zips the javadocs
- The POM zips everything into a distribution
-
mvn verifyruns the compile, test and package phases and ensures the result is valid -
mvn siteexecutes FindBugs, Checkstyle, PMD, and JavaDocs, then presents the results in an easy to read website (located in target/site)
Rather than executing a whole phase, you can also run an individual goal. For example, to run FindBugs without doing anything else, use
mvn findbugs:check
The same works for mvn pmd:check and mvn checkstyle:check. However, note that this does not
recompile the project. If you make changes to the source code, you should run mvn compile before
running any other checks.
If new dependencies are added to the project, they should be added into the POM and possibly into the .jar creation step in anttasks.build.xml. They also need to be added to the Eclipse classpath, which can be accomplished with
mvn eclipse:eclipse
This ensures that all dependencies listed in the POM are listed in the Eclipse classpath, with relative paths to the Maven repository. This is the best way to add any new dependency to the project because it ensures that it will work for any developer on any system.
Another useful tip is adding properties to Maven calls with the -D argument. For example, to run
the package phase without running JUnit tests, use
mvn package -DskipTests
This can be a great time saver if you've just run the test phase and now want to package. These properties can also affect the running of WattDepot. Say you want to test multiple configurations without changing the wattdepot-server.properties file between each test. Try
mvn test -Dwattdepot-server.db.impl=org.wattdepot.server.db.postgres.PostgresStorageImplementation
compared to
mvn test -Dwattdepot-server.db.impl=org.wattdepot.server.db.derby.DerbyStorageImplementation
Obviously using the wattdepot-server.properties file is best for long term configuration changes, but when doing individual tests it can be faster to just use Maven.
Change directory to your local clone of WattDepot. At the command prompt you can run mvn clean
clean the project. You should see something like:
[~/wattdepot] $ mvn clean
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building WattDepot 2.1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ wattdepot ---
[INFO] Deleting /Users/CSDL/wattdepot/target
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.169s
[INFO] Finished at: Mon Jul 15 13:04:16 HST 2013
[INFO] Final Memory: 2M/81M
[INFO] ------------------------------------------------------------------------
[~/wattdepot] $
To compile the code use mvn compile. The first time you run this command maven will download the
necessary dependencies.
[~/wattdepot] $ mvn compile
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building WattDepot 2.1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.4.3:resources (default-resources) @ wattdepot ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 36 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ wattdepot ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 117 source files to /Users/CSDL/wattdepot/target/classes
[WARNING] Note: /Users/wattdepot/src/main/java/org/wattdepot/server/WattDepotEnroler.java uses or overrides a deprecated API.
[WARNING] Note: Recompile with -Xlint:deprecation for details.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.097s
[INFO] Finished at: Mon Jul 15 13:08:58 HST 2013
[INFO] Final Memory: 12M/81M
[INFO] ------------------------------------------------------------------------
[~/wattdepot] $
To validate WattDepot run mvn test. The first time you run this command maven will download the
necessary dependencies.
[~/wattdepot] $ mvn test
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building WattDepot 2.1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.4.3:resources (default-resources) @ wattdepot ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 36 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ wattdepot ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 117 source files to /Users/CSDL/wattdepot/target/classes
[WARNING] Note: /Users/CSDL/wattdepot/src/main/java/org/wattdepot/server/WattDepotEnroler.java uses or overrides a deprecated API.
[WARNING] Note: Recompile with -Xlint:deprecation for details.
[INFO]
[INFO] --- maven-resources-plugin:2.4.3:testResources (default-testResources) @ wattdepot ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/CSDL/wattdepot/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ wattdepot ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 41 source files to /Users/CSDL/wattdepot/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.11:test (default-test) @ wattdepot ---
[INFO] Surefire report directory: /Users/CSDL/wattdepot/target/surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.wattdepot.datainput.TestOscarRowParser
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.05 sec
Running org.wattdepot.datainput.TestSensorLogDataConverter
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running org.wattdepot.datainput.TestSharkRowParser
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec
Running org.wattdepot.datainput.TestVerisRowParser
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running org.wattdepot.resource.carbon.TestCarbon
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running org.wattdepot.resource.carbon.TestCarbonResource
Loading Server properties from: /Users/CSDL/.wattdepot/server/wattdepot-server.properties
logDirString: /Users/CSDL/.wattdepot/server/logs/
2013/07/15 13:12:13 Starting WattDepot server.
2013/07/15 13:12:13 Host: http://localhost:8183/wattdepot/
2013/07/15 13:12:13 Server Properties:
wattdepot-server.admin.email = [email protected]
wattdepot-server.admin.password = ((hidden))
wattdepot-server.context.root = wattdepot
wattdepot-server.datainput.file = /Users/CSDL/.wattdepot/client/datainput.properties
wattdepot-server.datainput.start = false
wattdepot-server.db.berkeleydb.dir = /Users/CSDL/.wattdepot/server/testBerkeleyDb
wattdepot-server.db.dbName = testwattdepot
wattdepot-server.db.derby.dir = /Users/CSDL/.wattdepot/server/testDerby
wattdepot-server.db.derby.snapshot = /Users/CSDL/.wattdepot/server/testDerby-snapshot
wattdepot-server.db.hostname = localhost
wattdepot-server.db.impl = org.wattdepot.server.db.derby.DerbyStorageImplementation
wattdepot-server.db.password = ((hidden))
wattdepot-server.db.port = 5432
wattdepot-server.db.postgres.initialSize = 10
wattdepot-server.db.postgres.maxActive = 19
wattdepot-server.db.postgres.snapshot = /Users/CSDL/.wattdepot/server/testPostgres-snapshot
wattdepot-server.db.username = wattdepot
wattdepot-server.heroku = false
wattdepot-server.homedir = /Users/CSDL/.wattdepot/server
wattdepot-server.hostname = localhost
wattdepot-server.logging.level = INFO
wattdepot-server.maxthreads = 255
wattdepot-server.port = 8183
wattdepot-server.restlet.logging = false
wattdepot-server.smtp.host = mail.hawaii.edu
wattdepot-server.test.admin.email = [email protected]
wattdepot-server.test.admin.password = ((hidden))
wattdepot-server.test.db.berkeleydb.dir = /Users/CSDL/.wattdepot/server/testBerkeleyDb
wattdepot-server.test.db.dbName = testwattdepot
wattdepot-server.test.db.derby.dir = /Users/CSDL/.wattdepot/server/testDerby
wattdepot-server.test.db.derby.snapshot = /Users/CSDL/.wattdepot/server/testDerby-snapshot
wattdepot-server.test.db.hostname = localhost
wattdepot-server.test.db.port = 5432
wattdepot-server.test.db.postgres.snapshot = /Users/CSDL/.wattdepot/server/testPostgres-snapshot
wattdepot-server.test.domain = example.com
wattdepot-server.test.heroku = false
wattdepot-server.test.hostname = localhost
wattdepot-server.test.install = true
wattdepot-server.test.port = 8183
2013/07/15 13:12:14 Derby: previously initialized.
2013/07/15 13:12:14 Maximum Java heap size (MB): 129.957888
2013/07/15 13:12:14 WattDepot server (Version Development) now running.
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.798 sec
... SNIP ...
Results :
Tests run: 33, Failures: 0, Errors: 0, Skipped: 1
[INFO]
[INFO] >>> findbugs-maven-plugin:2.4.0:check (default) @ wattdepot >>>
[INFO]
[INFO] --- findbugs-maven-plugin:2.4.0:findbugs (findbugs) @ wattdepot ---
[INFO] Fork Value is true
[INFO] Done FindBugs Analysis....
[INFO]
[INFO] <<< findbugs-maven-plugin:2.4.0:check (default) @ wattdepot <<<
[INFO]
[INFO] --- findbugs-maven-plugin:2.4.0:check (default) @ wattdepot ---
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO]
[INFO] --- maven-checkstyle-plugin:2.8:check (default) @ wattdepot ---
[INFO]
[INFO]
[INFO] >>> maven-pmd-plugin:2.6:check (default) @ wattdepot >>>
[INFO]
[INFO] --- maven-pmd-plugin:2.6:pmd (pmd) @ wattdepot ---
[WARNING] Unable to locate Source XRef to link to - DISABLED
[INFO]
[INFO] <<< maven-pmd-plugin:2.6:check (default) @ wattdepot <<<
[INFO]
[INFO] --- maven-pmd-plugin:2.6:check (default) @ wattdepot ---
[INFO]
[INFO]
[INFO] >>> maven-javadoc-plugin:2.8.1:javadoc (default) @ wattdepot >>>
[INFO]
[INFO] <<< maven-javadoc-plugin:2.8.1:javadoc (default) @ wattdepot <<<
[INFO]
[INFO] --- maven-javadoc-plugin:2.8.1:javadoc (default) @ wattdepot ---
[INFO]
Loading source files for package org.wattdepot.client...
... SNIP ...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:13.000s
[INFO] Finished at: Mon Jul 15 13:13:20 HST 2013
[INFO] Final Memory: 27M/91M
[INFO] ------------------------------------------------------------------------
Before pushing a change to the GoogleCode repository, run mvn verify to ensure that everything is
valid. If that passes, create a commit with a detailed message of the change using
git commit -m MESSAGE
and then
git push origin master
To create a distribution, run mvn package.
Tag the most recent code revision that is included in the distribution. For example, if revision
8fef12173063 finalizes version 2.0.0504, use
git tag -a v2.0.0504 -m "Release 2.0.0504" 8fef12173063
Push the tag to GitHub with
git push origin v2.0.0504
Then upload the distribution .zip file in wattdepot/target/dist/ to the
Releases page. Include a summary of the
changes in the title and more detailed information in the description.
Add to the VersionHistory page describing the new version and what its changes mean for developers and clients. Include the tag name and a link to the download.
Post to the Discussion Group with a summary of the new revision and a link to the VersionHistory page.