Saturday, May 5, 2012

Configuration That Rocks! With Apache Commons

Using a modern library in your project is great fun, but often it is far better to resist the tempation and fallback to the tried and tested. In terms of storing configuration data in Java a proven solution is the Apache Commons Configuration framework.

What you will learn

You will learn how to:

  • fetch data from a XML file
  • access environment variables
  • combine different types of configuration (XML-based, environment-based, etc.)
  • automatically reload configuration after a change

In our example a database configuration will be stored in a XML file and the type of the environment (development, test, production, etc.) will be set as an environment variable. You will see more detail in a moment, but first lets configure Maven.

Maven setup

For our simple app we will need to add this dependencies to the pom.xml:

<dependencies>
    <dependency>
        <groupId>commons-configuration</groupId>
        <artifactId>commons-configuration</artifactId>
        <version>1.8</version>
    </dependency>
    <dependency>
        <groupId>commons-beanutils</groupId>
        <artifactId>commons-beanutils</artifactId>
        <version>1.8.0</version>
    </dependency>
    <dependency>
        <groupId>commons-jxpath</groupId>
        <artifactId>commons-jxpath</artifactId>
        <version>1.3</version>
    </dependency>
</dependencies>

Simple database configuration

Imagine we have a simple database configuration we want to store in a XML file:

<?xml version="1.0" encoding="UTF-8"?>
<!-- const.xml -->
<config>
    <database>
        <url>127.0.0.1</url>
        <port>1521</port>
        <login>admin</login>
        <password>pass</password>
    </database>
</config>

In order to fetch the url and port we can use this piece of code:

XMLConfiguration config = new XMLConfiguration("const.xml");

// 127.0.0.1
config.getString("database.url");  

// 1521
config.getString("database.port"); 

XMLConfiguration is an Apache Commons class that loads contents from the specified configuration file and provides a nice dot notation for accessing the values. For example the expression database.port maps to the node config/database/port with text "1521". Ofcourse there are more methods than getString to obtain data. Here are some other basic methods:

  • getBoolean
  • getByte
  • getDouble
  • getFloat
  • getInt
  • getInteger
  • getList
  • getLong
  • getStringArray
You can check more in the Apache Commons JavaDoc.

Evolving configuration

Suppose, after a while we decided that we would like to store configuration not only for single, but for multiple databases. We then change the const.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<!-- const.xml -->
<config>
    <databases>
        <database>
            <name>dev</name>
            <url>127.0.0.1</url>
            <port>1521</port>
            <login>admin</login>
            <password>pass</password>
        </database>
        <database>
            <name>production</name>
            <url>192.23.44.100</url>
            <port>1521</port>
            <login>admin</login>
            <password>not-so-easy-pass</password>
        </database>
    </databases>
</config>

Now to access the url values of the databases we can use this code:

XMLConfiguration config = new XMLConfiguration("const.xml");

// 127.0.0.1
config.getString("databases.database(0).url"); 

// 192.23.44.100
config.getString("databases.database(1).url"); 

As you can see to fetch consecutive values we used zero-based index inside parantheses, i.e. databases.database(1).url maps to the second node inside the databases parent.

Introducing XPath expressions

Dot notation is nice, but works only in simple cases. For complex, real-life situations a better solution may be to use the XPath expression language. The main advantage here is that you can write advanced queries against XML, while still being concise. Here is how you could rewrite the previous snippet:

XMLConfiguration config = new XMLConfiguration("const.xml");
config.setExpressionEngine(new XPathExpressionEngine());

// 127.0.0.1
config.getString("databases/database[name = 'dev']/url");        

// 192.23.44.100
config.getString("databases/database[name = 'production']/url"); 

...and here is an explanation of this two XPath queries [click to enlarge]:

Accessing the environment

With Apache Commons Configuration you can also read environment variables. Here is the code for getting the value of the ENV_TYPE environment variable:

EnvironmentConfiguration config = new EnvironmentConfiguration();
config.getString("ENV_TYPE");

It is assumed that the ENV_TYPE variable is set. To check whether you have done it correct, you can run the below script in the console:

echo %ENV_TYPE%        # for Windows
# or...
echo $ENV_TYPE         # for Linux/Mac OS

You should see as an output the value you have chosen for the ENV_TYPE variable.

Combining the configurations

Lets summarize what we have learned so far. Underneath is a getDbUrl method that does the following:

  1. checks the value of the environment variable called ENV_TYPE
  2. if the value is either dev or production it returns database url for the environment
  3. if the variable is not set it throws exception

public String getDbUrl() throws ConfigurationException {
    EnvironmentConfiguration envConfig = new EnvironmentConfiguration();
    String env = envConfig.getString("ENV_TYPE");
    if("dev".equals(env) || "production".equals(env)) {
        XMLConfiguration xmlConfig = new XMLConfiguration("const.xml");
        xmlConfig.setExpressionEngine(new XPathExpressionEngine());
        String xpath = "databases/database[name = '" + env + "']/url";
        return xmlConfig.getString(xpath);
    } else {
        String msg = "ENV_TYPE environment variable is " + 
                     "not properly set";
        throw new IllegalStateException(msg);
    }
}

Centralize your configuration

Creating multiple configuration objects for every configuration type seems a little bit awkward. What will happen, if we wanted to add another XML-based configuration? We should then create another XMLConfiguration object and we will end up with more and more distributed configuration code. A solution for the problem may be to centralize the configuration in a form of a single XML file. Here is an example of the XML file for our scenario:

<?xml version="1.0" encoding="UTF-8"?>
<!-- config.xml -->
<configuration>
  <env />
  <xml fileName="const.xml" />
</configuration>

To load this file you should use the DefaultConfigurationBuilder which will combine the env and xml configurations. The final version of the getDbUrl method would look like this:

public String getDbUrl() throws ConfigurationException {
    DefaultConfigurationBuilder builder = 
        new DefaultConfigurationBuilder("config.xml");
    boolean load = true;
    CombinedConfiguration config = builder.getConfiguration(load); 
    config.setExpressionEngine(new XPathExpressionEngine());
    String env = config.getString("ENV_TYPE");
    if("dev".equals(env) || "production".equals(env)) {
        String xpath = "databases/database[name = '" + env + "']/url";
        return config.getString(xpath);
    } else {
        String msg = "ENV_TYPE environment variable is " + 
                     "not properly set";
        throw new IllegalStateException(msg);
    }
}

Automatic reloading

There are many cool features in the Apache Commons Configuration. One of them is reloading file-based configuration when it gets changed. The framework will poll for changes every specified amount of time and if the contents of the file changed the responsible object is refreshed. To get it working you just have to add a file reloading strategy to your configuration. You can do it either programmatically:

XMLConfiguration config = new XMLConfiguration("const.xml");
ReloadingStrategy strategy = new FileChangedReloadingStrategy(); 
strategy.setRefreshDelay(5000);
config.setReloadingStrategy(strategy);

...or in the main configuration file:

<?xml version="1.0" encoding="UTF-8"?>
<!-- config.xml -->
<configuration>
  <env />
  <xml fileName="const.xml">
    <reloadingStrategy refreshDelay="5000"
      config-class="org.apache.commons.configuration.reloading.FileChangedReloadingStrategy"/>
  </xml>
</configuration>

This configuration results in a check for a file change every 5 seconds.

Want to try the examples yourself? Check out this github project.

Final thoughts

Apache Commons is my personal choice for the configuration-related code. I hope this article convinced you, that this framework may provide a very useful interface to your static data. What needs to be mentioned at the end is that not all features were covered in this article. Among many interesting are the abilities of the framework to:

  • load your configuration from different sources like properties files, ini files, a database, etc.
  • add new values to the configuration and save it back to the source it came from
  • listen for the configuration change event
  • automatically resolve the actual path to the configuration files (it does not matter whether you put them inside a jar or in the application folder)
I hope you enjoyed reading my text. Please, feel free to leave a comment.

16 comments:

  1. This is cool and simple.

    XML is more readable than JSON or YAML files. But some people get carried away with XML and start creating Schemas, JAXB bindings, Eclipse EMF bindings and stuff like that.

    This strikes a perfect balance, IMO - light, self-descriptive with free reload, env and include features.

    ReplyDelete
    Replies
    1. I cannot agree more. I especially liked the simplicity and ease of use. The first time I tried using it, it worked right away with no problems, which kind of surprised me ;].

      Delete
  2. You made my day. Thank You!!! :)

    ReplyDelete
  3. Thanks for writing about this. This is very useful for those applications that cannot use Spring or the newer frameworks that have built-in support for this.

    ReplyDelete
    Replies
    1. I presume the database configuration as an example lead you to the conclusion that Apache Commons Configuration can be only used for this. Well, it can do far more than that. I explained in the last section ("Final thoughts") what more can be accomplished with this framework. Maybe I should post another article presenting more features that will show Apache Commons Configuration from a different angle.

      Delete
  4. xml is more readable than json? really? xml has too much clutter for me.

    ReplyDelete
    Replies
    1. I think it all depends on your preferences and usually the preferences of your team/architects/team leaders (sometimes you do not have a choice).

      Personally, I would not bother which format I am using as long as it suits well for the whole team. XML fits here quite well, because everybody knows it.

      In my opinion it is far better here to leave your preferences behind and concentrate on good design, architecture and test coverage. At the end they are the things that matter the most to me.

      I want to be clear here. I did not mean to start a war. I wanted to present a tool I find really cool, which is only my opinion. It is up to you whether you will use it in your projects.

      Delete
  5. Excellent article. I would just add something on

    1] Multiple config options - Properties, XML, JNDI
    2] How the previous entry in the config file overrides the later ones.

    ReplyDelete
    Replies
    1. Thanks. Yes, you mentioned vital features of Apache Commons. I did not mention them in my article, because I wanted to keep the article short. I am aware that I only scratched the surface, because Apache Commons is such a great library with tons of features.

      By the way you could make a nice single article just about adding your own source of configuration :) (for example to store data in your company's proprietary format).

      Delete
  6. For Automatic reloading, if you set the auto-reload on the main file for a CombinedConfiguration, then you need to call "setForceReloadCheck(true)" on the CombineConfiguration object. Otherwise, changes in the included files (where the refresh option has been set on the top-level XML file) will not be detected. Refer to https://issues.apache.org/jira/browse/CONFIGURATION-240 for more detail.

    ReplyDelete
  7. thanks for a great write up, easy to follow, way to go apache

    ReplyDelete
  8. Thanks Mateusz! Just everything that I was looking for :P

    ReplyDelete
  9. Thanks Mateusz! I've found everything that I was looking for :P

    ReplyDelete
  10. Thank you very much for the tutorial.

    ReplyDelete
  11. Thanks a ton for a nice tutorial

    ReplyDelete