Category Archives: Java

Using Groovy Templates and Filters Java

I have written about the Groovy language a few times. It is a wonderful language and integrates seamlessly with Java. Recently, I have found that it can be very useful for templates and filters.

Using Groovy for Filtering

Consider an application where there are many events that fire on certain conditions. The Observer pattern is often used for these types of event handlers. Also consider that the user (not the developer) wants to restrict or filter out events so only the ones they want are fired by the handler. Essentially the user needs a filter language where they can express their business rules. This filter needs to exist outside of the regular codebase so the user can change it on the fly.

Groovy solves this problem very neatly. Consider an Event class with the following properties:

  • severity
  • timesCalled
  • message

Assume that an instance of this class is provided as the object ‘e’. In groovy we could express a filter script that takes events with a severity less than 2 or times called greater than 5 as the following:

e.severity < 2 || e.timesCalled > 5

The above script is pretty simple and is easily written by a technical savvy user. I have written a simple helper class that facilitates creating reusable scripts like our filter script. Here is the code:

public class ReusableScript {
    private GroovyClassLoader gcl = new GroovyClassLoader(getClass().getClassLoader());
    private Class scriptClass = null;
    public void setScriptString(String scriptString) throws UnsupportedEncodingException {
        if (scriptString == null || scriptString.trim().equals("")) {
        InputStream in = new ByteArrayInputStream(scriptString.getBytes("UTF-8"));
        scriptClass = gcl.parseClass(new GroovyCodeSource(in, "Filter", ""));
    public Script newScript() throws Exception {
        if (scriptClass == null) {
            return null;
        return (Script) scriptClass.newInstance();

To use this script to actually execute the filter, you would do something like this:

// Creating the filter script. Assumes filterString is set by the user (I use spring for this)
ReusableScript filter = new ReusableScript();

// Typically this code would be running in the event handler. Assumes that an object e is available that is the event
filterScript = filter.newScript();
// Bind the 'e' object into the binding context of the script
filterScript.setProperty("e", e);
Object result =;
if (result instanceof Boolean && ((Boolean) result).booleanValue() == true) {
    // The filter has passed.
    // ... Code that performs the operation

Very little code is actually needed to put this together. The ReusableScript class can be used many times. The groovy code is only compiled once so performance is very good.

Using Groovy for Templating

A similar problem is the problem of templating. Consider an application that sends emails on error conditions. The user may wish to customize the email message that is sent out. They also may want to be able to use information from the application to provide meaning to the message. There are many template and expression languages available for Java. I have looked at using Apache Commons EL and Velocity. My main complaint on these languages is their complexity. It requires learning a new language syntax and integrating them often requires quite a bit of code. Let’s consider a more groovy solution.

Using the same ReusableScript class as used for filtering, we can provide templating. Consider a data class called Event that contains the following properties:

  • message
  • detail
  • hostname

Consider the following Groovy script that creates the body of an email message:

"""[$e.hostname] An error occurred: $e.reason


This may produce a message that looks like this:

[localhost] An error occurred: File not found: abc.txt

FileNotFoundException ...

This is essentially making use of Groovy’s Templates that are built in to the language. The template engine can be used by itself standalone, but this provides a simpler integration. The other benefit is the if more complicated logic is required beyond a simple template, actual groovy code can be written beyond the simple script. The script is a full groovy script and would allow you to essentially do anything from it.

The template script would be loaded the same way as our previous filter script. This time instead of evaluating the script as a boolean, we get it as a string.

// ... Setup of filter script ...

Object result =;
if (result instanceof String) {
    // Code that uses the string

So that is two easy ways the Groovy can be used in a Java application to solve some of these simple problems.

When to do Unit Testing

I was listening to the Stackoverflow podcast number 41 today. One of the issues discussed was unit testing and when is it beneficial to do unit testing. I agreed with some of the points but not all of them. The point I especially found interesting was the “eating your own dogfood” concept where writing tests forces you to use the interfaces you created. I also never considered the perspective on tests serving as documentation or a spec on how the system should work.

There are lots of good reasons to do unit testing but there are a few issues I have run across with it that go beyond the obvious ones of time and effort.

  • Bugs in the tests – Let’s say a hypothetical programmer creates a bug in every 100 lines of code he writes. Logically, that would mean he would be creating a bug in every 100 lines of test code also. Unit tests do provide a double check to validate your code but a bug in the unit test could give you a false positive that your code works. A programmer who writes bad code will also write bad unit tests so even additional unit test coverage will result in poor code quality.
  • Tests are Bias – Unit tests are written by the programmer who wrote the code. If the programmer misunderstands how the code works or should be used (even if they wrote it), then it is likely that the test will also be built around this misunderstanding.
  • Test Failures due to design changes – Most times when I see unit tests fail it is because the code that was being tested was intentionally changed. Often the test is the last piece of code to be updated (right or wrong) to the new functionality of the rest of the system.
  • Code that Must be Maintained – Tests are additional code that developers need to maintain. I try to write as little code as possible to keep it clear and simple which makes it hard to justify writing a lot of extra code.

Writing tests requires effort. They take time to develop and maintain but they do have a payoff. I try to find a balance between the “100% test coverage” people and the “we don’t need any tests” people. The following is a list of criteria I use as a general guide to what I feel is most important to test. I choose the stuff that is most important areas to test.

  • Libraries – I always make sure I write tests for common libraries or utilities. These are generally easiest to test and are designed to be used in many places. Testing these has the biggest payoff.
  • APIs – APIs are a contract between systems. These are logical choices for tests as they should not be changing often and have strict rules.
  • Business Logic/Rules – Anytime there are business rules or logic that perform some defined function where there are clear inputs and outputs, these should be tested.

Libraries are usually number one on my testing priority. I usually aim for 90% or more coverage. GUI testing I have found to be of little benefit since the effort is so high and you end up with very little reward. Glue code or anything that ties different components together is hard to test and are more easily tested with basic smoke tests on the complete product.

Write as Little Code as Possible

The Zune Bug

A few weeks ago I heard of this issue where the Microsoft Zune crashes and won’t startup on December 31, 2008. The reason for this? A bug in the software that handled leap years. There are lots of articles on the original issue.

When I heard of this issue, I originally thought ‘How could that happen?’. Sure I could understand not handling things correctly when it is a leap year but causing a crash?

Well, if you like me were wondering how that could happen, you can now find out for yourself. This is a post someone made of the actual code that runs on the Zune. Look at line 249 and on.

If you missed it, there is a bug (obviously) where if the number of days passed in is 366, which is the case for December 31, 2008, the loop never terminates. The code checks to see if days is greater than 365 in a while loop. It then handles the greater than 366 condition but never checks if the days is equal to 366.

Write as Little Code as Possible

The less code you write the fewer bugs you will have in it. Most languages today have built in support for common tasks. One example of this is the Java Calendar object which would allow a developer to do what the zune code does using the platform APIs. Unfortunately, High level languages are usually not an option when writing code for a small embedded device.

Tips for writing as little code as possible:

  • Start with a high level language. Preferably use an agile and dynamic language if possible (Groovy, Ruby and Python etc). This of course depends largely on the requirements of the application.
  • Where possible using the built in platform APIs to do what you need.
  • Use open source software to fill in the gaps of platform APIs. Apache and Codehaus are great sources of open source software that are commercial friendly to ship.

Getting access to SOAP Headers from an Apache CXF service implementation

For one project I was working on, I needed to get access to the SOAP headers of web service call. I am using the Apache CXF services stack. There were quite a few threads on how to get access to SOAP headers from an interceptor. In my case, I needed the contents of the header inside the implementation.

wsdl2java does have the ability to create a java service class that provides access to the headers using the -exsh option. This was not an option for me since even with this flag on, the headers were not added to the service calls. I think it was how the WSDL I had was designed. This meant that I needed to do the work of pulling out the headers myself.

The SOAP headers can be retrieved from the JAX-WS SOAPMessageContext the easiest. Getting access to this was not trivial. I added the resource annotation to get access to the WebServiceContext. Unfortunately the MessageContext this gave me was not a SOAPMessageContext and provided no way to access the SOAP headers.

After looking through the code for the SoapMessage CXF class, I found how it gets the headers out of the Message. I came up with the following to access the headers from my implementation class:

private WebServiceContext context;

private List<Header> getHeaders() {
    MessageContext messageContext = context.getMessageContext();
    if (messageContext == null || !(messageContext instanceof WrappedMessageContext)) {
        return null;
    Message message = ((WrappedMessageContext) messageContext).getWrappedMessage();
    List<Header> headers = CastUtils.cast((List<?>) message.get(Header.HEADER_LIST));
    return headers;

This provides all of the headers available. To get the specific one I needed using JAXB I added the following to my code:

List<Header> headers = getHeaders();
if (headers != null) {
    for (Header h: headers) {
        Object o = h.getObject();
        // Unwrap the node using JAXB
        if (o instanceof Node) {
            o = getJaxb().createUnmarshaller().unmarshal((Node) o);
        if (o instanceof DesiredHeaderType) {
            // Do whatever is required with the header object instance

This way of accessing the headers turns out to be much simpler than writing an interceptor and trying to stuff the results of that into the request.

Detecting Empty XML Elements with JAXB

I use JAXB in conjunction with Apache CXF for web services. One requirement for the data model is to allow data fields to be “cleared”. This is done by entering no value in the XML. Data elements of the model can also be left out completely. This means that the value should not be set and is very different from a clear. Consider a JAXB class with one string field, one number field and one date field.


The previous XML indicates that the string, number and date values should all be cleared. The actual values you would get in the data model are as follows:

string-value: "" (empty string)
number-value: NULL
date-value: NULL

If no XML was passed in, the following would be the resulting values in the data model:

string-value: NULL
number-value: NULL
date-value: NULL

For string values, you can tell if the user intends to clear the value by checking if the value is empty string. The problem comes in when you look at the number and date fields. You get NULL if the XML element is present or not. This makes it impossible to tell the difference between no XML being specified and an empty XML element.

The solution I came up with was to use a special number and date to denote a clearable value. I wanted to pick numbers that should never occur in the data model so I chose the following:

Date: Epoch, 1970-01-01T00:00:00.0-00:00
Number: Integer.MIN_VALUE, -2147483648

In JAXB, I set these values to the “default value” in the XML schema. This means that if the user types in the XML element by itself with no text value, it should be treated as the default value. If the user leaves the element out, according the the schema spec, the default value should not be used. This is exactly the behaviour I want for the “clearable” fields.

On the data model class, I added the annotations on dates and numbers respectively (actually I used a constant):


Now, if you have the original XML example:


You will get the following values in the data model:

string-value: "" (empty string)
number-value: -2147483648
date-value: new Date(0) (1970-01-01T00:00:00.0-00:00)

Now we can easily differentiate between an empty XML element and leaving out the element altogether.

HTTP Basic Authentication with Apache CXF Revisited

I receive a lot of traffic to my post about HTTP Basic Authentication in Apache CXF. I decided to do a followup to that post to address some of the comments.

I have never tried to use this with Mule but if someone has, please let me know so I can update this post.

I have uploaded the Java code for the BasicAuthAuthorizationInterceptor class. There are a few changes over the original version. This one includes a Map of authorized users and their corresponding passwords. I believe the original example I created was for Apache CXF 2.0. This version works with Apache CXF 2.1.1.

In the original post, I also did not include a sample of how to use this code in a real application. The following section shows a sample of how to define the security interceptor and enable it on a simple endpoint.

<beans xmlns=""
    <bean id="securityInterceptor" class="BasicAuthAuthorizationInterceptor">
      <property name="users"> 
          <entry key="username" value="password"/>

    <bean id="service" class="sample.Service"/>
        <ref bean="securityInterceptor"/>

Building with Maven

I’ve decided to use Apache Maven for building the code for my new project. So far I have had a love-hate relationship with Maven. If you don’t know what maven is, the folks over at Apache say Maven is …

“… a software project management and comprehension tool. Based on the concept of a project object model (POM), Maven can manage a project’s build, reporting and documentation from a central piece of information.”

Unfortunately that definition is about as vague as you can get so I will explain what Maven gives me and my project. This also happens to be the list of what I love about Maven:

  • Maven enforces a standardized project structure to all modules in a project.
  • Maven handles dependencies between libraries.
  • Maven builds all of my code and creates a distribution package.

Essentially by using Maven, I can get all of these things without having to write and maintain ant scripts to do most of the work for me. Maven is not without its problems though.

  • Often times the number of lines of XML you need to write ends up to be just as complicated and long as an ant script to do the same thing.
  • I’ve found a lot of bugs particularly with the assembly module. It seems to include dependencies of modules that are set with the “compile” or “test” scope only.
  • The Groovy building plugin for maven does not yet work for Groovy 1.5.x.

I still think using maven was a good choice for my project. It may have some qwerks but I think that I am further ahead than if I had to build everything from scratch.

Using Maven to Generate a Build Number

Earlier I metioned that I use the subversion revision number as a build number. This is actually quite easy to do from Maven. The following plugin descriptor provides you access to the subversion revision number using the ${scm.revision} property.


Once you have access to the revision number, you can construct a build number using the product version and the revision number. I have added this build number to the Implementation-Version property of the jar manifest. The following plugin definition does this for me:


The resulting manifest in the jar looks something like this:

Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Created-By: Apache Maven
Built-By: cdail
Build-Jdk: 1.6.0_03
Implementation-Version: 1.0-SNAPSHOT.93

From Java code, this build number can be easily retrieved. The following code retrieves the Implementation-Version from the manifest:

Package p = getClass().getPackage();
String version = p.getImplementationVersion();

The result is quite elegant but it takes a bit of work to get things into place. Like with many other things to do with Maven, it takes a while to figure out how to do what you want and you need to write a bunch of XML. Once that was done, everything fell into place. It is hard to say at this point if this is better or worse than the chunk of ant script I used to use to perform the same function.

Apache CXF with HTTP Basic Authentication

Apache CXF is a great library for providing web services from Java. Their integration with the spring framework is the best of any web services library I have seen. But when it comes to documentation, they are a bit lacking in a few areas. Some tasks that you would think would be part of the framework are simply not provided.

There are two approaches to providing security for Web Services. You can secure the protocol itself (HTTP based security) or you can secure the web services call itself (the actual soap message). The new web services standard around security have opted to take the second approach. Web services and SOAP are designed to be separate from the protocol. The WS-Security specifications deal with securing the SOAP packet itself rather than relying on security provided by the underlying protocol (in many cases HTTP).

Apache CXF provides a WS-Security implementation for handling security and authentication for web services. They also provide a mechanism for setting up HTTPs to provide a secured HTTP connection. The one thing they do not provide is a mechanism to authenticate over HTTP. If you have the luxury to use WS-Security then that is not a problem. For a project I was working on, I needed to provide a web service older clients could access that may not know about WS-Security. The solution was to use HTTP Basic Authentication.

Apache CXF provides access to the AuthorizationPolicy from any message. This gives access to the authentication information used for HTTP. The approach I took was to create a CXF interceptor to intercept the message and check this information for HTTP Basic Authentication parameters. If an error occurs then a SOAP fault is thrown.

The problem with this approach was that a fault is returned to the client as an HTTP 500 response code. What we really want here is to use the 401/403 mechanism as a true web server would for HTTP based authentation. I found the following article on the CXF mailing lists that explained how to set some of that up.

I modified that code provide an CXF interceptor that provides HTTP Basic Authentication. When a message is received the HTTP headers are checked. If no user/password is provided, a 401 is returned. This indicates to the client the HTTP authentication is required. If the user/password is invalid, a 403 is returned indicating that this username/password is forbidden to access the service.

Here is an excerpt from the code. I took out some of the implementation details and left the basic “meat” of how to do this from CXF.

public class BasicAuthAuthorizationInterceptor extends 
        SoapHeaderInterceptor {
    private Map<String,String users;
    protected Logger log = Logger.getLogger(getClass());
    @Override public void handleMessage(Message message) 
            throws Fault {
        // This is set by CXF
        AuthorizationPolicy policy = message.get(
        // If the policy is not set, the user did not specify
        // credentials. A 401 is sent to the client to indicate
        // that authentication is required
        if (policy == null) {
        // Verify the password
        String realPassword = getAcualPassword(
        if (realPassword == null || 
                !realPassword.equals(policy.getPassword())) {
            log.warn("Invalid username or password for user: " 
                + policy.getUserName());
    private void sendErrorResponse(Message message, 
            int responseCode) {
        Message outMessage = getOutMessage(message);
        // Set the response headers
        Map responseHeaders = message.get(
        if (responseHeaders != null) {
                Arrays.asList(new String[]{"Basic realm=realm"}));
                Arrays.asList(new String[]{"0"}));
        try {
        } catch (IOException e) {
            log.warn(e.getMessage(), e);
    private Message getOutMessage(Message inMessage) {
        Exchange exchange = inMessage.getExchange();
        Message outMessage = exchange.getOutMessage();
        if (outMessage == null) {
            Endpoint endpoint = exchange.get(Endpoint.class);
            outMessage = endpoint.getBinding().createMessage();
        return outMessage;
    private Conduit getConduit(Message inMessage) 
            throws IOException {
        Exchange exchange = inMessage.getExchange();
        EndpointReferenceType target = exchange.get(
        Conduit conduit =
            inMessage, null, target);
        return conduit;
    private void close(Message outMessage) 
            throws IOException {
        OutputStream os = outMessage.getContent(

One More Level of Abstraction – Commons Logging

The typical way to solve most problems in languages such as Java is to create one more level of abstraction. One example of this is Apache Commons Logging. There are many different logging libraries used in Java. Log4j is the big one that has been around for many years. Developers had been looking for a logging library to be part of the Java platform for years and this finally happened in Java 1.4 with the java.util.logging package. At this time, log4j was already the defacto standard for logging on the Java platform. The java.util.logging package lacked in features and integration with existing logging infrastructure like log4j. To this day, not many people are using the built in logging capabilities because things like log4j are just so much better.

Apache Commons Logging enters the scene to provide a level of abstraction around different logging packages. It provides a thin layer around many different logging implementations like Log4j, java.util.logging, LogKit and others. This is a great solution if you are creating an embedded application where the logging needs to be done through some other application’s facility.

A lot of projects started using Commons Logging but encountered a lot of problems using it. This is specifically problematic in application servers when dealing with complex classloader hierarchies. The Think again before adopting the commons-logging API article explains some of these issues in more detail. There is even an apology by the author of Commons Logging.

“The purpose of Commons Logging is not to isolate your code from changes in the underlying logging framework. (That’s certainly easy enough to do on your own, and not really worth doing in the first place given the ease of switching from one logging framework to another.) The purpose of Commons Logging is not to somehow be more useful than actual logging frameworks by being more general. The purpose of Commons Logging is not to somehow take the logging world by storm. In fact, there are very limited circumstances in which Commons Logging is useful. If you’re building a stand-alone application, don’t use commons-logging. If you’re building an application server, don’t use commons-logging. If you’re building a moderately large framework, don’t use commons-logging. If however, like the Jakarta Commons project, you’re building a tiny little component that you intend for other developers to embed in their applications and frameworks, and you believe that logging information might be useful to those clients, and you can’t be sure what logging framework they’re going to want to use, then commons-logging might be useful to you.” –

Tomcat uses commons logging. This can create all sorts of strange classloading problems when you deploy a web application that also uses commons logging. There are different versions of commons logging to contend with and difficulties in commons logging to find the actual logging implementation because of the parent tomcat classloader.

It is clear that one more level of abstraction is not always the key to solving all of your problems. In this case, Commons Logging was created for one specific problem and that is for small embedded components. This abstraction may make sense for things like an web application server. When it comes right down to it though, if you are writing a web application server, it is up to you to dictate how logging is to be done for your application. Most Commons Logging implementations in software will end up using Log4j anyway. So if you control the application environment, just use Log4j.

Loading Java Applications

Executing an application in Java is hard. When you work with a language that compiles to native code, the artifact created from your build process is a native executable. If you are using an interpreted language or a language that compiles to bytecode, a runtime environment is required to run the program.

Invoking the Java runtime is simple but for a typical enterprise application, many parameters are required to be passed to the runtime. These parameters consist of arguments to the virtual machine (like memory options), program parameters and classpath parameters.

For a large application, there are often many .jar files that need to be included on the classpath. The software author can create a shell script or small native executable that calls Java with all of the appropriate jar files.

A good alternative to having to specify all of the .jar files in the script is to use a simple boot loader jar. The following two classes make a very simple boot loader that can be used to start a much larger application.

public class ClassLoaderFactory {
    public static ClassLoader createClassLoader(File[] files) {
        List<URL> urlList = new ArrayList<URL>();

        for (File file: files) {
            addURLs(file, urlList);
        URL[] urls = urlList.toArray(new URL[0]);
        return new URLClassLoader(urls, 

    private static void addURLs(File file, List<URL> list) {
        if (file.isDirectory()) {
            File[] files = file.listFiles();
            for (int i = 0; i < files.length; i++) {
                addURLs(files[i], list);
        else if (file.isFile() && file.getName().endsWith(".jar")) {
            try {
            catch (MalformedURLException e) {
                // This should not happen. Ignore this file.

public class Launcher {
    public static void main(String[] args) {
        String mainClass = null;
        List<File> libraryList = new ArrayList<File>();
        List<String> actualArgsList = new ArrayList<String>();

        for (String arg: args) {
            if (arg.startsWith("-l")) {
                String libraryPath = arg.substring(2);
                libraryList.add(new File(libraryPath));
            else if (mainClass == null) {
                mainClass = arg;
            else {

        if (mainClass == null) {
            System.err.println("No main class specified");

        // Load classloader based on the library path
        ClassLoader classLoader = 
            libraryList.toArray(new File[0]));

        // Execute the main class
        Class bootClass;
        try {
            bootClass = classLoader.loadClass(mainClass);
            Method main = bootClass.getMethod("main", 
                new Class[] {String[].class});
            main.invoke(null, new Object[] {
                actualArgsList.toArray(new String[0])});
        catch (Exception e) {

You will also need to create a META-INF/MANIFEST.MF file along with these classes. This should specify which is the main class to be run. The contents should be as follows:

Manifest-Version: 1.0
Implementation-Title: XML Transport Bootloader
Main-Class: com.whitehill.bootstrap.Launcher

Those classes and manifest should be packaged into a single jar file called boot.jar. Note that these should be the only classes in boot.jar and it should not include any classes for your running application. If not, you may end up with some classloading issues.

Now to run your application, create the following directory structure

/ – Root directory where the application is run from. You can place a batch file or .exe to start the software here.
/lib – Directory to contain all the .jar files for the application. You can break it up into subdirectories for organizational purposes.

Now to run the application, you can simply run the following:

	java -jar boot.jar -llib com.softwareco.MainClass

This command will start the Java Virtual Machine and run the boot.jar application. This application parses the arguments “-llib com.softwareco.MainClass” and adds all .jar files in the lib directory to the classpath and runs the com.softwareco.MainClass of your application. This command can be placed into a script to make it easier for the user and include any JVM options.