Writing Java Performance Tests in Groovy

In my previous post, I mentioned writing performance tests anytime you need to do optimization to slow areas of code. Writing effective performance tests can be tedious in Java. Every test you want to run in Java needs to have the same timing logic before and after the tests. Groovy‘s Closures make separating timing code from actual test implementations easy. I write all my performance tests in groovy because it simplifies the testing code logic and allows me to focus on what I am trying to test.

The basics of a performance test is to get the current time before the test, run the test and then get the time after. In Groovy, we can use a closure to express this.


def timeit = {String message, Closure cl->
    def startTime = System.currentTimeMillis()
    cl()
    def deltaTime = System.currentTimeMillis() - startTime
    println "$message: \ttime: $deltaTime" 
}

This allows you to call a test like this:


timeit("Test 1") {
    // This would be the code you want to test
    Math.pow(2, 6)
}

If you are going to be writing many tests, this format is much shorter than having to constantly repeat the currentTimeMillis() calls in Java. Also, no heavyweight testing framework is required. The ‘message’ that is passed in is a convience method so the output of the test can be distinguished. The results look like this:


Test 1: 	time: 0

Right away you will notice that a time of 0 milliseconds is not that useful. The code simply ran too fast to measure. Yes we could use nanoseconds and may get better results. What I prefer to do is to run the test many many time and to take the average. This way you get an average of how fast it is and it provides more repeatable numbers.

Updating the groovy closure, we end up with the following that runs the test 500 times.


def timeit = {String message, int count=500,  Closure cl->
    def startTime = System.currentTimeMillis()
    count.times { cl() }
    def deltaTime = System.currentTimeMillis() - startTime
    def average = deltaTime / count
    println "$message:\tcount: $count \ttime: $deltaTime \taverage: $average" 
}

The output of this looks like this:


Test 2:	count: 500 	time: 18 	average: 0.036

Another thing that should be considered in Java is discounting the first few runs. The first time Java executes a particular class, things are always slower. Some work has to be done by the Java VM to load all of the classes for the first time. Subsequent invocations of the same code get faster. To account for this, I include a warming period in the tests. Essentially I run the code to be testing a bunch of times before I record the time. This discards these initial runs that will be slower. The closure for this looks like this:


def timeit = {String message, int count=500, Closure cl->
    // Warming period
    20.times { cl() }
    def startTime = System.currentTimeMillis()
    count.times { cl() }
    def deltaTime = System.currentTimeMillis() - startTime
    def average = deltaTime / count
    println "$message:\tcount: $count \ttime: $deltaTime \taverage: $average" 
}

The output of this looks like this:


Test 3:	count: 500 	time: 6 	average: 0.012

Another thing you might want to do is to run a multi-threaded test. In Java, this would require quite a few extra classes. In groovy, a simple modification to this closure can allow it to be run in multiple threads. Here is the new closure and test call calling it with 5 separate threads:


def timeit = {String message, int numThreads=1, int count=500, Closure cl->
    // Warming period
    20.times { cl() }
    def startTime = System.currentTimeMillis()
    count.times {
        def threads = []
        numThreads.times { threads << new Thread(cl as Runnable) }
        threads*.start()
        threads*.join()
    }
    def deltaTime = System.currentTimeMillis() - startTime
    def average = deltaTime / count
    println "$message:\tcount: $count \ttime: $deltaTime \taverage: $average" 
}

timeit("Test 4", 5) {
    Math.pow(2, 6)
}

An extra parameter to the timeit closure allows you to pass the number of threads to concurrently execute. The results are the following:


Test 4:	count: 500 	time: 465 	average: 0.93

As you can see, groovy makes it much easier to write performance tests for Java code. The closure listed above can be used in all sorts of different projects to test performance. Hopefully this snippet will make your life easier when it comes to writing your own performance tests.

3 thoughts on “Writing Java Performance Tests in Groovy”

  1. Aren’t closures a beauty! They provide such elegant solutions to so many design situations (think command, strategy etc…) and seem such a natural approach. And coming from the overly verbose Java world, they really are a pleasure to use.

    Great post!

  2. @kodeninja Thanks for your comment and I agree. I love closures a lot and I especially enjoy how natural they feel in Groovy. I’m sorry that feature looks like it might miss JDK 7 but I guess that just means we all need to keep writing more Groovy!

Comments are closed.