Peformance Testing Your Java Methods With JUnit and CarrotSearch JUnitBenchmark Tool – With Source Code

I am always in the favor of Test Driven development (TDD), so in addition to following it myself, I always have my developers write a failing test before they begin the coding for any User Story. Once they write the code, we then run the unit test in order to pass them. Folks have a habit of ALWAYS writing unit tests for your code, that will make you a profesional developer.

In this article, I am not going to talk about TDD. But rather I will show you how you can performance test your methods using JUnit and Junit BenchMark tools. The idea behind is, lets say your wrote several methods, and you unit tested those methods using one of the several unit testing methodologies. But how do you make sure it performs well? You know what I mean. You don’t expect your methods to take forever, so unless you measure how much time it takes, you never know. So, I will share my experience with JUnit benchmark tool and show you how easy it is to performance test your individual method.

Unless you already are using these tools (JUnit and JUnit Benchmark Tools), you will need the following two jar files added to the build path of your project.

  1. JUnit Framework Jar File (Visit their site here)
  2. JUnit Benchmark Jar File (visit their site here)

The jar files I am using areĀ junit-benchmarks-0.3.0.jar and junit-4.8.2.2.jar

The following LoopPerformance.java class contains few simple methods. I am not only interested in writing unit tests for those methods, I am also interested to find out if these methods perform well.

package com.kushal.tools.junitbenchmark;

/**
 * @author Kushal Paudyal
 * www.sanjaal.com/java
 * www.icodejava.com
 * 
 * Last Modified on: 2012-11-27
 * 
 * This is just a simple class that will be unit and performance
 * tested using JUnit and JUnit Benchmark tool
 */
public class LoopPerformance {

    private final int MAX_ITERATIONS = 100;

    // Do some stuffs in for loop
    public void doForLoop() {

        for (int i = 0; i < MAX_ITERATIONS; i++) {
            /**
             * Some statements. Not intended to have any meanings
             */
            String str = "Hello";
            str += i;
        }
    }

    // Do some stuffs in while loop
    public void doWhileLoop() {

        int i = 0;
        while (i != MAX_ITERATIONS) {
            String str = "Hello";
            str += i;
            i++;
        }
    }

}


The following is a unit test class that I wrote. But it also has some annotations that attach bench marking code to the test.
For example the @Rule annotation is telling the Junit runner to use BenchmarkRule. Although this much is enough to do the performance testing your method using the JUnit test methods, we can further fine tune the test methods with additional annotations.

One example annotation that you can add in from of your unit test method declaration with how many times the methods should run and how many initial warm up rounds are done is:
@BenchmarkOptions(benchmarkRounds = 100000, warmupRounds = 10)

What this means is your test method will be run for 100,010 times but the intial 10 times are considered warm up rounds, so their performance data will not be considered.

Here is another annotation example with parameter to disable garbage collection via callgc parameter.
@BenchmarkOptions(callgc = false, benchmarkRounds = 20, warmupRounds = 3)

package com.kushal.tools.junitbenchmark;

/**
 * @author Kushal Paudyal 
 * www.sanjaal.com/java 
 * www.icodejava.com
 * 
 * Created On: 2012-10-22 
 * Last Modified On: 2012-10-22
 */
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.MethodRule;

import com.carrotsearch.junitbenchmarks.BenchmarkOptions;
import com.carrotsearch.junitbenchmarks.BenchmarkRule;

public class JUnitBenchMarkDemoTest {

    // Add a JUnit4 rule that tells JUnit runners to attach benchmarking code to
    // your tests
    @Rule
    public MethodRule benchmarkRun = new BenchmarkRule();

    /**
     * We can fine tune the benchmark with additional 
     * annotated parameters using benchmark options.
     */
    @BenchmarkOptions(benchmarkRounds = 100000, warmupRounds = 10)
    @Test
    public void testDoForLoop() throws Exception {

        LoopPerformance performance = new LoopPerformance();
        performance.doForLoop();
    }

    @BenchmarkOptions(benchmarkRounds = 100000, warmupRounds = 10)
    @Test
    public void testDoWhileLoop() throws Exception {

        LoopPerformance performance = new LoopPerformance();
        performance.doWhileLoop();
    }
}


Here is the output of the performance test run. The output shows what is the total time taken, how many times the test was run for, how much time was spent in garbage collection etc.

JUnitBenchMarkDemoTest.testDoForLoop: [measured 100000 out of 100010 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.01], round.gc: 0.00 [+- 0.00], GC.calls: 1503, GC.time: 0.34, time.total: 5.62, time.warmup: 2.85, time.bench: 2.76
JUnitBenchMarkDemoTest.testDoWhileLoop: [measured 100000 out of 100010 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 1503, GC.time: 0.35, time.total: 2.54, time.warmup: 0.00, time.bench: 2.54

Please comment on this post (make sure you have registered to my blog), and leave me your feedback.