Wrong time of method execution

I want to test the time of adding and getting item in simple and generic hashmap:

public void TestHashGeneric(){


        Map hashsimple = new HashMap();             

        long startTime = System.currentTimeMillis();



        for (int i = 0; i < 100000; i++) {
            hashsimple.put("key"+i, "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" );
        }

        for (int i = 0; i < 100000; i++) {
            String ret =(String)hashsimple.get("key"+i);
        }
        long endTime =System.currentTimeMillis();

        System.out.println("Hash Time " + (endTime - startTime) + " millisec");

        Map<String,String> hm = new HashMap<String,String>();

        startTime =  System.currentTimeMillis();



        for (int i = 0; i < 100000; i++) {
            hm.put("key"+i, "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" );
        }

        for (int i = 0; i < 100000; i++) {
            String ret = hm.get("key"+i);
        }
        endTime = System.currentTimeMillis();

        System.out.println("Hash generic Time " + (endTime - startTime) + " millisec");         

    }

The problem is that I get different time if I change places between hashmap's code section! if i put the loops (with the time print ofcourse) of generic below the simple I get better time for generic and if i put simple below the generic I get better time for simple!

Same happens if I use different methods for this.

Answers


The JIT will compile and optimise your program while it is running, so the 2nd run will always be faster.

You should make the following modifications:

  1. Run both tests untimed first, then re-run them timed so that you don't get affected by the JIT.
  2. You should use System.nanoTime() as this is more accurate for timing (you should never get a diff of 0).
  3. You should also test against some empty methods, as you are also timing the string concatenation operation in each loop.

Also note that in Java generic types are erased, so there should be no runtime difference at all.


When you have a single loop which reaches the compile threshold (about 10K) the whole method is compiled. This can make either the first loop appear faster (as it is optimised correctly as the second loop has no counter information) or the second loop appear after (as its compiled before it is even started)

This simplest way to fix this is to place each test in its own method and they will be compiled and optimised independently. (The order can still matter but is less important) I would still suggest running the test a number of times to see how the results vary.


The Java Runtime is pretty sophistocated - it does some learning/optimisation while it's running. You might be able to get the test you want by "warming up" the JVM first. Try calling TestHashGeneric() twice and see what the second set of results gives you.

You've also got twice as much stuff in memory on the second run. There's all sorts of variables that can affect this test.


This is not the correct way to perform micro-benchmarks as it is very involved (Why?). I suggest you use Caliper framework for this.


Need Your Help

Scala's SBT and REST Java Client for JIRA

scala sbt jira jira-rest-api jira-rest-java-api

I am trying to add REST Java Client for JIRA to my Scala project.

Cross server websockets

apache http-headers websocket

I get this error when I try to connect to my websocket server:

About UNIX Resources Network

Original, collect and organize Developers related documents, information and materials, contains jQuery, Html, CSS, MySQL, .NET, ASP.NET, SQL, objective-c, iPhone, Ruby on Rails, C, SQL Server, Ruby, Arrays, Regex, ASP.NET MVC, WPF, XML, Ajax, DataBase, and so on.