EasyMock Tests Are Usually Overspecified

Summary

EasyMock is a fantastic tool for making unit testing easier and better. It lets you create dummy objects that implement an interface or simulate a full class with just a single line of code:

EasyMock.create(LineInputReader.class)

However, when specifying the calls expected by the class and the return values, it is easy to get into a trap of over-specification, where the tiniest change to the internal implementation of the unit being tested causes a failure in the unit test.

Details

When specifying the expected method invocations and return values for the mock objects, EasyMock remembers the count and order of the calls. Later, after resetting the mock objects via their (strangely named) replay() method, you can confirm that they were called the expected number of times with the expected parameters by invoking verify() on the mock objects.

The problem comes when you verify each and every mock object down to the finest detail. You can encounter circumstances in which you don't care whether a getter is called 2 or 3 times, or if getA() is called before or after getB(). However, since EasyMock checks all of these things, if you use verify() on the fine-grained objects, minor implementation changes (that still produce correct behavior) can cause unit test failures.

Solution

The solution is to reserve calls to verify() for the less-fine-grained mock objects, one where the count and order of invocations is critical. An alternative is to use anyTimes() to turn off the exact count matching for a mock object's method, but leave count checking in place for other methods.

Last modified on 27 Jan 2009 by AO