To Mock Or Not To Mock

Noel Rappin, Friday, July 10, 2009 @ 4:55 pm

Today, bereft of post ideas, I asked my vast Twitter army (which, if you subtract the bots, probably numbers in the dozens...) for topics, and the leading (not to mention only) vote-getter was "a good rant on mocking vs not-mocking"... Well, I can't promise a good rant, but here goes nothin'

Here's my starting point: Mock objects are best used in the following 2 1/2 situations:
1. To simulate an external object that would otherwise be prohibitive or impossible to use during testing. The classic example is a web service such as PayPal or an OpenId server or something.

2. As a shortcut to setting up a system state that would be difficult or impossible to set up step by step. My favorite example at the moment is Timecop, which is basically a specialized mock object package for setting and freezing the system time.

Here's the half situation:

2 1/2. Sometimes it's appropriate to use a mock object to fake a method call from another layer in the program, for example, mocking out model methods from controller tests. This turns around and bites me roughly half the time I try it.

The situation that I don't like using mocks is the one where you are just trying to isolate the method under test from the rest of the system, in what Martin Fowler called a behavioral test, rather than a state test. In my experience, I regret trying this about 110% percent of the time. I grant that some people really like and are successful with this style.

How do mocks not work for me? Generally, one or more of the following problems.

1. Philosophical. When I'm adding a lot of mocked methods to a test, I can't escape the feeling that the test knows too much about the underlying methods.

2. Practical. Typing in all the mock methods for a method under test that makes a lot of calls can be a real pain in the fingers. This is what turned me off RSpec the first time I tried it -- I felt like I was going so slowly.

3. Maintenance. Inevitably, the underlying API of the models changes, and the mocks have to change to keep up. For me, this has been a bigger maintenance drag than I normally have with tests.

4. Skin deep. One problem with faking a system state with the surface API is that eventually you may need to know about the object in more depth. For example, I recently was testing a message rate limiting system, and mocked just the boolean "can the user send messages" method. Inevitably, I actually needed to check the user's raw message count, which hadn't been set because it was beyond the facade. Essentially, I had to rewrite the entire test.

That's the gist of it -- I'm not a strong mock user by any means, but I do find them very helpful in certain spots. Next week, I'll dig deeper into that message system and talk about how I tested a real feature in practice.

Related posts:

1. Comparing Ruby Mock Object Libraries
2. Walk-Through Test Coverage
3. Using Mocha for ActiveRecord Partial Mocks with Finders
4. Lesser Known Test Processes
5. Review of fixture_replacement2 plugin – Update

Comments (5)

Software Development

Comments: 5 so far

• Leave a comment
• Comment feed for this post
• TrackBack URL

1. [...] Agile Ajax » To Mock Or Not To Mock » Pathfinder Development [...]

   Pingback by Ennuyer.net » Blog Archive » Rails Reading backlog, Saturday, July 11, 2009 @ 3:20 am

2. 

   I’ve a heavy mocker (all 2.5 reasons) but I’ve also been bitten a lot when mocking out the different layers. I’ve been doing a lot of integration tests lately with no mocking, which makes things a bit easier to maintain.

   The thing I like about mocking layers, is that I can test a wide range of inputs to a method easily (behavioral).

   Comment by Eric Davis, Saturday, July 11, 2009 @ 5:14 pm

3. 

   I’ve had a similar love/hate relationship with mocks. There are plenty of scary looking tests that I’ve found on my travels that seem to suggest I’m not the only one that wasn’t sure if it was right or not. #3 on your list sounds just like the issues that were clearly visible, tests failing when something internally changes. Should the tests really care about this? These days I still use them, but they are very much along the same lines you describe. Good post!

   [WORDPRESS HASHCASH] The poster sent us ‘0 which is not a hashcash value.

   Comment by Pramatr, Tuesday, July 14, 2009 @ 8:20 am

4. 

   I am starting to dislike mocking as well due to the brittleness of the resulting test.

   I am starting to lean towards creating fakes instead of setting mocked expectations. I find that the tests I create with this is less brittle and much cleaner.

   However, I still find that behavior verification can sometimes still be the way. Although I can currently trying to use spies instead for the same reason.

   Cheers,
   Franz

   Comment by Franz See, Tuesday, July 14, 2009 @ 10:01 am

5. [...] sort-of promised in last week’s post, I’m going to work through a real-world test example, with an eye toward explaining how and [...]

   Pingback by Agile Ajax » A Real Testing Example » Pathfinder Development, Friday, July 17, 2009 @ 3:36 pm

Leave a comment