Elements of Testing Style

Noel Rappin, Friday, July 3, 2009 @ 7:28 am

Cover image from Amazon.com

It's been way too long since I blathered on about style issues. Today I'd like to talk about testing style. This article assumes you are already writing tests and already using something approaching a Test-Driven Development process -- I'm not here to argue about process, at least not today.

Today the topic is the actual construction of individual tests, how to name them, how to group them, where to get data from and the like.

I suspect I'll think of five more things right after I post this, so look for an update sometime in the future. The update will also address all the places where everybody tells me that I'm totally wrong.

Name tests after their expected behavior

The first part of each test is its name. By now, there's no excuse for still using the def test_underscore_instead_of_spaces naming convention. Rails 2.3.x, Shoulda, and RSpec all support a style where the test name can be an actual string with actual readable spaces and everything.

A test should be named after the expected behavior it verifies. Being specific is helpful here -- the name is potentially important documentation.

GOOD: test "should use last_name, first_name as display"

LESS GOOD: test "should correctly display names"

BAD: test "name display"

If you can't think of an easy name to describe the behavior being tested, this is a strong hint that the test is either a) richly deserving of being split into multiple parts or b) unneeded. Most of the time, it's (a).

Use contexts to group related tests

I'm also at the point of feeling like there's no good reason not to use contexts. If you don't want the weight of using Shoulda or RSpec, then just use the Context gem to get simple contexts.

So, what gets grouped in a context?

• A set of tests that all share a common data setup
• A set of tests that exercise a common method, as in controller tests that all work on the same action

Obviously, those two options tend to go together.

The context name should reflect the commonality -- either the name of the method being exercised, or something denoting the common data setup, like "as an administrative user".

Nesting

Contexts can also be nested, allowing for setups to be extended. I don't have very strong ideas on when nesting contexts is advisable, my current guidelines look like this:

• Only nest a context if the inner context uses all of the outer context setup and then adds on (similar to the guideline of only creating an Object subclass if the subclass uses 100% of the parent class data)
• Try to keep the distance between the different setup methods as small as possible -- the problem is that having to trace setup code up multiple pages and across several generations of contexts is a real pain.

Actually, the problem I'd really like a clean solution for is what to do if I want the inner context setup to be run first -- for example, a controller action that I want tested for multiple user types. The controller action is logically in the parent context, but the login as the various user type, logically in the inner context, needs to happen first. Putting a large context for each user type separates the tests for each action, which I'd rather avoid. Anybody smart got this one solved yet?

Setting Up Data

The guideline for creating data for tests is simple: create the minimum amount of unique data that you can while still having a meaningful test. In many cases, a model unit test will only require one object, and in many of those cases only a small number of attributes are actually relevant to the test. (One of the advantages of factory tools is the ability to highlight just those relevant attributes).

Even search and report methods can often be unit-tested with only two models in the database -- one to be found, and one to be ignored. (You'll often need more than one test, but each individual test is small and focused on one unit).

Setup should be as close to the actual test as possible, granted that it often makes sense to group repeated patterns into setup methods.

Single Assertion Style

Shoulda has popularized a testing style where most of the test is in the setup and each individual testing method has a single assertion. I have mixed feelings about this style, though I use it sometimes.

Guidelines:

• If you are going to do single-line tests, definitely use a tool that optimizes single-line tests like Zebra or Testbed.
• I think single-assertion tests work best when setting up relatively simple model tests that don't require a lot of nested contexts.
• Especially in controller tests, single assertion tests can lead to a lot of extra calls to the setup and a slower test suite, so that's something to keep an eye on.

When tests should be DRY and other fascinating questions?

The key thing about tests is that they are also code, so pretty much every style issue that applies to regular code also applies to tests.

That said, tests do have a special place in your application in that they are the part of your code that doesn't actually have tests of their own. So there are some special guidelines for tests.

Don't get cute. There's an old programing truism that if debugging is harder than coding, then by definition any code that is as clever as you can write it is impossible to debug. Similarly, any test written at the edge of your ability is going to be impossible to verify. It's not that metaprogramming and other fancy stuff has no place in testing -- Shoula macros depend on metaprogramming, for example. It's that you probably don't want an individual test to depend on the subtleties of Ruby's object system.

You can repeat yourself a little. I place a higher value in tests on being able to read the whole test in one glance than I do in regular code (I expect regular code to be spread across various methods, I expect a test to be self-contained). As such, I tend to allow somewhat more code duplication in tests than I do in application code, especially in setups. That said, keeping setups short has a value as well, and when the setup is creating enough objects that it's obscuring the point of the test, I'll refactor the setup to a helper method and reuse that. But in application code, I'll generally blast duplication immediately, but in test code, I'll often wait until I have some pain. Is that a good thing? Not sure.

Your turn. Where am I wrong?

Related Services: Ruby on Rails Development, Custom Software Development

Related posts:

1. The Testing Interviews
2. Elements of Ruby Style
3. Nested Contexts and Test Structure
4. Real Testing Example, Part Two
5. Functional Testing Annoyances, Wapcaplet, And You

Comments (2)

software development best practices

Comments: 2 so far

• Leave a comment
• Comment feed for this post
• TrackBack URL

1. 

   be sure to check out Contest, a great gem for nested contexts, extremely light and efficient: http://blog.citrusbyte.com/2009/05/19/introducing-contest/

   Comment by ehqhvm, Friday, July 3, 2009 @ 9:17 am

2. 

   Greetings,
   You mention that Shoulda macros rely on metaprogramming, and that’s my problem with them. I like Shoulda a lot; it’s my basic testing platform, but I feel strongly that the macro system encourages a bad test design.

   Basically, because the macro ‘creates’ code in the macro definition file, there’s no association with a particular line of code in the original test file. This leads to stacktraces (on error) which appear to be completely unrelated to the actual test file, and potentially a lot of digging to figure out where the problem is.

   So while I agree that Shoulda (and similar libraries) are very useful for context and the ’should’ testing syntax, I’d warn against the macro functionality as having a good chance of falling into the ‘clever’ trap.

   For the ‘inner setup first’, I think what might work is something like ‘acts_like “{other context}”‘, which would run the setup from the named context before running the tests. I _think_ the Context gem has (or had) something like that, but Context has other problems that made it not work well for me, so I fell back to Shoulda, and never explored that too much.

   – Morgan

   Comment by Morgan, Tuesday, July 7, 2009 @ 2:18 am

Leave a comment