A lot of pair programming chatter this week. Starting with a New York times article describing pair programming at Hashrocket. It's an interesting article, with a tone that could be described as "anthropologist describing the strange, yet quaint customs of the native tribe"

Obie Fernandez followed up with a list of 10 reasons why pairing doesn't work in most cases. It's actually a list of the things that Hashrocket does to support pairing, although entries like "2. Most software developers just don't want to work that hard" and "1. Most software shops don't really care about excellence" do have a certain, "aren't we great" vibe to them, causing Mike Gunderloy to dryly observe: "Funny, Extreme Programming Explained never said anything about fancy hw or being awesome as a prerequisite for pair programming."

C'mon Mike -- everybody knows that being awesome is a prerequisite for everything in XP.

Josh Susser adds that pair programming isn't right for all projects, particularly projects that have long compile times that force the pair to stare blankly at the screen.

I'd also add this interview with Kent Beck because a) every programmer could use some more Kent Beck in their life and b) because he talks about XP as being concerned with the the social context of programmers, with pairing being a part of that.

Now you are caught up. Here's the part where I talk.

I've had a running debate with Dietrich for two years now. He thinks that Pair Programming is the number one most important part of an agile team. I think it's testing. That said, I do think pairing can be a nutritious part of your agile breakfast. But it's tricky to do right and easy to do wrong.

Pairing is for the long haul. Like pretty much everything in the Agile toolkit, the real gain in pairing happens over time and is hard to quantify because it's the absence of friction later in the project. Code is cleaner and easier to change. System knowledge is more distributed, so you are less likely to freak out when Fred catches H1N1 a week before.

Actually, I'd describe most of the XP practices as "we thought we were trading short-term productivity for long-term productivity, then found that we got short-term gains as well." Certainly can apply to TDD.

Project size and make up matters. A team of three developers, for example, is a challenge for an always-pairing environment. Even a team of two is a problem -- some tasks really don't lend themselves to pairing. If people on the team also has non-developer responsibilities, that's another challenge.

Some people really don't like it. And I think it's glib to say "those people are lazy" or "those people don't care about excellence". Even I find pair programming really tiring, but in a "I worked hard today, plus I had to deal with another person all day." It's not unusual for programmers to be very strong Meyers-Briggs introvert-intuitives, and it's not surprising that personality type would find pairing a challenge. It's not insurmountable, but you do need to structure the pairing with people in mind -- the Hashrocket 25 minutes on/ 5 minutes off thing is a good start.

While I'm here, I think you can really overstate the "people work harder with somebody next to them" thing. Pairing is great and helpful because of the continual review and talking about what you are doing. Conceiving pairing as a way to use peer pressure to keep your coders off of Twitter is, I submit, not very much in keeping with the spirit of the Agile Manifesto.

The physical layout is really important. A lot of Obie's piece is about this. I just want to emphasize first that pairing is a lot louder than solo programming, and this can be an issue in open office environments, and also that pairing really does work better on a neutral environment that isn't either developer's home machine.

Pairing is hard to reconcile with offsite practices And I don't mean just when you're working with six guys in Krakow, but also any kind of work at home, flexible hours, programmer-friendly kind of time structure is a real challenge to integrate with pairing.

Hmm... A lot of challenges, not much guidance. Which matches my feelings. When put in the right context, pairing is fantastic. But the context can be elusive, even for people with the best of intentions and talent.

Would this have been a better blog post if I had paired with a co-worker?

Related posts:

1. The Importance of Pair Programming
2. Growing Into Pair Programming
3. Pair Programming with VNC

Please continue reading after the next sentence.

I installed Snow Leopard a couple of weeks ago.

Wait -- don't stop reading. This isn't a post about how to install MySQL or a post about whether or not Snow Leopard is the Greatest Thing Ever. There are plenty of other places on the Internet where you can get that information.

I wanted to talk about two cool corners of the Ruby universe that I started using as a result of my Snow Leopard installation: MacRuby and RVM

RVM: Ruby Version Manager

One of the Ruby changes in Snow Leopard was an upgrade of the system Ruby version to 1.8.7. All fine, except that one of the projects I work on is on an older version of Rails that is not compatible. Clearly, I needed to get an 1.8.6 Ruby on my machine, ideally without messing up the system installation that other applications are using.

Enter Ruby Version Manager (RVM), a handy command line tool for installing and managing multiple Ruby interpreters on your system. In theory, usage is as simple as this:

$ gem install rvm
$ rvm-install

At this point RVM asks you to make a slight change to your shell files. Then just this:

$ rvm use 1.8.6

"But you didn't have 1.8.6", I hear you cry. True. RVM will go out, download, compile, and install Ruby 1.8.6 to my local machine without interfering with anything else on the system (it installs in ~/.rvm). You can install Ruby 1.8.x, Ruby 1.9.x, Ruby Enterprise, JRuby, Rubinius, and (I think) MacRuby.

(Okay -- that's the ideal. In practice, it took a little bit of system tweaks and command line tweaks. Mike Gunderloy's Snow Leopard guide was helpful here. Although I've been able to get all the main Ruby versions to install, I'm still struggling some of the more esoteric version. Once installation is done, the command line interface is really good and things just work.)

Now, the Ruby in my shell is the new 1.8.6. But any other running terminal shells are unaffected -- especially helpful if you are working on multiple projects. Each Ruby maintains its own Gem listing, although there's work ongoing to make it easy to share gems.

To get back to the system Ruby, just

rvm use system

You can also specify any Ruby version as the default.

This is outstandingly cool, not just to solve my problem, but also as an easy way to create a harness to test your Ruby program or library against multiple Ruby setups. There are other useful features about setting different versions and managing shells, check it out.

MacRuby

Here's the thing. For the last several years I've had this script that communicates with iTunes via AppleScript, and creates a bunch of random playlists according to criteria that is a bit more complex that an iTunes smart playlist. For instance, it can create a playlist made up of two-song blocks by the same artist. Okay, it's wildly overdone, but I like it.

It broke in Snow Leopard. I don't know why. It seems like the Scripting Bridge framework occasionally decides to go out for a cup of coffee, and my script times out.

This seemed like as good a time as any to investigate MacRuby. MacRuby is an implementation of Ruby in Mac OS X Objective-C. Unlike a lot of hybrid language/vm tools, MacRuby gives you direct access to the native objects. So, if you ask for a string, you get an object that acts as both a Ruby string and a Cocoa NSString, responding to methods of either. MacRuby uses Ruby 1.9 key/value arguments to translate Objective-C method names.

[person name];
[person setName:name];
[person setFirstName:first lastName:last];

In Ruby (this example is from a tutorial on the MacRuby site)

person.name
person.setName(name)
person.setFirstName(first, lastName:last)

Unlike regular Ruby, the order of the keyword arguments must match the Cocoa method selector.

For most people, this allows writing Cocoa applications in Ruby, including integration with XCode and Interface Builder.

That's extremely useful, and I plan on trying it soon. For my purposes, the point is that it uses the Scripting Bridge directly, and I hoped that would allow it to bypass whatever weirdness was breaking my original script. (The original script was in Python, but I had a 75% functional Ruby version that I never actually built the I/O on, so it was largely a matter of learning the MacRuby way to communicate with iTunes.) Here's a sample, cobbled together from various parts of the script:

def itunes
  @itunes ||= SBApplication.applicationWithBundleIdentifier(
    "com.apple.itunes")
end

def library
  @library ||= itunes.sources.objectWithName("Library")
end

def all_music
  @all_music ||= library.userPlaylists.objectWithName("Music")
end

all_music.fileTracks.each_with_index do |track, index|
  # stuff here
end

def create_itunes_playlist
  playlist = itunes.classForScriptingClass(
      "playlist").alloc.initWithProperties(
      {'name' => name})
  library.playlists.addObject(playlist)
  playlist
end

#chosen_tracks are my object wrappers around Cocoa
#itunes_track is the actual cocoa object
playlist = create_itunes_playlist
chosen_tracks.each_with_index do |track, index|
  track.itunes_track.duplicateTo(playlist)
end

Overall, everything works as advertised (it seems as though MacRuby is better able to deal with whatever happens to cause the Scripting Bridge to take a nap. I'm using the pre-release MacRuby 0.5, so there's the occasional feature glitch (gem installation is a little dicey, for example). But the MacRuby team is actively, even furiously, pushing forward, and this looks like it'll be very useful, very soon.

Related posts:

1. Keeping Up With The Joneses: Keeping Rails and its extensions up to date
2. Rails Development on Windows: Native or Virtualize
3. Installing Edge Ferret/acts_as_ferret

Sometimes I write these just for me.

As I've mentioned a couple of times, tomorrow I'll be speaking at WindyCityRails, and I need a checklist of all things I don't want to forget, and all thing things I want to do to make the talk great.

Preparing the Talk

Figure out what you want to say first. The outline view in PowerPoint and Keynote is a great way to organize what you want to say without worrying about how it looks.

Determine a small number of things that you think are key items that you want the audience to walk away with. Mention them at the start, summarize them at the end.

The classic advice is not to put too much text on your slides, and avoid reading the text directly. A pragmatic way to manage this is to set your body text font pretty large (I use 44 point fonts for my body text if I can get away with it), and break up any slide that threatens to overrun.

That said, you can make the opposite mistake as well -- I find it tiring to go to a talk where all the slides are two-words and a non-sequitur picture. People may be viewing just your slides online, they need a little bit of context. Even people in the room often need a bit of context on the slides to help follow what you are saying.

Building up the slides during the talk point by point using animation can be a good way to keep from reading too much text on the slides. And it can help time a "punch line" by keeping it from the audience until you are ready. But don't use it everywhere. (I've always wanted to give a talk that used bullet points like Colbert does in The Word segment, as ironic commentary. But that's very hard, and I'm not sure it would work.)

I do like to separate off major sections of the talk with title-only slides. I've started tying these sections together by putting images on the title slide, and carrying those images in the background of the slides in the section at something like 20% opaque. It makes it easier for people in the audience to follow along.

Keep the number of moving parts to a minimum -- rather than switch to an editor to show code, copy the code into your slides. TextMate has a "Copy to RTF" bundle that preserves syntax highlighting, or take a screen shot.

Similarly, try to avoid having to run code during your talk. (Unless the interactivity is the point, as in a longer workshop).

Try not to depend on Internet access for a demo, you never know how flakey the Internet is going to be at a venue. At the very least, have a bunch of screenshots in hand as a plan B.

Practice. Do a dry run by yourself. If you see something wrong in your slides or missing in your talk, write it down. But do try and get a clean dry run at some point, so you get a sense of how long the talk is.

If possible, practice in front of other people. Nearly every talk that somebody at Pathfinder gives has a practice run as a company brown bag. This is invaluable feedback as to what is interesting in your talk and what is not.

In many situations, your last slide will be up for a while while you answer questions or something. Put something useful on there, generally either a summary or places to go for more information.

Doing the Talk

Make sure you bring everything you need. Laptop. Power Cord. Monitor Adapter. Remote Control. Don't assume that anything will be as you expect in the actual setup. As I type this I'm going to put my monitor adapter in my backpack. Put a copy of your talk on a thumb drive. Put a copy online using Dropbox or Gmail it to yourself.

If at all possible, scope out the space well before your talk. Where will you stand? Will you be able to see your own slides? Will you need a remote, or will you need to stand in front of your presentation machine?

Try and set up early if you can (if you are part of a conference program, you're limited because there's a previous speaker). Still, the less the audience sees you fumbling with cables the better off you are. It helps if you can check to see if the display resolution matches your slides.

Before the talk, shut down as many programs as you can on your laptop. Especially shut down notifiers -- you don't want an IM from your mom popping up in the middle.

I use a Mac utility called Caffeine to keep my computer from going to sleep in the middle of the talk, because it's annoying when that happens.

The point of all this is to reduce the amount of things that can go wrong so that you can be as relaxed and have fun during the talk.

Getting started can be awkward. Sometimes it helps to get the audience involved by asking a question or two right off the bat -- this can also help you gauge how much the audience already knows about your topic. It can also be an unobtrusive way to make sure that everybody can hear you.

At the end, if there's time, take questions. Get in the habit of repeating the question before you answer it. There's a good chance that most of the audience didn't hear the question. If you are miked and being recorded, the recording probably won't pick up the question unless you repeat it.

Stick around afterword if you can, often there will be one or two more people who have questions.

Related posts:

1. Artifacts of a Presentation
2. Ruby Code Audit Checklist
3. Windy City Rails

It's time to play "Ask A Tester Person", where I answer questions that I've gotten via email or otherwise about Rails Testing topics.

If you have a question for Ask A Tester Person, send it to railsprescriptions at gmail.com.

I've got two questions today:

Question 1: I'm writing too many tests!

One subject I still have questions
about is how to test authorization in controller tests. In my
applications I've been testing three different user cases for every
controller action, but this leads to, for example:

test_edit_by_anonymous_user
test_edit_by_unauthorized_user
test_edit_by_authorized_user

So every action has at least three tests, which obviously means a
*ton* of tests for the entire application. This has always felt like
overdoing it but I haven't yet been able to convince myself that just
testing the authorization part of the code is sufficient. If I could
isolate the authorization mechanism for testing that might convince
me, but I'm not really able to do that (maybe my authorization process
needs to be redesigned?), and I'm not sure I'd be convinced
anyway...how the application responds in all three cases seems like
something that *should* be tested for every action...

I doubt that your authorization process needs to be redesigned, but then I've never seen your authentication process.

I'd recommend a couple of different things to clean this up a bit. On the assumption that unauthorized access has largely the same behavior across the application, you can create a boilerplate test like this example -- the example uses Shoulda, but the basic idea should work in any framework.

  self.def should_block_access_for_anonymous_user(*actions)
    actions.each do |action|
      should "block anonymous access for #{action}" do
        logout_current_user
        get action
        assert_redirected_to root_path
      end
    end
	end

Which you would then use as:

  should_block_access_for_anonymous_user :edit, :update, :delete

The implementation shown here is probably a little too simplistic for full use (you might need to pass more information than just the action for each request), but the simple version could easily cover the most basic authentication issues with very little fuss.

If your authentication system is more complicated, the other option is nested contexts -- again, this is in Shoulda, but can be adapted to RSpec or the Context gem:

context "GET edit" do

  setup do
    # generic edit setup here
  end

  context "with an anonymous user" do
    setup do
      logout
      get edit
    end

    should "not allow access" do
      assert_redirected_to root_path
    end
  end

  context "with an admin user" do
    setup do
      login_as_admin
      get edit
    end

    should "allow access" do
      assert_response :success
    end
  end

end

The upside of this compared with what you are probably doing is that it consolidate the setup between the different options. The downside is that it can be kind of verbose and hard to follow. But it's still potentially easier than writing three separate tests for each action.

Question two: I'm creating too many objects!

Okay, I actually can't find the email that this came from, which is driving me crazy, because how do you lose an email message these days? Anyway, the gist was that this person's tests were running very slowly because he or she was creating so many objects for each test -- I distinctly remember the number 50 to 100 being tossed around. So the question was how to avoid creating so many objects?

The bottom line is that there's no way that you should need to create 50 to 100 objects for every TDD test, or even for any TDD test -- it's pretty rare that a unit-level test really needs that much data to work. There are a lot of potential issues here. Three that spring to mind are:

• You are trying to test too much code at one time. It's possible (though still not very likely) that an integration test might need that many objects, but testing a single method almost never does
• You've transitioned from fixtures to factories, but you are still writing your tests as though you were using fixtures.
• You have perhaps an unnecessary fixation on providing "realistic" data in unit tests.

There's some overlap here. One cause of this is problem is that in fixtureland, there's very little marginal cost to creating new objects, so the tendency is to create fairly large aggregations of objects that cover all possibilities and have that be the universe for all the tests. When you transition to factories, though, keeping that big data blob around is not necessary. Since you can and should be custom-creating the data for each unit test, most model methods only need one model with specific attributes in order to specify the logic. Sometimes that model will need associated objects, but the factory tool can be set to create those automatically.

A special find method or named scope can be tested with as little as two objects -- one to be found and one to be skipped. Sort logic similarly can be tested with two or maybe three objects. If the logic is really complicated, it's better to do multiple tests with small amounts of data than one test with the whole shebang. The idea of a "unit" test is to verify one small piece of logic with as little data as possible, not to check program behavior under realistic data load -- that's what integration tests, performance tests, and actual user acceptance is for.

I feel like I'm answering the question "How do I create fewer objects in my tests?" with "By creating fewer objects in your tests", I hope this answer gives you some tools for minimizing the number of objects you need to create in your tests.

Related Services: Ruby on Rails Development, Custom Software Development, Testing and Quality Assurance

Related posts:

1. Elements of Testing Style
2. Rails Testing Frequently Asked Questions — The Non-Code Version
3. Rails Test Prescriptions is now on sale

To make a cheap joke and paraphrase a common quote, web developers and web designers are two groups separated by common languages. In our case, the languages are HTML and CSS, which are the output of both the web design process and the web development process. Developers and designers produce their HTML/CSS in different ways and with different goals. Here are some ideas for bridging the gap so that the developers and designers on your team can work together smoothly.

Designers and developers obviously have different goals for their HTML -- developers have issues of reducing duplication, organization, and performance that are largely not the designer's concerns. The designer is primarily concerned with how the HTML looks and behaves to the user.

By the way, I'm absolutely not trying to make this some kind of left brain/right brain thing. It's more of a software needs vs. domain expertise thing. Once upon a time, I was writing scripts that outputted router configuration files, and I had exactly the same issues with the router domain experts -- my software engineering desire to structure the code without duplication conflicted with the way the router experts liked to structure their hand-written configuration instructions.

Our teams have had success with getting everybody on the team using common tools as much as possible. This means putting designs and code in the same code repository, and it means the development team supports the designers in creating a set up to run the current development version of the app locally. (By the way, this article by Mindy Wagner might be helpful if you are trying to convert everybody to Git.)

From the developer perspective, if you are working with HTML provided by designers, it's important to keep the view layer of your code accessible to the HTML providers. Exactly what this means is subject to negotiation. Left to my own devices, I'd be putting all kinds of HTML generation in Ruby via helpers or something more esoteric. That didn't work out well when the designer needed to go mucking about in metaprogrammed Ruby code to start changing CSS classes. We do better with putting pure logical stuff in helpers and using partials to split view logic. I'm pretty sure that if I were to suggest Haml for a project, the designers would veto it -- Haml barely meshes with the way I think of HTML, the designers I've shown it to have basically recoiled in horror.

That said, everybody likes Less CSS, which seems to augment CSS in ways that seem very intuitive to CSS designers, and which are very satisfying to coders. It does all the things that you would expect CSS to do if it was a real language, but vanilla CSS works just fine. It really caught on quickly here.

From the designer perspective, get everything out of photoshop and into HTML/CSS as early as possible. It's just too easy to put stuff into a photoshop image that represents hours of development work, leaving the developers in the position of trying to determine which parts of the impossible image are vital, and which are just chrome. Doing the wireframes in HTML/CSS keeps the design honest.

We use this little hack to integrate wireframes into the development app, which is really nice for developers when working with in-progress designs.

Ultimately what it comes down to is for everybody in the team to take some responsibility for making the team work together. The developers need to make the code base accessible to designers and to be alert to basic design issues and flexible in adapting wireframes into the site. Designers need to help place their deliverables in a format that keeps the developer from having to guess how things are supposed to work -- nobody wants that.

Related Services: Ruby on Rails Development, User Experience Design, Ajax Rich Internet Applications

Related posts:

1. A New Workflow for Web Designers
2. Down with HTML + Code Markup!
3. Designers on Joel

So, I tried Blue Ridge for the first time yesterday and I thought I'd write down some quick impressions. Hence, Rails Testing First Look.

Disclaimer: We came into this tool so cold our toes froze. We fumbled, we made mistakes, we probably missed really great ways of doing things. I look forward to being enlightened.

Let's do this question-and-answer style: What is Blue Ridge? Glad you asked. It's an framework for testing JavaScript from inside a Rails application. For the most part, it's a cohesive bundle of existing JavaScript test tools, such as Screw.Unit. The value-add of Blue Ridge is an easy installation and nice integration with Rails testing tasks.

Why did we start using it? We had some simple JavaScript stuff to do, and thought it would be a good time to start out with what looks like a solid way to test.

Is it easy to install? Very. Install as a plugin ./script/plugin install git://github.com/relevance/blue-ridge.git, then generate the Blue Ridge files with script/generate blue_ridge.

Blue Ridge will install a test/javascript directory. In that directory will be a sample test file application_spec.js, a helper file, and a fixture directory. Each javascript test file is assumed to have a matching HTML file in the fixture directory which is loaded to provide a sample DOM for test purposes.

Does it make any other assumptions? BlueRidge assumes you are using jQuery as your library. It's theoretically possible to use Prototype, but we ran into continual difficulties (possibly because we were doing it wrong), and actually wound up switching to jQuery for the project, easy enough to do since we just started.

How do you write tests? Tests are written using Screw.Unit syntax, and if you are familiar with RSpec, the syntax will look pretty similar. Here's the final draft of the first couple of tests we wrote, for a search box with default text that goes away when focused. The TextMate Screw.Unit bundle was very helpful, but other than that, the tests are pretty straightforward.

Screw.Unit(function() {
  describe("With my search box and default", function() {
    it("should switch the default", function() {
      search_focus($('#unified_search'));
      expect($('#unified_search').attr('value')).to(equal, '');
    });

    it("should switch a non-default", function() {
      $('#unified_search').addClass('search_entry');
      search_blur($('#unified_search'));
      expect($('#unified_search').attr('value')).to(equal, default_text);
    });

    it("should not switch if there is a value", function() {
      $('#unified_search').attr('value', 'Fred');
      search_blur($('#unified_search'));
      expect($('#unified_search').attr('value')).to(equal, 'Fred');
    });

  });
});

How do you run tests? Two ways. Blue Ridge provides a test:javascripts Rake task that will run all your Javascript tests in the terminal, this is also suitable for including in your continuous integration build, for example. A very nice feature from Screw.Unit is the ability to run tests in browser by simply opening the fixture HTML file in the browser of your choice. As long as your choice isn't Safari, which isn't supported at the moment. The ability to run the tests in console and in browser is very useful.

Any gotchas? Oy. We had what appeared to be some minor differences between the Rhino implementation that powers the console tests and the browser tests, leading to tests passing in the browser and failing in the console. We weren't quite able to do what amounted to integration tests -- we tried to trigger the JavaScript events so that the jQuery functions tested above were triggered, but couldn't get that to work. You have to be careful that the fixture HTML file actually has all the DOM elements you need, and remember that fixing the actual view doesn't change the fixture. Screw.Unit's error messages on failure aren't as helpful as they might be, there also seemed to be a thing where the fixture DOM wasn't actually being reset between tests, but I'm not 100% sure about that, we wound up working around it.

And in the end? That said, I did feel better about my JavaScript after having tested it. Fixing the JavaScript code, it was nice to see that other functions hadn't broken. There's a lot to like here, and I'm hoping to get past my initial problems to a tool that will improve the JavaScript parts of my code.

Related Services: Ruby on Rails Development, Custom Software Development

Related posts:

1. Rails Testing Frequently Asked Questions — The Non-Code Version
2. Rails Test Prescriptions is now on sale
3. Getting Started With Rails Testing Guide Now Available

I'm writing a functional test:

should "do something functional"
  get :search,  rder_id => @order.id, :user_id => @user.id
  # and so on
end

The get call in that test simulates a browser request. Intuitively, you would (well, I would) expect this request to be identical to a request coming from the actual view, via a helper like link_to("search", :action => :search, rder_id => @order.id, :user_id => @user.id). At least, you'd expect that parameters hash in the controller to be the same between the

Makes sense, right? The testing call should set up the same environment as the actual call being tested.Well, if you've gotten this far, read the title that said "Annoyances", or have ever read any blogs, you know that isn't how Rails works. Specifically, the actual browser call has all its arguments converted to strings as part of going through the HTTP wire, but Rails, oddly, does not similarly convert the arguments in the test.

Most of the time it makes no difference -- if you pass the string or the integer directly to an ActiveRecord find method, everything works swimmingly. If you do a direct equality check on the parameter, though, you can get code that passes tests, but fails in the browser because the string value is no longer equal to the integer value.

For some time, I've had this on my list of annoyances that I don't have the energy to fix, right next to the slightly misaligned light over my desk, and the fact that they bought the wrong kind of Kleenex at the office that one time.

This week, Adam Milligan at Pivotal Labs tried to do something about it. Specifically, he created a Rails patch, then a plugin, that can raise a warning or an error when a non-string argument is passed to an HTTP method in a functional test.

Adam chose to fail or warn rather than silently convert the parameter because he feels that some arguments that might get placed in an HTTP method don't have clear correct conversions (his example is false), and he's trying to avoid setting up a whole different kind of difficult-to-diagnose test to app misalignments.

As much as I love that Adam did this, I kind of think that it's more consistent with Rails design to silently convert, to make the browser call and the test consistent. Still, having the code warn or fail (your choice) is a pretty good way to get into good habits and avoid having to track down weird test errors. So, thanks, Adam.

Related Services: Ruby on Rails Development, Custom Software Development

Related posts:

1. Rails, Ajax, RJS, and Testing
2. Rails Testing First Look: Blue Ridge
3. Elements of Testing Style

Dot, dot, dot, dot, dot -- tests are passing, looks like it's time for lunch -- dot, dot, dot, dot, F. F? F? But the code works. I know it does. I think it does. Why is my test failing?

One of the most frustrating times as a TDD developer is that moment when a test is failing and you don't know why, as opposed to the more normal case where the test fails as expected. Here's a grab bag of tips, tricks, hints, and thoughts to get us all through that difficult time.

Something Must Have Changed

This may be the most obvious piece of advice in the history of ever, but I find it's worth repeating, mantra-like, when confronted with a bad bug:

When a formerly-passing test fails, it means something changed.

It may be in the code, or the system, or the test. But it's probably not sunspots, and it's probably not evil spirits possessing your MacBook. (Unless you are either living in a Charles Stross novel or writing Perl, but I digress...)

Looking through recent changes can help figure out what the cause of the failure is. Git's bisect tool can do this automatically, or you can just look through recent changes in your source control viewer of choice. If the test was passing at one time, there's a good chance the answer is in there somewhere.

This is a great argument in favor of committing to your source control very, very frequently (especially when you are using git and can do local commits), so that your changes are very granular.

Isolate

When looking at a small number of failing tests, it's helpful to be able to run just those tests. Autotest is outstanding for this, since it will run the failing tests over and over until they pass. This is especially helpful if you have a number of failing tests that are not in the same test class.

This little code and terminal snippet is very helpful for quickly running one class at a time, which is almost like isolating a failing test, or at least close enough to be useful. Depending on your IDE and test framework of choice, you may also be able to run individual tests from the IDE.

Isolating tests makes the tests run faster when you are focused on just a few tests, and also makes any diagnostics you insert easier to interpret.

Two tips that I've stolen from listening to and reading Kent Beck:

• Back out your entire most recent change since your last passing test and start over. This works best if you work in very small increments, but it gets you out of the "I know I typed something wrong but I just can't see it" nightmare
• Replace all the expressions in the method under test with literals -- if that passes, then put the expressions back one by one until you find the culprit.

Diagnose

I have to say, I'm not a big fan of using stop-and-step debuggers. I've used them when I've been in an IDE, I've never really used the Rails command line debugger, but mostly I've found that not to be a great experience.

Normally, to diagnose what's going on in a test, I usually either add additional assertions in the test or have the code print information to the console. If I diagnose via assertions, generally I'm testing the values of variables in more detail.

For some reason, I see a lot of people using Ruby's puts method to write to the console -- I recommend p, which calls inspect on the object before printing, and generally results in more informative output. As a matter of course, I put require pp, which allows me to use pp to get pretty-printed output, which is nice for nested data structures. Also, y gives a YAML representation of the output -- very readable for ActiveRecord objects.

>> x = {1 => ['a', 'b'], 2 => 'c'}
>> puts x
1ab2c

>> p x
{1=>["a", "b"], 2=>"c"}

>> pp x
{1=>["a", "b"], 2=>"c"}

>> y x
---
1:
- a
- b
2: c

Especially if I have autotest running just the one test, I've been known to bury print statements all over the place -- controllers, Rails itself (often educational). Just remember to take them out when you are done.

Clear Your Head

Take a walk. Force your pair to solve the problem. Get a cup of coffee (actually, I hate coffee, get a Diet Coke). Take a nap. All those silly clear your head things really do work sometimes.

Band Aids

It's tempting sometimes to comment out the offending test and then your suite passes and all is well with the world again. That's generally a bad idea (although sometimes a major refactoring can genuinely make tests obsolete.

Hope this helps. What do you do to fix stubborn tests?

Related Services: Ruby on Rails Development, Custom Software Development

Related posts:

1. Rails Test Prescriptions is now on sale
2. Lesser Known Test Processes
3. Again With The Test Driven Development

What with upward of two people saying nice things about last week’s post, I’ve decided to keep going with part two of a look at some real testing code.

Most code-heavy tutorials show the code but not the tests — I’m doing the opposite here, and showing the tests, but not much of the code. Also, although I’m presenting these tests in chunks, you should realize that there was a lot of back-and-forth from Cucumber to tests to code and some backtracking, most of which I’ll spare you from having to wade through.

At the end of last week, I had run through the tests for spam-prevention code which worked by limiting the rate at which a user could send messages to other users of a particular social networking site. Cucumber was involved, and I think I went off on a tangent about writing lots of tests.

Once the basic rate limiting code was in place, the client and I came up with a couple of special cases. The application allows a user to explicitly reply to a specific message sent to them. We agreed that these replies should not count towards the rate limit on the grounds that a reply was, pretty much by definition, not spam. Similarly, we decided that any message between users with a friend relationship on the site also doesn’t count toward the rate limit. More subtly, we decided that if a user was inadvertently blocked, then reinstated by an administrator, that their rate count should drop back to zero, so the user doesn’t just get immediately re-blocked.

The Cucumber tests:

Scenario: Reply behavior
  Given I am a user who is not a new member
  When I send 4 messages in a day
  And I reply to a message
  And I send 1 message in a day
  Then 6 messages are sent
  And I am not blocked from sending further messages
  And the administrator does not get an email

Scenario: Friend Behavior
  Given I am a user who is not a new member
  When I send 4 messages in a day
  And I send a message to a friend
  And I send 1 message in a day
  Then 6 messages are sent
  And I am not blocked from sending further messages
  And the administrator does not get an email

Scenario: Unblocked behavior
  Given I am a user who is not a new member
  When I send 6 messages in a day
  Then I am blocked from sending further messages
  When I am unblocked by an administrator
  And I send 5 messages in a day
  Then 5 messages are sent
  And I am not blocked from sending further messages
  And the administrator does not get an email

These tests are quite similar in structure to the tests I started with last week. I might start thinking about combining some of the steps, but in Cucumber, I bias in favor of having the actual test be as clear and readable as I can get it. I don’t think these scenarios are so complicated that they require simplification.

Most of these steps have already been defined, here are a couple that aren’t. The first two steps are slight variants on the message sending step definition:

When /^I reply to a message$/ do
  original_message = Factory.create(:message, :sender => @recipient,
      :recipient => @sender)
  message = Factory.attributes_for(:message, :sender => @user)
  message[:in_reply_to] = original_message.id
  visit(messages_path,
      :post, {:recipient => @recipient.id, :message => message})
end

When /^I send a message to a friend$/ do
  @user.become_friends_with(@recipient)
  visit(messages_path,
      :post,
      {:recipient => @recipient.id,
          :message => Factory.attributes_for(:message, :sender => @user)})
end

One thing that you may have noticed about these step definitions. Despite all my jumping up and down about Cucumber being a black-box system, both of these steps bypass the UI to complete the setup. The first step creates a factory message and the second step creates a friend relationship. In both cases, the actual message send being tested passes through the UI.

There are basically two reasons for bypassing the UI in the setup. One is, of course, that it’s the setup and not the action being tested, the second is that being a purist all the time is a pain, and it’s much easier to get the background info out of the way directly.

The final missing step, however, is completely through the UI — the step logs the user out, logs in as an administrator, navigates to the page, unblocks the user, and then logs the administrator out and the original user back in.

When /^I am unblocked by an administrator$/ do
  @blocked = @user
  visit "/logout"
  Given "I am a logged in administrator"
  visit path_to("the admin messaging page")
  click_link("Unblock")
  @user = @blocked
  visit "/logout"
  Given "I am logged in"
  reset_mailer
end

A little on the Rube Goldberg side, perhaps, but it works.

With the Cucumber step definitions in place, the question becomes what tests and code need to be written to make these pass. For the first two scenarios that adjust the definition of what makes a message count toward the rate limit, the code change will be in the User and Message models. Since it’s a thin controller that defers to the models for data, there shouldn’t be any code change in the controller at all.

Now, that’s a much easier determination to make in hindsight than up front, and as it happens I did write controller tests for both of those scenarios, largely because they were so similar to the controller context chunks in last week’s post that it only took about five minutes to set them both up.

The real test action was in the user and message models. I added a database column for messages called counts_toward_spam. The idea is that this would be true for most messages, but false for replies, friend messages, or messages otherwise cleared by the admins. This requires some unit tests in the message class to support it. Again, I’m using Matchy here, and testing the boolean spam_message?, which is basically an alias for counts_toward_spam that I added for readability. In retrospect, adding the method was probably unnecessary.

context "spam count" do

  setup do
    @sender = Factory.create(:user)
    @recipient = Factory.create(:user)
  end

  should "give an ordinary message a spam count" do
    @message = Message.new_from_params(
        {:subject => "fred", :body => "body"}, @sender, @recipient)
    @message.sender_id.should == @sender.id
    @message.recipient_id.should == @recipient.id
    @message.should be_spam_message
  end

  should "not give an ordinary message a spam count if it is a reply" do
    @message = @message = Message.new_from_params(
        {:subject => "fred", :body => "body", :in_reply_to => 11},
        @sender, @recipient)
    @message.should be_reply
    @message.should_not be_spam_message
  end

  should "not make a message between mutual friends a spam count" do
    @sender.become_friends_with(@recipient)
    @message = Message.new_from_params(
        {:subject => "fred", :body => "body"}, @sender, @recipient)
    @message.should be_friend_message
    @message.should_not be_spam_message
  end
end

The message tests simply set up the different classes of message and verify that they are classified appropriately, they are pretty straightforward.

I also added a test to user to verify that the spam count was being used appropriately — this is part of the same context as last week’s user tests. The user has already sent four messages in the setup.

should "not count a reply message" do
  message = Factory.create(:message, :sender => @user,
      :created_at => 10.minutes.ago)
  message.in_reply_to = 3
  message.update_count_toward_spam
  message.save!
  @user.spam_message_count.should == 4
  @user.update_message_block_status.should be_nil
  @user.should be_able_to_send_messages
end

It’s interesting what you see when you go back over a chunk of code to explain it to other people. In this case what I notice is the update_count_toward_spam method, which is part of Message that unsets the count_toward_spam_ field if the message is a reply and is automatically called as part of the controller create message code. However, strictly speaking, it should be tested in the message class rather than here (although, the message controller tests would also exercise it).

That’s the bulk of the testing for those features — at the time, I didn’t think there were any other special cases.

The part about unblocking a user requires controller tests, because the unblock action needs to be written in the controller. I’m making it part of the message controller, although I know that RESTfully speaking it probably should be a separate resource — the legacy app is such a REST tangle that it’s not worth it.

The controller test context looks like this — the setup creates a blocked user and calls the unblock method.

context "GET unblock" do

  setup do
    ActionMailer::Base.deliveries.clear
    @recipient = Factory.create(:user)
    @user = Factory.create(:user,
      :message_sending_status => User::BLOCKED_STATUS)
    @user.should_not be_able_to_send_messages
    admin!
    get :unblock, :user_id => @user.id.to_s
    @user.reload
  end

  expect { @user.should be_able_to_send_messages }
  expect { assert_redirected_to :action => :admin }

  should "send email to unblocked user" do
    assert_sent_email do |email|
      email.to.first == @user.email &&
      email.from.first == "do_not_reply@singlestravelintl.com"
    end
  end
end

The assertion tests verify that the user can send messages again, and that the user gets an email from the system to that effect. The associated user model test validates that the user’s spam message count is reset back to zero, which was the specific point of this round of tests (in the actual application, there were other Cucumber scenarios supporting basic administrative behavior).

context "unblocking a user" do

  setup do
    Message.new_from_params(Factory.attributes_for(:message,
        :created_at => 10.minutes.ago), @user,
         Factory.create(:user)).save!
    @user.update_message_block_status.should == "blocked"
    @user.should_not be_able_to_send_messages
    @user.unblock_message_status
  end

  expect { @user.should be_able_to_send_messages }
  expect { @user.spam_message_count.should == 0 }

  should "reblock" do
    @user.block_message_status
    @user.should_not be_able_to_send_messages
  end

end

Anyway, the code that I wrote along side these tests to make them pass seemed to work fine, and we deployed to production.

Shortly thereafter, we got a bug report. I haven’t stressed it here, but new users were limited to a single message in their earliest time on the system. We heard from a new user who sent a message, received a reply, and then replied back to that message, only to find that he was blocked, even though the reply message shouldn’t count toward the rate limit.

Interesting. It’s one of those bugs that almost turns philosophical — exactly when in this process does a user become blocked? In the code that I originally wrote, a user was blocked as soon as he or she hit the rate limit, and before sending another message. In fact, the code should wait until the user tries to send that next message to consider that user blocked, because a user who is at the rate limit border should still be able to send replies and friend messages without being blocked.

So, bug. Since the bug involves multiple interactions, I started in Cucumber:

Scenario: New User Behavior on reply
  Given I am a new user
  When I send 1 message in a day
  And I reply to a message
  Then 2 messages are sent
  And I am not blocked from sending further messages

All those step definitions exist, but the last two steps fail without further work. I wasn’t initially sure whether the code change for this would wind up in the controller or not — as it happened, it turned out to be a minor controller change and minor model change but it took me a few tries to get it right.

Since I already had a decent controller harness for similar scenarios, it took almost no time to adapt to the new condition:

context "with a borderline new user and a reply" do

  setup do
    reply = Factory.create(:message, :recipient => @user,
        :sender => Factory.create(:user))
    flexmock(User).should_receive(:new_user?).and_return(true)
    flexmock(Message).should_receive(
        :toward_spam_in_last_24_hours).and_return(1)
    post :create, :recipient => @recipient.id,
        :message => Factory.attributes_for(:message, :sender => @user,
            :in_reply_to => reply.id.to_s)
    @user.reload
  end

  expect { @user.messages_sent.size.should == 1 }
  expect { @user.spam_message_count.should == 1 }
  expect { assigns(:message).should_not be_new_record }
  expect { @user.should_not be_message_sending_blocked }
  expect { assert_no_email_to_administrator }
end

I have to say, in retrospect, that I’m not 100% sold on the use of the mock package here — it’s a clear user of mocking to limit the test’s exposure to the model layer, but I’m not convinced it makes the test more clear or readable. This setup and tests is very similar to the other bundles of controller test and makes most of the same assertions.

The main change was in the user method, triggered by the following tests — one of which tests the positive sequence, one the negative — and yes, the setups probably should have been combined.

should "move a new user to blocked mode after first message" do
  Timecop.freeze(Date.today)
  new_user = Factory.create(:user, :created_at => 1.day.ago)
  m = Message.new_from_params(Factory.attributes_for(:message),
      new_user, Factory.create(:user))
  m.created_at = 2.hours.ago
  m.save!
  assert_equal(2.hours.ago, new_user.last_message_sent_time)
  assert_equal("blocked", new_user.update_message_block_status(true))
  assert new_user.message_sending_blocked?
end

should "allow a new user a second message that is a reply" do
  Timecop.freeze(Date.today)
  new_user = Factory.create(:user, :created_at => 1.day.ago)
  m = Message.new_from_params(Factory.attributes_for(:message),
      new_user, Factory.create(:user))
  m.created_at = 2.hours.ago
  m.save!
  assert_equal(2.hours.ago, new_user.last_message_sent_time)
  assert_nil new_user.update_message_block_status(false)
  assert !new_user.message_sending_blocked?
end

The key here — which would be easier to see if the setups were combined — is the next to last line where update_message_block_status is called. That’s the method called by the controller when a new message is sent, and the false argument means that the new message does not count toward the rate limit. So in the second test, the user can still send messages, while in the first test, the user does move to a blocked status.

Having the existing suite of tests around the controller and model behaviors was a big relief when fixing this bug — it’d be easy to make a fix for this issue that affected one of the other scenarios (I know because I made fixes that broke a lot of tests…). This was a case where the tests clearly made me more confident in the fix that I made.

Related Services: Ruby on Rails Development, Full Life Cycle Software Development

Related posts:

1. A Real Testing Example
2. Elements of Testing Style
3. The Testing Interviews

As sort-of promised in last week’s post, I’m going to work through a real-world test example, with an eye toward explaining how and why I tested the way I did. Hopefully, I’ll be able to do this at blog-post length. If not, well, there’s always next week.

This site, which was a legacy rescue, allows users to send messages to each other within the site without having to give away their other contact information. The problem is that nefarious spammer types were creating logins and immediately sending messages to large numbers of the user population, irritating them. After some deliberation, the client decided on a rate-limiting strategy, where a member could only send a certain number of messages in a day, and a new member could send even fewer messages a day. Messages above that point would require administrative action to unblock the user’s privileges.

I started with Cucumber tests — the first two are representative of the initial batch. Everything you see here is actual code, only slightly tweaked to anonymize details of the site. But I’m trying not to present simplified “example” code. That said, this is the final state of the code, there were some intermediate steps and false trails that I’m sparing you from having to read about. I’m also focusing on the tests here, rather than the resulting application code.

Background:
  Given the user database is cleared
  Given a group of recipients.

Scenario: Normal User behavior
  Given I am a user who is not a new member
  When I try to send 5 messages in a day
  Then 5 messages are sent
  And the administrator does not get an email

Scenario: Excessive User behavior
  Given I am a user who is not a new member
  When I try to send 6 messages in a day
  Then 5 messages are sent
  And I am blocked from sending further messages
  And the administrator gets an email

This test uses an implicit style, so a lot of the details are in the step definitions. I start by writing the step definitions one by one until I get to one that I can’t make work without writing new code. As it happens, since not sending email is the current state of the system, the entire first scenario should work without any new code. It’s very important to include that scenario, though, to ensure that the new changes don’t break the basic behavior.

Here’s what the steps look like — the user who is not a new member defers to the boilerplate RESTful Authentication step for logging in:

Given /^I am a user who is not a new member$/ do
  @user = Factory.create(:user, :created_at => 5.months.ago)
    Given "I am logged in"
end

Given /^I am logged in$/ do
  visit "/login"
  fill_in("email", :with => @user.email)
  fill_in("password", :with => @user.password)
  click_button("Sign in")
end

Next up, sending messages. Since the point of Cucumber is to treat the application as a black box, the step definition uses Webrat to simulate number of posts to the create message RESTful action. (The @recipient is created in the background action, which I didn’t show here because it’s not very interesting.)

When /^I try to send (.*) message(.*) in a day$/ do |count, plural|
  count.to_i.times do
    visit(messages_path, :post, {:recipient => @recipient.id,
             :message => Factory.attributes_for(:message,
                                :sender => @user)})
  end
end

So this step definition allows you to match “send 3 messages” and “send 1 message” by adding that little group at the end of the message, that group needs to have an associated variable in the block argument list, but it’s just ignored. Somebody with better offhand regular expression skills could easily make it so the end of the expression only matches “message” or “messages”, but I can live with it being a little overly matchy for now.

Closing out the normal case, the step definition to test emails sent uses the excellent
email_spec plugin, which creates RSpec matchers and Cucumber step definitions for email testing. This one checks an all_emails method, filters out any methods sent to the administrator, and makes sure the right number exist.

Then /^(.*) message(.*) (is|are) sent$/ do |count, plural, verb|
  emails_out = all_emails.select do |e|
    !e.to.include?("admin@admin.com")
  end
  assert_equal(count.to_i, emails_out.size)
end

The administrator email step definition uses the email_spec steps directly — and yes, I could have included those steps explicitly in the Cucumber scenario. I chose not to, on the perhaps dubious grounds that I wanted to keep explicit string literals out of the scenario. But it doesn’t make that much difference either way.

Then /^the administrator (gets|does not get) an email$/ do |status|
  if status == "gets"
    Then '"admin@admin.com" should receive an email'
  else
    Then '"admin@admin.com" should not receive an email'
  end
end

Everything here passes for the normal case. So far, my main note is that all the step definitions are really simple — it’s almost impossible to misinterpret them.

At this point, I move on to the blocking case. There’s one more step definition to write. In actuality, the “then 5 messages are sent” step will fail, since the blocking isn’t in the code. So, I would normally jump to writing regular tests at that point, but since I’m here, I’ll present the last step definition.

Cucumber is a black box, therefore in order to detect if the user is blocked, we need to find some place in the application that will display it. After their initial notification, the user doesn’t see anything confirming their block status. But the administrator does, via an admin screen that actually hasn’t been written yet. So, the cucumber tests logs the user out, logs an admin in, and checks the admin page for a row associated with the user.

Then /^I am blocked from sending further messages$/ do
  @sender = @user
  visit "/logout"
  Given "I am a logged in administrator"
  @admin = @user
  visit path_to("the admin messaging page")
  assert_select("tr#?", dom_id(@sender, :blocked_row), :count => 1)
end

This step definition is a bit more complicated than the others, and it’s also taking it slightly on faith that the row with the correct DOM id will actually have the information the admin needs. (The tradeoff in view testing, as always is faith an flexibility vs. certainty and brittleness…)

Okay, there are genuinely failing Cucumber steps, it’s time to make them pass.

A couple of things to note:

• After going back and forth on this some, I’ve decided that Cucumber tests don’t really replace controller tests, except in maybe the simplest cases. My rationale is that Cucumber and controller tests look at different error conditions, and that Cucumber tests are really (partially) replacing the hand-testing that I would be doing to verify that the feature is working. Cucumber is the 30,000 foot view of the application, I still feel the need to test from the ground.
• This app was a legacy with effectively no tests, leading me to have to write some extra tests to cover the normal, non-blocking case. If the whole app was TDD, then that functionality would already have tests, but since I’m working on the new features, I need to go back and cover the base functionality.

The basic ideas is write the tests in the class where the code is going to go. I can guess that code will need to go in the MessageController, the Message model, and the User model. In fact, I’m going to need a new field in the user model, a string representing message sending status. (It’s a string and not a boolean, because I know from a later requirement that there will be more than two states). I actually will wait to write the migration, though, until a test compels it.

My preference is to start with the controller tests — it’s the easiest way into the system for me.

Here’s the batch of tests I wrote to cover the normal sending a message and not getting blocked functionality. These tests will probably look weird to just about everybody since I wrote them using a combination of Shoulda, Zebra, and Matchy. Zebra gives nice one-line tests, and Matchy gives a sort-of RSpec syntax that I sometimes like.

context "POST CREATE" do
  setup do
    ActionMailer::Base.deliveries.clear
    @recipient = Factory.create(:user)
    @user = login!
  end

  context "with a clean user" do
    setup do
      post :create, :recipient => @recipient.id,
          :message => Factory.attributes_for(:message,
                :sender_id => @user.id)
    end

    expect { @user.messages_sent.size.should == 1 }
    expect { @user.spam_message_count.should == 1 }
    expect { assigns(:message).should_not be_new_record }
    expect { @user.should_not be_message_sending_blocked }
    expect { assert_no_email_to_administrator }

    should "create and send" do
      assert_sent_email do |email|
        email.to.first == @recipient.email &&
        email.from.first == "do_not_reply@singlestravelintl.com"
      end
    end

  end

### Outer context continues

What to say about these tests…

• In controller tests, I organize contexts by action. The outer context setup clears the email and creates a couple of users — the login! method creates a factory user and simulates a login.
• The internal context posts the message
• The various expect blocks each resolve into a test that passes if the block returns true. The first one checks that a message has actually gone into the database (messages_sent is an association on user). The spam_message_count is what is used to determine if a user is blocked — that’s not going to pass yet, because the concept of a spam count is new to the app. The third tests that the message assigned in the controller is actually saved. The fourth checks the message_sending_blocked? method of a user, which is also going to need to be written. The last calls a helper method to determine if an email is sent to the administrator, which should only happen if the user is blocked.

Many of these tests pass as is. The ones that don’t are dependent on the new user features. Which means we need user tests.

I realize that some of you are starting to think that having all these tests is ridiculous overkill — I mean, you’ve got your Cucumber tests, you’ve got your controller tests, and now model tests. That’s, like, triple the work, isn’t it?

No, I don’t think it is. My rationale for doing all this testing goes something like this:

• In my head — which is, admittedly a strange place to be — these are all different tests covering different aspects of the same feature. They could all fail independently of each other, at least in theory. Each of them is exercising a different part of the code, and crucially, each of them is responsible for potentially testing error conditions at in their own space.
• It’s not that much extra time — I’m presenting the code in larger chunks than I actually wrote it (can you imagine how long this would be otherwise?). In the rhythm of testing, it’s a very short piece of test, followed by a very short bit of code, and the amount of tests grows over time.

Anyway, the user tests cover the case where a user has four messages, and then the transition to five messages. The setup is pretty similar (another reason why the extra tests don’t take as long to write as you might think)

context "rate limiting" do

  setup do
    ActionMailer::Base.deliveries.clear
    Timecop.freeze(Date.today)
    @user = Factory.create(:user, :created_at => 1.month.ago)
    4.times do |i|
      Message.new_from_params(
          Factory.attributes_for(:message,
              :created_at => i.hours.ago, :body => "Message #{i + 1}"),
          @user, Factory.create(:user)).save!
    end
  end

  should "correctly count messages" do
    @user.spam_message_count.should == 4
    @user.update_message_block_status.should be_nil
    @user.should be_able_to_send_messages
    assert_number_of_emails_to_administrator(0)
  end

  should "move user to blocked mode after fifth message" do
    Message.new_from_params(Factory.attributes_for(:message,
        :created_at => 10.minutes.ago), @user,
        Factory.create(:user)).save!
    @user.update_message_block_status.should be("blocked")
    @user.should_not be_able_to_send_messages
    assert_number_of_emails_to_administrator(1)
  end

## Outer context continues

Most of this should be clear given all my blabbering so far (and I have no real clear reason for abandoning Zebra in this section of tests). One thing is that the method update_message_block_status is actually responsible for changing the user status after a message is sent.

This is getting way long, so back next with with how this played out with new requirements, and a kind of interesting bug.

Quibbles with my testing style or process should go in the comments.

Related Services: Ruby on Rails Development, Custom Software Development

Related posts:

1. Real Testing Example, Part Two
2. The Return of the Cucumber
3. Using Cucumber for Acceptance Testing

Here's my starting point: Mock objects are best used in the following 2 1/2 situations:
1. To simulate an external object that would otherwise be prohibitive or impossible to use during testing. The classic example is a web service such as PayPal or an OpenId server or something.

2. As a shortcut to setting up a system state that would be difficult or impossible to set up step by step. My favorite example at the moment is Timecop, which is basically a specialized mock object package for setting and freezing the system time.

Here's the half situation:

2 1/2. Sometimes it's appropriate to use a mock object to fake a method call from another layer in the program, for example, mocking out model methods from controller tests. This turns around and bites me roughly half the time I try it.

The situation that I don't like using mocks is the one where you are just trying to isolate the method under test from the rest of the system, in what Martin Fowler called a behavioral test, rather than a state test. In my experience, I regret trying this about 110% percent of the time. I grant that some people really like and are successful with this style.

How do mocks not work for me? Generally, one or more of the following problems.

1. Philosophical. When I'm adding a lot of mocked methods to a test, I can't escape the feeling that the test knows too much about the underlying methods.

2. Practical. Typing in all the mock methods for a method under test that makes a lot of calls can be a real pain in the fingers. This is what turned me off RSpec the first time I tried it -- I felt like I was going so slowly.

3. Maintenance. Inevitably, the underlying API of the models changes, and the mocks have to change to keep up. For me, this has been a bigger maintenance drag than I normally have with tests.

4. Skin deep. One problem with faking a system state with the surface API is that eventually you may need to know about the object in more depth. For example, I recently was testing a message rate limiting system, and mocked just the boolean "can the user send messages" method. Inevitably, I actually needed to check the user's raw message count, which hadn't been set because it was beyond the facade. Essentially, I had to rewrite the entire test.

That's the gist of it -- I'm not a strong mock user by any means, but I do find them very helpful in certain spots. Next week, I'll dig deeper into that message system and talk about how I tested a real feature in practice.

Related posts:

1. Comparing Ruby Mock Object Libraries
2. Walk-Through Test Coverage
3. Using Mocha for ActiveRecord Partial Mocks with Finders

Cover image from Amazon.com

It's been way too long since I blathered on about style issues. Today I'd like to talk about testing style. This article assumes you are already writing tests and already using something approaching a Test-Driven Development process -- I'm not here to argue about process, at least not today.

Today the topic is the actual construction of individual tests, how to name them, how to group them, where to get data from and the like.

I suspect I'll think of five more things right after I post this, so look for an update sometime in the future. The update will also address all the places where everybody tells me that I'm totally wrong.

Name tests after their expected behavior

The first part of each test is its name. By now, there's no excuse for still using the def test_underscore_instead_of_spaces naming convention. Rails 2.3.x, Shoulda, and RSpec all support a style where the test name can be an actual string with actual readable spaces and everything.

A test should be named after the expected behavior it verifies. Being specific is helpful here -- the name is potentially important documentation.

GOOD: test "should use last_name, first_name as display"

LESS GOOD: test "should correctly display names"

BAD: test "name display"

If you can't think of an easy name to describe the behavior being tested, this is a strong hint that the test is either a) richly deserving of being split into multiple parts or b) unneeded. Most of the time, it's (a).

Use contexts to group related tests

I'm also at the point of feeling like there's no good reason not to use contexts. If you don't want the weight of using Shoulda or RSpec, then just use the Context gem to get simple contexts.

So, what gets grouped in a context?

• A set of tests that all share a common data setup
• A set of tests that exercise a common method, as in controller tests that all work on the same action

Obviously, those two options tend to go together.

The context name should reflect the commonality -- either the name of the method being exercised, or something denoting the common data setup, like "as an administrative user".

Nesting

Contexts can also be nested, allowing for setups to be extended. I don't have very strong ideas on when nesting contexts is advisable, my current guidelines look like this:

• Only nest a context if the inner context uses all of the outer context setup and then adds on (similar to the guideline of only creating an Object subclass if the subclass uses 100% of the parent class data)
• Try to keep the distance between the different setup methods as small as possible -- the problem is that having to trace setup code up multiple pages and across several generations of contexts is a real pain.

Actually, the problem I'd really like a clean solution for is what to do if I want the inner context setup to be run first -- for example, a controller action that I want tested for multiple user types. The controller action is logically in the parent context, but the login as the various user type, logically in the inner context, needs to happen first. Putting a large context for each user type separates the tests for each action, which I'd rather avoid. Anybody smart got this one solved yet?

Setting Up Data

The guideline for creating data for tests is simple: create the minimum amount of unique data that you can while still having a meaningful test. In many cases, a model unit test will only require one object, and in many of those cases only a small number of attributes are actually relevant to the test. (One of the advantages of factory tools is the ability to highlight just those relevant attributes).

Even search and report methods can often be unit-tested with only two models in the database -- one to be found, and one to be ignored. (You'll often need more than one test, but each individual test is small and focused on one unit).

Setup should be as close to the actual test as possible, granted that it often makes sense to group repeated patterns into setup methods.

Single Assertion Style

Shoulda has popularized a testing style where most of the test is in the setup and each individual testing method has a single assertion. I have mixed feelings about this style, though I use it sometimes.

Guidelines:

• If you are going to do single-line tests, definitely use a tool that optimizes single-line tests like Zebra or Testbed.
• I think single-assertion tests work best when setting up relatively simple model tests that don't require a lot of nested contexts.
• Especially in controller tests, single assertion tests can lead to a lot of extra calls to the setup and a slower test suite, so that's something to keep an eye on.

When tests should be DRY and other fascinating questions?

The key thing about tests is that they are also code, so pretty much every style issue that applies to regular code also applies to tests.

That said, tests do have a special place in your application in that they are the part of your code that doesn't actually have tests of their own. So there are some special guidelines for tests.

Don't get cute. There's an old programing truism that if debugging is harder than coding, then by definition any code that is as clever as you can write it is impossible to debug. Similarly, any test written at the edge of your ability is going to be impossible to verify. It's not that metaprogramming and other fancy stuff has no place in testing -- Shoula macros depend on metaprogramming, for example. It's that you probably don't want an individual test to depend on the subtleties of Ruby's object system.

You can repeat yourself a little. I place a higher value in tests on being able to read the whole test in one glance than I do in regular code (I expect regular code to be spread across various methods, I expect a test to be self-contained). As such, I tend to allow somewhat more code duplication in tests than I do in application code, especially in setups. That said, keeping setups short has a value as well, and when the setup is creating enough objects that it's obscuring the point of the test, I'll refactor the setup to a helper method and reuse that. But in application code, I'll generally blast duplication immediately, but in test code, I'll often wait until I have some pain. Is that a good thing? Not sure.

Your turn. Where am I wrong?

Related Services: Ruby on Rails Development, Custom Software Development

Related posts:

1. The Testing Interviews
2. Ask A Rails Tester Person
3. Elements of Ruby Style

I'm unabashedly happy with my phone, because it's strengths and weaknesses mesh pretty well with my actual needs. It's not that great a phone, but I don't use the phone that much. On the other hand, there's never been a better gizmo for whiling away a long train commute.

So, here's some stuff I use:

Instapaper (free light version, $9.99 pro version currently on sale for $4.99)

Instapaper has probably changed my web reading habits more than any other app since I started using RSS readers. It's so simple that its almost hard to believe how useful it is. When I come across an article on the web I want to save for later, I click a bookmarklet. Later, launching Instapaper on the phone, the article shows up, with the images stripped away, and the text presented in a reader friendly format. (Sometimes you can help Instapaper out by invoking it from the printer-friendly version of a page...)

Using Instapaper has become kind of second nature -- I always have a few articles ready to go. The Pro version adds a few nice features, including control of the display font and the ability to scroll the article by tilting the phone. The tilt-scroll sounds like you'll need to be a gymnast or something to read an article, but in practice it's a super-clean interface for reading long articles and letting the text scroll at your reading speed. Great app.

Birdhouse ($3.99)

Very well designed little app for what seems like a dumb use case -- saving drafts of posts intended for Twitter. I mean, how much do you need to polish that tweet about the ham sandwich you had for lunch, amirite?

But, if you are trying to do a tip a day on Twitter (@railsrx), then having a nice place to store up ideas for future tips is great. The app also works as a slightly structured note-taking app, since it can email it's existing draft population back to you.

Stanza / Kindle (Free)

The two leading general-purpose eBook readers, both of them are easy to use, and manage the task of making text readable on an iPhone. It'd be nice if there was some consistency in formats between the two apps, and also if you could buy books directly from the Kindle App (presumably that's coming).

MLB At Bat ($9.99)

Obviously only if you are a baseball fan, but the app gives you access to live box score and play-by-play data, live audio stream of radio broadcast, video highlights, special goodies like condensed game videos a few hours after games and, plus live video streaming on a currently limited basis. That's a lot of stuff. Add in the fact that the app is pretty enough to have won an Apple Design Award, and it's a pretty fabulous package.

Twitteriffic (Free lite, $3.99 no-ads)

There are something like a zillion Twitter clients on the iPhone at last count, and which one to pick is basically idiosyncratic. It's a mark of how fast iPhone development is going that Twitteriffic 1.0 won an Apple Design Award in 2008 and was completely blown away by newer clients six months later before regaining strong status with the 2.0 release. I find this has a nice blend of features and interface. (Tweetie, which is my desktop client, is also very good on the iPhone).

Of course, this all sounds serious -- the big winners on the iPhone have all been games, there are all kinds of inexpensive, addictive little games I play. Here are a few: Defender Chronicles, The Creeps, Drop 7, Flight Control, Frenzic, Galcon, Peggle, Strategery.

Related Services: iPhone Application Development

Related posts:

1. “Developing iPhone applications using Ruby on Rails and Eclipse, Part 3: Developing advanced views for iPhone” now available on IBM Developerworks
2. Chiphone Meeting: Chicago iPhone user group gets its TDD on
3. “Developing iPhone applications using Ruby on Rails and Eclipse, Part 2: Displaying iPhone content to the client” Also up on IBM Developerworks

Windy City Rails is September 12, 2009

Here are a couple of upcoming Ruby and Rails-based appearances by Pathfinder personnel:

On Tuesday, June 23, Noel Rappin (referring to himself in the third person) will be the guest speaker at the Chicago Ruby.org monthly meeting. The meeting starts at 6:00 at Chicago Ruby's downtown loop meeting location, see the link for details. The working title of my talk is "I'd Like To Start Testing. Now What?" and it'll be an informal discussion of testing tools and good practice.

The schedule for WindyCityRails was announced this week. The conference is September 12, 2009 at the Westin Chicago River North. John McCaffrey from Pathfinder will be presenting "Super-easy PDF Generation with Prawn and Prawnto", and I'll be up there with "How To Test Absolutely Anything".

Other speakers include Ryan Singer from 37signals, Ben Scofield from Viget Labs, and Yehuda Katz from Engine Yard.

Registration through August 1st is $99, there are a couple of tutorial sessions also available for purchase.

This was a very well-run regional conference last year, and I'm excited for this year's edition. Hope to see you there.

Related posts:

1. Why Chicago is Rails-town, USA
2. Upcoming No Fluff Just Stuff conference
3. Pathfinder Launches Beer Hunter, A New Flex + Ruby RIA

I mentioned last week that the RubyMine post was replacing what I had meant to write about. Well, this week we finally get to it...

It's been about ten weeks since I wrote about Cucumber the first time and the second time. Since then, I've continued to use Cucumber and now seemed like a good time to update some thoughts on how and why it seems to be working for us.

The big headline, of course, is that I'm still using it after ten weeks. I'm pretty quick to abandon tools that aren't pulling their weight, so just the fact that Cucumber is still in the toolbox means that despite the time that it takes to write Cucumber tests and step definitions, I'm finding the process of writing the tests and the tests themselves to be valuable.

Where I'm using it

Right now, I've been using Cucumber primarily on two different projects.

• An internal project that has a well-defined testing process, and 100% coverage for unit tests on models and for functional tests on controllers.
• A client project that we took over from the original coders, who wrote no tests at all. All new features come in with tests, but we haven't

Cucumber plays out a little differently in each case.

How I'm Using it

I struggled a bit with how to integrate Cucumber into the testing process, and eventually came to the following rule of thumb:

Just as all lines of code should be in response to a failing unit or functional test, all new unit or functional tests should be in response to a failing line of Cucumber.

The process seems to go something like this:

1. In conjunction with a stakeholder of some kind, write the cucumber tests for the feature, usually this will be multiple scenarios. At this stage, don't worry about setting up data, and I think you're better off focusing on making the test as human-readible as possible, but see below for some thoughts in that direction.




2. Start writing step definitions for the Cucumber tests, step by step. In many cases, you'll find yourself adding some steps to create data (especially to the background step of the feature). Especially on a mature project, you'll often be able to write several steps without needing new code in the app itself -- "Given I am a logged in user", "When I visit the project page"... in essence, the setup.




3. In the fullness of time, you'll come to the meat of the feature, and you won't be able to make the Cucumber line pass without writing application code. At this point, you drop into a normal TDD process, with the goal of moving the application forward enough to make this line of Cucumber succeed.




4. What happens here depends a little bit on personal taste and the project team. For the internal project, which has a goal of 100% functional test coverage, I do write functional tests even if they overlap the Cucumber test strongly. On the legacy project, I do sometimes drop do the model/unit tests from the Cucumber test if there doesn't seem to be a substantial difference between the acceptance and functional test.




5. When I think the regular TDD process has gotten me enough code to pass the Cucumber line, then I run Cucumber again to verify, and move on to the next line.

This process has a lot going for it. The TDD part is always focused on a specific, definable goal -- the next line of Cucumber test. The cycles between acceptance and regular testing are short enough that the two feed into each other (sometimes the Cucumber tests change in response to the shape of the code, and sometimes vice versa), and they work together to keep the code simple, complete, and covered.

Cucumber vs. Functional testing

The big question is to what extent the Cucumber tests duplicate or replace functional tests. At the moment, my answer is somewhat, but not completely. I see the Cucumber tests as being the way to go for testing a work flow of more than one one action, but I still find it a little easier to set up data in the functional test framework. That said, I don't think that adding the functional tests has cost me that much more time, and there are still enough gaps in Cucumber/Webrat (most notably Ajax) that I'm reluctant to give up controller tests entirely. I expect my thinking on this will change as we have projects that had Cucumber from the very beginning.

General vs. Specific

I'm having a stylistic issue with the Cucumber tests between putting the specifics in the Cucumber or in the step definition. This came up most sharply in the legacy project. We were working on a feature that made a distinction between new users and old users. My first pass at the Cucumber test started off like this:

Given a user named "Noel Rappin" who has been a member for 5 months

With a step definition that was something like this:

Given /^A user named "(.*)" who has been a member for (.*) months$/ do |name, mo|
  @user = Factory.create(:user, :name => name,
      :created_at => mo.months.ago)
end

The idea was that "5 months" was just a stand in for "an old user", but the client, quite understandably, wanted to know what the deal was with five months -- the specificity of the rule seemed to imply that it was important functionally. Eventually, I changed the rule to be more like this:

Given /^A user who is not a new member$/ do
  @user = Factory.create(:user, :created_at => 5.months.ago)
end

Once I saw this issue, I started seeing it everywhere. The second form has the advantage of being more general and more requirements like, and the disadvantage of hiding a lot of magic detail in the step definition. The original version has the advantage of putting specific data front and center and the disadvantage of not making it clear which details are important.

I'm not aware that there's a consensus on the style issue. My tendency, right now, is to favor the general form on the theory that it's a good thing to make the Cucumber test as readable as possible and let the step definitions sort it out. On the other hand, this seems like a good way to get really messed up by having the step definition not quite match what you expect, and have that be hidden.

Related Services: Ruby on Rails Development, Custom Software Development, Agile Development

Related posts:

1. Cucumber Rocks – But it’s not a replacement for unit tests
2. Using Cucumber for Acceptance Testing
3. Again With The Cucumbers

This actually wasn't the post I was planning to write this week, but a couple of days ago I got it in my head to try the latest version of RubyMine, the Ruby and Rails IDE from JetBrains.

Now, I tried this with the beta version back in December, and thought it had a long way to go. However, there's been a lot of online chatter about it over the last few months. (Gregg Pollack was particularly enthusiastic in his RailsConf talk), and I thought I'd go back and try again. And I started Twittering about it again, and got some responses, so I thought I'd do a longer write up.

The executive summary is that I started using this on Wednesday and haven't stopped yet. It's come a very long way, and although there are still annoyances, it felt better to me than the other Ruby IDE's. I think it's not quite as solid as my normal TextMate/termial prompt combination, but it's improving rapidly. Overall, this is the first time that JetBrains' Ruby IDE gave me that pleasant "this is like IntelliJ" feeling.

One thing I definitely learned from my years as a happy IntelliJ user back when I wrote more Java code, is that JetBrains releases often, their betas are usually very solid, and they add new stuff really, really rapidly. So my expectations are high that RubyMine will continue to improve.Here are some more specific comments. A lot of these are going to sound negative -- there a many cases where RubyMine has about 85% of the functionality I want, just enough to make me really miss the 15% that's not there:

Navigating

File navigation is very nice, one area where RubyMine just blows TextMate away. The fuzzy file search for file names is much crisper, you can click on a symbol to go to its definition, open file tab management is actually sensible. It works very nicely for getting around your project. Also, the search for symbol feature is outstanding.

I've also always loved the navigation bar that shows you the path to the current file.

Features

One of my favorite features is local history, so you can go back and check the file against all saved versions locally over the past few days, even if they were not separate checkins to source control. This is amazingly useful, and very easy to use.

The auto-completion is nice, but a little fragmented -- there are four different auto-completion commands, which is probably three too many. A unified approach would probably be better here. (I still miss the TextMate/RubyAMP complete against all open files command).

There's an integrated irb console which is nice -- I'd love to also see a regular command line.

I can't really say much about the git/subversion integration -- I have a long-standing, probably irrational aversion to the JetBrains UI for source control integration, so I stuck with my command line for git.

My all-time favorite JetBrains feature is the help menu item that shows you a whole bunch of JetBrains features and flags ones you haven't used. I browse that from time to time to try and pick up useful tricks.

Test Support

As of RubyMine 1.1, it syntax colors and runs Cucumber files in the IDE, this is very nice.

Although there are commands to bounce between associated model, controller, view, and helper files, there's no way to bounce to the test file for an app file. That's a really annoying oversight.

You can run any file open file from its tab, including tests, which is great. It'd be really great if you could run a test file directly from its associated app file.

All though RSpec files are correctly parsed, and the file structure window lets you browse them, Shoulda files are not, and neither are the Rails tests that use test "do something" do end.

UI

A lot of the UI feels a little clunky, over designed, and kind of, dare I say, Java-y. Examples:

• All the side decoration windows -- project window, test runner, etc, take up a lot of space. It's a problem on a laptop screen because eventually you don't have much room for code. All these windows can be detached, but in the detached state, as far as I can tell, they only float over the main window, they won't act like normal windows. This is a real pain.
• The debugger is nifty, but a little hard to use, I had a little problem creating watches, and inspecting really introspective objects like ActiveRecord models is awkward at best.
• It's the only application I use that has enough menu items that it blocks my system-side menu items on the right. That's annoying.
• Similarly, I think the test result display window could use some polish.
• The preference dialog is really, really complicated.

Intelligence

The live templates and surround with templates are pretty good -- a hair behind what TextMate offers, but still pretty useful.

As nice as the autocompletion is, the warning and code intelligence systems still have a ways to go. Here's a partial list of things that triggered spurious code warnings from the IDE:

• Any call to a FixtureReplacement create method. I haven't tested on
• All RESTful routes
• Any call to one of the fixture methods
• Dynamic find methods
• Anything of the form model.association.select
• Any call to Timecop.freeze comes up as an error, not a warning.

That's a lot of stuff, and it's a lot of stuff I use regularly -- the problem being that having a lot of spurious warnings makes it difficult to recognize a real one. I realize it's a challenge to verify methods that are created dynamically. I'd at least like an option to say "hey, this method is really there", or better, have the IDE say "you just passed a test that covered this warning -- I guess that method is there, I'll shut down the warning.

That said, it does a decent job flagging syntax errors.

Conclusion

I've got a tough decision coming up when my demo runs out. Right now, I still think my TextMate workflow is a little more efficient. But give me a few months of RubyMine getting better, and me getting back acquainted with the key shortcuts and what's there -- I dunno. It'd be really close.

Meantime, there's no reason for you not to try the 30-day free trial and use it for a day or two.

Related posts:

1. Trying Out RubyMine
2. My IDE
3. Ruby Internals: Overriding Object.require() to watch files as they are loaded

Continuing on the theme of comparing similar things that I started last week, this week I'll be taking on Mock Object libraries.

The purpose of a mock object library is to allow you to create "fake" objects that can take the place of the regular objects in your application during testing (you'll sometimes see them called "test doubles", by analogy with a stunt double, another kind of stand-in).

There are a several reasons why you might want to use a test double in your tests, the two most common are probably these:

• To take the place of a hard to create or expensive to access object or method call, such as a web services call. Using the test double lets the test system pretend that the expensive object is there, but at a much lower time cost.
• To isolate an object being tested from the details of the rest of the system. In Rails, for example, a controller test might create test doubles for methods in the model so that the controller test can pass or fail separately from whether the model implementation is correct.

Ruby's open object model and duck-typing makes creating test double objects relatively easy compared to stricter languages like Java. There are four major mock objects packages in Ruby:

1. FlexMock is the original Ruby mock object package
2. Mocha is quasi-official in that the Rails team uses it for their tests.
3. RSpec defines its own mock package
4. RRis the newest entry, with shorter syntax and a couple of new features

Here's a tour of what each package looks like, and when you might use each feature.

Before we start, some important points:

• This is in no way a comprehensive look at any of these libraries.
• Some of these libraries have advanced or other features that don't really map to anything the other ones do (especially RR), check the full docs for each package for full info.

The first step is installing and integrating with Test::Unit. The RSpec mocks, of course, can't be integrated with Test::Unit, however, any of the other packages can be integrated with RSpec by adding the line config.mock_with :rr or :flexmock, or :mocha in the spec_helper.rb file.

The most basic thing any of these packages can do is create a single dummy object that you can specify one or more methods with return values. When those methods are called, the specified values are returned. If the methods are not called, nothing happens. In general, once these objects are created, they are treated exactly the same as pre-existing objects, and all the later filters and expectation methods apply.

Most of the time, though, you'll want to replace methods on existing objects. In Flexmock and RR, this involves calling a special method with the object as an argument, as in Flexmock's flexmock(object), while in Mocha and RSPec, this involves calling a method on the object, as in object.stubs. In either case, further information about the method being stubbed and its return value is usually chained after the declaration of the stub.

Remember, classes are just another kind of object in Ruby, so class methods can be treated like any other method stub(User).should_receive(:find).

All these packages allow you to set an expectation that the method being replaced will actually be called one or more times. This is typically called a mock, as opposed to a stub. If a method is set up with an expectation that it will be called, then if that method is not called during the test, the test will fail.

The biggest difference here is that FlexMock does not have a separate syntax for creating objects with expectations, any doubled object can have an expectation added by appending a method like once to the description chain. In Mocha, RR, and RSpec, test doubles that will have expectations must be declared as such, using mock or (in Mocha) expects. In those libraries, specifying an method with mock implicitly assumes that the method will be called exactly once.

Each library offers options to change the expected number of times a method will be called.

RR has this unique feature called a proxy, where the method is actually called (as opposed to the return value being set by RR), but you can still specify an expectation on how many times the method is called.

All four libraries offer similar syntax for specifying arguments that must match for the doubled method to be invoked. This allows you to specify different return values based on the arguments. For example, you could specify multiple stubs of the find method, each returning a different model object. In addition to matching based on the exact value of the arguments, each library offers some more generic matchers based on class, or matching a regular expression or whatnot.

I said a little bit ago that classes can be doubled just like any other object. Three of the libraries also have special syntax that allow you to specify stub or mock behavior for any instance of the class that is created subsequently to that declaration. (Be careful, instances of the class previously created will not have the double behavior). Typically, after the method declares that this double applies to all new instances, any other filter or expectation can be applied.

A couple of the libraries offer special features for ActiveRecord.

Also, and this is a little more advanced, some of the libraries offer the ability to set a stub or mock on an entire chain of method calls in one line, without having to explicitly set the intermediate mock object.

Conclusions

I sure hope that made sense, I find the terminology surrounding mock objects to be hopelessly and needlessly confusing. Please discuss confusion in the comments, and I'll change the text as needed.

All of these tools are very solid, flexible, and providing functionality above and beyond what you are likely to need in your testing. I think it's perfectly reasonable to make a choice based on liking a particular syntax or needing one particular feature.

I've personally used FlexMock the most, but recently switched to Mocha on the theory that it was almost a Rails default. I think RR is very cool in concept, but I've found it hard to get used to in practice, it's almost too succinct.

Related Services:
Ruby on Rails Development, Custom Software Development

Related posts:

1. To Mock Or Not To Mock
2. Ruby Internals: Overriding Object.require() to watch files as they are loaded
3. Using Mocha for ActiveRecord Partial Mocks with Finders

Continuing on the general theme of blog posts that people actually asked me to do... this week, I'll be taking a comparative look at fixture replacement tools. The general request was to have one place that compared the features of the various libraries so that a prospective fixture replacer might make a more informed choice.

So here goes, I'll try to keep the editorializing to a minimum, but no promises.

Fixture replacement or factory tools replace the global nature of Rails fixture data with the ability to easily create new objects based on sample valid data previously defined by the developer. Factory tools have several advantages, including keeping each test independent of other tests, since you are no longer using global fixtures, and allowing the important data for each test to be defined in that test.

Updated to correct some errors caught in the first few comments.

Self-promotion alert: A full description of why to use fixture replacements and an in-depth look at one of these tools is available in the full version of Rails Test Prescriptions available wherever fine PDF books are sold as long as you think the only place fine PDF books are sold is http://www.railsrx.com.

The competitors

As I write this, there seem to be four factory tools that have some user base:

• Factory Girl, from Thoughtbot, at version 1.2.1 right now
• Fixture Replacement, by Scott Taylor, version 2.1
• Machinist, from Pete Yandell, currently at version 0.3.1
• Object Daddy, by Rick Bradley and Yossef Medelssohn, currently at version 0.4.1

Each of these tools has subtle or more advanced features that are outside of the scope of this quick guide -- be sure and check out the tools that look appealing to you.

Defining Factories

The basic structure of each of these tools is similar. You create a file made up of factories, each of which defines the basic data for a model. First, let's discuss that file. Each tool places it in a different location.

Just as a note, my personal preference here is to keep the tools in the test directory -- the placement for FixtureReplacement in the db directory bugs me. Object Daddy is the only tool that doesn't expect the factories to be defined outside the model.

Within the factory file, the general idea is that multiple blocks are defined, one to each model, more or less. This table contains the syntax for creating a block -- we'll cover what goes inside that block in a moment. Let's assume here a model called Project

There's a slight difference in structure here. Since Factory Girl and Fixture Replacement are essentially defined in their own namespaces, the factory blocks take a single argument, which is the model being created. Machinist adds a class method to ActiveRecord base, so it doesn't need that argument -- the block will be evaluated inside the class that the blueprint method is called on.

Defining Attribute Defaults

Within each factory block, individual attributes can have default values specified. Typically these are either static values, or random values (the Faker gem is very helpful here). Each tool has a slightly different syntax for defining basic attribute defaults.

Unless otherwise specified, these lines defining the defaults would go inside the factory blocks listed above. Also, unless otherwise specified, you can do other arbitrary code inside the factory definition to calculate any intermediate values you want.

Some slight difference here between Fixture Replacement's use of an assignment statement, Machinist using a method that takes a block, and FactoryGirl using a method that takes a value or a block. Object Data does everything a little differently

Defining Sequences of data

One nice feature of these factory tools is the ability to create sequences of related non-repeating data, whether that's just calling projects "Test 1", "Test 2" and so on, or a more complex sequence. Here is how to define a data sequence in each tool. Note that even though these examples take an index and are sequential, they can also be used with just random data.

Defining Associations

A factory also allows you to define default associations with dependent objects. This isn't always what you want -- my experience is that you are frequently better off specifying the associations in the actual test. For the purposes of the example, let's assume that the Project belongs to a Client, and again, all this code is in the block defining the example, unless otherwise specified. These also assume that there is a separate factory definition for Client.

Also note that has_many style relationships can not be created directly in any of these tools, because ActiveRecord needs the object to be saved first. These relationships need to be created in the actual test.

Creating Instances

Having spent all this time defining attributes, you are probably going to want to actually declare some in your tests.

First, you actually need to include or require some things to allow your tests to be aware of the factory replacement tool.

Then you need to actually write the lines in your test that will create your data.

In general, all these methods allow you to override the defined default values by passing them as key/value attributes to the method creating the object. In general, this works for attributes as well as associations. The examples below will show creating the object with the name field overridden, just to give a sense of the syntax. Some of the tools have different options depending on whether you want the object saved to the database or not.

To me the biggest difference here is that Object Daddy and Machinist call new class methods of the actual class, while Fixture Replacement creates methods of Test::Unit, and Factory Girl uses its own object.

And In Conclusion, I'm Done

That's a quick look at the available tools for using factories to replace fixtures. I've had personal success using Fixture Replacement, Factory Girl, and Machinist -- I haven't had a chance to use Object Daddy yet. I slightly prefer Machinist, but not by enough that you should take that seriously if one of the others looks good to you. (I should mention that Factory girl will allow you to use some of the syntax from the other tools if you've taken a liking to it).

The biggest thing to remember when using these tools is to avoid setting up large amounts of data for each test -- that'll slow things down a bit. Instead, you want focused unit tests that create a minimal amount of data for each test.

Related posts:

1. Review of fixture_replacement2 plugin – Update
2. ChicagoRuby meeting ‘Test Prescriptions’ recap
3. Live Ruby: Testbed, part 1

Between my sessions at RailsConf, and the book promotion at Ruby Learning I've been fielding a lot of questions about Test-Driven Development and Rails. Here are five of the most common general questions.

Self promotional note: many of these questions and more are covered in detail in Rails Test Prescriptions, only $9 dollars for a book that covers all kinds of testing topics...

What test tools should I use? Should I use RSpec?

This is, I think, an increasingly common question as the number of available Rails testing tools increases. Most of these tools were created to fill specific needs that were unserved by the Test::Unit and fixture tools provided as the default by Rails.

The important thing is not to get hung up on finding the absolute best test stack before you start -- for one thing, you won't know which tools are best for you until you get going. For another, the big frameworks don't have all that much functional difference any more.

If you are new to testing in Rails, I recommend you start with the default stack -- it's easiest to install, simplest to use, and has the most documentation. Eventually, you will discover the same limits that led people to move to new tools.

At that point, if you start hitting the limitations of fixtures, you can move to FactoryGirl or Machinist. If you start wanting groups of tests with the same setup, you can add Shoulda or Matchy. If you want mock objects, you have choices. You can also go to RSpec to get most of these features as well. But you'll get the most out of the extension tools if you come to them from a position of genuine need rather than a position of "all the cool kids are using this".

I have an entire app with no tests, what should I do now?

Start your TDD process on new features immediately. Don't worry so much about going back and fixing the rest of the program in one fell swoop -- it'll wind up being a huge time sink and the risk of adding errors is high. (This advice also applies if you have tests, but think you can do better now...)

Do start using TDD on new code. Also add tests to cover any reported bugs, this will help you build up coverage throughout the program. Use mock objects to keep the part of the code you are testing separate from the untested parts.

If you want to get some coverage on the legacy parts of the code, try using a more black-box tool like Rails integration tests or Cucumber to try and verify high-level features from the level of user interaction. Since these tests aren't as dependent on the structure of the code, they may be easier to manage.

Why should I test? Won't I still need human testers?

Test-Driven Development is not a substitute for human acceptance testing or review by a domain expert. Just because the tests pass is no proof that the tests are testing the right functionality.

However, TDD has a lot of benefits. Code written using TDD tends to be simpler, easier to manage, and easier to debug and maintain going forward. I develop much more quickly using tests then when I am not writing tests. This is in part, but only in part, because I execute the code in the browser less frequently. It's also because the TDD process focuses my activity efficiently. A program written with tests also has a lot of protection against introducing bugs or breakage in later development.

When should I test? Is there such a thing as too much testing?

You can always overdo a good thing, and it's certainly possible to spend time writing tests beyond the point where you are getting much benefit from it. I think that most of the code and developers I see are, to put it mildly, not near that point.

My feeling is that you should test any code whose correctness matters to you. Sometimes, the correctness doesn't matter -- you know it's a one-off script, or you are just experimenting, or who knows what. AddedSometimes it's genuinely the case that it's not worth the time to add a test for some weird corner case or something.

EDITDED SLIGHTLY
Remember, though, when you do write tests, it's much easier to write the tests first. Easier to write the tests, easier to write the code.

I don't know what to test. How do I start?

The great part about TDD is that it doesn't matter all that much where you start. So pick one specific thing about your program that you know it needs to do. You can start at the controller level: "When the activate controller method is called, the user is activated", or at the model "The user model updates when it's activate method is called" or at the user level "When the admin clicks the activate check box, then the associated user can log in". Some of those are higher-level then others, but all of those will lead you in the direction of the other ones as you move forward.

Things to keep in mind.

• No matter where you start, start with one simple test. One assertion, if you can manage it.
• Write the code to make that test pass
• Then refactor -- this part is critical, this is where the creative design is with a TDD process.
• Then write the next test.
• In Rails, you'll often start in the controller then realize that code needs to be written in the model (or vice-versa). Write a separate test for the model -- testing models from the controller makes it hard to test all the model logic.
• It's okay to plan the tests in advance, but you should only work on one test at a time. (Sometimes I'll write the series of tests, then comment out all but one)

Related posts:

1. The Testing Interviews
2. Rails Testing First Look: Blue Ridge
3. Ask A Rails Tester Person

A rambling set of notes and things while it's still in my head.

Overall Notes

Starting with the positive... I had a great time. There's nothing like getting to be around hundreds of people who share the same excitement for programming, Rails, learning new things, and doing things well. The sessions were quite good overall, it was great to meet several people I had only interacted with online. There are always things to grumble about, but overall it was well-organized and exciting and fun.

Attendance was definitely down. I don't have exact numbers, but it was about 1/3 less then last year. The exhibit hall seemed down even further. On the other hand, there seemed to be an unusually large percentage of first time attendees. Not sure how to explain that, although it did seem like the Las Vegas location turned people away.

Insecurity over doing things "the right way" was a common concern, from the tutorials, to DHH saying in his keynote that even he doesn't know all the Rails internals any more, to the questions I was getting on testing, etc...

On the plus side, all the keynotes were recorded professionally, and are being turned around very quickly. DHH's keynote Tuesday morning, the Tim Ferris interview Tuesday night, and Chris Wanstrath's talk Wednesday morning are online as I write this. Robert Martin's just went up.

On the down side, unlike a lot of the regional conferences, the individual sessions were not recorded. O'Reilly tried to organize the community to record sessions, but there was nothing like enough of a critical mass of people with Flip cameras to make that work. Although, who knows, maybe things will trickle in.

All available session presentation slides are available at http://www.railsconf.com/presentations.

Rails 3

Rails 3 was a common topic, although an alpha version was not released. New features discussed included:

• Brand new router, with a cleaner DSL and the ability to be more modular
• A change in output to where the new default is to escape output to prevent XSS attacks, and raw output must be explicitly specified
• New, unobtrusive JavaScript and Ajax handlers.
• Easier support for different ORM and template engines

DHH assured everybody that the Rails defaults will still "kick ass" (Kathy Sierra fan alert!), but that developers with specialized needs will find it easier to meet their needs.

The JavaScript mechanism looks particularly nice. Using the new HTML 5 "data-" prefix for custom attributes, a remote call like the following:

<%= link_to "Delete", @comment, :remote => true, :method => :delete %>

Would be turned into the following HTML

<a href="/comments/1" data-remote="true" data-method="delete">Destroy</a>

From there, it's easy to see how an unobtrusive JavaScript library like jQuery could latch on to those custom attributes. In fact, the plain is for Rails to provide adapters for common JavaScript toolkits in the same way that adapters for databases are provided.

Rack

Lot of discussion of Rack, Rails Metal, and Sinatra this year, including two sessions that had very similar focuses, plus a significant section of the Rails Envy session (which dug into the actual Rails core Rack middleware code), plus the announcement that next year's Rails Rumble will be open to any Rack-based application.

There's a general consensus that Rack is very powerful stuff, and that we are just starting to see the beginnings of very cool tools. At the same time, it seems like nobody is quite sure what web applications will look like when everything that can be refactored to Rack is refactored to Rack.

Testing

Testing was a big theme, with a lot of sessions on testing tools and processes, including my own. Bob Martin's keynote, which you should definitely check out, was very emphatic on the idea that testing is part of what makes programming a profession. (It was also an interesting counterpoint to recent discussions about "professionalism" in the Rails community).

Cucumber was very prevalent. In addition to a dedicated session from tool maintainer Aslak Hellesøy, it was mentioned in several other talks, including mine.

I learned about some new tools.

Dave Chelimsky announced stubble, a new API for stubbing ActiveRecord objects away from the database for the purpose of testing. Pretty cool.

I only saw part of the talk, but Relevance presented Blue Ridge, a plugin for testing JavaScript code.

Non Rails-Core keynotes

Tim Ferris... I'm sure you may have seen some of the reaction online to this talk. While I appreciate the general idea of bringing in speakers from outside our little bubble, and I can understand the potential appeal of Ferris from a lifestyle or productivity standpoint. I think the nicest thing I can say about the talk was that it didn't work. The conversation format veered off into topics of dubious relevance, and worse, Ferris veered between low-energy and coming off as kind of arrogant. And not in a good way.

Robert Martin, on the other hand, was outstanding. I was just skimming through that talk again for the purposes of pulling out the best parts to show people here, and it's hard to limit it. Smart, funny, and inspiring in the sense of uplifting the daily programmer routine into something professional and profound.

Entrepreneurship

Side projects and entrepreneurship were less of an explicit theme this year than last.

I went to both Obie Fernandez' session on being a successful Rails consulting company as well as the panel on entrepreneurship prominently featuring him and DHH. I appreciated Obie's willingness to give out actual numbers, that was very useful, and the interaction between him and DHH on how much a dedicated business person is needed was interesting, but probably more reflective of those two specific personalities than general guidelines.

Related posts:

1. RailsConf 2009: The RailsConfening
2. RailsConf 2008: A Belated Look Back
3. Welcome To RailsConf