Rails Testing Frequently Asked Questions — The Non-Code Version

Noel Rappin, Friday, May 15, 2009 @ 11:50 am

Test-Taking Robot picture by slavin fpo

Between my sessions at RailsConf, and the book promotion at Ruby Learning I've been fielding a lot of questions about Test-Driven Development and Rails. Here are five of the most common general questions.

Self promotional note: many of these questions and more are covered in detail in Rails Test Prescriptions, only $9 dollars for a book that covers all kinds of testing topics...

What test tools should I use? Should I use RSpec?

This is, I think, an increasingly common question as the number of available Rails testing tools increases. Most of these tools were created to fill specific needs that were unserved by the Test::Unit and fixture tools provided as the default by Rails.

The important thing is not to get hung up on finding the absolute best test stack before you start -- for one thing, you won't know which tools are best for you until you get going. For another, the big frameworks don't have all that much functional difference any more.

If you are new to testing in Rails, I recommend you start with the default stack -- it's easiest to install, simplest to use, and has the most documentation. Eventually, you will discover the same limits that led people to move to new tools.

At that point, if you start hitting the limitations of fixtures, you can move to FactoryGirl or Machinist. If you start wanting groups of tests with the same setup, you can add Shoulda or Matchy. If you want mock objects, you have choices. You can also go to RSpec to get most of these features as well. But you'll get the most out of the extension tools if you come to them from a position of genuine need rather than a position of "all the cool kids are using this".

I have an entire app with no tests, what should I do now?

Start your TDD process on new features immediately. Don't worry so much about going back and fixing the rest of the program in one fell swoop -- it'll wind up being a huge time sink and the risk of adding errors is high. (This advice also applies if you have tests, but think you can do better now...)

Do start using TDD on new code. Also add tests to cover any reported bugs, this will help you build up coverage throughout the program. Use mock objects to keep the part of the code you are testing separate from the untested parts.

If you want to get some coverage on the legacy parts of the code, try using a more black-box tool like Rails integration tests or Cucumber to try and verify high-level features from the level of user interaction. Since these tests aren't as dependent on the structure of the code, they may be easier to manage.

Why should I test? Won't I still need human testers?

Test-Driven Development is not a substitute for human acceptance testing or review by a domain expert. Just because the tests pass is no proof that the tests are testing the right functionality.

However, TDD has a lot of benefits. Code written using TDD tends to be simpler, easier to manage, and easier to debug and maintain going forward. I develop much more quickly using tests then when I am not writing tests. This is in part, but only in part, because I execute the code in the browser less frequently. It's also because the TDD process focuses my activity efficiently. A program written with tests also has a lot of protection against introducing bugs or breakage in later development.

When should I test? Is there such a thing as too much testing?

You can always overdo a good thing, and it's certainly possible to spend time writing tests beyond the point where you are getting much benefit from it. I think that most of the code and developers I see are, to put it mildly, not near that point.

My feeling is that you should test any code whose correctness matters to you. Sometimes, the correctness doesn't matter -- you know it's a one-off script, or you are just experimenting, or who knows what. AddedSometimes it's genuinely the case that it's not worth the time to add a test for some weird corner case or something.

EDITDED SLIGHTLY
Remember, though, when you do write tests, it's much easier to write the tests first. Easier to write the tests, easier to write the code.

I don't know what to test. How do I start?

The great part about TDD is that it doesn't matter all that much where you start. So pick one specific thing about your program that you know it needs to do. You can start at the controller level: "When the activate controller method is called, the user is activated", or at the model "The user model updates when it's activate method is called" or at the user level "When the admin clicks the activate check box, then the associated user can log in". Some of those are higher-level then others, but all of those will lead you in the direction of the other ones as you move forward.

Things to keep in mind.

• No matter where you start, start with one simple test. One assertion, if you can manage it.
• Write the code to make that test pass
• Then refactor -- this part is critical, this is where the creative design is with a TDD process.
• Then write the next test.
• In Rails, you'll often start in the controller then realize that code needs to be written in the model (or vice-versa). Write a separate test for the model -- testing models from the controller makes it hard to test all the model logic.
• It's okay to plan the tests in advance, but you should only work on one test at a time. (Sometimes I'll write the series of tests, then comment out all but one)

Related posts:

1. The Testing Interviews
2. Rails Testing First Look: Blue Ridge
3. Ask A Rails Tester Person
4. Rails Test Prescriptions is now on sale
5. Rails Performance, Code Metrics, and Locking Down your Application: Tips & Tricks from Windy City Rails 2008

Comments (9)

Software Development

Comments: 9 so far

• Leave a comment
• Comment feed for this post
• TrackBack URL

1. 

   That’s a pretty good rundown. Excellent point about using tools because you need them, not just because they are the new hotness (not that it’s necessarily a bad idea to keep up with such things).

   I recently wrote up a detailed breakdown of my own TDD setup with explanations of why I use what; maybe someone will find it useful:
   http://code.isdangero.us/posts/My-test-driven-Ruby-setup

   The post includes a subheading “TATFT?” wherein I question the notion of 100% test coverage. I think, as you put it, it’s really a question left for each developer or team to consider on a case-by-case basis. I think the line is still being drawn on this matter, and the true answer certainly won’t lie in overzealousness nor in a lack of testing altogether.

   Comment by Chris Vincent, Friday, May 15, 2009 @ 12:50 pm

2. 

   I think we have to be careful to not start a “test for testing’s sake” movement. The cult of rails likes to rush to whatever kool aid du jour is being poured out, but sometimes kool aid can kill you.

   To be clear, I am a huge proponent of TDD/BDD and think that code should be tested, but I also think unless you know why you are testing you should ask answer that question before continuing.

   Comment by Derek Neighbors, Friday, May 15, 2009 @ 3:12 pm

3. 

   @Derek It’s interesting that often when I write about TDD, somebody will mention the cult of Rails testing.

   It’s interesting, because I’ve been doing TDD since about 2000, and had written xUnit style tests in at least six different programming languages before I even touched Rails. Okay, that sounds whiney, but the point still holds, my TDD advocacy only overlaps with my Rails advocacy in that Rails was designed to be a very friendly environment for testers.

   Comment by Noel Rappin, Friday, May 15, 2009 @ 4:06 pm

4. 

   @Chris, the test stack you list is pretty similar to the tools that I push for in the book (Machinist over Factory Girl, Mocha over the others for now, Shoulda over RSpec)…

   Comment by Noel Rappin, Friday, May 15, 2009 @ 4:07 pm

5. 

   Thanks, Noel, for answering my questions at RubyLearning. Your effort on the book, I belive, will give people a better understanding and skillset about TDD.

   Comment by Takaaki Kato, Saturday, May 16, 2009 @ 2:23 am

6. 

   I like your suggested progression of testing tools. This is similar to how we teach testing in thoughtbot’s Rails training program.

   * introduce Ruby
   * introduce Four-Phase Test
   * introduce Test::Unit
   * unit test a Ruby class with Test::Unit
   * Rails testing with Test::Unit
   * as needs expand, introduce Shoulda & Factory Girl

   In the advanced class, we get into mocking and different styles of testing (integration, isolation), and talk about RSPec’s separation of testing views & controllers as a reason you might then layer on RSpec on top of code you’ve already written. (Test::Unit & Shoulda code can run nicely inside RSpec)

   I’ve found progression is important for Rails developers new to testing. There’s already a ton of information thrown at Rails developers, so building up piece-by-piece lowers the barrier to entry.

   Comment by Dan Croak, Wednesday, May 20, 2009 @ 1:32 pm

7. 

   TATFT is thinking about testing all the time. Not actually doing it.

   Comment by bryanl, Friday, May 22, 2009 @ 1:23 pm

8. [...] http://www.pathf.com/blogs/2009/05/rails-testing-frequently-asked-questions-the-non-code-version/ : une FAQ sur les tests dans Rails [...]

   Pingback by Dev Blog AF83 » Blog Archive » Veille technologique : Conditions d’utilisation, Lecture, HTML5, Mozilla, Vim, Git, Ruby, Rails, Javascript, Microframeworks, PHP, Python, Bases de données, Tuesday, June 9, 2009 @ 5:03 am

9. [...] Rappin’s talk covered some of the most common questions he gets through the Pathfinder Development Blog, and his own site RailsPrescriptions site [...]

   Pingback by Agile Ajax » ChicagoRuby meeting ‘Test Prescriptions’ recap » Pathfinder Development, Wednesday, June 24, 2009 @ 12:52 pm

Leave a comment