ChicagoRuby meeting ‘Test Prescriptions’ recap
The ChicagoRuby users group (not to be confused with chirb.org another great Chicago Ruby user group) held their second meeting at their downtown location.
While the meetings out in Elmhurst are always informative and helpful, the downtown location may allow for a bigger crowd, and the weekday time might work better for more people. Plus, the Illinois Technology Association - Tech Nexus is right next to Union Station which works great for people that still have to go out to the suburbs.
Noel Rappin's talk covered some of the most common questions he gets through the Pathfinder Development Blog, and his own site RailsPrescriptions site RailsRX.com dedicated exclusivly to testing.
There's too much detail to cover here, but at a high-level, Noel covered:
- The why, how, when, and where questions that make the foundation of a good testing approach
- Testing techniques for legacy projects
- A good review of the different testing frameworks, what their differences are, and some discussion about each (test::unit, shoulda, rspec, cucumber)
- A solid review of different factory tools that go beyond standard fixtures (Factory Girl, Fixture Replacement, Machinist, and Object Daddy)
- Strategies for migrating from Fixtures
- Differences between the mocking frameworks: Flexmock, mocha, rspec, and RR (double ruby)
- Finding the right balance, and avoiding the pitfalls of 'over mocking'
- Cucumber testing workflow (and where cucumber doesn't work well)
The audience seemed to have a wide range of experience, but all had opinions to share about what they've learned about testing. After the talk everyone seemed fell into small groups to network and exchange ideas and contact info, and a group gathered around Ray and Noel as they tossed around some ideas on potential future meeting topics "Ruby IDE/Editor review", "Rails Jumpstart", "Coding Dojo".
The organizers took a stab at hosting the meeting virtually over gotomeeting (not sure if it was recorded or not).
Their next meeting is scheduled for July 18th, details can be found in their meetup.com group.
The organizers of the ChicagoRuby.com group are also the organizers of the 2nd annual WindyCityRails Conference right here in Chicago, on Sept, 12th, which Noel will be presenting at.
Topics: chicagoruby, chirb windycityrails, factory, fixturereplacement, fixtures, flexmock, mocking, rails, railsrx, rspec, ruby, shoulda, tdd, test::unit, Testing
The Return of the Cucumber
I mentioned last week that the RubyMine post was replacing what I had meant to write about. Well, this week we finally get to it...
It's been about ten weeks since I wrote about Cucumber the first time and the second time. Since then, I've continued to use Cucumber and now seemed like a good time to update some thoughts on how and why it seems to be working for us.
The big headline, of course, is that I'm still using it after ten weeks. I'm pretty quick to abandon tools that aren't pulling their weight, so just the fact that Cucumber is still in the toolbox means that despite the time that it takes to write Cucumber tests and step definitions, I'm finding the process of writing the tests and the tests themselves to be valuable.
Self-promotion alert: More details about Cucumber and anything relating to Rails testing can be found at Rails Test Prescriptions, there's a free getting started tutorial which contains an extensive section on Cucumber, and a nearly 300 pages and counting full book for $9. Thanks. Also, follow @railsrx on Twitter for a testing tip every weekday.
Topics: cucumber, Ruby on Rails, Testing
Avoid the last minute security review
Security is hard
Security is often an after thought, slated towards the end of a project, or after some big issue has been discovered, but the nature of security functionality, rules, roles, auditing, etc make it hard to layer in to an existing codebase effectively.
Oh, and if the code base isn't sufficiently tested, you're in for a world of hurt.
If you are a developer that was just asked to 'do a quick security check and plug any holes', you are now in the difficult position of managing the expectation that a two-week security review means "we're covered". Be realistic about what you can accomplish, setting out some short-term and long-term goals.
Do More With Less. Start with a research 'Spike'
Instead of pushing for more time to be able to 'cover it all' (even though you have no idea what 'it all' is yet), it might be better to start with a shorter analysis phase, where you detail your findings, identify any trends, and include recommendations for process change. I've found that even the most demanding manager is appreciative and understanding when you ask for a small amount of time in order to produce a better estimate, than to just immediately demand more time without any evidence as to why.
Plan for success
With your analysis and recommendations in hand,
Continue reading »
Can your Selenium do that? Testing flash/flex and silverlight in web apps with iMacros
Having learned a long time ago the value of automated testing tools like Selenium, jMeter, and soapUI, I'm always on the lookout for new improvements in these tools. While I love Selenium and other frameworks like it, it has the limitation of not being able to test Flash/Flex/Silverlight or Java Applets. But if you need to test flash and silverlight components of your web app, in an automated way, the iMacros testing tool might be worth checking out.
No Free Ride
While the free version of the iMacros plugins for InternetExplorer and Firefox allow powerful web scripting similar to Selenium, to be able to do the flash/flex and silverlight, you have to get the paid version or the 30-day trial. I downloaded the trial version to see how it compares to Selenium and what kind of damage I could to.
Going through some of the online demos, Continue reading »
Unit Testing: Can You Afford Not To?
50 Cent's unit test based savings
Unit testing is fairly common these days as more and more people understand the benefits; however, you may still run into a client or manager who is not convinced. These people tend to think unit testing increases development time and cost, or they feel that unit tests are redundant since a QA department is also testing the application. If you've ever heard "we don't have time for unit tests", "you're a developer, leave testing to QA", or "I'm not paying you to write tests" then you know what I'm talking about.
These encounters can be quite frustrating for a developer or project manager who wants to leverage unit testing. While there might be a temptation to just unit test anyway without the blessing of a client or manager, it is more useful to help these people understand why they are mistaken about unit tests. I have found that this misunderstanding is often because they don't realize that testing is actually a cost saver instead of just an additional cost.
But how to vocalize this point in a way they will understand?
Topics: Testing
Again With The Cucumbers
My cucumber obsession continues unabated. I've spent a lot of the last week adding Cucumber to projects, writing about it in a very long section for the Rails Test Prescriptions book, and generally trying to figure out how to use this tool, not to mention trying to figure out why this tool has been so interesting to me this week.
Some follow-up thoughts since last week's post.
Continue reading »
Topics: Ruby on Rails, tdd, Test Driven Development, Testing
Using Cucumber for Acceptance Testing
An consistent nuisance problem when testing Rails applications is the "unit test gap". This happens when the model test passes and the controller test passes, but the application as a whole fails because there's a mismatch between the output produced by the model and the input given to the controller test. In theory, Rails integration tests can solve this problem, but they aren't really designed for it, and nobody uses them much anyway.
An easy-to-use tool that solves that end-to-end testing would be great. Somebody should really write a blog post about that.
So, as I was saying. We continually run into an issue with our clients over defining requirements at the level that developers need to keep going, without getting bogged down in long stretches of design. There's a big gap between "users should be able to upload photos to their pages" and all the different details of permissions, validation and the like than need to be answered at some point. The Agile process suggests that those requirements be created as closely as possible to the code, which leads to the question of how best to keep the customer in the loop while the developers need these decisions to be made.
An easy-to-use tool that lets users read or create requirements that developers can build from and run as acceptance tests would be great. Somebody should definitely write a blog post about that.
Which brings us -- finally -- to Cucumber, a tool for creating automated acceptance tests. It's flexible enough to solve both problems. It can be used as a developer tool to drive regular TDD testing, and as a client tool for managing requirements.
Self promotion: If you want to read more about Cucumber and all kinds of Ruby testing, buy the beta version of Rails Test Prescriptions. Cucumber isn't in the book yet, but will be in the next release, schedule for the end of March. Buy now, and get all future updates free of charge.
Topics: Ruby on Rails, tdd, Test Driven Development, Testing
Rails Test Prescriptions is now on sale
Rails Test Prescriptions: Keeping your Application healthy is now on sale. This is a beta, partial release.
Rails Test Prescriptions is a comprehensive guide to testing your Rails application, covering both the mechanics of writing tests and the style for writing good and useful tests.
I'm excited to be publishing this electronically in a way that allows me to keep the book up to date as Rails changes over the upcoming months and years. Your $9 purchase entitles you to all updated versions for the life of the book.
Continue reading »
Topics: Ruby on Rails, tdd, Test Driven Development, Testing
Notification Testing with PureMVC and FlexUnit
One of the nice extensions available for FlexUnit is the ability to easily unit test event behavior using EventfulTestCase.
This library has support for testing Cairngorm, but unfortunately support is lacking for PureMVC. I searched for existing solutions which added PureMVC notification support, but was not satisfied with their APIs. Luckily, it was not difficult to write some code to support PureMVC which mirrored the API of EventfulTestCase.
Topics: actionscript, Flex, flexunit, PureMVC, Testing
Agile Development for Product Managers: Why Agile Testing Rocks
All software has bugs. I don't care if you're Apple, Microsoft, IBM, or a smaller, leaner ISV. Your software has bugs in it. Once you accept this fact, that into each software product a little crap must fall, it becomes clear that what differentiates one software development organization from another is how they manage those bugs. What do you do to prevent them in the first place, find them, fix them, measure them, celebrate their squashing?
Most of the developers at Pathfinder Development are old hands. We've been around the block a few times. We've worked on quite a number of software projects over the years. We've all gravitated toward agile development not because it's the latest buzzword, or because it feels so good (though it does when it's done right), but because everything else feels so bad.
Take testing, for example. Testing girds you, the product manager, for the moment of truth -- deploying or shipping a new release. In the monolithic or waterfal processes, we would often wait until the end, after all of the development was done to "QA" the application. Often you'd throw in some performance testing to see if this monstrosity you'd just built would actually handle the load you were going to throw at it.
After weeks or months of testing, fixing, hoping, praying and integrating, you'd finally deploy. Odds are you'd ship with some pretty serious bugs. But the quality of the software, in terms of bugs and errors, was pretty poor and the expense and effort of the developers fixing the bugs was akin to the contractor applying spackle to a drywall that was more hole than whole.
Let me go through some of the testing we do today, what the previous alternative was, and what the benefit for your product is.
Again With The Test Driven Development
So, if you're lucky, you've mostly missed another Internet pile-up about unit tests and so on, this one started by Joel Spolsky and some comments he made on his podcast.
I've already written a little bit about what Joel said, and I don't want to rehash that. But I have seen a recurring theme in arguments that testing is overrated -- namely that the testing that those people are writing about doesn't seem to be related to the testing that I actually do.
I'm going to put the summary up front:
- "Unit Testing" and "Test Driven Development" are not identical concepts
- It would seem, on the available evidence, that those of use who are TDD or BDD backers haven't done a great job of explaining this.
- TDD has code quality benefits that are completely separate from whatever function it has as a code verification engine.
- It would seem, on the available evidence, that those of use who are TDD or BDD backers have done a really poor job of explaining this.
Topics: Ruby on Rails, tdd, Test Driven Development, Testing
Review of fixture_replacement2 plugin - Update
UPDATE:
So, after working on my project for months, and getting other developers involved, I need to change my mind. I still like fixture replacement, but most of my tests require such an elaborate data set, that my tests are PAINFULLY slow. What was I thinking back in November? I guess I wasn't focused due to the Obamathon that was running through the nation at the time.
My application is an internal tool we use for staffing our projects, and is going to manage time entries and invoicing. There are many other features in the pipeline, but I'm making sure that we keep our eye on the ball, and releasing the most important features first. The problems I run into, are that most of my model methods are doing complicated queries on a data set that has a lot of possible values. To test some of my reports, I need to test the cases where there are people on various projects, vacations, holidays, out of office, etc. To ensure that the method responds correctly with all of these possible values, my data set is complicated. I have a number of test helper methods that set up some or all of my data set depending on what I am testing. This has effectively replicated fixtures, and I have lost my speed.
What are my issues:
1. Fixtures are wicked fast. Fixture replacement is not when you have large data sets. Why? Fixtures essentially load the data in the tables, then start a transaction before each test and rolls it back after each test. Fixture replacement does not get this benefit, as all data is loaded after the transaction is started. So my tests take forever....
2. Because I need complicated data sets that are reusable, My test helpers set up the data in stages, and I only load what I need. One of the problems I had with fixtures, was I always had to go the fixture files to see what data was loaded. When using fixture replacement, my test data is normally defined in a setup context of at the beginning of each test, so i can see exactly what data is on the model I'm testing. Due to my test helper setup methods, I now am constantly referring to them instead of the fixture files.
My plan:
As my project takes a few minutes to run the tests, I need to fix this, as my application will begin to be used by the entire company for time tracking, and the features will need to roll out quickly. I can't have any developer waiting minutes before each check in. This will only lead to test mutiny (checking in without running rake). So, my plan is:
1. Create a new data set in fixtures that includes an updated view of my projects (the current one is stuck in July of last year). Use mocks to mock out Date.today. I was setting an effective date as a member variable in my tests, but I like the idea of mocking it out better. I will also create some sort of visual depiction of my data set, as I am constantly trying to figure out what the application should generate, as I created my data set months ago, and generally don't have any idea of what it looks like when I am adding new features.
2. Slowly convert my tests to use my new data set.
3. Kill my old fixture replacement setup helpers
4. Only use fixture replacement for small unit tests. Most of my simple model methods can use a very simple setup with fixture replacement. The model methods that are crazy complicated can use the fixtures and date mocks.
So - I still like fixture replacement, but It is not good for large data sets. I think fixtures are still useful and their speed only sweetens the deal.....
Original blog post:
We have been using fixture_replacement2 on some of our Ruby on Rails projects. I am writing an internal application and finally switched over from using fixtures. It took a bit of time to get the tests that I had written modified to stop using the fixture data, but now I am completely fixture free. The tests are a lot easier to write, and I created a few helper methods that setup common datasets. This way, I can call a setup method to configure my data, and then - it's all good. At first I called the method test_data_blah, and was wondering why I was getting an error running any test, so now it's called data_blah. Doh!
At first I didn't like the fact that I needed to explicitly create the data for what I was working on, but then I realized that (especially for unit tests) you don't need much dependent data; usually it's one or two records. For functional tests, I found that the data setup methods rocked, as I could quickly get the data set up that I needed.I could quickly assert the content of the response, based on the data in my setup. I missed fixtures at first, but now realize that over time, they are a pain.
Topics: fixture replacement, fixtures, plugin, plugins, Ruby on Rails, Testing
More Testing Interviews
Just a quick note that the next two testing interviews are up on the Rails Prescriptions Blog:
Geoffrey Grosenbach of PeepCode and Topfunky.
Gregg Pollack of Rails Envy, the Rails Envy Podcast and Rails Activist fame.
Enjoy!
Topics: Ruby on Rails, tdd, Test Driven Development, Testing
OCMock: Handling Boolean Return Types with NSValue
Testing your iPhone application, you may at some point in time need to stub a method call whose return type is a boolean (or some scalar value, or C based structure). The way to implement this (as described in a discussion on CocoaDev here) is to wrap the boolean in an NSValue using the OCMOCK_VALUE() macro defined in OCMockRecorder.h.
While this solution technically works, I find that doing this within the test class itself makes the test cases a little less readable along the way. Instead, I found it useful to extend OCMock to do this type of conversion for me, keeping my test code a bit cleaner as a result.
Let's take a look at an example by examining the set up for a simple test case..
Topics: iPhone, mock objects, obj-c, ocmock, Testing
The Testing Interviews
Over at the Rails Prescriptions blog, I'm going to be posting a series of interviews with various and sundry Rails folks -- I'm cross-posting the first one here. To get the series started off, I've decided to use myself as a guinea pig and see if I can hold up to the grilling.
A couple of quick notes before I get to it.
- The free e-book Getting Started With Rails Testing is now available. Check it out.
- Those of you lucky enough to be in the sub-zero Chicago area can come to theChicago Ruby.org meetup Saturday, January 17th to hear lots of interesting stuff, including me talking about testing and Rails some more.
And on with the interview.
Continue reading »
Topics: Ruby on Rails, tdd, Test Driven Development, Testing
About Pathfinder
Follow the Blog
-
Get a monthly update on best practices for delivering successful software.
Subscribe via email
Subscribe via RSS
Categories
Topics
.NET
.NET Browser Control
2d physics
3d
3D GPS
3D physics
37signals
Acceptance Tests
Accessibility
ActionMailer
actionscript
activerecord
acts_as_ferret
Add new tag
Adium
ADO.NET Entity Framework
Adobe
Adobe AIR
adobe flex
Adobe Illustrator
Advertising
aggregation
agile
Agile Development
agile thinking
AIR
Ajax
Ajax Applications
Ajax Bookmarking
Ajax Components
Ajax Development
Ajax Examples
Ajax Experience
Ajax Frameworks
Ajax history management
Ajax Intervention
Ajax libraries
Ajax library
AJAX Obfuscation
Ajax Performance
Ajax Products
Ajax toolkit
Ajax Tools
Ajax Widgets
A list apart
Amazon
Amazon CDN
Amazon Web Services
amf
Analysis
Android
animation
Announcement
Announcements
antennae
Antipatterns
Apache
Apollo
apple
Application Architecture
Application Development
architecture
AS3
ask a UI guy
ASP.NET
ASP.NET
asterisk
Asynchronous Processing
authorization
awards
axiis
Azure
Back Button
backups
bandwidth
bandwidth profiling
Beans
beer
Benchmarking
Best Practices
BitmapData.draw
BJAX
blackberry
Blaze Advisor
blender
blog
blogging
book review
Books
braindump
browser
Browsers
Bugs
Business
Business Reasons for Ajax
Business Rules
C#
caching
campfire
Canvas
capistrano
Case Studies
CFO
Charles
checklist
chess
Chesspresso
Chicago
chicagoruby
chirb windycityrails
CIO
Cloud Computing
CloudFront
CMS
COBOL
Cocoa
code
code art
Code Generation
code generator
Color
COMET
Conference
Confluence
Consistency
Content Management
continuous integration
converget appliances
core animation
CRM
cruise
CruiseControl
CSS
cucumber
Custom Application Development
Custom Flex Component
Data Mapper
data visualization
Degrafa
deployment
deprec
Design
Design Patterns
design thinking
Desktop
Desktop RIA
Developer's Notebook
development
DHTML
Diagnose
Dojo
Domain Knowledge
don norman
Drools
drupal
dynamic languages
ease of use
EC2
Echo2
Echo3
Editorial
Entrepreneur
erb
ERP
Estimating
estimation
Ethnographic Research
events
everyblock
Excel
externalinterface
Ext JS
Extreme Programming
eye tracking
Facebook
factory
Feedback Loop
ferret
FileReference
Firebug
Firefox
Firefox Extensions
fixturereplacement
fixture replacement
fixtures
Flare
Flash
flash awards
Flash Platform
flash player
flash player 10
Flash Player optimization
Flash Remoting
Flex
Flex3
flex code generator
flex css
flexmock
Flex optimization
flex skins
flexunit
Flickr
Flock
Flow
Fluent
forms
Frameworks
FriendFeed
front end
front end development
fulltext search
functional
Games
Gauge Component
gem
getting things done
Git
github
gitignore
Golf
Google
Google Analytics
Google Analytics for Flash
Google Analytics for Flex
google android
Google calendar
google docs
Google Gadgets
Google Gears
google maps
GORM
government
g phone
Grails
Graphics
Greasemonkey
Groovy
GStreamer
GTD
Gwittir
GWT
h.264
haml
hardware
Healthcare
heuristic evaluation
Hibernate
hosting
HTML
Hudson
IBM
IDE
Ideation
IE
IE6
IE7
IE8
iGoogle
illustrator cs3
ILog
ILOG JRules
imacros
importing graphics to flex
Information Architecture
infrastructure
Innovation
Instructional Design
Interaction Design
interaction patterns design
Internship
Interview
iPhone
iPhone SDK
iPod
irb
iteration
IT Mill Toolkit
iTunes
jakob nielson
Java
javafx
Javascript
JavaScript frameworks
Javascript Libraries
JBoss Rules
Jess
Jetty
JIT
jmeter
Jobs
jQuery
JSF
JSON
JSP
JSR-94
JsUnit
laptop
Lazlo
Legacy Systems
lightweight
LinkedIn
LINQ
logging
Logical Model and Conceptual Model
Low Pro
Mac
Malware
mapping
Mash Note
Mashups
math
Meebo
metal
metaprogramming
MetaWidget
Methodology
Microformats
microsite
Microsoft
migrations
minimalism
Mobile
mobile platform
mocking
mock objects
modeling
mod_rails
monitoring
Mootools
mouse
mouse scroll
mouse wheel
Mozilla
Music
MVC
MySql
natural key
neal ford
NetNewsWire
networking
news
newspapers
nfjs
NHibernate
nokia
notebook
NSURLProtocol
obj-c
Object-Oriented
Objective-C
Object Relation Mapping (ORM)
ocmock
Office
OmniGraffle
online spreadsheets
OOAD
OOP
opengl
Open Screen
OpenSocial
Open Source
opensource
Opera
Oracle
ORM
osx
OS X
pagination
Pair Programming
palm
papervision3d
Pathfinder Development
Patterns
Peer Creation
Performance
Personas
PGN
PHP
Phusion Passenger
physics
physics engines
planning
plugin
plugins
portableapps
pragmatic
Predictions
preloader
primary key
process Web/Tech
Product Definition
Production Support
productive programer
productivity
product launch
Progressive Enhancement
project concept
project management
Project Website
Prototype
Prototyping
PureMVC
PV3D
pyro
QA
qooxdoo
Radiant CMS
rails
railscasts
Rails Environment Tests
railsrx
Really Simple History
Refactor
refactoring
References
regex
regular expressions
Requirements
Requirements
Alice Toth
Requirements Visualization
resesign
Restlet
RETE
Review
rfp
ria
Rich Interactions
rich internet applicaiton
rich internet applications
ROI
rspec
ruby
rubyamf
Ruby on Rails
Ruby on Rails testing role
S3
SaaS
Safari
San Francisco
Scalability
Scenarios
Scriptaculous
Scrum
SDLC
Search
Secretariat
Security
Selenium
SeleniumIDE
Semantic web
SEO
Server Side
shoulda
Silverlight
simplicity
skins
SOA
soapUI
Social Networking
software develoment
Software Development
Software Engineering
Software Processes
Songbird
SpiderMonkey
Sprajax
Spreadsheets
StageScaleMode
Standards
standish
starting projects
Startups
static typing
Stencils
STI
Story Telling
Structured Design
Struts
sun
surrogate key
Swing
tabs
tag
taglib
Tamarin
Tank Engine
Task Flows
tdd
teams
telephony
Tellurium
test::unit
Test Driven Development
Testing
tether
textmate
The Ajax Experience
throttling
Tilt Component
Tools
touch screen kiosk
TraceMonkey
Training
Trends
Tumblr
Tutorial
Tutorials
Twitter
ubuntu
UI
UIViewController
uml
unit testing
Unit Tests
unity3d
Usability
Usability Testing
user driven agile
User Experience
user experience design
user groups
user interface
User Interface Standards
User Research
UXD
value
Venture Capital
Video
Vision
visual analytics
visual design
visual documentation
Visualization
VLC
VML
Volta
waterfall
watij
watir
web
Web/Tech
Web 2.0
web app
Web Design
Web Development
web forms
web hosting
web infrastructure
Webkit
Weblogs
Web Services
WebSockets
Web Standards
WebTest
Widgets
will_paginate
Windows
windows mobile
WinForms
Wireframes
WordPress
workflow
work life balance
xcode
XML
XML Metadata
xp
XUL
Yahoo Map AS3 API
YUI
Zeigarnik
Zeigarnik Effect
ZendAMF
ZK
Archives
- July 2009
- June 2009
- May 2009
- April 2009
- March 2009
- February 2009
- January 2009
- December 2008
- November 2008
- October 2008
- September 2008
- August 2008
- July 2008
- June 2008
- May 2008
- April 2008
- March 2008
- February 2008
- January 2008
- December 2007
- November 2007
- October 2007
- September 2007
- August 2007
- July 2007
- June 2007
- May 2007
- April 2007
- March 2007
- February 2007
- January 2007
- December 2006
- November 2006
- October 2006
- September 2006
- August 2006
- July 2006
- June 2006
- May 2006
- April 2006
- March 2006
Blogroll
Recent
- Elements of Testing Style
- Aesthetics and Web Design
- Asterisk-Java Testing with Groovy
- 3 Misuses of Code Comments
- Fluently NHibernate
- Digging a Hole and Covering it with Leaves — The Software Development Version
- The Importance of User Experience - Do You Understand It in Your Bones?
- Writing Your Own Protocol With NSURLProtocol
- What’s In Your Dock: iPhone edition
- Feature Fatigue