photo credit: victoriapeckham

We have reached the most critical point on a project I'm working on. After a few months we think we know enough about the domain and application to build a product road map that will take us to the first public release. The proof of concept is complete. The design team has created a remarkable, genera changing product. Additionally, the system is designed around real users we have been able to talk to and get feedback from. We have put together an unbelievably good development team and built a backlog of stories with estimates. We have been here before. Putting together a design and backlog of stories is something we have done countless times...

The easy part is over. Now the hard part begins.

Our research and user feedback tells us we have multiple potentialcustomer groups we can build the system for. On one hand this is great news. We have a number of potential markets to choose from. On the other, we don't have an infinite amount of time and money to build it for all of these groups. We have to commit and go all in with one group. Right now, these are just some of the questions we are asking ourselves now:

• What customer group do we value the most?
• What features do they value the most?
• How expensive is it to build the ultimate product for each group?
• What is the minimum viable product we can build for each group?
• Which group is most likely to give feedback and partner with us to help refine our product?
• How much feedback is this group likely to give you?
• Are we missing some market window by passing on one group v.s. another?

This is a critical point in the product's design. Whichever user group we choose will be our customers. Or another way of saying it: They will be our ONLY customers. Other customer groups aren't likely to be interested because we aren't building any features for them yet.

When designing a product do you consider what customer groups you are including and excluding? Are you going to be happy with that choice for the foreseeable future?

Related posts:

1. Your SDLC or Your Product – You Decide
2. User Driven Product Development
3. “Build half a product, not a half-assed product” – tips on clarity and focus from Jason Fried of 37Signals

photo credit: moonlightbulb

I started my first real Agile software development project in 1999. I'd been doing more traditional software development before then all the way back to 1980. I won't bore you with the details of those earlier projects, but my feeling was that there had to be a better way of developing software that didn't involve a senior technologist (me) telling a whole bunch of junior technologists what to do. It turns out I was right.

I started my first real Agile software development project in 1999. I'd been doing more traditional software development before then all the way back to 1980. I won't bore you with the details of those earlier projects, but my feeling was that there had to be a better way of developing software that didn't involve a senior technologist (me) telling a whole bunch of junior technologists what to do. It turns out I was right.

But almost from the start I got pushback from other people in the development organizations I worked in that Agile development was horribly wasteful. They pointed to Test Driven Development ("all those tests more than double your effort"), pair programming ("two developers doing the work of one?"), and refactoring ("you're rewriting the software every time at enormous cost"). Of course all of these objections were born not just out of a misunderstanding of Agile development, but a fundamental misunderstanding of how their own software development processes actually worked.

The issue of refactoring was particularly mysterious to my colleagues. If you took the time and designed software properly up front, they reasoned, you won't have to do expensive rewrites. Of course anyone who has ever maintained code knows that this isn't ever true. All code, regardless of how it is developed, changes over time for bug fixes and to meet the needs of new requirements. If you aren't thoughtful about how you change your software, you can easily end up with a big mess. Further, very little software is design with a perfect understanding of the requirements, leading to imperfect designs and things like the second and third system effects.

If you are thoughtful about how you change your software and think about the sorts of "code smells" that crop up in your code over time, you end up making more fundamental changes, rather than adding a method here or a instance variable there. You might see that two methods are always called in conjunction and decide they should be folded into one method (a design win, see State Patterns), or you may have extended your design and ended up with two parallel hierarchies of classes. You break the Go4 glass and pull out the Bridge Pattern to solve that burgeoning design problem. Through constant refactoring (or rewriting, take your pick) you avoid painting yourself into a corner as your requirements change.

The fact is that good developers are constantly redesigning and rewriting their code. Those same developers will tell you that the longer you leave problems (technical debt) before refactoring, the bigger and more expensive they become, until you might be better off just scrapping the system and rewriting from scratch.

Agile development takes this normal refactoring to it's logical extreme. Rather than doing a big design up front and then doing a series of expensive changes when they are inevitably found to be wrong, Agile teams accept that their design will be imperfect and then build comparatively inexpensive refactoring into every iteration. It may seem counterintuitive, but all that refactoring results in code that is cheaper to develop, easier to maintain (because it's easier to understand, change and debug).

Among professional writers the golden rule is that the key to all good writing is rewriting. The same is certainly true of software.

Related posts:

1. GWT and the Static Versus Dynamic Religious War
2. Has Many has_many: A Refactoring Story
3. Requirements Set in Stone and Software Made of Concrete

photo credit: Rich B-S

Gwarred Mountain over at Climax Studios has posted a very thoughtful blog post about software development methods and the appropriateness of Agile Software Development. I was ready not to like this article, what with the title and things like this:

If I have to sit through another meeting with some little "agile" toe-rag defending their train wreck of a project then I may end up forcibly ramming a kanban where the scrum does not shine.

But then I thought about all of those fresh-faced management consultants we've run into recently -- who have read a book about agile -- trying to teach us how to do it. Well, yes. I've had some uncharitable thoughts myself.

Surprisingly, I felt myself agreeing with most of what he wrote. No, Agile is no panacea. Yes, it works best on small to medium sized projects. Yes, you have to have an experienced team to do it well. Yes, there are other methods that are more appropriate in some cases and, yes, you have to recognize when to use something else than an agile approach.

His breakdown of people vs process is one of the many nicely coherent gems in the post:

So what is so great about process? Well, it gives us:

• Repeatable and predictable results
• Quality Assurances (through the above)
• Cost savings through the ability to optimise work flows
• Defined work flow allows us to use cheaper labour
• The promotion of best practices and conceptual integrity
• The ability to scale to large numbers
• A means to effectively track our progress against the objectives

McDonalds is a great example of successful process. No matter where you are in the world, you know what you are going to get and you get it quickly and cheaply. This process has successfully scaled to thousands of restaurants. Whether you consider this good or bad it is hard to argue with the results.

Nevertheless, software development is much harder than frying beef burgers. Process is sometimes inappropriate or unconstructive.

I'd say the lesson from his post is that you need to be thoughtful about the software development and project management technique you adopt. Know why you are doing things. Any process that has magical rituals rather than purposeful activities has a good chance of devolving into a software development farce.

There is one thing with which I take issue. He states that Test Driven Development (TDD) adds from 15%-35% to development effort. He cites an empirical Microsoft Research study. The paper, authored by Nagappan, et. al., is one that I am well familiar with. The study looks at four case studies, one at IBM and three at Microsoft, where two teams are given the same system to implement. Everything is the same except in one team incorporates TDD into their process. Neither team is told they are part of a study.

It isn't clear from the paper, but in my experience teams that adopt a new technique or tool usually are not as efficient the first time around. Certainly that is born out by our own empirical observations -- developers that are performing TDD for the first time are anywhere from 10%-30% less productive. That starts to move toward 0% after the first project. It becomes second nature. And as the study itself makes clear, the productivity losses in the study would be more than made up for in the maintenance phase of the software life-cycle, where 80% of software product costs land anyway.

Let's see, lower defect rates, no productivity loss after a little bit of developer training and reduced cost in the maintenance phase? Sounds like a no-brainer to me. Of course I don't know Mr. Mountain's industry. It may be that there is no maintenance in the game development space, just code and release. No updates. So rush to release, bugs be damned may be the best way to go (not being facetious here).

The fact that this study comes out of Microsoft already sets some alarm bells ringing for me. Although we're headquartered in Chicago now, we actually were founded in Seattle, physically and intellectually not far from the Microsoft campus. Based on what we saw working at and for Microsoft, we soon began a physical and intellectual journey away from there that has landed us in Chicago as a premier User Driven Software Product Development shop. We use experienced teams, User Experience Design (UXD) and are thoughtful about how and why we practice Agile.

We may not be everyone's cup of tea, but if you have a consumer or business software product or service that you need to get out to the marketplace swiftly, reliably and with high quality, then we may well be a fit for you.

Related posts:

1. Agile 2009: A reminder of why each team needs leadership
2. Agile Development and Play: Understanding the Value
3. Building a High Performance Agile Team: Assume You Will Be a One Hit Wonder

photo credit: Hugo90

This morning I sat through two pitches by two startups looking for funding. I won't get into the details, but they both had clever ideas at their root. But while one company was attractive and poised for success, the other was mediocre and not getting much traction. Why was that? They both had clever ideas, no?

Over the years I've looked at a lot of business plans for Venture Funds. The first lesson that I learned was that cool ideas didn't equal successful companies. While I would get all hot and bothered by a particularly elegant software solution, the VC's I was consulting to preferred the plans that understood the market and the customers in it (and had a kick ass management team, natch).

The "good" company from this morning had a clever idea that was a clear solution to a customer problem. The mediocre company had a clever idea that didn't really solve a specific customer problem.  What accounted for this difference? Well, the "good" company spun out of a larger business, while the mediocre came out of a government funded not-for-profit.

The easy conclusion here is that business folks have the discipline of competition working for them, while NFP's don't. It's true that I've seen companies that have no valid competitors flounder when it comes to a business plan, but that's a disease that's not restricted the the NFP realm.

No, instead this seemed to be a different ivory tower problem: lack of a customer. As a result their product definition was crap. Who would want this thing? Always start with the customer.

If your startup idea includes a software component, you should make sure that your software development practice also keeps the customer front and center. It may be tempting to start with architecture, to see if your software solution can "scale." But if you get the customer wrong, you won't need to scale.

That why with most software products that we develop, we start with user modeling. We do move on, eventually, to data modeling, process modeling, etc., etc., etc., but only after we've set the tone, so to speak, with user modeling -- personas, user stories, task flows, etc. Everything that follows is then motivated by the customer.

So, before you spend a million dollars on that cool idea, validate it with some customers to see if they actually want it.

Related posts:

1. Upcoming Talk at RIApalooza: Fast. Smart. Agile. User Experience Driven Agile Development
2. Web 2.0 Expo: Behavior-Driven Development with Rails and RSpec
3. Gmail, agile development and user experience design

Once you have sufficient experience managing projects with Agile practices you should feel comfortable adapting those practices to your own teams and projects. As long as you are still following Agile principals this is okay. In general, this is what’s going on in smaller companies. Having coached a number of large organizations transitioning to Agile I can say this isn’t how they look at things. The problems start when you adapt the practices, but try to deliver all projects in an identical manner. This makes sense for waterfall-like delivery methods. But, when an organization comes up with its own version of “Agile” it can only work for the subset of projects it is tested on. Rolling this out to the entire organization as “the way” to deliver projects from them on is a failure pattern. The principals are lost and so is the adaptability of the organization. The time spent to move over to agile is immediately wasted.

Photo Credit:
kevindooley
under a Creative Commons Attribution License

Related posts:

1. Agile 2009: A reminder of why each team needs leadership
2. Building a High Performance Agile Team: Assume You Will Be a One Hit Wonder
3. Effective vs Efficient Teams

If you walk away with one concept after reading this book it should be to believe that success comes from people. The best companies focus on developing problem solving skills and local decision making. These companies favor adaptability over efficiency.  These companies make money to survive rather than simply surviving to make money.

The book starts out by defining the concept of frames, “the unspoken mental constructs that shape our perspectives and control our behavior in ways we rarely notice.”  A useful way to look at software development. I was happy to see their description of Agile as evolutionary rather than revolutionary. This is why when you explain the set of Agile practices to those with extensive experience they usually nod their heads and say they are already doing them. I have been telling people that Agile is a collection of best practices that good software shops have been using for years. Now I have a reference to support this.

Software craftsmen should read chapter 2 about technical excellence. The chapter goes through each engineering practices, explains it such that a non-practitioner can understand and gives examples. After they are done they should make their manager’s read it, then their managers.

Conclusion

I wish this book had been written 5 years ago. It would have saved me considerable effort and time trying to define what a well run software organization looks like. Your mileage my vary, but I will say that I intend to keep this on my bookshelf right next to Managing The Professional Service Firm.

Related posts:

1. Software Development and Wasted Motion
2. What’s the value of Agile out of the box?
3. Agile 2009: A reminder of why each team needs leadership

photo credit: LKaestner

Back in college I used to work at the main campus computer center. That was back in the days of time sharing Unix and mainframe computers, so it was a great way to get some extra CPU time for my various projects. Besides locking up at closing time (I loved to work the 10pm-3am shift), I had to change toner cartridges, support the brick macs, windows 3.1 machines, and the various Unix and Mainframe terminal users.

Over time, you learned to identify various user types. The vast majority of users had simple "hey, how do I do this" or "this ain't working" types of issues. Of the rest there were two main problem user types: the "helpless bunny" and the "walking crisis."

The Bunny displayed profound helplessness in order to get you to do their work for them. The first time I had a Bunny, I was caught unawares and spent the better part of my shift formatting their document in Wordperfect. With subsequent Bunnies, I used tough love and gave them manuals and man pages to read.

The Walking Crisis was typified by the grad student who waited until the last moment to add end notes to their masters dissertation, and dammit, if they were going to suffer, you were going to suffer. Their technique was to try to convince you that their problem was your problem, i.e. the fact that they couldn't do two months of work in five hours was a shortcoming of the software, and thus a support issue.

The more enthusiastic you were about helping users, the more susceptible you were of being dragged down by the Bunny and the Crisis.

Professional Software Development

When I left school for the big time of professional software development, I quickly found the Walking Crisis existed there too. When these problem types were in position of influence or even the project stakeholder, then they didn't just make the life of one person miserable, they could whipsaw and demoralize an entire team. And while they were poison to a team, by themselves they were capable of pulling heroic all-nighters to make up for their lack of discipline. This perceived capacity for hard work (minus the self-inflicted need for it), often puts the Walking Crisis into positions of power.

It was around this time that I became a fan of professional project managers. If you were lucky enough to get one of these rare beasts on one of your projects, you at least had a chance of surviving. If the Walking Crisis procrastinated decisions, a good project manager could ring the alarm bells in advance of any software death march.

Agile is not Undisciplined

When I began to embrace the joys of Agile Software Development, the Walking Crisis made a strong comeback. Since Agile software development methods embrace change, the Crisis can delay all the hard decisions and work until the last few iterations and slip them in as some basic change.

Of course the same things could happen in a more traditional waterfall model of software development, but somehow lots of folks have the idea that Agile is just fast and loose and undisciplined. Nothing could be further from the truth. In Agile you grapple early and often with schedule and scope. From the second or third iteration you are dancing with velocity and the backlog, making decisions on what is in or out.

Be Careful in Choosing Your Product Manager

If your organization is developing software, you may be in the position to choose the product manager or software development liaison. Pick someone you already know works well with teams and has a good grasp of your business and the marketplace. Be careful that you don't mistake their heroic "Walking Crisis" efforts for competence, else you may be condemning your software product to failure.

Related posts:

1. Agile Development for Product Managers: Why Agile Testing Rocks
2. Agile 2009: A reminder of why each team needs leadership
3. Agile Software Development and the Lazy Client Trap

For instance, I always have Action Script Language Reference, Wikipedia, some library API site(s), Gmail and a dozen other ones open + the debug version of the app at hand.

So what used to happen when you but a breakpoint in Flex Builder with all these tabs? They would be unavailable and any process happening inside of them could not be relied on. Since not all code runs well on first attempt, if the app crashed while testing ( think 3D, data intensive apps, etc.) the browser and all the tabs went down with it.

My solution so far was to use Firefox as a development browser and Safari ( since I'm Mac based ) as a browser for references and everything else. For crashing resolution, Firefox has a nice "Restore" option but it's not fun waiting for 15 tabs to reload.

So Google Chrome recently came out for Mac. It didn't impress me on Vista so I didn't care much. I guess I was in between of curious and bored so I decided to give it a spin.

What a pleasant surprise to see every tab running in a different process. My workflow feels so much better now that I'm not afraid that a bad line of code is going to take down my whole browser.

I've heard that IE8 also runs tabs as different processes but I'm not crazy about returning to development on Windows. I did try out Chrome on Windows 7 as a result of the Mac test and all the issues I've seen the first time around have been addressed. Kudos to Chrome development team.

Let's not forget to mention all the features that are missing on Google Chrome for Mac, primarily the lack of Bookmark Management, but Google Bookmarks or any online bookmarking service will do for now.

I can not wait to see more development being done on Google Chrome for Mac and it getting out of beta. I will not uninstall Firefox anytime soon but as a Flex developer I give Google Chrome for Mac high scores for beta.

Related posts:

1. Why Chrome OS is the Future of Netbooks
2. How much Excel can we get in Flex?
3. Using Adobe Flex Builder 3 on a Mac

It sounds great in theory: You produce requirements up front, getting them so detailed that they cannot be misunderstood, you vet them with a full committee of stakeholders who will be impacted (except your customers.) Then you give it to the developes. The result: you can get a really precise cost and timeline, you'll save on expensive development time, and your CFO will sleep soundly at night.

It's a nice fantasy, but it's wrong. It's a fundamental misinterpretation of how software development works. In fact, it's a guaranteed way of increasing your costs and producing bad software.

Here's why:
If you're developing software, you're coming up with a new solution to a problem, or solving a new problem all together. Otherwise, why are you building software, rather than buying it?

Traditional requirements definition processes are a spectacularly bad way of understanding stakeholder needs, and coming up with, conveying and vetting a solution for these types of problems. You will miss certain requirements, you will over specify others, and some of them will never get used. Your stakeholders will only be able to provide feedback of limited value until the working software is actually in their hands. At that point, you will finally start getting valuable feedback that you can push through as change orders.

When you're developing new software, you can't know everything up front. Software development projects are characterized by progressive learning. The further we progress in the design, specification and development of a software product, the more we understand the user’s needs, the technical implementation, the availability of new technologies, the competitive market, the responsiveness of third party vendors and a host of other factors.

In an agile process, you do just enough design up front to get a high level iteration plan and start the first development iteration. The developers interact with the designers, so they can advise on what kind of designs are difficult to implement and which are easier. This saves time and money, and can improve the user experience. Getting working software in front of stakeholders and end users early and often is a much more effective way of vetting requirements and eliciting feedback.

Related posts:

1. Agile Software Development and the Lazy Client Trap
2. Software Development and the Construction Analogy
3. Startups, Software and the Vision Thing

photo credit: tallkev

Last weekend I was watching a movie with my kids. In the movie there was a chain of monkeys that needed to pass on the message from one character to one on the other side of the chain. The message went something like, “Don’t throw us over the wall. There must be another way. We will all be killed.” As it went through the chain and the receiver heard, “Throw us over the wall. It’s the only way. Banana.” The scenario seems ridiculous, but its roughly equivalent to how many companies approach software product design. Often times companies don’t realize they are creating a product at all. They think they are just running a project and focus only on delivery of that project as if it is the only artifact of their work.

The problem stems from the fact that when organizations reach a sufficiently large size they must focus on consistency of delivery and efficiently using people’s time. For large organizations this is part of the mix that makes up their competitive advantage. However, the sheer size and number of moving parts required to enable clocklike consistent delivery leads to the most knowledgeable people about the customer never directly speak to the people responsible for building the product. Or translated into a traditional SDLC, the definition/high level design team isn’t communicating with the build team. In my experience they are usually two different groups of people. I’ll give you an example:

A while back, I was leading a software development team creating a product to be used by all 170,000 of my customer’s employees on a daily basis. They happened to have a team of user experience designers and wanted to take on the “big picture” part of the design themselves. This company could afford the best and the brightest talent - and was able to attract them. Individually the folks on this team were talented and knew their craft well. I actually learned a lot just from my brief time with them. However, once we got the design in hand it was obvious that the usability team’s artifacts weren’t going to work for the project. They didn’t meet the end user’s needs nor were they implementable within the time we had available for the project. The client’s design team literately spent months of time showing users lo-fi prototypes, running focus groups, and understanding usage statistics from similar applications. But, the simplicity the end users craved didn’t match the complexity of the business rules required. Upon further investigation the customer’s design team never was given a business level view of the problem to be solved. We tried to merge the business requirements with good usability, but ultimately the franken-design didn’t work. We had to throw out the big picture design and use them as ”guidelines” instead. Clearly it was a waste of talent and a haphazard way to build a product.

In hindsight the design team should have been presented the complex business rules so that their design could incorporate them from the beginning. However, the customer’s SDLC only allowed the design team to be engaged in the definition/high design phase of the project. Once we got to the design phase they were hard to find. By the time we got into the build phase the development team was simply a distraction from other work for these designers. A better model would have kept the designers on the project as each piece is built. I’m not suggesting full dedication to the team – 40 hours a week. That would be nice, but that’s not likely possible in most organizations. I’m suggesting a small time commitment over a long period of time.

Most of the time projects are actually building products. If you are building a product, but focusing SDLC metrics and efficiency, keep in mind that your phases are likely making walls around teams and causing ineffective communication between them. As Matt from 37Signals points out, “Inefficiencies are what make you special.”

Related posts:

1. Who values your product and do you value them?
2. Design Doesn’t Just Mean Color
3. Tech Terms that Drive Business People Crazy

photo credit: Ramón Peco

For any software development professional who has contracted out from time to time, this story must seem familiar. You interview and are told that they practice agile development. Once you show up at the job site, they tell you they practice their own "brand" of agile, which turns into "sort of agile" or "we've take the best parts of a couple of different methods" or "it isn't exactly" -- wild and crazy hand gestures -- "XP."

It's a classic case of people following the form or ritual of a process without understanding the why of it. Take the morning scrum, for instance. The ritual around these usually calls for them to be done standing up. We've of course all heard of the standups that run an hour or two. Someone on that team doesn't understand the function and is following the ritual.

So, what is a scrum supposed to accomplish? It is a quick way for a self-organized team to stay on the same page and surface issues or obstacles. You cover three things: what you did yesterday, what you are doing today, and any blockers or obstacles you might have. If an obstacle comes up, the scrum master facilitates the removing of that obstacle after the meeting. You do them standing up so that participants don't feel the need to bloviate on and on about something. If you get comfortable standing for longer periods, have them standing on one leg, or on tippy toes, or with a book balanced on your head.

If you're having 2 hour standups, it can point to one of several things. First, your team isn't self organizing but rather a traditional top down, "cut the food for your kids" kind of team. Then you need longer status meetings so you can remote control your subordinates. That's fine, but stop kidding yourself that you're doing agile development. Second, you may have too large of a team. Even at 30 seconds a crack, a team of 30 will make for a long scrum. Consider breaking your team into smaller units. Third, you may not be doing real iterative development. If you're talking about scope and slipping things into the next iteration or expanding your iteration by a day or two in almost every scrum, then you're not doing agile.

There are a number of other problems that may be causing those marathon scrums, but those are usually at a deeper level. I'll tackle another one in my next installment of "Fake Agile: Embracing Failure."

Related posts:

1. Getting a team over the fear of daily scrums
2. Post-Agile in the Game Development World?
3. Building a High Performance Agile Team: Assume You Will Be a One Hit Wonder

photo credit: Daquella manera

When you're in software development you sometimes get called in via the Bat Signal -- a project is going sideways and management feels they need more bodies or different bodies to get the thing back on track. For a while now we have turned down "gasoline" work (see The Mythical Man Month on throwing gasoline on the fire by adding more developers) but still do a fair amount of toxic software project remediation.

In one particular case the system in question was several months late and substantially over budget. After a few days of reading documents, interviewing developers and meeting with stakeholders, one thing stood out. This system -- we'll call it "Cool Beans" -- was in fact "Cool Beans 5," i.e. the fifth version of this system. I asked the product manager the obvious question: "What was wrong with versions one through four?"

He told me that none of them had ever been adopted by the business units they were built for and that it was concluded each time that the technology was to blame. This time they were writing a web application, so all of their problems should be solved.

About a week later it was clear to me that the motivation for the latest incarnation of "Cool Beans" was that central corporate management wanted a way to enforce workflow and reporting on various semi-autonomous offices around the country. All of the previous systems had been heavy on prescriptive workflow and extensive reporting -- for the benefit of corporate management -- and offered precious little to the actual users of the system. No wonder they had never been adopted.

Why Companies are Bad at Developing Software

When we talk about the shortcomings of corporate software development processes, we often focus on the difference between waterfall and agile. That comparison certainly still holds, but there are other, more critical factors that bend these projects toward failure.

In the above case, the failing was an all too common one: they had the wrong people giving requirements for the system. Instead of asking the actual end users what would help them do their jobs, they instead had someone from corporate designing a system to enforce a new way of working, one that probably didn't make much sense or work for the end users.

That's why you see lots of three, four, five and so on versions of the same system in corporate environments -- they keep making the same mistake of having the wrong people give requirements. On the one hand it may seem obvious, but I've personally seen this mistake committed more than two dozen times in a variety of industries.

Now what's true for internal company systems is also true for public-facing software. If you don't let the user tell you what they want -- if you substitute your own judgement for that of your customers-- you run the risk of building a system based on your misunderstanding of their needs.

It may seem inconvenient and untidy to have so many outsiders in your software development project, but what you may see as untidy results in actual end users modifying your product vision and concept into something that people will actually want. Yes, the process can be messy, but there are ways to deal with that (more later on how to use the practices of User Experience Design to make the process of requirement gathering from end users go more smoothly).

So, if you are developing version 1, make sure you get it right and not wait for version 5; get the right people in the room from the beginning and involve the end users.

Related posts:

1. ILog Aquired by IBM
2. Digging a Hole and Covering it with Leaves — The Software Development Version
3. The Incredible Rising Version Number

For developers to understand why this is important, just think about the client/stakeholder you've worked with who has been the most willfully ignorant of the software development process -- the sort of client that demands things that are hard or impossible to do with software and then throws a tantrum when someone tries to explain this to them. Who wants to work with that person? No one. Think of how ineffective that appdev team is with this person gumming up the works and whipsawing the team with unreasonable demands.

Now put yourself in the shoes of a business person who is dealing with developers who are ignorant of basic business concepts. Why would you entrust such folks with any degree of authority for developing a new software product for your company? You wouldn't.

So, the homework for developers is to become conversant with common business concepts so they can become real partners in the product development process. How do we do this homework? I sometimes have developers ask me whether they should get an MBA. There are good reasons to get an MBA, such as building up a network of classmates with whom you will do business, but if what you really want to do is "understand the business better," then you can get that for a few nights of reading and less than $30.

Competitive Strategy by Michael Porter is just about the most accessible introduction to business concepts -- how companies compete in a marketplace -- that I've come across. If you want to come up to speed on concepts like "barriers to entry" and "market fragmentation," this is the book for you.

One example of how understanding business concepts can be helpful to developers is the different ways of competing: on cost, on quality, or on specialization. Suppose that the company you work for competes on cost. Normally, developers will want to build the best, most complete product possible. But if the resulting product will be too expensive for me to continue competing on cost, then I have to compete on specialization or quality, and changing your stripes, i.e. your brand, overnight is not an easy thing to do. If you understand the constraints on the product and, more importantly, the business reasons for these constraints, business people and developers can work cooperatively to optimize the product so it is both packed with features and inexpensive.

Another example concerns a business plan I looked at a while back. The basic idea was that a company would be delivering a service to laptop users. The state of the art at the time was to use cc:Mail to distribute the content of this service to the laptops and to reassemble it there. This was a complex and difficult part of the application that only folks with some experience with the technology could hope to develop -- a real, serious "barrier to entry," i.e. other folks wanted to get into the marketplace, but very few actually would.

If a technologist had been looking at the business aspect of this deal (I was just looking at the technical issues, though I did raise this as a business issue at the time), they could have quickly identified the Internet and Web as a threat to this business plan. Once this technology became ubiquitous, one major barrier to entry would be removed. As it was, the investment went ahead and was, predictably, not a success.

So, if you want to be treated as an adult, read this book and start thinking about your employer's business as if its success really mattered to you.

Related posts:

1. Tech Terms that Drive Business People Crazy
2. Innovation and Treating Developers Like Adults
3. Thinking about starting a SaaS or eCommerce business?

Well-well look. I already told you: I deal with the god damn customers so the engineers don't have to. I have people skills; I am good at dealing with people. Can't you understand that? What the hell is wrong with you people?
-- Tom Smykowski

photo credit: sdminor81

I get a lot of grief when I talk about agile practices and how it ends some inefficiencies, i.e. less personal activity during work hours by developers. One commenter on my last article on the benefits of pair programming put it thusly:

It is amazing how insulting people can get when waxing on about the benefits of Pair programming. It is one thing to make the business case to an organization that Pair programming can be beneficial to an organization, but it becomes downright mean when developers get classified as being lazy because they are not paired up. As someone who has spent many a night working (alone) for as long as it takes to complete a project on time without anyone noticing, I seriously disagree with the notion that by not pair programming, developers are lazy.

He's right, but not for the reason he states. Pair programming is still better than solo programming in almost all cases. Treating developers like children, however, is a sin that is pervasive in many businesses. In my advocacy for pair programming, I was subconsciously pitching my message to product and development managers in large organizations.

Why large organizations? Certainly developers are treated poorly in both large organizations and small, but nowhere have I seen developers so systematically infantilized as in the large, corporate software development organization. Tally up all the times you've heard things like "We don't want developers making business decisions," or "we don't want developers talking to users" and you get a sense of the scope of this problem. I have some ideas on why this infantilization takes place, but, more importantly, I believe that any organization with this sort of culture will not be able to innovate new software products and services.

Why Developers are Treated Like Children

To understand why developers are often treated like children, consider a similar scenario. Say you're a politician and you're arguing with a social scientist about public policy. This scientist is an expert. He's studied this domain backwards and forwards. He has all the statistics and arguments and studies at his fingertips. You are not an expert. You're likely going to lose most arguments with this guy on the merits. So you decide to cut him down to size. His ideas are from the "ivory tower" and impractical. Sure, he sounds impressive, but he's got feet of clay. Sensing an advantage, you go a little further and point out that some of his colleagues are a little weird, wear sandals everywhere or talk to themselves. Suddenly this leading expert has become an idiot savant, an intellectual buffet where you can take or leave his arguments and opinions like shrimp or brussels sprouts.

Much the same happens in software development environments. Software developers -- highly skilled experts in crafting software -- are declared business incompetents. They may understand bits and bytes, but they don't understand business. The results are predictable: the developers check out of the project and become clock watchers.

One Solution: Self-Organizing Teams

At Pathfinder Development, we've been called on to introduce Agile practices into projects and organizations. One of the more subtle changes we introduce is that of the self-organizing team. This is where developers (and BA's, IA's, etc.) share responsibility for organizing their own work, rather than depend on managers telling them what to do. At the same time they also become accountable for this work and the ultimate success of the project. The former "managers" become obstacle removers, freeing up the team members to do their best work.

This constitutes a seismic shift in thought for most corporate IT environments. In one case I had to repeat the concept of the self-organizing team three times until it sunk in. "You mean we will be making the decision on what to work on instead of a VP?"

This is the hardest of all agile lessons to learn. Once it takes however, some amazing things happen. Developers begin to work a lot harder. It's amazing how giving someone a big say in what they do every day will impact their motivation. They also start to take and avid interest in the underlying business problems their software is to solve.

Since one of the biggest sources of bugs and consequent cost is misunderstanding between stakeholders and developers, it pays to reduce that misunderstanding. That's the purpose of sprint planning meetings and user stories. If you infantilize developers and deliver business specs to them in the equivalent of business baby-talk, is it really surprising that the result is a comedy of errors?

Innovation

Beyond productivity, there is another cost to locking a significant part of your workforce out of thinking about the business process. Some of your most creative, most analytical thinkers, the software developers in your organization, are being discounted. If less than 5% of your workforce is being asked to innovate, to come up with that great new product or service idea, or just a better way of doing something old, then odds are they won't think of it. Harness those talented developers, however, and you'll discover human capital you never knew you had.

If, by now, you're still shuddering at the thought of that developer -- a shy, introvert who sits in a cube farm over by the break room -- taking an active role in understanding your business, then you're going to be paying lots of management consultants a ton of money to try to splice innovation into the "DNA" of your company, all to no effect.

Related posts:

1. Advice to Developers: Try Understanding the Business
2. Software Development and Wasted Motion
3. Pair Programming: Lone Programmer Shoots Back

We're currently in the process of interviewing candidates for 1-2 Senior, and 2-3 junior level Rails Developers, and I'm wondering about the skills that are most critical, and how best to identify the best candidates.

My normal interview process flows like this:

1. Quick Phone screen evaluating basic development skills, background, availability, personality
2. Basic coding problem (less than 1hr to complete)
3. Phone call to review coding problem, 'how would you handle X?' questions, etc
4. On site interview targeting: Communication, Communication, Communication, (and tech)
5. Final decision

Independent of the language they will be working with, I've found decent success with having candidates answer some standard dev questions, solve a basic coding problem, and demonstrate their ability to whiteboard a design.

The problem is that it doesn't scale. When we've opened up positions in the past, either Senior or Junior, there have been so many applications that its hard to contact them all. This leads to a reshuffling of the tasks and matching criteria, in an attempt to identify the 'best' matches, and 'filter out' the others.  So instead of calling each candidate first, we might simply reply to them with the details of the coding assignment. I've seen 40 people send the "I'm an extremely hard worker, very interested in your position and would do anything to join your wonderful company" cover letter and resume, and then get whittled down to only 10 candidates that actually submit the assignment.  (Note to applicants: 'showing up' is an important first step!)

I came across an interesting post titled '11 tips on hiring a rails developer' which mentioned some 'filtering' criteria to identify the best candidates

You can read the full description of each one, but there are a few of these that I wanted to highlight:

1. Don't use Monster.com or recruitment agencies.
2. Poach Talent from other companies!
   
3. Don't hire someone that doesn't know Rails at all.
4. Look for open source contributions.
5. A personal Rails blog is required.
6. A university degree is not important.
7. Be wary of holes in proficiency.
8. Avoid brand-name superstars.
9. Hire perpetually.
10. Have a company Rails blog
11. Special compensation.

So each of these tips is meant as a heuristic or proxy of the true underlying ability. Any time you are making generalizations you are going to miss out on a few exceptions, but hopefully you end up with what you are looking for.

Starting with the most controversial first, (#5) while I like the idea of giving a strong preference to someone that has a rails blog, and loves  Ruby, Rails, and Web Development enough to be opinionated and spend their own time ranting about it, I'm not sure I could make it a filtering criteria. Doesn't that seem a bit strong?

I feel the same about (#4) Open Source contribution. I think its great if they have it, but I'm not sure I would reject any candidates that haven't contributed to open source.

While I've met many systems administrators that (#6) don't have a degree, and are very good at their craft, I haven't really encountered many solid developers that don't have a degree. That's not to say they aren't out there, I'm just saying I've never encountered them, and I'm reluctant to use it as a filter. Do you think that there are many strong Rails developers out there without degrees? (I don't care if they have a CS degree, but I think I prefer that they have some kind of degree)

As it relates to (#1) and (#11), I know for sure that the economics of finding a candidate through your own network and contacts can be less expensive than the recruiter fees (assuming you find a decent candidate and don't waste a lot of your own time doing it), and if you are able to find someone directly, it frees up cash for such amenities as 'New Macbook Pro', 'RailsConf', and 'Fridge full of XXX'.

If I was in charge of everything

I really like what Andy Singleton describes regarding how his team at Assembla is organized and tackles Agile Development, and on the right project, I'd really like take his advice and try this one:

"Don’t interview. Just pay people to join a project, pull a task from the queue, and find out what they can do."

Anyways,  I'd be very interested in your thoughts regarding what makes for a Solid Rails developer, where you find them, how you keep them, and what you've learned.

(Oh, and if you know any passionate Rails Developers in the Chicago area, send them over!)

Related posts:

1. 20 useful Facebook/FBJS developer resources
2. Faking the art of interrogation
3. Interview: Songbird developer evangelist Stephen Lau

Never get involved in a land war in Asia.

.

Also, never get involved in a religious war about statically versus dynamically typed languages. Well, maybe just this once.

Periodically, an angry Javascript developer will let loose and flame GWT as a misbegotten spawn of evil. Then all the GWT developers point and chuckle and move on to developing more cool applications. Every so often, though, someone will make a thoughtful comment about GWT, and then we have a fruitful discussion that helps clarify what GWT is and what it does and doesn't do well.

William Shields has either posted such a thoughtful comment or a very high end version of a flame, entitled Lost in Translation or Why GWT Isn’t the Future of Web Development. It is well worth reading, along with Google's Joel's somewhat heated response.

What I want to take issue with is Shields' statement that "Between Javascript, PHP, Python, Perl, Ruby and other languages over the last decade (and yes some have a history going far earlier than that) have clearly demonstrated that indeed the sky hasn’t fallen with loose and dynamic typing."

I'm very much a right tool for the job kind of guy. I love my Groovy and my Ruby, but I also like C# and Java for some things. So, has the sky fallen? I would say yes.

Over the last decade, I've run a software development company that practices agile development. One crucial aspect of our practice is Test Driven Development (TDD). We strive for a high level of code coverage, typically over 85%.  With this approach we deliver software where the number of bugs post-release converges rapidly toward zero, even with enhancements and new features being added.

Now while TDD is great for all software development projects, it is absolutely essential of projects developed with dynamically typed languages. Where statically typed source code without unit tests is like a toxic wasted dump, dynamically typed source code without unit tests is like a nuclear holocaust. Those types of projects just throw off new bugs like a frat house in summer.

Given that many JavaScript and PHP (and Rails, for that matter) projects are developed without unit tests, either because the practice of TDD wasn't common in the organization, or because dynamically typed languages are often the choice of amateurs and, well, "what's a unit test?"

We often take over code developed by other development shops. Sometimes that code has no tests and the client is predictably upset about the bugginess and fragility of the code. I'm usually calm when we take over a statically typed code base, but I'm extremely nervous when the code base is dynamically typed. That's not religious, that's just pragmatic.

Related posts:

1. Josh Bloch Intro Q&A
2. Remain Static or Go Dynamic?
3. Static Typing and the Paranoid Style of Programming

photo credit: bobistraveling

We have a few simple principles when it comes to software guarantees: either we have developed the software according to our rigorous testing standards or the third party code we being asked to support meets some basic criterea:

1. It has full functional specifications.
2. It has unit tests that provide better than 85% line and branch coverage.
3. It has automated functional tests that exercise 100% of the user stories.

That may seem pretty extreme, but if we're promising to fix bugs on our dime, we can't have it any other way. Doing without these basic criteria is like guaranteeing you can drive from New York to Los Angeles in 30 hours, keeping under the speed limit without a speedometer and without being told what the speed limit is. Spelling it out a little more, if you can't tell me what your application should do, how do I know if it is "broken?" If I can't run and inspect unit tests to tell if individual classes and methods are doing the right thing, how can I quickly tell if my changes have broken the system?

There's lots of damning studies of the software development industry out there, but the NIST study from 2002 is one of my favorites. In it there is a nice little table that shows that in the 1960's and 1970's fully 80% of software development effort was spent on Coding and Unit Testing, another 10% on Integration and System Testing. Only 10% was spent on the whole requirements and design shooting match. Fast forward to the 1990's, when only 30% of effort was spent on what previously consumed 90%.

The story isn't, of course, that programmers became less professional and more sloppy. Rather, people discovered that if you developed the wrong software it didn't really matter how bug free it was. So more effort was spent up front on requirements. In that sense, Agile software development is a little bit of a retro movement since it reasserts the primacy of unit and integration testing, though with Test Driven Development (TDD) and continuous integration you get there in a very different way than before.

Of course most of the 3rd party software we are asked to take over were developed using a waterfall approach. They usually have functional specifications, though they're typically out of date, and they usually have no unit tests and spotty functional tests. There's another nice table in the NIST study that shows that defects that are from 440 to 880 times as expensive to repair in the operation and maintenance phase of a software applications life-cycle as they were in the original development. I think that's too pessimistic. I think that 80% is a good figure. In other words, if you spent $200k developing your original application, you should expect to spend $800k on maintenance programming and testing over the application's life span.

If you've got good unit tests, functional tests and functional specs, you might carve that down to 50%, in other words, expect to spend another $200k over the lifespan of the application. If you've followed an Agile development approach, you should be able to achieve those criteria. Your choice.

Related posts:

1. 800 on Your Math SAT, Software Development and Bugs
2. Agile Development Improves ROI – But RFP Processes are Stuck in Waterfall.
3. Cucumber Rocks – But it’s not a replacement for unit tests

The C# compiler itself is setup to extract documentation which can then be piped through one of a number of documentation generation apps. My preference is a new(ish) project called SandCastle from Microsoft, which aims to provide much of the featureset that NDoc once did. Unfortunately this application provides no GUI. A fine gentleman named Eric Woodruff stepped in to wrap this application in an easy to use GUI for us aptly named Sandcastle Help File Builder. Through the use of good comments written WHILE we wrote the code, we were able to pop out some documentation for every bit of code we wrote in a matter of minutes. Add to this the nice information we get when using our code via intellisense, and it simply doesn't make sense not to strictly enforce documentation standards on your project.

I often see code written with little or no attention paid to comments. Sometimes the comments are fairly haphazard and appear to follow no standards whatsoever. By following documentation standards, you too can auto generate documentation instead of wasting hours or days going back through and trying to remember what your code does. The following are some of the more important tags available for C# documentation, presented in a roughly organized fashion. You can find a more complete list here.

Method/Type Description:
summary - To describe a type or type member
param name='name' - To describe a parameter of a method
example - To show an example of code usage
returns - To describe the data to be returned from the method

Formatting:
code - Denotes what is contained is code
c - Similar to 'code' but used within comments
para - Usually nested in another tag, allows formatted text

References:
see cref='member' - To specify a link to another type
seealso cref='member' - To specify text that may appear in a "See Also section" (I honestly have no clue what that means)
paramref name='name' - To reference a parameter in comments

There are other tags available, but with these in your arsenal, you should be able to build some nice documentation. Here's an example of what I would consider well commented code:

/// <summary>
/// This is a class description
/// </summary>
public class MyClass
{
  /// <summary>
  /// This method gets something
  /// </summary>
  ///
<param name="number">An integer input</param>
  /// <returns>Returns a <see cref="string"> based on the provided
  /// number
  /// </returns>
  public string GetSomething(int number)
  {
    ...
  }
 

That's not that painful is it?

Related posts:

1. Agile Development, Documentation and Bringing Projects back from the Dead
2. Integrating Design and Agile Development
3. ZK – Documentation & Tools

Design patterns. I think they are the one of the most intriguing areas of object oriented application design and development. There are so many out there that can puzzle you each and every time you try to take a crack at them (I can name a few of them that I still can't figure how they are to be used or implemented).  But a thing that most programmers would agree is if used wisely and appropriately, these design patterns can provide really powerful benefits that can enhance one's programming experience and also the software that is being built. There are several patterns I have used/implemented in my projects that I think are awesome. I ll touch upon how I used some of them and why every Object oriented programmer needs to master them.

Factory Method

This must be one of the most commonly used patterns out there. We indeed have used it the most in our projects. The crux of this pattern is essentially delegating the task of creating objects to a method that basically takes in information about what kind of object is to be created etc.  There were a number of instances in the applications we built where there was a base domain object that had a bunch of derived types. In most of these instances, single method/interface that returned the base domain object type but created instances of the appropriate the derived type was used. The benefit? The object creator is decoupled from the users of the objects and makes the code more reusable and maintainable.

Dependency Injection

I m absolutely in love with this design pattern. It is a very effective design approach in my opinion. What it advocates is lazy/runtime injection of object dependencies or modules instead of hard-wiring them.  There are a number of third party dependency injection frameworks like Spring, Object Builder etc that let you implement dependency injection out of the box. A very simple example of how I used this on a recent project : Consider the application needs to talk to a remote service for what ever reason. We also built out a fake service that mocked the real remote service to enable us to test/run the application when the real one was unavailable or being developed. We needed a way for the application to swap dependencies (basically the real service and the fake one) at run time. Spring .NET's dependency injection enabled me to implement this.

Command Pattern

A really powerful design pattern that helped us a great deal in implementing an undo/redo feature  in one of our desktop application projects. The idea is pretty simple. Every operation that you want to be undone/redone is encapsulated in the form of a command object that contains information about the method or delegate that needs to be invoked for that operation, parameters for that method or delegate, the undo method etc. Every time, any such command is executed, it is pushed to an undo stack. If the command needs to be undone, it is simply popped off the stack and the undo method is invoked. There is a redo stack for handling redo operations.

Other patterns that I have used widely are the Singleton, Facade, MVC, Pure MVC and MVP. These are effective in their own different ways.

Related posts:

1. Using Design Patterns
2. Bean of the Devil: Why Getters, Setters and Such are Evil
3. Defining RIA Interaction Patterns on time in Flex Agile Development

photo credit: Johnathan!

I got a few angry and scolding comments on my last post on pair programming. Let me address each of the issues in turn:

1. Programmers are not lazy - Rather, they are no more or less lazy than any other type of worker. Why do the peak usage times of most web sites, including Facebook and the like, coincide with business hours? You can look at the studies that reinforce this finding, but those traffic graphs speak volumes. Workers -- developers included -- that pair, spend less time on personal stuff than those that don't.
2. Nobody ever gets hit by a bus - True, but people do leave projects. Maybe the whole bus (or Pie Truck, if you prefer) is a read herring. The real issue isn't developers leaving a project. It's that you have less flexibility when one developer knows a piece of the code and the other doesn't. I can't double down on a part of the code that needs more resources, since then I'm in Mythical Man Month land and am throwing gasoline on the fire.
3. Nobody can prove that pairs are more productive - There are studies pro and con on this. I can speak from experience. I've been on a number of agile projects that did not use pair programming. We had a good history on what the velocity for each iteration was. We changed to pair programming and got anywhere from a 30% to 50% increase in velocity going forward, along with a large decrease in bug reports per iteration. That's pretty convincing.

It's easy for folks to misunderstand pair programming and to dismiss it as useless, weird or wasteful. It's one of the first things that gets ditched when the going gets tough, yet it's one of the most important, along with TDD. That's why I wanted to make as strong of a statement as possible in favor of pair programming and against singleton programming.

Related posts:

1. A Pair of Kings Beats A Single Ace: Pair Programming, Agile Rails, and You
2. The Importance of Pair Programming
3. Pair Programming in the New York Times