-
Get a monthly update on best practices for delivering successful software.
I started my first real Agile software development project in 1999. I'd been doing more traditional software development before then all the way back to 1980. I won't bore you with the details of those earlier projects, but my feeling was that there had to be a better way of developing software that didn't involve a senior technologist (me) telling a whole bunch of junior technologists what to do. It turns out I was right.
But almost from the start I got pushback from other people in the development organizations I worked in that Agile development was horribly wasteful. They pointed to Test Driven Development ("all those tests more than double your effort"), pair programming ("two developers doing the work of one?"), and refactoring ("you're rewriting the software every time at enormous cost"). Of course all of these objections were born not just out of a misunderstanding of Agile development, but a fundamental misunderstanding of how their own software development processes actually worked.
Topics: agile, refactoring, Technical Debt
Gwarred Mountain over at Climax Studios has posted a very thoughtful blog post about software development methods and the appropriateness of Agile Software Development. I was ready not to like this article, what with the title and things like this:
If I have to sit through another meeting with some little "agile" toe-rag defending their train wreck of a project then I may end up forcibly ramming a kanban where the scrum does not shine.
But then I thought about all of those fresh-faced management consultants we've run into recently -- who have read a book about agile -- trying to teach us how to do it. Well, yes. I've had some uncharitable thoughts myself. Continue reading »
We've discussed the benefits of Agile development before and that the iterative approach to building the architecture -- where you explore architectural issues (very few apps are completely new and unknown) a little bit through each iteration -- is an effective method for arriving at a good application architecture. What is less obvious is the psychological benefit to working in this way.
It's frankly been a while since I've participated in a large waterfall project directly (one benefit of working for a firm that does agile software product development), but I regularly talk with folks who are still in the corporate trenches doing things the old fashioned way. One thing that hasn't changed is the BIG ARCHITECTURE wrestling match up front. Management wants to know the architecture, the guys with "architect" in their job titles want to know the architecture (so they can criticize, natch), the project manager(s) want to know the architecture. How will we scale? How will we ensure security? More useless brainpower is spent on this ultimately fruitless task -- solving problems that end up being no problem at all -- than almost any other activity in the project.
Topics: agile, Divide and Conquer, Stress, waterfall
Academic research is incredibly inefficient when it comes to producing products and services. Grad students, post-docs and professors work on "problems" that some collection of graybeards has deemed "interesting." Their "solutions" -- sometimes successful, sometimes not -- are published as research and then, occasionally, if the stars align, is turned into a product or service through the application of venture capital via a startup.
The only thing less efficient in producing innovative products and service is the corporate R&D department. In most places you have a phase-gate process (waterfall under a different name) of conceptualization, feasibility testing, definition/specification, development and launch. They are typically bloated, bureaucratic monstrosities with huge documentation requirements and endless committee meetings that do more to stifle innovation than promote it.
There is a way, however, of applying Agile principles to the concept phase, and it relies on two of the basic tennets of Agile: failing fast and self organizing teams.
Topics: agile, Innovation
In Leading Lean Software Development, Mary and Tom Poppendieck present a handbook for how to run a software development group, top to bottom. I intended for this to be a simple review of concepts known to me for years, but the book offered much more. The book’s jacket describes it better than I can: They “show software leaders and team members exactly how to drive high-value change throughout a software organization—and make it stick.” If you are completely new to agile and lean you the book might move a little fast for you. If this is the case, I suggest you spend some quick time getting agile and lean 101 elsewhere first.
If you walk away with one concept after reading this book it should be to believe that success comes from people. The best companies focus on developing problem solving skills and local decision making. These companies favor adaptability over efficiency. These companies make money to survive rather than simply surviving to make money.
The book starts out by defining the concept of frames, “the unspoken mental constructs that shape our perspectives and control our behavior in ways we rarely notice.” A useful way to look at software development. I was happy to see their description of Agile as evolutionary rather than revolutionary. This is why when you explain the set of Agile practices to those with extensive experience they usually nod their heads and say they are already doing them. I have been telling people that Agile is a collection of best practices that good software shops have been using for years. Now I have a reference to support this.
Software craftsmen should read chapter 2 about technical excellence. The chapter goes through each engineering practices, explains it such that a non-practitioner can understand and gives examples. After they are done they should make their manager’s read it, then their managers.
Conclusion
I wish this book had been written 5 years ago. It would have saved me considerable effort and time trying to define what a well run software organization looks like. Your mileage my vary, but I will say that I intend to keep this on my bookshelf right next to Managing The Professional Service Firm.
Topics: agile, Agile Development, Best Practices
Twas the night before Christmas, and 500 pages of detailed requirements had been produced. No line of code had been written, not even by a mouse.
It sounds great in theory: You produce requirements up front, getting them so detailed that they cannot be misunderstood, you vet them with a full committee of stakeholders who will be impacted (except your customers.) Then you give it to the developes. The result: you can get a really precise cost and timeline, you'll save on expensive development time, and your CFO will sleep soundly at night.
It's a nice fantasy, but it's wrong. It's a fundamental misinterpretation of how software development works. In fact, it's a guaranteed way of increasing your costs and producing bad software.
Here's why:
If you're developing software, you're coming up with a new solution to a problem, or solving a new problem all together. Otherwise, why are you building software, rather than buying it?
Traditional requirements definition processes are a spectacularly bad way of understanding stakeholder needs, and coming up with, conveying and vetting a solution for these types of problems. You will miss certain requirements, you will over specify others, and some of them will never get used. Your stakeholders will only be able to provide feedback of limited value until the working software is actually in their hands. At that point, you will finally start getting valuable feedback that you can push through as change orders.
Continue reading »

By now almost everyone has figured out that traditional fixed bid doesn’t work for software development projects, especially where effective, intuitive user interaction is a significant component of success. Over time there has been a lot of effort devoted to finding the underlying reasons for project failure. The Standish Group Chaos Report for 2002 found that 45% of features developed were never used, their 2008 Chaos Report found that typical software projects come in at 189% of original estimates (there is an obvious correlation there), and many studies over the years have concluded that an uncomfortably small percentage of development projects are viewed as meeting or exceeding expectations or providing competitive advantage.
Hidden implications of “fixed bid”
Although you need to provide a target budget to get projects approved, “fixed bid” for application development starts of with a faulty basic assumption – that user, functional and system requirements are fully known and documented in a way that enables efficient development. True fixed bid also implies “waterfall” development and two assumptions: 1.) you have completed a thorough, efficient design phase, and 2.) requirements won’t change during the development process. Neither of these assumptions are realistic.
Topics: agile, failure, fixed bid, Software Development Best Practices
…or the telephone game
Last weekend I was watching a movie with my kids. In the movie there was a chain of monkeys that needed to pass on the message from one character to one on the other side of the chain. The message went something like, “Don’t throw us over the wall. There must be another way. We will all be killed.” As it went through the chain and the receiver heard, “Throw us over the wall. It’s the only way. Banana.” The scenario seems ridiculous, but its roughly equivalent to how many companies approach software product design. Often times companies don’t realize they are creating a product at all. They think they are just running a project and focus only on delivery of that project as if it is the only artifact of their work.
The problem stems from the fact that when organizations reach a sufficiently large size they must focus on consistency of delivery and efficiently using people’s time. For large organizations this is part of the mix that makes up their competitive advantage. However, the sheer size and number of moving parts required to enable clocklike consistent delivery leads to the most knowledgeable people about the customer never directly speak to the people responsible for building the product. Or translated into a traditional SDLC, the definition/high level design team isn’t communicating with the build team. In my experience they are usually two different groups of people. I’ll give you an example:
A while back, I was leading a software development team creating a product to be used by all 170,000 of my customer’s employees on a daily basis. They happened to have a team of user experience designers and wanted to take on the “big picture” part of the design themselves. This company could afford the best and the brightest talent - and was able to attract them. Individually the folks on this team were talented and knew their craft well. I actually learned a lot just from my brief time with them. However, once we got the design in hand it was obvious that the usability team’s artifacts weren’t going to work for the project. They didn’t meet the end user’s needs nor were they implementable within the time we had available for the project. The client’s design team literately spent months of time showing users lo-fi prototypes, running focus groups, and understanding usage statistics from similar applications. But, the simplicity the end users craved didn’t match the complexity of the business rules required. Upon further investigation the customer’s design team never was given a business level view of the problem to be solved. We tried to merge the business requirements with good usability, but ultimately the franken-design didn’t work. We had to throw out the big picture design and use them as ”guidelines” instead. Clearly it was a waste of talent and a haphazard way to build a product.
In hindsight the design team should have been presented the complex business rules so that their design could incorporate them from the beginning. However, the customer’s SDLC only allowed the design team to be engaged in the definition/high design phase of the project. Once we got to the design phase they were hard to find. By the time we got into the build phase the development team was simply a distraction from other work for these designers. A better model would have kept the designers on the project as each piece is built. I’m not suggesting full dedication to the team – 40 hours a week. That would be nice, but that’s not likely possible in most organizations. I’m suggesting a small time commitment over a long period of time.
Most of the time projects are actually building products. If you are building a product, but focusing SDLC metrics and efficiency, keep in mind that your phases are likely making walls around teams and causing ineffective communication between them. As Matt from 37Signals points out, “Inefficiencies are what make you special.”
Well-well look. I already told you: I deal with the god damn customers so the engineers don't have to. I have people skills; I am good at dealing with people. Can't you understand that? What the hell is wrong with you people?
-- Tom Smykowski
I get a lot of grief when I talk about agile practices and how it ends some inefficiencies, i.e. less personal activity during work hours by developers. One commenter on my last article on the benefits of pair programming put it thusly:
It is amazing how insulting people can get when waxing on about the benefits of Pair programming. It is one thing to make the business case to an organization that Pair programming can be beneficial to an organization, but it becomes downright mean when developers get classified as being lazy because they are not paired up. As someone who has spent many a night working (alone) for as long as it takes to complete a project on time without anyone noticing, I seriously disagree with the notion that by not pair programming, developers are lazy.
He's right, but not for the reason he states. Pair programming is still better than solo programming in almost all cases. Treating developers like children, however, is a sin that is pervasive in many businesses. In my advocacy for pair programming, I was subconsciously pitching my message to product and development managers in large organizations.
Continue reading »
We have a few simple principles when it comes to software guarantees: either we have developed the software according to our rigorous testing standards or the third party code we being asked to support meets some basic criterea:
That may seem pretty extreme, but if we're promising to fix bugs on our dime, we can't have it any other way. Doing without these basic criteria is like guaranteeing you can drive from New York to Los Angeles in 30 hours, keeping under the speed limit without a speedometer and without being told what the speed limit is. Spelling it out a little more, if you can't tell me what your application should do, how do I know if it is "broken?" If I can't run and inspect unit tests to tell if individual classes and methods are doing the right thing, how can I quickly tell if my changes have broken the system? Continue reading »
Topics: agile, Agile Development, software develoment, waterfall

A lot of pair programming chatter this week. Starting with a New York times article describing pair programming at Hashrocket. It's an interesting article, with a tone that could be described as "anthropologist describing the strange, yet quaint customs of the native tribe"
Obie Fernandez followed up with a list of 10 reasons why pairing doesn't work in most cases. It's actually a list of the things that Hashrocket does to support pairing, although entries like "2. Most software developers just don't want to work that hard" and "1. Most software shops don't really care about excellence" do have a certain, "aren't we great" vibe to them, causing Mike Gunderloy to dryly observe: "Funny, Extreme Programming Explained never said anything about fancy hw or being awesome as a prerequisite for pair programming."
C'mon Mike -- everybody knows that being awesome is a prerequisite for everything in XP.
Josh Susser adds that pair programming isn't right for all projects, particularly projects that have long compile times that force the pair to stare blankly at the screen.
I'd also add this interview with Kent Beck because a) every programmer could use some more Kent Beck in their life and b) because he talks about XP as being concerned with the the social context of programmers, with pairing being a part of that.
Now you are caught up. Here's the part where I talk.
Topics: agile, Ruby on Rails
If my previous post about the value of agile meetings over traditional status meetings got you interested, I want to share a common pattern of behavior I often see from teams trying scrums for the first time. Hopefully you can avoid these and save yourself some time.
For new teams to Agile the statuses given in scrums are generally … well … lies. “Yep, on time. No obstacles.” I was once told by a colleague that, “You can’t hide on an Agile team.” This is true. However, this kind of exposure can be extremely uncomfortable for individuals to get used to. In traditional software teams people aren’t used to their peers asking them direct questions and paying close attention to their progress.
Continue reading »
I'm addicted to TED talks. There have been many nights where I've stayed up way too late watching them. For those who are unaware, TED is a yearly conference where world leaders and thinkers gather to share their ideas and spread their passions and work. I'm someone who gets fired up listening to outher people's passions. I love hearing about what gets other people excited, what makes them tick, what makes them want to share with the world.
One of the more interesting talks I've listened to recently is by Tim Brown at the Serious Play conference of 2008. Tim talks about how the elements of play can improve creativity and productivity in our workplace and life. I found the lessons learned from this talk something that can be very directly applied to software development, and taking some of these nuggets of teaching can help me be a better thinker and worker.
Code Ownership is a well known term in software development. Depending on how you define it, it may be a good thing or bad. When a developer sees code-ownership as him/her owning a piece of codebase that only he/she understands enough to make changes, it is generally a bad thing. It is only when everybody is free to modify the code with a sense of responsibility that he/she should leave the code cleaner than how they found it, it is a good thing. In my view, code-ownership is a good thing when viewed as a responsibilty as opposed to a right. I view it as a Collective Code Ownership where code is not owned by a single person or pair but is owned by an entire team.
So, the question is: How to determine if your project/organization has that collective code ownership culture. And what team members (including managers
) can do to create/encourage it.
Does your project have collective code ownership?
Here are few things you may want to ask yourself to determine if your organization/project has collective ownership culture.
Topics: agile, Software Development Best Practices
Last week I presented at Agile 2009 a workshop for those new to Agile entitled: The Agile Game: An Experiential Workshop. I love this workshop because it allows those new to Agile to experience the rhythm of an agile project in action and learn first hand many of the practices in a non-threatening, non-technical, and fun way. I have used this workshop for clients a number of times and it works. The feedback from this session was overwhelmingly positive too. Comments such as, “Fun game, good to demonstrate how teams do and don’t work together” and “Very, very, practical way to get concept through” are always great to see.
Recently I had been wondering if Project Management was a questionable career choice. I have spent the last couple of years surrounded by talented individuals who seemed to be able to work fine without me. The test is always when the project manager (me) goes on vacation. Has the team fallen apart? Are they forgetting all of the practices they were doing? Is stuff getting delivered to clients? I was finding that I had a backlog of chores when I came back, but overall things were still humming along.