The Staggered Iterative Waterfall (Anti-)Pattern, Part 1

Foreword: This blog entry was originally intended to be a single article. In writing it I found that I had trouble trying to communicate all the information in one sitting. As a result I’ve decided that this will be a 2 part article. The first part (presented here) discuss the Iterative Waterfall pattern and the reasons why this pattern of software development process is detrimental. In the second part I’ll discuss possible approaches to moving away from this pattern.

Over the last 3 years I’ve noticed a reoccuring anti-pattern amongst companies trying to implement Agile. This has happened with development teams that are particularly experienced with RUP [for reasons that I’ll discuss towards the end of this article].

There are two variations of this anti-pattern, which I’ve called Iterative Waterfall and Staggered Iterative Waterfall. The Staggered Iterative Waterfall anti-pattern is simply the Iterative Waterfall anti-pattern taken to it’s extreme. Throughout this article I’ve used the term Iterative Waterfall to refer to both variations except in situations where I need to distriguish between the two.

So, what are the characteristics of these anti-patterns?

A description of the problem
The Iterative Waterfall anti-pattern may arise in when consecutive timeboxed iterations are devoted to consecutive phases from the traditional waterfall software lifecyle (SDLC).

As an example, consider a project that has been started with iterations that have been timeboxed to 2 week intervals. Each iteration is dedicated to one phase from the traditional waterfall SDLC; either Analysis, Design, Construction or Testing. Frequently Design and construction are combined. So in our example Analysis is done in the first iteration, Design and Contruction in the second iteration and Testing in the third iteration. This is then repeated for the duration of the project.

A Gantt chart for this project is shown below:

Iterative Waterfall Gantt
Project managers will look at this Gantt chart and will immediately notice that there is room for improvement. The Business Analysts that worked in iteration 1 are not fully productive again until iteration 4. Similarly for the developers and testers. If we were to stagger iterations we could utilise every team member at 100%!

Here’s what a Gantt chart that shows the Staggered Iterative Waterfall anti-pattern:

Staggered Iterative Waterfall Gantt

Reasons why this approach is undesirable
The Iterative Waterfall model of software development has a number of drawbacks. The most serious of these problems is the impact on the teams ability to communicate and collaborate. The separation of the understanding of the problem (Analysis), from the solution (Design/Coding), and from the quality (Testing) is arbitrary. One of the most significant advantages of Agile methods is that this seperation of knowledge is elimated, or at the very least blurred. This is done by having multi-skilled teams and by having continuous communication between team members. To re-introduce this separation of knowledge into an Agile project is counter-productive.

Let me list the reasons why I believe this model to be very dangerous:

  • Breaks the collaboration model of Agile software development. There is a contract in any handoff point, in this case the requirements documention is the contract document (although it may be implicit). Agile software development is more effective in environments where there is continuous dialog between the customer and the developer. Introducing a written document into this dialog is, at best, not helpful.
  • Does not improve the visibility of progress made. Measuring a teams progress can be difficult at the best of times. The Iterative Waterfall model complicates the measurement of progress. When is a particular piece of work complete; at the end of Construction? at the end of Testing? How much progress has been made at the end of Analysis?
  • Deminishes flexibility for changing business requirements. If I were to start analysing a problem immediately (iteration 1), I would not recieve functional software until the end testing (iteration 3). In our example (where we had iterations timeboxed to 2 weeks) this would equate to an elapsed time of 6 weeks. This may be an improvement on a Waterfall model but if you have the option of delivering every 2 weeks, why would you not want to do that?

How does this anti-pattern come about
Part of the problem is that there are several terms that are common between RUP and Agile that are overloaded. That is, they have different meanings when used in different contexts. The term “Iteration” is especially confusing.

Let us look at how RUP defines an “Iteration”:

Each pass, which is like a mini waterfall, explores a new portion of the requirements set and offers a chance to correct defects and rework the result of the previous iteration.” – Introducing the IBM Rational Unified Process essentials by analogy.

The diagram from this page is particularly illuminating. A RUP “Iteration” is a complete cycle of each of the waterfall phases.

The Agile concept of an “Iteration” is quite different. It’s more appropriate to say that Agile iterations are timeboxes, during which work is completed. Within the Agile “Iteration” there is no separation of the phases. Even though the development team will perform some of the same activities, the separation and ordering is not there. My collegue Victor said that, “We take waterfall and put it in the blender.” I personally think that’s quite a good analogy.
So, how does this anti-pattern evolve? I believe its comes about when a team tries to introduce agile with a preconception of what is meant by the word “Iteration”.

In summary
I’ve outline two variations of an anti-pattern that I’m calling Iterative Waterfall and Staggered Iterative Waterfall. These patterns of software development have all the disadvantages of traditional Waterfall without any of the advantages of Agile.

But recognising an anti-pattern and actively addressing it are two very different things. In my next blog entry I’ll discuss some approaches to mitigating the Iterative Waterfall model of software development, and putting Agile back on track.

If you have comments on this blog I’d appreciate your feedback. You can contract me at Kane at Danube dot Com

Download the PDF version: Staggered Iterative Waterfall 1 blog

Posted in Agile
4 comments on “The Staggered Iterative Waterfall (Anti-)Pattern, Part 1
  1. Jason Lewis says:

    Many people who have avoid this anti-pattern still fall into the story waterfall anti-pattern. That is, in the iteration, development does not start until we have all the requirements. Testing does not start until all the code is complete. This is especially true in teams that have very specific roles and responsibilities vs. teams where the responsibilities are shared. Just like the staggered effect, this still slows down the feedback cycle on working software. It is important that requirements, testing and coding happen simultaneously and continuously until the acceptance criteria are met…

  2. Erics says:

    Great artile, the concept of anti-patterns is readily applied to engineering in general. waiting for your next blog about an anti-pattern and actively addressing. Thanks.

    ————-
    eric dbdeveloper 2008

  3. Bill Krueger says:

    From the “Unified Software Development Process” by Booch, Jacobson and Rumbaugh, it states that an iteration is “a more or less complete traversal of all the core workflows”, meaning an iteration includes requirements, analysis, design, implementation and testing. This also seems to be what is said on the IBM RUP site you link to. But you state that an iteration is dedicated to one of these phases which seems counter to what I’ve learned. Any comment on this would be appreciated. Also note that there must be some agreement on how long a work product is stabilized (i.e. I need to know that the current design is what I’m coding to). If you want to go back and redesign then do that in the next iteration.

  4. Michael James says:

    In theory, the RUP should be done as an Agile process. Unfortunately it lends itself to misinterpretation. The places we see doing RUP have interpreted it as a refinement of waterfall. There’s probably someone doing RUP well somewhere, but those people don’t call us for help.

    –mj

Leave a Reply

Your email address will not be published. Required fields are marked *

*