Forecasting in Agile

This question came in and I was asked to see about answering it:

We are struggling with how to forecast feature delivery to stakeholders / customers in Agile – esp for a client data migration project where we seem to keep finding bugs and exceptions. How is forecasting handled in Agile?

I spend a lot time answering this question, primarily because our software ScrumWorks assists in doing this in a variety of ways, and it’s my job to sell ScrumWorks. The underlying concepts and methods are not software dependent however, just easier. So I am going to run through some of the basics here in Agile forecasting.

What you will Need:

A team needs to stable to be measurable. If our team is constantly changing members they will not only fail progress into norming (much less performing) but they will never be able to provide a consistent Velocity. Keep in mind that Velocity is simply a measurement, the average rate that your team completes work measured in whatever relative estimation scale you are using. If you were trying to measure the average number of miles you get on a gallon of gas you wouldn’t have a very useful value if you changed your car every 4 miles. The same applies to a team, the more stable they can be the more reliable your Velocity will be.

Backlog items need to be estimated in a relative fashion, as in; “Item A is a 2, and Item B is twice as hard, therefore it is a 4”. Over time this type of estimation, with a stable team, allows for more consistent estimation. Forecasting in Scrum isn’t about accurate estimation, just consistent estimation. Estimates are guesses, and probably wrong. If we are consistently wrong we can still use that data to forecast.

  • A Product Backlog that isn’t just User Stories.

A misconception I run into constantly these days is that the Product Backlog is just user stories. This couldn’t be more wrong. The product backlog is a prioritized list of Everything the team needs to do to complete the project. Defects need to be prioritized into this list, development spikes, technical debt, functional requirements. If the team is going to do work on it and it is related to this project then you should be putting it on the backlog, estimating it, and committing to it, just like your User Stories. Without a relatively clear idea of all the work you will have to do forecasting becomes a lot less certain.

 

How you go about Forecasting:

Once you are set with a stable velocity and a backlog populated with all the work we need to do you will want to try to get as much of it estimated as possible. These estimates do not need to be solid until the team is ready to commit to them, high level estimates are good enough as you go down in the backlog. I usually suggest time boxing the estimation of items, ~5 minutes per item with a team that has a good handle on what an estimate means to them can be good enough for decent forecasting. A sold ScrumMaster can really help when it comes to this type of work. It is very likely that the items will be re-estimated when they are closer to being committed to in a Sprint.

The most basic form of forecasting just requires some simple math. For example: our team has a velocity of 50. A sprint is 2 weeks. This means that a 2 month release is about 200 points. If you have more then 200 points within the release it’s unlikely that you will finish. Less then 200 you are looking good. This doesn’t really help us visualize, or forecast dates, but it can tell us quickly if we are going to meet a target date.

When we want to actually forecast we need to plot 3 things: The points a team completes in a sprint, the total points on the backlog, the rate at which our scope changes. The third one is the hardest, it requires us to record each change made to a backlog item’s estimate, but it is a very important piece of this.

Visually we can create a chart that looks like this:

The actual ScrumWorks 2.0 release back in ’06

 

On this report the Blue Line jumps up every time the Team completed a backlog item and the Product Owner accepted it as complete. The trend line is the team’s Velocity.

The Red Line rises or falls any time backlog items are added, removed, or re-estimated. The big drop between July 15th and 30th was when the Product Owner removed work from the backlog, and then on August 29th appears to have put it back in, or added different work. The red trend line represents the scope change. In this case the scope was trending upward.

Where the red and blue trends converge you have a potential release date. You could also use such a report to note that the velocity trend will converge with the baseline (which in this report’s case is 450) between February 10th and the 25th and say that would be the date all the work should be finished assuming no more work is added (which is pretty safe to say based on the recent trend).

This particular report is very simple and can be improved with a broad variety of options.  There are some traps people fall into when they are practicing agile that will make this report less valuable however:

  • Changing estimates after they are scheduled in a sprint is something you do not want to do. Keep in mind that we are comparing the rate at which we complete points to the points that we have not completed yet. If we change our estimates after the fact we are suddenly comparing reality to estimates, and defeating the purpose. We want to compare the rate that we consistently complete flawed estimates to the total flawed estimates we haven’t started yet. If we are consistently wrong we can still generate a solid forecast because we are comparing apples to apples.
  • Partial Credit is just waste. Don’t bother doing it. Velocity is the average, if one sprint results in 10 points completed and 40 partially completed, and the next sprint we complete 90 points, the average is still 50. If you look at the report above you can see an example of where this occurred, right at the November 27th mark. You have one really bad looking sprint, followed by one really good looking sprint. That’s reality, the report will reflect it correctly. Any consumer of such a report needs to understand that it is the average that matters, not what happened in any specific sprint.
  • Estimation is a ritual the team uses to surface questions about a Backlog Item that need to be answered before the team can commit to completing that item within a sprint. Estimation is not a way to get estimates, estimates are simply a happy and valuable by-product of Estimation. Good Agile reporting is a passive thing that stays out of the team’s way, even as it answers important questions.
  • Time Estimates technically work with this report, but not for the process of Estimation. Remember that Relative Estimation is a requirement for this type of forecasting to be successful, it’s not just a suggestion. If you are estimating your story points in time you are going to have trouble.

 

This blog entry is really just a taste of Forecasting in Agile. I’ve personally seen it work, surprisingly well in fact, and this is typically why this method is what I immediately go to when someone asks me about Forecasting, but it isn’t the only way to do it.

 

Caleb Brown

Caleb Brown is an Agile trainer, organizational coach, and Certified Scrum Professional with a focus on the the “People Side” of software. While practices and technology are part of what enable organizations to succeed with Agile, Caleb likes to concentrate on using the cultural piece supported by whatever practice, process, framework, or technology is needed to be successful. Having worked on the technical side of the software industry since 1999, and as CollabNet’s Product Owner and Master ScrumWorks Trainer since 2007, Caleb has gained deep insight into the inherent challenges with Agile adoption. His trainings and expertise in helping global organizations understand where process ends and tooling picks up, is well respected and highly sought after.

Tagged with: , ,
Posted in Agile
2 comments on “Forecasting in Agile
  1. I like your “switching the car all 4 miles” metaphor as the stable team pre-requirement is typically forgotten. Same goes for trying to apply the same story point size across multiple teams. Do you have a good metaphor for this misconception as well?

    • Caleb Brown says:

      If multiple teams were working on similar work and understood what the scale means, they could in fact use the same story point scale across multiple teams.

      Keep in mind that we are talking about relative estimates scaled linearly, based on an understanding of the effort it took to complete a past piece of work. If 2 teams had a good idea of what a piece of work takes to complete, and they can estimate another piece of work based upon this (i.e. this is 2x or 4x harder) then they could be using the same scale.

      Ultimately what this would mean is that an item estimated at 2 story points, might take 3 days of development time for one team, whereas another team would only spend 1 day on that. A 4 story point item would be 6 days or 2 days, and so on.

      This is why relative estimation is so powerful. If we had estimated our piece of work based on “Ideal Days” then the estimate would be two different things for two different teams.

      To use a Driving Analogy again:

      Car A is driven at an average speed of 50 miles per hour. Car B is driven at an average speed of 100mph. Because our distance is estimated to be 100 miles we can say that Car A will arrive in 2 hours and Car B in 1 hour.

      If we had estimated in hours, and not miles, the estimate would have been 2 hours and 1 hour, specific to each driver. The distance estimate is based on a linear scale independent of the speed anyone might drive.

      To take it a step further, and illustrate why relative estimate is so great, lets say that car B has a problem and can only drive 50mph. The estimate of 100 miles still works, it just means that we know now that it will take 2 hours. If we had estimated 1 hour we would have to re-estimate.

      The exact same thing holds true for Relative Effort Estimates with software. If we had estimated something in “Ideal Days” and then lost a team member, we would have to go through and re-estimate everything. If we had estimated on a relative scale on the other hand, the estimates are fine, they are simply going to take longer to complete then we originally calculated. To revise our forecast we simply have to revise our velocity, not go through and re-estimate everything on the backlog.

Leave a Reply

Your email address will not be published. Required fields are marked *

*