This question came in and I was asked to see about answering it:
We are struggling with how to forecast feature delivery to stakeholders / customers in Agile – esp for a client data migration project where we seem to keep finding bugs and exceptions. How is forecasting handled in Agile?
I spend a lot time answering this question, primarily because our software ScrumWorks assists in doing this in a variety of ways, and it’s my job to sell ScrumWorks. The underlying concepts and methods are not software dependent however, just easier. So I am going to run through some of the basics here in Agile forecasting.
What you will Need:
- Stable, Cross Functional Scrum teams.
A team needs to stable to be measurable. If our team is constantly changing members they will not only fail progress into norming (much less performing) but they will never be able to provide a consistent Velocity. Keep in mind that Velocity is simply a measurement, the average rate that your team completes work measured in whatever relative estimation scale you are using. If you were trying to measure the average number of miles you get on a gallon of gas you wouldn’t have a very useful value if you changed your car every 4 miles. The same applies to a team, the more stable they can be the more reliable your Velocity will be.
- Consistent Relative Estimation of your Backlog Items.
Backlog items need to be estimated in a relative fashion, as in; “Item A is a 2, and Item B is twice as hard, therefore it is a 4”. Over time this type of estimation, with a stable team, allows for more consistent estimation. Forecasting in Scrum isn’t about accurate estimation, just consistent estimation. Estimates are guesses, and probably wrong. If we are consistently wrong we can still use that data to forecast.
- A Product Backlog that isn’t just User Stories.
A misconception I run into constantly these days is that the Product Backlog is just user stories. This couldn’t be more wrong. The product backlog is a prioritized list of Everything the team needs to do to complete the project. Defects need to be prioritized into this list, development spikes, technical debt, functional requirements. If the team is going to do work on it and it is related to this project then you should be putting it on the backlog, estimating it, and committing to it, just like your User Stories. Without a relatively clear idea of all the work you will have to do forecasting becomes a lot less certain.
How you go about Forecasting:
Once you are set with a stable velocity and a backlog populated with all the work we need to do you will want to try to get as much of it estimated as possible. These estimates do not need to be solid until the team is ready to commit to them, high level estimates are good enough as you go down in the backlog. I usually suggest time boxing the estimation of items, ~5 minutes per item with a team that has a good handle on what an estimate means to them can be good enough for decent forecasting. A sold ScrumMaster can really help when it comes to this type of work. It is very likely that the items will be re-estimated when they are closer to being committed to in a Sprint.
The most basic form of forecasting just requires some simple math. For example: our team has a velocity of 50. A sprint is 2 weeks. This means that a 2 month release is about 200 points. If you have more then 200 points within the release it’s unlikely that you will finish. Less then 200 you are looking good. This doesn’t really help us visualize, or forecast dates, but it can tell us quickly if we are going to meet a target date.
When we want to actually forecast we need to plot 3 things: The points a team completes in a sprint, the total points on the backlog, the rate at which our scope changes. The third one is the hardest, it requires us to record each change made to a backlog item’s estimate, but it is a very important piece of this.
Visually we can create a chart that looks like this:
On this report the Blue Line jumps up every time the Team completed a backlog item and the Product Owner accepted it as complete. The trend line is the team’s Velocity.
The Red Line rises or falls any time backlog items are added, removed, or re-estimated. The big drop between July 15th and 30th was when the Product Owner removed work from the backlog, and then on August 29th appears to have put it back in, or added different work. The red trend line represents the scope change. In this case the scope was trending upward.
Where the red and blue trends converge you have a potential release date. You could also use such a report to note that the velocity trend will converge with the baseline (which in this report’s case is 450) between February 10th and the 25th and say that would be the date all the work should be finished assuming no more work is added (which is pretty safe to say based on the recent trend).
This particular report is very simple and can be improved with a broad variety of options. There are some traps people fall into when they are practicing agile that will make this report less valuable however:
- Changing estimates after they are scheduled in a sprint is something you do not want to do. Keep in mind that we are comparing the rate at which we complete points to the points that we have not completed yet. If we change our estimates after the fact we are suddenly comparing reality to estimates, and defeating the purpose. We want to compare the rate that we consistently complete flawed estimates to the total flawed estimates we haven’t started yet. If we are consistently wrong we can still generate a solid forecast because we are comparing apples to apples.
- Partial Credit is just waste. Don’t bother doing it. Velocity is the average, if one sprint results in 10 points completed and 40 partially completed, and the next sprint we complete 90 points, the average is still 50. If you look at the report above you can see an example of where this occurred, right at the November 27th mark. You have one really bad looking sprint, followed by one really good looking sprint. That’s reality, the report will reflect it correctly. Any consumer of such a report needs to understand that it is the average that matters, not what happened in any specific sprint.
- Estimation is a ritual the team uses to surface questions about a Backlog Item that need to be answered before the team can commit to completing that item within a sprint. Estimation is not a way to get estimates, estimates are simply a happy and valuable by-product of Estimation. Good Agile reporting is a passive thing that stays out of the team’s way, even as it answers important questions.
- Time Estimates technically work with this report, but not for the process of Estimation. Remember that Relative Estimation is a requirement for this type of forecasting to be successful, it’s not just a suggestion. If you are estimating your story points in time you are going to have trouble.
This blog entry is really just a taste of Forecasting in Agile. I’ve personally seen it work, surprisingly well in fact, and this is typically why this method is what I immediately go to when someone asks me about Forecasting, but it isn’t the only way to do it.