Should We Estimate Software Projects… At All?

(first published on April 12th 2012, migrated from my deprecated Wordpress site)

Introduction

After a year or two of “having a hunch” about this, and after many years of either estimating work or working to someone else’s estimates, I’ve now finally come to the conclusion that the use of estimation of any kind in a project is not only a waste of time but is actually destructive.

I am fully aware this is an extremely controversial statement, so I am going to be as thorough as I can in explaining how I came to this conclusion via experience, data and validation. Indeed, when I read Duarte Vasco’s post about this several months ago, I saw his “point” (no pun intended) but also argued the merits of using story point estimation for the purposes of:

• Up-front sizing of a project to determine its validity within a given budget or timeframe
• Increasing shared understanding and knowledge within the team based on the discussions that arise from a Planning Poker session
• Allowing the PO to make trade-off decisions between different sized stories (based on ROI)
• Measuring team velocity
• To continually validate the initial project sizing by predicting scope-fit within a given release date
• To allow the team to measure and improve its performance

Why shouldn’t we estimate?

I have since come to the conclusion that some of these things do not need to be done at all, and the other things can be done without the need for estimating (guesswork) of any kind. I would now additionally argue that even if you acknowledge the shortcomings of estimation and use ranges, account for uncertainty, etc., the act of estimation in itself is destructive for the following reasons:

• “Fixed” scope project delivery expectations are often (always?) based on an up-front estimate of scope (guess) and how long that scope will take to be delivered (another guess), leading to the obvious dysfunctions like death-marches, low quality, etc.

If the budget is fixed, there is no way of going “over budget” in order to deliver the fixed scope. Yet “over budget” is a common term used when describing failed projects. If your budget is truly a constraint then you will only deliver what can be delivered. Agile methods ensure that what you deliver is of the highest value to the business.

I chatted to a team member earlier and he complained of feeling pressure to increase velocity. I asked him where this pressure was coming from and he said that it stemmed from the concern that the project will fail if the team isn’t able to deliver more stories more quickly. No one is actually specifically asking the team to deliver more, but there is an implied pressure to do so because they are aware the budget is running out. This mindset comes from years of poorly funded, gated projects, death marches, focus on productivity rather than quality and canned or failed projects.

• Asking teams to estimate how long their work will take (or how many points they will deliver in a Sprint or a Release, same thing) has connotations that their output is being measured by an external party (manager), creating an environment of fear and massaging figures to reflect what is desired rather than what is predicted

To increase velocity the team simply needs to over-estimate stories to give the illusion of delivering more. They may not consciously do this but it may happen sub-consciously. The project manager pats them on the back, but all that has happened is the same amount of “done” working software has been delivered.

It’s time to get real and use real data to reflect real progress, whether it’s good news or bad.

• We shouldn’t be defining all our scope up front, meaning we shouldn’t estimate all our scope up front, meaning we shouldn’t be defining our delivery date based on our scope

We should be fixing our Release 1 delivery date and aiming to build the best possible product by that date (variable scope).

As soon as we introduce the word “estimation”, the default mindset is to consider “how long will this project take?” (if this isn’t asked explicitly). This causes us to consider the complete scope and duration of the project (this is anti-Agile and I won’t go into why it’s a bad idea because enough has been written about that already elsewhere)

How do we size a project?

Short answer — you shouldn’t. If you don’t have a firm deadline for your project (e.g. day 1 of the Grand Prix for a Grand Prix app), you will have a budget for your project (set by the PMO or the external customer), from which you can derive a deadline. The smart thing to do is to then plan an interim release (say at the halfway point) where you can gauge how the project is going based on the working software measure.

For example, if your budget gives you enough cash for ten 2-week Sprints (given a fixed, 100% allocated team), clearly you need to assume that your go-live date is in 20 weeks time. But the aim should be to get working software in a production environment in 2 weeks time (after Sprint 1). You should then iterate over the product, allowing requirements (scope) to emerge and shape the direction the product takes, and take time to reassess after Sprint 5.

These things are not predictable up front — estimation will set you up with a load of scope (expectations) that will not get delivered and will only create unnecessary analysis time (money) and pressure.

How does the team get shared understanding of a story?

Simple. When a new item is added to the top of the product backlog, the team will discuss it in Sprint Planning and break it down if necessary. If it doesn’t need breaking down then it is likely already well understood. If it does then the act of breaking it down will necessitate conversations around the implementation detail that will facilitate shared understanding.

In short, the team does not need to be in an estimation session to discuss and break down a story.

How can the PO make trade-off decisions?

The PO probably needs to know the ROI of a story when introducing it to the team to be delivered. In order to calculate the ROI she needs to know how much it will cost to be delivered (how long).

Here a team would estimate the item using story points and then the PO, armed with the team’s velocity, can estimate the item’s ROI. But without story points how can this be done?

This is where the concept of “implicit estimation” comes into play. In order to create predictability in the flow of work, the team will break down stories just-in-time (in Sprint Planning) so that they are all roughly the same size. This is something that happens naturally throughout the course of the project. Over time the size of stories normalises because the team naturally wants bite-size chunks to work on in the short time period of the Sprint. They get used to delivering a certain number of stories, give or take, in a Sprint.

So for the PO to cost the item, she just needs to ask the team if it is understood or needs breaking down. If the PO considers it high enough priority she will want to introduce it in Sprint Planning so that it gets built right away, if it makes sense to do so. Sprint Planning is the place for the team to break down the story if required and decide if it can be delivered in the Sprint. If it can, the cost of the item is essentially 2 weeks of team wages (assuming production deployment is done at the end of the Sprint — a continuous delivery model can improve speed to market and ROI, but that’s a discussion for another day).

If the item can’t be delivered in the Sprint, the PO can simply look at how many stories have been spawned from the epic item and determine the likelihood of it being delivered in the next Sprint or the Sprint after, based on how many stories the team usually gets through. This leads me nicely on to the topic of how we measure velocity in the absence of story points.

How do we measure velocity?

Now I’m moving firmly into Duarte territory. The answer is we count stories rather than accumulate story points, hence negating the need to estimate. As I mentioned before, teams break stories down into roughly the same size, so counting how many stories are delivered in each Sprint makes for a satisfactory measure of velocity. If the team usually delivers 5 stories with zero defects and then one Sprint delivers 6 or 7 stories with zero defects, an improvement has been made (disregarding variance, which exists whatever unit you use to measure velocity).

Due to the hunch I mentioned earlier, I have been tracking velocity as both story count and points for my current team and making projections using both methods. As I suspected (and as Duarte points out with much supporting data), story count provides just as good, if not better a measure of progress and predictability as story points do. Therefore why spend all the time, cost and effort on estimation sessions and velocity calculations?

While story count works great for velocity, I would still warn against using this or any other velocity measure as a way of predicting when you can deliver. You should know when you are delivering and only be predicting what you can deliver at that date. Don’t leave your delivery date to chance, even if you are using historical data rather than guesswork to predict how many stories can be done.

What you can do, however, is use velocity to help the PO understand scoping trade-offs in the backlog (“the data tells me the team can deliver 20 more stories before the release date, so I’ll make sure the most important 20 are at the top of the backlog“).

Conclusion

It’s taken me several years to come to this conclusion. But, if you think about it, people laugh and joke about estimates all the time. Everyone knows they’re a guess. Everyone knows they’re wrong. Yet we continue to do them. I believe it is time for us to acknowledge that it makes far more sense to eliminate the risk and cost of estimation completely and use only empirical data (as Agile and Scrum promotes) to make predicitions.

In a world without estimation overhead the team is likely to be more happy and productive, the inefficiency of spending time on estimating rather than delivering working software is eliminated and the PO will have real data with which to make decisions rather than guesses made under pressure.

To summarise:

• Don’t estimate your delivery date — base it on your budget or a firm deadline
• Don’t estimate your scope — allow it to emerge in order to reap the benefits of building products with agility
• Don’t explicitly estimate product backlog items (stories)
• Use historical data (story count) to predict scope delivery on a given date
• Use just-in-time implicit estimation (story breakdown in Sprint Planning) and past data to estimate cost (ROI) of story delivery

I don’t like to guess, but I predict that not estimating your projects will make success far more probable :).

Thanks for reading! If you are looking for help with your software or product delivery, I provide agile coaching, public training (both theory and practical) up to executive management level, and more. As well as public events, I can also run training internally in your organisation for a massively reduced cost, so please ✍ get in touch.