#NOESTIMATES

2/5/2014 How to improve software estimation

Since 1968 - the year of the Garmisch conference that originated

the term "software crisis" - our industry has been struggling with
predictability and estimation. We all know of the horror stories of
software project run amok with unbelievable cost and time overruns.
Can we do it better? Can we estimate better software projects?
 #NoEstimates

#NoEstimates
HOW TO IMPROVE SOFTWARE ESTIMATION

INTRODUCTION
I am proud to be part of the Agile community.

In the Agile community we’ve been fighting accepted truths in the field of software development for many,
many years.

Kent Beck with Extreme Programming (XP) fought many entrenched truths like: tests are
only written after the requirements. In XP they inverted that order and created cards that
included the tests (acceptance criteria) as a way to express requirements. At that time they were
mostly ignored, and somewhat scorned by the software development community. Today, many
teams write their requirements in the form of User Stories with Acceptance Criteria. Kent Beck
and the early advocates of XP have changed our world.

Ken Schwaber with Scrum fought many entrenched truths like: software projects should
follow deterministic processes where the plan determines what will happen long after
the plan is created. Ken and others defended the idea of using Empirical processes,
where plans are changed constantly and never defined with detail beyond a short
period of time (4 weeks then, 2 weeks now!). They were also scorned, so much so
that Ken Schwaber got thrown out of a company after the first day of Scrum training in the early days of
Scrum. Today all agile teams know the concept of a Timebox, delayed requirements elicitation and constantly
delivering working software according to a Definition of Done. Ken and other Scrum early adopters changed
our world.

But this evolution really goes further back with Taiichi Ohno and Edwards W. Deming.
Taiichi Ohno is the person behind the Toyota Production System. He and the people at
Toyota used different ways to organize work and defined what is – arguably – the
most productive manufacturing system in the world. This transformed Toyota into
the super power in the auto industry that it is today. In the process, Lean emerged,
and Lean changed the world of software development thanks to people like Mary and
Tom Poppendieck. Today we look positively towards a reduction of burocratic
overhead (reducing waste), unlike a few years ago when adding another process
step to a waterfall project was seen as a sign of sophistication. Remember RUP
(Rational Unified Process) and how it took the world by storm because it “disciplined”
cowboy coders?

However, Taiichi Ohno was able to transform Toyota in a period when Deming was a true
leader of change in Japan. Today the Japanese industry is regarded as the standard in
high quality manufacturing industry because of the work that Deming did, and from which
many – like Ohno – followed.

The work the people listed here did, changed our world, and continues to have
impact in how we design work today.

Page 1
#NoEstimates

“I have seen further by standing in the shoulders of giants”

– Newton

THE WORLD HAS CHANGED

Not many years ago, having a very long Requirements phase in a software project was considered a must. I –
and many others – learned that a good project would use about one third of the calendar time working
on planning and requirements gathering. This was the norm when non-incremental software development
was the most common approach to software projects.

The software industry used the phased waterfall approach. This approach called for a linear sequence of
steps as the prevailing model. Today we start to define requirements through tests (acceptance criteria), back
then tests were something we did at the end of the project. Remember the “validation” phase?

Today we write software with a keyboard and mouse, but back then paper was where software was
developed, do you remember that? Flow-diagrams, UML, etc.

At that time Programming or Coding was considered a “lowly” profession, only for juniors. The real software
developers used paper, and the cool kids were all over Rational Rose! Code generation was the keyword! No
self-respecting architect would touch real code. Any tool (or fool) could write code, the value was in the magic
models that generated all the necessary code.

To the right is a photo of an actual code-monkey (just pay them with bananas), but you
don’t see many actual code monkeys these days. In fact, a little known fact is: today
most software developers are human as of today! (I just saw the latest statistics.)

However, the change is not complete. Today many people still look at developers as
code monkeys. Or worse: Code Anarchists! In the era of waterfall the Agile practitioners
were called undisciplined, chaotic, lazy, idealists. You name it! For many, Agile was
untouchable, Agile was chaos.

It took our community many years to get Agile accepted as a “serious” way to develop software.

And as we can see from this email screen-shot. It now really is main-stream, when
even the traditionalists at PMI think it is worth their attention (no doubt because they
saw how popular Scrum had become within their own community.)

Today, I write to you as an advocate of the #NoEstimates approach to software

development. #NoEstimates is just a minor Family in the Agile Species of software
development approaches.

What have I observed? I’m also labeled: incompetent, unaware of customer needs, undisciplined, idealist.

However, what I am proposing is a natural evolution for Agile. Simply, in my experience #NoEstimates is
the natural way to implement two of the four Agile Values:

Customer collaboration over Contract negotiation

Page 2
 #NoEstimates

Respond to change over following a plan

Estimates naturally enforce a plan. What if, during the project, you find a better way to develop that project?
Should you stick to the old plan and be late (guaranteed!) or change your plan and make those estimates
obsolete? Many would choose to stick to the plan. This is one way in which estimates marry you to a plan.

Plans should change, but so should your understanding of the product and customers. As you develop your
product your understanding of customer and technology evolves, why should you prevent yourself from
discovering what is really important as you go? When you estimate a project (after creating the plan) you
assume a certain meaning and value for the stories or requirements you are developing. What if your
understanding changes? Do you go back and discuss all the estimates again? Or do you just continue because
we’ve already planned this in anyway?

#NoEstimates is about focusing on what matters, focusing on what is the next most important thing to do.
Instead of trying to guess at all the things you need to do and spending a lot of time trying to guess how long
they will take to implement.

The software community started to adopt Agile because it wanted to focus on what matters. Not on accessory,
unnecessary work. Remember, BDUF (Big Design Upfront) was once considered essential and even critical for
successful software projects. (Some say it still is).

When Christopher Columbus was dining with a group of rich and important men, after having returned from
America, he was told that “discovering the Americas was no great accomplishment!”

After one person did it (Columbus) all others knew it was possible to do, therefore no one gave it the credit it
deserved. After all what Columbus did was to travel through an Ocean without a map! We knew only his
direction (west).

What those important men dining with Columbus did not realize was that they had fell victims to Retrospective
Coherence: once something is known it becomes obvious (also known as Common Sense).

As an advocate of #NoEstimats I feel like Christopher Columbus did in the famous Egg of Columbus situation. I
know how to balance the egg (develop software without estimates), and once you experience the solution you
too will understand how obvious it is. But no-one can see it before trying it. Agile was once considered
marginal and chaotic but now is main-stream. The same will happen with #NoEstimates.

#NOESTIMATES IS EASY
I propose to you that #NoEstimates is easy. So easy in fact, that I can explain it in 4 simple steps.

1. Select the most important piece of work you need to do (highest value first): Don’t worry about all
requirements you need to deliver. Have a clear direction (Vision) and decide what is the most
important piece of work when you know more. Prioritize your work, but be ready to change the
priorities. Select the most important piece of work at any given time, not simply the one at the top of
your backlog.

2. Break that piece of work down into risk-neutral chunks of work: Risk Neutral simply means that each
piece should be small enough that failing to deliver it at first attempt will not jeopardize the project

Page 3
#NoEstimates
objectives. For example, if your piece of work requires many man-days of effort and many weeks of
calendar time failing to deliver it will have a great impact on the likelihood of success for your
project. However, if you slice your work down to very small increments (my rule of thumb: one day,
one person), failing to deliver that will have very little impact on the project success, and you can
immediately adjust based on that knowledge.

3. Develop each piece of work: following the Definition of Done. Deliver that work to a production-like
environment. Work is only done when it is ready to be used by real customers. Any other definition
causes many unforeseen dependencies to be discovered too late and jeopardizes your project with or
without #NoEstimates.

4. Iterate and refactor: The first implementation can only reflect a first step in understanding what needs
to be done. Learn from the first implementation and improve it until it reflects your definition of a
high-quality product. This step builds on practices that many still don’t believe in like Emergent
Architecture where coding is a key/critical skill and cannot be delegated to code monkeys.

That is how easy it is...

Of course there are a few questions to answer before we can reliably apply this approach. One of those
questions comes from understanding that what we are trying to do with #NoEstimates is learning to live
without trying to impose an artificial control on the development system. We must try to understand deeply
our System of Development, and use its inherent capability, instead of tampering and therefore unpredictably
changing it. What makes sense to do next in a project will change as we learn more.

DEVELOPING STABLE PROJECTS

In other words, if a system is stable and reliably churning out a certain number of User Stories / Requirements
per time period (throughput), it is of no use to set higher (or lower) targets for that system. Conversely, if the
system is not stable, setting a target is useless because you cannot predict reliably enough the output of the
system.

Let me give you an example. Remember Office Space the movie that came out in 1999 (what a fitting year!)
and starred Jenifer Aniston, Ron Livingston and
others? I AM GOING TO GO
AHEAD AND ASK
YOU TO DELIVER 10
Here’s an hypothetical scene from that movie: STORIES NEXT
SPRINT...

You know the boss, right? He was the one that

constantly requested the TPS report from his “trusted”
employees? In the picture we see him requesting a
team member to deliver 10 User Stories during the
next sprint. This request he just made is very common.
When I ask at conferences how many people have
been in a situation like this, typically 70% of the
audience raises their hands – and the other 30% are
either too young or too ashamed to admit it. The truth
is that if you give a manager a number he will want more of it! Or less, depending on which is better from his
point of view.

Page 4
 #NoEstimates
What if your team’s throughput looked like the one on the picture? (Blue line being the actual throughput for
your team)

Asking that team to deliver 10 User Stories on their 22nd sprint,

WTF!!!!!
after consistently delivering between 2 and 4 every Sprint is !#%&!
completely missing the point. The System of Development is
telling us that the team has a stable environment. We should
use that data, not try to manipulate it (which Deming would call
Tampering).

After this explanation I’m betting your reaction to the

manager’s request would be similar to Milton’s reaction…

Page 5
#NoEstimates
People who have studied systems for many years tell us that we have to learn to dance with the system, and
#NoEstimates is one way to help you dance with the system. Just like when you don't drink milk because it
messes with your system – if you are Lactose intolerant. In my years of experience with and without
#NoEstimates I have come to realize that most software development is highly Estimates-intolerant, so
estimates must be used with care and best avoided altogether.

When struggling with predicting the output of a project, I asked the question: Is the System of
Development ”stable” enough for me to make predictions? Stable means that we can trust its throughput
to be stable over a long period of time (i.e. until proven otherwise). I came up with the idea of a stable
System of Development after having asked myself: can we use the data we observe (no estimates) to predict
system throughput and detect changes that make the system unstable? If the System of Development is not
stable, we cannot make useful predictions about long term output.

Over time, and to verify if a particular System of Development is stable I’ve developed 2 rules, based
Deming’s and Shewart’s SPC (Statistical Process Control) rules:

1- If velocity (# of stories Done in one sprint) falls outside the control limits more than 3 times in a row
(”outside limits”): this would suggest that the System of Development is not yet stable and will slowly
converge to a stable state over time (unless we tamper with it constantly – like estimates and plans
tend to do).
2- If there are 5 or more velocity points in a sequence (ascending or descending) (”Run test”): this would
suggest that the teams involved in the System of Development are still learning their way around the
project, organization or technology. These sequences can be increasing or decreasing.

If a team’s velocity over time breaks any of these rules, you should not use the data to make long term
predictions. You should investigate what is making the system unstable, and help the teams find a stable
rhythm of development (what Scrum calls Sustainable Pace).

I could have developed a lot more rules and I encourage you to develop rules for yourself based on your
understanding of SPC and your knowledge of your system.

Let’s look at some sample projects to evaluate if these rules apply or not to a wide variety of projects (small,
large, distributed, co-located, etc.) In the graphs below: LCL refers to Lower Control Limit as defined by
Shewart and Deming within one Sigma (standard deviation) of the average team velocity; UCL refers to
Upper Control Limit (also one sigma); and average is the simple arithmetic average of the team’s velocity
over N sprints.

Page 6
#NoEstimates

Page 7
#NoEstimates

A CASE STUDY IN #NOESTIMATES

Now let’s take a look at one example of how we can use #NoEstimates to predict what will be delivered
during a relatively long project. The question I’m asking with this experiment is: When we consider Counting
Stories (#NoEstimates) vs. Estimated Story Points (Estimates), which ”metric” is more accurate when compared
to what actually was delivered during the project?

This example reflects a relatively long project: 24 sprints. And the question is: which metric predicted most
accurately the output of the whole project?

a) Considering the data we had after only the first 3 Sprints

b) Considering the data we had after only the first 5 Sprints

Disclaimer...
This is only one project!
Find 22 more at: http://bit.ly/NoEstimatesProjectsDB

Page 8
 #NoEstimates
Here is the result when we consider the data after just 3 sprints

Here is the result when we consider the data after just 5 sprints

Page 9
#NoEstimates

PS: the fact that counting the number of stories predicted the same output in both cases is just a
coincidence. It could as easily have predicted a different output.

What was the answer to the original experiment question: Which ”metric” is more accurate when compared to
what actually happened in the project?

Even though the Story Point estimation improved the prediction when considering 5, instead of 3 sprints, it is
clear that at least in this project, counting the number of stories was enough to predict very accurately (4%
deviation) the output of that project.

ONE MORE THING…

One more thing: when presenting #NoEstimates I am constantly challenged on one point. Skeptics point out
that only “bad estimates” are bad. There are ways to produce “good” estimates and I should focus on how to
improve estimation, not getting rid of it – they say. Fair enough. In that case let’s look at the track record in
the software industry. How well have we performed over the years? Certainly if estimates can be improved,
or done well, we will find examples that we should follow.

Here’s a chart with the evolving results of the Software Project survey known as the Chaos report.

Source: Software Estimation by Steve McConnell

The result: 80% of projects are consistently late or failed. And this data is over many years. This is not a
problem of yesteryear or a recent trend. It has always been this bad!

Page 10
 #NoEstimates
Another chart, the rate of failure and late delivery changes with project size.

The larger
the project,
the bigger
the
problem

Source: Software Estimation by Steve

McConnell
We, in the software industry, consider the correlation between project size and failure to be “self-evident” or
common-sense. Yet we continue to invest obscene amounts of money in large (and, therefore more likely to
fail) projects! We have learned nothing from history.

Below is a chart that visualizes a possible explanation for this failure of estimates. My argument here is that
estimates are a self-defeating process. So much so that we even have names for that self-defeating process.
We call it Parkinson's Law, or the Student Syndrome or Hofstadter's Law. Projects are late because we are
human, and estimating projects actually amplifies some of the patterns that lead projects to be late.

Likelihood of a project being on time

# of projects

Early
On Time More late

Page 11
#NoEstimates

“Work expands so as to fill the time available for its completion.”

Parkinson’s Law

“Students always study for their exams the night before the exam.”
Student Syndrome

“Projects always take longer than you expect, even when you take into account
Hofstadter's Law.”
Hofstadter’s Law

Below is another chart that illustrates this point clearly. This chart compares 17 projects ending between 2001
and 2003. You see two bars for each project. The first (left bar) depicts the lateness of a project (in %) when
compared to the estimated created when the project was first approved. The second bar depicts the lateness
of the a project when compared to the estimate created when all requirements were considered “final”. The
average lateness for this population of projects is 62%. Would you bet your company on a method that
would be wrong 62% on average? Many software companies are doing this, today!

CONCLUSION
Christopher Columbus was wrong about finding India by navigating west. And maybe I am also wrong. But
maybe in 500 years people will look back and understand why it is necessary to be wrong and experiment

Page 12
 #NoEstimates
to evolve our industry and profession, not just to stick to what we already know does not work well! Estimates
do not work well. Just look at the evidence!

My message to you is simple: Agile is alive and kicking. Take #NoEstimates and experiment! Be Agile!

Contact
You can follow me on twitter: @duarte_vasco

Find out more about my experience here: LinkedIn profile

Or email me at: Vasco.Duarte@oikosofy.com

Know more about #NoEstimates

You can find out more about #NoEstimates in the following resources:

Blog posts
The #NoEstimates How-to: Where I explain a bit more than what is in the slideshare

How #NoEstimates helps teams focus on Value: Here I present how #NoEstimates helps us focus on Value, and
how that happens. I’ve included a short video about a similar approach by Arlo Belshee.

YouTube videos
Story points considered harmful: An exposition of why Story Points are the wrong method to estimate Agile
and Scrum projects.

How to estimate better a software project, the #NoEstimates view: We have better ways to estimate
software projects. In this video I share my insights and a few cases.

A Hangout where me and other Agile Coaches explore the ideas behind #NoEstimates: Some of the theories
and experiences on which #NoEstimates is built

Other blogs that have articles on #NoEstimates

Woody Zuill’s blog

Neil Killick’s blog

Henri Karhatsu’s blog

Page 13