Estimation Is a Core Competency

A few uncomfortable truths, and keys for success, for people who build software.

Illustration by Stephen Carlson
  • The more pressure you’ll feel to ship on time.
  • The more pain you’ll feel watching deadlines whiz by unmet.
  • The more complex or vague tasks you will be asked to estimate.
  • The better you’ll want to get at giving estimates and expressing how certain you are about them.

The Mistake Many Engineers Make

There’s a certain type of engineer — I have been this person! — who is technically competent and experienced enough to know how to complete common engineering tasks, loves technology and shipping software, and wants to please their boss. However, this responsible, productive, and lovely engineer is also so ruthlessly optimistic, they err too often on the side of “if all goes well, we can have this done this week!” We’ll call this person the “This Should Be Easy” engineer.

Uncomfortable Truths

An estimate is:

  • an informed guesstimate.
  • a commitment to complete the task.

I’ve always been far too optimistic in my estimates, and it can put a lot of stress into your life, especially when you are a young programmer without the experience and self-confidence to tell bosses uncomfortable truths.

Stack Overflow user RB

So what do you do? There’s plenty of formulaic advice around making your estimates more realistic, like: “Take a guess at how long you think something will take, then triple it.” Sorry pal, that doesn’t work. When you give an estimate, you have to justify it, and “I tripled it” doesn’t fly. There are no tricks or shortcuts for better estimation.

xkcd on estimating projects

1. Make It Smaller

The most effective way to rationalize an estimate is to calculate it as the sum of its parts. Start by reducing the task at hand down to its smallest pieces and estimate each piece.

T-shirt sizing tickets, Postlight style.

2. Narrow the Cone of Uncertainty

The more uncertain software requirements are, the more uncertain time and cost estimates to build that software will be. Certainty increases over time, as software requirements become clear. The Cone of Uncertainty is a useful project management concept that you can use to rationalize a wider, less certain estimate range. Frame your estimate certainty around where the requirements are in the cone. The more decisions get made, the narrower the cone becomes, and the more accurate an estimate becomes.

“Software organizations routinely sabotage their own projects by making commitments too early in the Cone of Uncertainty. If you commit at Initial Concept or Product Definition time, you will have a factor of 2x to 4x error in your estimates.”

— Steve McConnell, Software Estimation: Demystifying the Black Art

3. Err on the Side of Overestimating

Many engineers are naturally optimistic about what’s possible to accomplish with technology quickly. Related: the software industry suffers from an epidemic of missed ship dates and projects that run over budget. You never hear about software projects that ship early. The success state is on time, and the failure state is late.

Watch Out For “Just”

Once upon a time I worked on a software team that had Slack set up so that anytime anyone said the word “just” a bot responded, “just.” It was a reminder: when someone says “just,” there’s a chance they are minimizing what something entails. Train your conservative estimation ear to hear phrases like:

  • “Just write a script”
  • “Just cache it”
  • “Just create a batch process”

That Will Take Longer Than You Think

Keep a working list of things that almost always take longer to implement than you think they will at first blush. Be disciplined about it. Every time you come out of a feature or ticket thinking, “Wow, I didn’t think that would take so long,” add it to your list.

  • Internationalization: Translation support across alphabets with various directionality, localization, character encoding, and geo-IP targeting introduce complexity across the stack, from the server architecture to the wireframes.
  • Syncing: Keeping multiple copies of a changing dataset up-to-date— whether that’s providing offline support on the client, or setting up server-side caching — and propagating those changes quickly and smartly so no source is out of date is non-trivial.
  • Third-party libraries or APIs: There’s always the risk of an unexpected gotcha, performance problem, or security vulnerability with a third-party service or tool.
  • Sending mass email: Spam filters are highly-evolved and basically sentient at this point. This is why businesses like Mailchimp and Sendgrid exist.
  • DevOps: Mature cloud services and tooling make it really easy to say, “We’ll just spin this up on AWS.” (Just!) But despite incredible tools and services, devops can easily take more effort than you think (which is why managed hosting services are also a thriving business).

4. Back It Up with Research

Finally, when someone asks you how long something will take, don’t immediately respond with your top-of-mind hunch. No matter how experienced you are, every software task is unique, so take a beat and say, “Let me do a little bit of research and let you know.” When all else fails, try one of the following approaches to inform your guess:

  • Talk to other engineers: If anyone on your team or in your network has built something similar or has experience in an area you don’t, ask for their thoughts, or what they learned in their work. You’ll be surprised at how productive a quick chat with another engineer can shed light on things you hadn’t thought of.
  • Rubber-duck it: Just as rubber-ducking can help with debugging, try explaining the thought process behind your estimate to another engineer. “I think it will take 3 months. Here’s my breakdown” can increase the accuracy of your estimate and spread good practices around building rationale behind estimates within your team.

Technology, culture, representation, and self-improvement. Once upon a time I started Lifehacker.