Every engineering estimate has a ghost in it. Ask a team how long a feature will take, and they'll picture it going smoothly, name a number, and be wrong on the low side, again. This happens to honest, experienced teams. It's a well-documented cognitive bias, one that helped win a Nobel Prize, and it explains why software is almost always late.
Key Takeaways
- The planning fallacy (Kahneman & Tversky) is the systematic tendency to underestimate time, cost, and risk (planning fallacy).
- It holds across cultures, experience levels, and personality, and knowing about it doesn't cure it (PMI).
- The cause is taking an "inside view", imagining this specific project going well, instead of looking at how similar projects actually went.
- The fix is reference-class forecasting: estimate from the track record of similar past work.
What the Research Says
Daniel Kahneman and Amos Tversky named the planning fallacy in 1979: people consistently underestimate how long their own tasks will take, even with plenty of experience of similar tasks running long. Kahneman later expanded it, we underestimate time, cost, and risk while overestimating benefits (planning fallacy). The unsettling part is how stubborn it is. The bias holds across cultures, across personality types, and across experience levels, and simply knowing about it does not make it go away (PMI on the planning fallacy). Senior engineers are not immune; they underestimate with more confidence.
Why We Underestimate
Kahneman's explanation is the "inside view." When you estimate a project, you build a mental model of it going right: you list the steps, imagine each one working, and add them up. That model leaves out everything that isn't a step, the sick day, the dependency that wasn't ready, the "quick" change that took a week, the integration that surprised everyone. Real projects are dominated by exactly those unplanned events, and the inside view is blind to them by construction. So the estimate describes a project where nothing goes wrong, a project that has never existed (reference-class forecasting).
| Inside view (biased) | Outside view (calibrated) |
|---|---|
| "Here are the steps, they'll go fine" | "How did similar projects actually go?" |
| Ignores surprises | Prices surprises in from history |
| Optimistic, precise, wrong | Less precise, more accurate |
A Concrete Version
A team estimates a new integration at three weeks by listing the tasks and assuming each goes smoothly. Anyone who has shipped integrations knows how this ends: the third-party API is under-documented, the auth flow has an undocumented quirk, and a "quick" data migration eats four days. It ships in seven weeks. The team didn't lack skill; they estimated the version of the project where nothing surprised them. Their last three integrations also took roughly twice their estimate, which nobody looked at.
The Fix: Estimate From History
The reliable antidote is reference-class forecasting: instead of estimating this project from the inside, look at how a class of similar projects actually turned out and start from that (reference-class forecasting). If your last five features of this size took an average of six weeks, the honest estimate for the next one is around six weeks, whatever the task breakdown suggests. It feels less rigorous than a detailed bottom-up plan and comes out far more accurate, because it captures all the surprises you can't itemize in advance. This is the same reason we argue teams should measure delivery, not estimates: the track record tells the truth the plan won't.
What This Means for Planning
Two practical habits follow. Pad estimates toward your historical overrun instead of your optimistic breakdown, and keep a simple record of estimate-versus-actual so you have a reference class at all. And treat any plan that assumes everything goes right as fiction, because the research guarantees it will not. Experienced engineers who have been burned by the planning fallacy estimate more humbly and build in slack, which connects to why 100% utilization backfires: a plan with no room for surprises breaks on contact with the first one. See available engineers.
Frequently Asked Questions
What is the planning fallacy?
A cognitive bias, identified by Kahneman and Tversky, where people systematically underestimate the time, cost, and risk of their own projects, even when they have experience with similar work running long.
Why do experienced teams still underestimate?
Because the bias holds across experience levels, and knowing about it doesn't cure it. Experience often just adds confidence to the underestimate. It comes from taking an "inside view" that ignores unplanned events.
How do I estimate more accurately?
Use reference-class forecasting: base your estimate on how similar past projects actually turned out, not on a task-by-task breakdown of this one. Keep a record of estimate-versus-actual to build that reference class.
Should we just stop estimating?
Not necessarily, but treat estimates as ranges anchored in history rather than precise commitments. The track record of similar work predicts far better than a detailed plan of the current one.
The Bottom Line
Software runs late for a reason older than software: the planning fallacy makes us underestimate systematically, and experience doesn't fix it. The escape is to stop estimating from the inside and start from the record of how similar work actually went. Anchor plans in your real history, build in slack for the surprises you can't list, and your estimates stop being wishes.
Roberto Espinoza is CEO of Ruzora, which helps US startups hire pre-vetted senior LATAM engineers in 72 hours. See available engineers.
