Somewhere a dashboard says "85% test coverage" in green, and a team feels safe. The research says that number tells you a lot less than you think. A large study of real projects found almost no correlation between how much code your tests cover and how many bugs ship. Coverage measures how much code your tests touch, not whether they'd catch anything going wrong.
Key Takeaways
- A large Microsoft study found coverage has an insignificant correlation with post-release defects at the project level, and none at the file level (Microsoft Research).
- A separate study of unit testing and code quality found only none-to-weak correlations between coverage and defects (empirical study).
- Returns plateau past ~90% coverage; chasing 100% is mostly wasted effort.
- What you test, and whether the test would actually catch a failure, matters far more than the percentage.
What the Research Found
Coverage feels like it should predict quality: more tested code, fewer bugs. The data doesn't cooperate. A large-scale Microsoft Research study of open-source projects found that code coverage has an insignificant correlation with post-release defects at the project level and no correlation at the file level (Microsoft Research). A separate study of unit testing and code quality found only none-to-weak correlations between coverage and defects, which makes the assumed "tests cause quality" story look shaky (empirical study).
That's not "tests don't matter", they obviously do. It's that the percentage is a weak proxy. Coverage tells you a line got executed during a test. It says nothing about whether the test actually asserted the right thing, or would notice if that line started returning garbage. You can hit 100% coverage with tests that assert almost nothing.
Why the Number Misleads
A test can run a line of code and check nothing meaningful about it. Cover a function with a test that only asserts "it didn't throw," and you've bought a green number and near-zero protection. Worse, chasing a coverage target pushes people to write exactly those low-value tests, the ones that game the metric, so the number goes up while real safety doesn't. It's Goodhart's law again: once coverage is a target, it stops measuring quality.
| Coverage tells you | Coverage doesn't tell you |
|---|---|
| Lines were executed in a test | Whether the test asserts anything useful |
| How much code is touched | Whether it'd catch a real bug |
| A single easy-to-game number | Whether the important paths are tested |
What Actually Matters
Two things beat coverage percentage. First, test the risky and important paths deliberately, the payment logic, the auth check, the data migration, rather than spreading thin effort to hit a number. Second, respect diminishing returns: research shows quality metrics plateau past about 90% coverage, so the effort to go from 90% to 100% is usually better spent writing a few high-value tests for the scary parts. A focused suite that hammers the ten things that would actually hurt beats a 100% suite of assertions that assert nothing.
A Concrete Version
A team mandates 90% coverage, so engineers backfill tests on getters, setters, and trivial code to hit the number, because that's the cheapest coverage to buy. The dashboard goes green. Meanwhile the gnarly payment-retry logic has one shallow test that checks a happy path, and a subtle bug in its error handling ships anyway, fully "covered." The number said safe; the suite wasn't. Spend the same hours writing five hard tests for the retry edge cases instead, and coverage might read lower while the software is genuinely safer.
The Honest Counterpoint
Low coverage is still a red flag, so don't read this as permission to skip testing. Coverage is a useful floor: 20% on a critical service genuinely tells you something's wrong, and coverage tools are great at showing which important code has no tests at all. The mistake is treating it as a ceiling or a goal, optimizing the percentage instead of the protection. Use coverage to find untested risky code, not as the score you're trying to max out.
What This Means for Teams
Judging a test suite by its coverage number is easy; judging whether it tests the right things takes engineering judgment, which is exactly what senior engineers bring and juniors chasing a metric often don't. It's the same theme as why story points and velocity mislead: a number that's easy to game replaces the harder question of real quality. Measuring the right thing, and hiring people with the judgment to know what that is, beats optimizing a dashboard. See available engineers.
Frequently Asked Questions
Does higher test coverage mean fewer bugs?
Not reliably. A large Microsoft study found near-zero correlation between coverage and post-release defects. Coverage shows how much code your tests execute, not whether those tests would catch a real failure.
What coverage percentage should we aim for?
There's no magic number, and returns plateau past about 90%. It's better to test the risky, important paths thoroughly than to chase 100% with shallow tests.
Why is coverage a weak quality signal?
Because a test can execute a line without asserting anything meaningful about it. You can hit high coverage with tests that would never catch a bug, and chasing a target encourages exactly those.
Is test coverage useless then?
No. It's a useful floor for spotting critical code with no tests at all. The mistake is treating it as a goal to maximize rather than a tool to find untested risk.
The Bottom Line
Test coverage is a comfortable number that barely predicts quality; the research shows almost no link between coverage and shipped defects. What matters is testing the paths that would actually hurt, and whether your tests would catch a failure at all. Use coverage to find untested risk, not as the score you optimize.
Roberto Espinoza is CEO of Ruzora, which helps US startups hire pre-vetted senior LATAM engineers in 72 hours. See available engineers.
