There's a famous line, usually credited to Phil Karlton: there are only two hard things in computer science, cache invalidation and naming things. The joke lands because caching looks easy, store a result, serve it fast, and turns out to hide some of the most maddening bugs in software. Knowing what to cache, and when to throw it away, is a genuinely senior skill.
Key Takeaways
- "Two hard things: cache invalidation and naming things," attributed to Phil Karlton (Two Hard Things).
- Caching is often the cheapest large speedup available, and the easiest way to introduce subtle staleness bugs.
- The core tension is freshness vs performance: shorter TTLs are fresher but slower; longer ones are faster but staler (caching tradeoffs).
- Invalidation is hard because cached data can be changed from many places at once.
Why It's Hard
Storing a cached value is trivial. Knowing when that value has gone stale is the hard part (Two Hard Things). Cache invalidation is difficult because the underlying data can be updated from many different sources, and it's easy to forget which cached entries a given update should invalidate. You cache a user's profile; then their name gets changed by an admin tool, a billing job, and a support script, and only one of those remembered to bust the cache. Now two of your servers show the old name and one shows the new, and the bug only reproduces sometimes. Those "only sometimes" bugs are the expensive ones.
The Freshness/Performance Tension
Every caching decision trades freshness against speed, and there's no free answer (the latency-vs-complexity tradeoffs). A short time-to-live keeps data fresh but causes more cache misses, so you lose the speedup you cached for. A long TTL gives great hit rates but risks serving stale data. Write-through caching (updating cache and database together) keeps things fresh but adds latency to every write. Eventual consistency scales beautifully but tolerates a window of staleness. The right choice depends entirely on the data: a stale stock price is a bug; a stale "number of likes" is fine for a minute.
| Choice | Fresher | Faster |
|---|---|---|
| Short TTL | Yes | No (more misses) |
| Long TTL | No | Yes |
| Write-through | Yes | Slower writes |
| Eventual consistency | Eventually | Yes |
A Concrete Version
A team caches product prices to speed up their catalog, with a generous TTL because prices "rarely change." Then a flash sale drops prices, but the cache serves the old prices for another hour, so customers see one price on the listing and a different one at checkout. Support tickets and chargebacks follow. The fix wasn't to stop caching; it was to recognize that price is exactly the kind of data where staleness is a bug, so it needs active invalidation on change rather than a lazy TTL. The catalog description, which almost never changes and doesn't matter if slightly stale, can keep its long TTL.
The Honest Counterpoint
The dangers of caching lead some teams to avoid it, which is usually the bigger mistake. Caching is often the single highest-return performance optimization available, turning a slow, expensive database query into a microsecond lookup, and refusing to cache to dodge invalidation bugs means paying for slowness and load forever. The answer is to cache deliberately: cache the data that's read far more than it's written, be honest about how much staleness each piece of data can tolerate, and invalidate actively where staleness is a real bug. Caching is a scalpel, not something to fear or something to sprinkle everywhere.
What This Means for Teams
The reason "cache invalidation" became a running joke among engineers is that it looks trivial and repeatedly isn't, which makes it a decent proxy for experience. A senior engineer asks, before caching anything, "how stale can this get before it's a bug, and everywhere this data changes, will the cache know?" That habit of thinking through the failure mode before writing the happy path is the judgment we screen for in how to verify a senior engineer, and it pairs with the naming research, the other famous hard thing. See available engineers.
Frequently Asked Questions
Why is cache invalidation considered hard?
Because the underlying data can be updated from many different places, and it's easy to miss which cached entries an update should invalidate. The resulting staleness bugs are intermittent and hard to reproduce.
What's the tradeoff in caching?
Freshness versus performance. Short TTLs keep data fresh but cause more cache misses; long TTLs are faster but risk stale data. Write-through is fresh but slows writes; eventual consistency scales but tolerates staleness.
Should I avoid caching to dodge these bugs?
Usually not. Caching is often the highest-return performance optimization available. The better approach is to cache deliberately, matching each piece of data's staleness tolerance to the right strategy.
How do I decide what to cache?
Cache data that's read far more than written, and for each piece ask how much staleness is acceptable. Use active invalidation where staleness is a real bug (prices, permissions) and lazy TTLs where it isn't (descriptions, counts).
The Bottom Line
Caching is the cheapest large speedup in software and the source of some of its most maddening intermittent bugs, which is why "cache invalidation" earned its spot among computing's two hard things. Don't avoid caching; do it deliberately. Match each piece of data's staleness tolerance to a strategy, and invalidate actively wherever stale data would be a real bug.
Roberto Espinoza is CEO of Ruzora, which helps US startups hire pre-vetted senior LATAM engineers, with a vetted shortlist in 72 hours. See available engineers.
