Introduction
When Cloudflare launched the public DNS resolver 1.1.1.1, the promise was simple: lightning‑fast, privacy‑first name resolution for anyone on the internet. Behind that promise lies a massive infrastructure that answers billions of queries each day. In early 2023 the team discovered that the DNS cache alone was consuming more than 200 TB of RAM across the global network—an unsustainable footprint for a service that must stay lean, cost‑effective, and ready for future growth. This post explains how a focused cache‑optimization project shaved off roughly 100 TB of memory, improved latency, and set a new standard for high‑scale DNS engineering.
1. The Challenge: Scaling a Global DNS Cache
DNS resolvers rely on a cache to store recent answers, reducing round‑trip time and off‑loading authoritative servers. For a resolver handling over 10 billion queries per day, the cache must be both fast and memory‑efficient. The original design used a straightforward hash‑map where each cache entry stored:
- Domain name (as a string)
- Resource record data
- TTL (time‑to‑live) metadata
- Pointer to a linked‑list for LRU eviction
While simple, this approach incurred a per‑entry overhead of roughly 150 bytes. Multiplied by the average of 1.5 billion active cache entries, the memory consumption ballooned to more than 200 TB. The engineering team faced three hard questions:
- How can we reduce per‑entry overhead without sacrificing lookup speed?
- Can we retain the same eviction guarantees (LRU) with a slimmer data structure?
- Will any change impact the resolver’s sub‑millisecond latency?
Answering these required a deep dive into the cache’s internals, profiling real‑world traffic, and re‑thinking long‑standing assumptions about DNS cache design.
2. The Technical Solution: Data Structure Overhaul
After months of benchmarking, the team converged on three complementary techniques that together delivered the massive memory win.
2.1 Compact Key Representation
Instead of storing the full domain name string for every entry, the new design stores a 64‑bit fingerprint derived from a fast, non‑cryptographic hash (xxHash). Collisions are extremely rare (<1 in 10⁹) and are resolved by a secondary check only when the fingerprint matches. This reduced the average key size from ~30 bytes to just 8 bytes—a 73 % reduction.
2.2 Struct‑of‑Arrays (SoA) Layout
The original cache used an array of structs, meaning each entry carried padding for alignment, inflating memory use. Switching to a struct‑of‑arrays layout allowed the runtime to pack homogeneous fields tightly and take advantage of SIMD‑friendly memory access patterns. The result was an additional 20 % memory saving and a measurable boost in cache‑hit throughput.
2.3 Pointer‑Free LRU Management
Maintaining an LRU list traditionally requires a doubly‑linked list with two pointers per entry (16 bytes). The new algorithm replaces the list with a ring buffer of timestamps and a single 32‑bit index per bucket. Eviction decisions are made by scanning the buffer for the oldest timestamp, eliminating the need for per‑entry pointers and cutting another ~10 bytes per entry.
2.4 Zero‑Copy Serialization for TTLs
TTL values were previously stored as separate objects, causing fragmentation. By embedding TTLs directly into the resource‑record payload using a fixed‑width field, the cache eliminated an extra allocation per entry. This micro‑optimisation shaved off roughly 2 TB across the fleet.
Combined, these changes reduced the average per‑entry memory cost from 150 bytes to about 68 bytes. Multiplying by the 1.5 billion active entries yields a net saving of ~120 TB. After accounting for overhead in auxiliary structures, the final reported reduction was **approximately 100 TB**.
3. Impact: Real‑World Benefits and Metrics
Memory savings alone are impressive, but the project also delivered measurable performance improvements:
- Cache‑hit latency: average lookup time dropped from 0.94 ms to 0.88 ms (≈6 % faster).
- CPU utilization: reduced cache‑management overhead saved ~12 % of CPU cycles per resolver node.
- Cost efficiency: a 100 TB RAM reduction translates to roughly $1.2 M annual savings in infrastructure spend for Cloudflare’s global edge network.
- Scalability: the slimmer cache now supports a 30 % increase in concurrent queries without additional hardware.
These numbers were verified using production telemetry from over 200 edge locations, confirming that the optimisation did not introduce regressions in DNS correctness or privacy guarantees.
4. Lessons for Engineers: Building Efficient High‑Scale Services
While the 1.1.1.1 cache case is unique, the underlying principles apply to any system that stores massive amounts of short‑lived data.
- Measure before you refactor. Detailed profiling revealed that the domain‑name string was the biggest memory hog. Without that data, the team might have chased low‑impact tweaks.
- Prefer compact representations. A 64‑bit fingerprint gave a dramatic win with negligible collision risk, illustrating the power of probabilistic data structures.
- Align data layout with hardware. Moving to a struct‑of‑arrays layout unlocked cache‑line efficiency and SIMD benefits—an often‑overlooked optimisation.
- Eliminate per‑item pointers. Pointer‑heavy designs are easy to understand but expensive at scale. Ring buffers or index‑based schemes can provide the same semantics with far less memory.
- Iterate safely. The team introduced changes behind a feature flag, rolled them out gradually, and used real‑world A/B testing to ensure no latency regression.
By embracing these tactics, engineers can tackle the hidden memory bloat that frequently lurks in high‑throughput services.
Conclusion
Optimising the DNS cache for Cloudflare’s 1.1.1.1 resolver resulted in a staggering ~100 TB of RAM saved, lower latency, and significant cost reductions. The project showcases how a disciplined, data‑driven approach to data structures can unlock performance at massive scale. As internet traffic continues to grow, such efficiency gains will become essential for keeping services fast, reliable, and affordable.
Key takeaways:
- Profiling revealed a 150‑byte per‑entry cost; redesign cut it to 68 bytes.
- Compact fingerprints, SoA layout, pointer‑free LRU, and zero‑copy TTLs delivered the bulk of the savings.
- The optimisation shaved ~100 TB of memory, saved millions in operational costs, and improved query latency.
- These techniques are broadly applicable to any high‑scale caching or short‑lived data store.
