Why Vary on User Agent Wrecks Caching

A server that changes its response based on the user agent must declare that variation to caches. Declaring it on this particular header has consequences that surprise people.

The declaration creates a cache key

Caches key entries on the request path plus whichever headers a response says it varies on. Adding a header to that key multiplies the number of possible entries.

For a header with few values this is harmless. For one with an effectively unbounded value space it is close to disabling the cache.

User agent strings vary by browser, version, platform, platform version and device. The distinct-value count across real traffic runs to very large numbers.

Cache hit rates collapse quietly

Nothing errors. Responses are still cached and still served; they are simply rarely reused, because the next request usually carries a slightly different string.

Origin load rises and edge caches fill with near-identical entries. The symptom looks like a capacity problem rather than a configuration one.

Because the change is a single header, it often arrives with a feature nobody associates with caching, which makes the cause hard to find.

Normalisation is the standard remedy

Edges commonly derive a small classification from the string and vary on that instead: a device class, a browser family, or a single mobile flag.

Two or three possible values keep the cache key small while still allowing the response to differ where it must.

The classification must be computed consistently everywhere, or the same client receives different entries depending on which node handled the request.

Client hints have the same hazard

Varying on a hint header creates the same fragmentation. High-entropy hints are precise, which is exactly what makes them poor cache keys.

The low-entropy mobile flag is a good key because it has two values. The platform hint is workable; an exact version hint is not.

The general rule is that a cache key should be the classification the response depends on, not the evidence used to derive it.

Avoiding the variation entirely

The cheapest fix is often to stop varying. Serving one response that adapts in the browser keeps a single cache entry and usually costs less than the fragmentation did.

Where variation is genuinely necessary, separate URLs are more transparent than an invisible header-based split and are easier to reason about at every layer.

Reserving header variation for the cases that truly require it keeps caching predictable, which matters more for most sites than any detection refinement.