KV cache: why repeating a prompt is cheaper than the first one

When a model generates text it computes a "key" and a "value" for every input token, and later attention steps depend on them. The KV cache holds those values so earlier tokens are never rebuilt for the next token. This dry technical mechanism sits behind one of the least-known discounts on your bill: if a prompt prefix is cached, the model does not recompute it — and providers charge correspondingly less.
The arithmetic is simple and merciless: the first answer pays the full prefill, while every later answer on the same prefix pays almost only for new tokens. On some providers a cache read costs up to 90 percent less than recomputing. The practical rule: keep the fixed part first and the variable part last — system prompt and tools stay byte-identical on top, and only the tail grows; one edit at the front burns the whole cache.
So the economic question for system design becomes: how much of my workload has a stable prefix? A support bot with one handbook, a summarizer with fixed instructions, a chat whose history only grows — all are cache candidates. The site’s pricing table lists input and output token prices side by side; before signing, compute the share of your bill the cache can rescue. The 2026 skill is not buying a cheaper model; it is designing a prompt that processes cheaper.