Skip to content

Search across models, tools, services, companies, providers, APIs, plans, articles and news.

Context Rot: why a bigger window does not mean a bigger memory

Context Rot: why a bigger window does not mean a bigger memory

On the box of today’s models sits a dazzling number: a 1-million-token context window. Yet the experience of any developer who has worked with a coding agent for a long session says otherwise: the agent gets confused mid-task, forgets files read at the start and repeats early instructions. This gap between the box number and reality has an official name: Context Rot.

Research in 2026 has measured this precisely: attention is a finite budget and each extra token dilutes the rest. In practice, an advertised 128k window sometimes delivers only 20–40k tokens of effective context depending on the task; information in the middle of the window is recalled worse than the edges, and complex tasks degrade sooner than simple ones.

For the everyday user this means three things: restart long agent sessions every few steps and carry a context summary into the new one; reference critical files at the end of your prompt — models attend to the window’s edges more than its middle; and distrust "infinite context" marketing — the real metric is accuracy on your task, not the window number.

The industry is testing serious fixes: context engineering (automatic session pruning and summarization), selective retrieval instead of dumping everything into the window, and recursive architectures that process long documents in chunks. The 2026 race is less about "bigger windows" and more about "smarter use of the window".

Back to blog