Context rot is what happens when an agent has room for more tokens but the added material makes the answer worse. The fix is not a larger prompt by default. Curate evidence, retrieve source-linked facts, remove stale context, and move durable memory outside the working window until it is needed.
Last updated: June 20, 2026.
What is context rot?
Context rot is prompt degradation caused by poor context selection. The model may still be under its token limit, but the context is crowded with stale summaries, unrelated documents, duplicate turns, tool output, and conflicting instructions. The answer gets worse because the signal-to-noise ratio falls.
Anthropic’s context engineering guidance is useful because it treats context as an engineered input, not a dumping ground. The question is not “how much can fit?” It is “which information should the agent receive for this task?”
Why does context rot happen below the token limit?
Token capacity is not the same as relevance. A long prompt can contain three different versions of a policy, a stale ticket, a transcript summary that lost the source, and a current document buried near the end. The model has enough tokens, but not enough clean structure.
Common causes:
| Cause | What it looks like | Memory-aware fix |
|---|---|---|
| Stale facts | Old policy competes with new policy | Supersede old source artifacts |
| Duplicate context | Same decision appears in many summaries | Store one source-linked memory |
| Unscoped retrieval | Agent sees other project or tenant facts | Apply scope before retrieval |
| Raw transcript stuffing | Every turn enters the prompt | Write durable decisions, not every message |
| Citation loss | Summary has no source id | Preserve source lineage and citation spans |
| Context bloat | Prompt includes low-value facts | Retrieve only task-relevant evidence |
OpenAI’s conversation state guide puts state management in the application. That means the app is responsible for deciding which prior state returns to the model. If the app blindly resends everything, context rot is expected.
How is context rot different from forgetting?
Forgetting means the fact was not available when the agent needed it. Context rot means the fact may be present, but it is diluted or contradicted by other material. The first is a persistence problem. The second is a selection and coherence problem.
They often combine. A team may paste a huge project summary into every session because the agent has no durable memory. The summary grows, gets stale, and starts to contradict itself. The system then both forgets specific source facts and suffers context rot from the workaround.
How does memory help?
Memory helps by keeping durable facts outside the prompt until retrieval says they matter. A source-aware memory layer can store “the billing migration was delayed” with source, timestamp, tenant scope, and lifecycle state. The context layer can retrieve that fact only when the agent asks about billing or migration.
That is different from pasting an entire meeting transcript into every prompt. The transcript remains a source artifact. The memory record becomes a precise handle with provenance.
LangChain’s memory docs separate short-term and long-term memory. Use that distinction operationally: short-term context is for the current task, while long-term memory is recalled only when useful.
How should you inspect context rot?
Inspect the prompt assembly pipeline:
- What user question triggered retrieval?
- Which source artifacts were candidates?
- Which candidates were ranked highest?
- Which candidates were current, superseded, or deleted?
- Which snippets entered the final prompt?
- Which citations appeared in the answer?
- Which useful sources were missing?
If you cannot answer those questions, you have an inspectability problem. The failure-mode catalog treats context rot as a trace problem because the fix depends on seeing what the agent actually saw.
What should you remove from context?
Remove repeated summaries, outdated instructions, raw logs that were already distilled into source-linked facts, and documents outside the tenant or library scope. Remove evidence that is relevant to a different user or time period. Remove old facts that have been superseded unless the user asks for history.
Do not remove source lineage. If the agent needs a current fact, include enough citation detail to prove where it came from.
What should you store instead?
Store compact, source-linked memories:
- Current decisions with source references.
- Supersession edges from old fact to new fact.
- Procedural rules that apply across sessions.
- Project preferences that are stable.
- Deletion tombstones that prevent stale recall.
Then retrieve those memories when the task calls for them. This is why Memory needs a write path is the architectural fix, while “make the prompt bigger” is only a temporary patch.
What should you read next?
Read Context Engineering vs Memory, Agent memory keeps growing, and RAG returns wrong results. For the resource version, use the Memory Failure-Mode Catalog.