Agent memory is the durable, source-aware layer that lets an AI agent reuse facts across sessions without stuffing every fact into the prompt. Good memory records where a fact came from, who can see it, whether a newer fact replaced it, and how a human can inspect or delete it.
What is agent memory?
Agent memory is not a single database table or a feature checkbox. It is the set of write, store, retrieve, inspect, and delete behaviors that let an agent carry useful context forward. The memory layer sits outside the model and serves the agent through an interface such as an API, CLI, or Model Context Protocol server.
The reason this layer exists is simple: model calls have boundaries. OpenAI’s API docs describe conversation state as something applications manage explicitly, either by sending prior context or using stored response state (OpenAI conversation state guide). Framework docs make the same architectural split by separating short-term thread state from longer-lived memory that must be stored and retrieved by the application (LangChain memory concepts).
For builders, the lesson is that memory is an application architecture problem. The model may reason over context, but your product decides what should be written, what should be retrieved, and what evidence should be exposed back to the user.
How is memory different from a context window?
A context window is temporary working space. It is where the model reads instructions, user messages, tool results, and retrieved snippets for a single answer or a bounded conversation. It is valuable, but it is not durable by itself.
Anthropic describes context engineering as the work of choosing what information goes into the model’s context so an agent can act effectively (Anthropic context engineering). Agent memory is one source for that context. It decides what survives beyond the current run and what should be recalled later.
The difference matters because teams often try to solve forgetting by sending more text. That can help temporarily, but it does not answer the deeper questions:
- Which facts are worth carrying forward?
- Which source proves the fact?
- Which user or tenant can access it?
- What happens when the fact changes?
- Can a human inspect why it was retrieved?
- Can a human ask the system to forget it?
Those are memory questions, not prompt-length questions.
How is memory different from RAG?
Retrieval-augmented generation, or RAG, retrieves source material and passes it to the model. Agent memory can use RAG internally, but memory is broader. RAG usually starts with “find relevant chunks.” Memory starts with “what should be remembered, under what scope, and with what lifecycle?”
| Layer | Primary job | What it does not solve alone |
|---|---|---|
| Context window | Holds the current working set for one answer or conversation | Persistence, source lifecycle, deletion, cross-tool recall |
| RAG | Retrieves source material for a query | Write policy, memory conflict handling, supersession, inspection |
| Fine-tuning | Changes model behavior through training | Per-user facts, source citations, deletion of specific memories |
| Agent memory | Stores and recalls scoped evidence across sessions | It still needs good retrieval, evaluation, and product controls |
LangChain’s memory docs make a similar distinction between short-term state inside a thread and long-term memory stored across conversations (LangChain memory concepts). Databricks’ stateful agent docs also treat memory as external state an agent can persist and retrieve, not as a synonym for one prompt (Databricks stateful agents).
What types of memory do production agents need?
The vocabulary varies by framework, but the useful taxonomy is stable:
| Type | What it stores | Example |
|---|---|---|
| Working memory | Immediate task state inside one run | ”The user is debugging the billing route right now.” |
| Episodic memory | Events or interactions | ”On Monday, the user decided to delay the Stripe dashboard task.” |
| Semantic memory | Facts about the world, product, or user | ”The API host is engine.answeragent.ai.” |
| Procedural memory | Preferred ways to do work | ”Use the project smoke test before closing launch issues.” |
| Source memory | Documents, tickets, pages, and artifacts with provenance | ”This answer came from docs/gtm/CONTENT_PLAN.md.” |
LangChain’s overview of agent memory uses semantic, episodic, and procedural memory as practical categories for long-running agents (LangChain memory for agents). Research surveys of LLM-agent memory systems also separate memory by lifetime, content type, and retrieval/update mechanism (survey of memory in LLM agents).
For production, the source memory row is easy to underrate. If the agent cannot point back to the document, page, ticket, or transcript that justified an answer, the user cannot debug the answer. That is why inspectable agent memory matters.
What should happen when an agent writes memory?
A write path is where memory becomes a product rather than a search index. The system should decide whether an observation deserves to be remembered, attach source and scope metadata, and preserve enough lineage to explain the memory later.
A minimal write path should capture:
- Source: the document, transcript, ticket, URL, or tool result that produced the memory.
- Scope: tenant, workspace, user, library, or client boundary.
- Type: semantic fact, event, preference, instruction, or source artifact.
- Currency: whether this memory is current, superseded, draft, or stale.
- Evidence: citation spans, document IDs, artifact IDs, or line references.
- Deletion handle: the key that lets the system remove the memory and its derived artifacts later.
This is where memory differs from “just store chat history.” Chat history may be a source, but production memory needs deliberate write rules.
What should happen when an agent reads memory?
Read paths should be source-aware. A useful memory response should include the recalled fact, the source that supports it, and enough retrieval metadata for a developer or user to understand why it appeared.
The common shape is:
- Receive a question or tool request.
- Build a scoped retrieval query.
- Retrieve candidates from source memory.
- Re-rank or filter candidates.
- Return the evidence with citations.
- Let the model answer from that evidence.
- Log what was retrieved so teams can inspect or evaluate misses.
This does not require publishing an accuracy number before you have a reproducible evaluation set. It does require the product to be honest about what it can inspect today and what remains roadmap work.
What makes memory trustworthy?
Trustworthy memory is memory you can audit. The important dimensions are:
| Requirement | Why it matters |
|---|---|
| Inspectability | Users can see what was remembered and why it was retrieved. |
| Supersession | Changed facts do not compete silently with old facts. |
| Deletion posture | Users can request removal of specific memories and derived source artifacts. |
| Tenant isolation | One tenant’s facts do not leak into another tenant’s answers. |
| Portability | Teams can move their memory layer between agents and clients. |
| No training by default | Product memory should not imply model training on customer data. |
The deletion dimension has a legal and product side. GDPR Article 17 defines the right to erasure for personal data in qualifying circumstances (GDPR Article 17). The EU AI Act also includes logging and record-keeping obligations for high-risk systems (Regulation (EU) 2024/1689). Even when your agent memory product is not a high-risk system, those regulatory patterns set expectations: memory needs traceability and lifecycle controls.
Where does Answer Engine fit?
Answer Engine is the memory and context layer for agents that need grounded, tenant-isolated recall from real sources. It exposes memory through MCP so Claude Code, Cursor, and other MCP-compatible clients can use the same memory without one framework lock-in.
The honest current wedge is not “the model magically improves itself.” The wedge is simpler and more useful: persistent, inspectable, source-aware memory for agents, with roadmap-labeled work for deeper inspection and evaluation. The product should be judged by whether it can answer from your sources, show citations, keep tenant boundaries, and avoid making unverified benchmark claims.
How should you evaluate an agent memory system?
Use this checklist before you buy or build:
- Can it ingest the sources your agent actually uses?
- Can your agent call it through MCP, API, or CLI without rewriting the agent stack?
- Can you inspect what was retrieved for an answer?
- Can it distinguish current facts from superseded facts?
- Can it delete a memory and its derived artifacts?
- Can it export or migrate memory later?
- Does it avoid claiming a benchmark number without a reproducible harness?
- Does it keep tenant and user scope in the retrieval layer?
If a system only answers “we have a vector database,” you are evaluating retrieval infrastructure, not memory.
What should you read next?
Read Inspectable Agent Memory to learn the audit surface, Supersession-Aware Memory for changing facts, and Provable Forgetting for deletion and compliance. Keep the Agent Memory Glossary open as the shared vocabulary.