Memory and context: remember the useful parts, retire the rest

Name the kinds of state

Request context is the material available to the model for one response. Conversation history is a stored sequence of turns that may be included in later requests. Durable memory stores selected information across sessions. None of these necessarily changes the base model's parameters.

Working state may include an unresolved order ID, a pending clarification, and the latest tool result. A user preference such as concise answers has different retention needs from an order status. The latter can become stale quickly and should normally be refreshed from its system of record.

Build a budget before the context overflows

Reserve space for trusted instructions, the current question, essential state, relevant evidence, and the expected output. Measure with the selected model's tokenizer or documented accounting tools. A rough character estimate may help planning, but should not be presented as an exact token count.

When history grows, consider keeping recent turns and a checked summary of older context. Preserve unresolved questions, explicit user choices, source references, and important constraints. Do not silently drop the very instruction that limits an agent's authority. If essential information cannot fit, ask a focused clarification or reduce the task.

Treat summaries as fallible records

A summary can turn “the customer asked whether delivery takes five days” into “delivery takes five days.” That changes a question into a fact. Keep attribution and uncertainty in the summary. For important state, structured fields are easier to inspect than a free-form paragraph.

{
  "user_id": "demo-alice",
  "preference": "concise_answers",
  "source": "explicit_user_request",
  "created_at": "2026-09-21T09:00:00Z",
  "expires_at": "2026-10-21T09:00:00Z"
}

The dates are an illustrative retention choice, not a prescribed policy. The application must enforce ownership and expiry when reading as well as writing. A memory record should be editable or removable according to the product's actual controls.

Retrieve memory without mixing people or authority

Semantic retrieval can find relevant prior information, but scope it to the correct user and purpose before it reaches a prompt. Shared caches and vector stores need the same access discipline as ordinary databases. A stale remembered preference should not override a new explicit request.

Avoid storing every conversation detail just because storage is cheap. Select information with a clear future use, record where it came from, and define when it becomes invalid. Test a user correction, a expired record, two users discussing similar orders, and a long conversation that forces summarization. Useful memory improves continuity without converting guesses into permanent facts.

References and further reading

Use these primary references for deeper study and current API details. Examples in this lesson use fictional Northstar data.

• MCP architecture — architecture