Memory & State
Short-term context, long-term memory, and how agents remember across sessions.
By the end of this lesson, distinguish the layers of agent memory and choose what to persist versus recompute.
Agent 'memory' isn't one thing — it's layers with different lifetimes. Getting them right is what lets an agent hold a coherent conversation, recall past sessions, and still fit inside a finite context window.
First, see the whole system. Then we’ll open it up.
Memory layers
- Working memory is the current context window; short-term is the session history fed into it; long-term memory is a persistent store the agent retrieves from and writes salient facts back to.
flowchart TB work[Working: current context window] --> agent[Agent] short[Short-term: session history] --> work long[(Long-term: vector / DB store)] -->|retrieve relevant| work agent -->|write salient facts| long
Working memory is whatever currently fits in the context window. Short-term memory is the running conversation, which eventually overflows and must be summarized or truncated. Long-term memory persists across sessions in a store you retrieve from — usually RAG over past interactions and learned facts. The core engineering decision is what to persist and re-retrieve versus what to recompute.
This is where the abstraction starts leaking.
The context window is a hard wall, not elastic memory. As a conversation grows you must drop or compress history, and summarization silently loses detail. 'The agent forgot what I said earlier' is usually the window filling up, not a reasoning failure.
More context (longer history, more retrieved memories) improves continuity but costs tokens, latency, and can dilute the model's attention on what matters. Aggressive summarization saves cost but risks dropping the one detail that mattered.
Why must short-term memory eventually be summarized or truncated?
Because the context window is finite; a growing conversation overflows it, forcing compression or dropping of older history.