Advanced30 min

Agent Runtimes & State

The engine that runs the loop, persists state, and lets an agent pause, resume, and recover.

By the end of this lesson, explain what an agent runtime provides and why durable state is what makes agents reliable.

How deep?

How the pieces actually move.

The model doesn't run itself. A runtime (or harness) drives the loop: it assembles context, invokes the model, parses tool calls, executes tools, enforces limits and permissions, and persists what happened. The difference between a fragile demo and a production agent is almost entirely in the runtime, not the model.

First, the whole system

First, see the whole system. Then we’ll open it up.

Runtime around the model

The runtime builds context, invokes the model, parses the tool call, checks permissions and limits, executes the tool, and persists state before looping.

flowchart TB
  subgraph Runtime
    ctx[Build context] --> model[Invoke model]
    model --> parse[Parse tool call]
    parse --> guard[Check permissions/limits]
    guard --> exec[Execute tool]
    exec --> persist[(Persist state)]
    persist --> ctx
  end

The critical capability is durable execution: checkpointing state after each step so a long-running agent can survive a crash, a deploy, or a pause and resume exactly where it left off — instead of restarting and repeating side effects. This is the same reliability problem as sagas and idempotency: steps must be replay-safe, because a resumed agent may re-attempt the step it died on.

Abstraction leak

This is where the abstraction starts leaking.

Statelessness is the leak. Each model call is independent; the illusion of a continuous, remembering agent is entirely reconstructed by the runtime re-feeding prior state into every request. Lose or corrupt that state and the 'agent' has amnesia mid-task.

State

Durable, checkpointed runtimes give reliability and resumability at the cost of persistence infrastructure and the discipline of replay-safe (idempotent) steps. A purely in-memory runtime is simpler but loses everything on any interruption.

How it connects

Why must an agent runtime's steps be replay-safe?

Because durable execution may resume after a crash and re-attempt the step it died on; without idempotency, that repeats side effects.