Short-Term Memory: What an Agent Actually Needs Within One Run

Short-term memory is everything an agent must hold for the current session so the next decision is grounded in what already happened.

It is not a vector store of lifelong facts.

It is the working set: transcript turns, intermediate plans, and reusable tool results that keep one multi-turn run coherent.

Summary

Within one run, agents need three distinct short-term stores: conversation history (what was said), scratchpad / working state (task progress), and tool-result cache (observations you may reuse without re-calling tools).
Insight: Stuffing everything into one growing message list burns tokens, leaks noise into decisions, and makes multi-tenant bugs hard to see. Separating stores makes truncation, security, and debugging intentional.
Key Concepts: session, context window, message transcript, scratchpad, tool observations, TTL, isolation key.
When to Use: Multi-turn chat, multi-step tool loops, or any agent that pauses between HTTP requests and must resume with prior state.
Limitations/Trade-offs: Short-term memory dies with the session (or TTL). Facts that should survive across days belong in long-term memory or durable product databases.
Related Topics: context budgets, Redis session state, summarization, session isolation.

Foundations

A run or session is a bounded unit of work: one user conversation, one support ticket thread, or one autonomous job with a session id.

Everything the model sees on the next model call is working memory.

That working set is finite because the model has a context window (input plus output tokens per call).

Three buckets of short-term data solve different jobs:

Conversation history - user and assistant messages (and often system policy). Needed so the agent does not re-ask what the user already said.
Scratchpad / task state - structured progress: checklist of steps, chosen plan, intermediate variables (ticket_id, files_touched, confidence). Needed so multi-step work does not rely on the model re-deriving status from a messy transcript.
Tool-result cache - recent observations keyed by tool name and arguments (or a content hash). Needed so you avoid redundant expensive calls and can re-inject a compact form of a large result.

These are not three chat roles.

They are three storage concerns that you may serialize into messages when you call the model, but you should manage them separately outside the prompt when possible.

Long-term memory (preferences, CRM facts, embeddings) is a different product surface.

If a fact must matter next week for a different session, design an extraction and retrieval path.

If a fact only matters until the user closes the chat, keep it short-term.

Mechanics & Interactions

How the runtime uses short-term memory

Typical turn flow:

Load session by (tenant_id, user_id, session_id) - never by user message content alone.
Assemble context: system policy + optional long-term snippets + recent history + scratchpad summary + any forced tool results.
Call the model (and tools as needed).
Append new messages; update scratchpad fields; cache tool results with TTL.
Enforce budget: drop, compress, or summarize oldest material before the next call.

# Conceptual session shape (not a framework API)
session = {
    "messages": [],      # conversation history
    "scratchpad": {},    # structured task state
    "tool_cache": {},    # key -> {value, expires_at}
}

Conversation history

History is usually a list of role-tagged messages the chat API accepts.

Include tool-call and tool-result messages when the provider requires them for multi-turn tool use.

Do not treat every log line as history.

Debug traces, raw HTML, and multi-megabyte blobs belong in storage with a short pointer in the prompt, not as full pastes every turn.

Scratchpad

The scratchpad is authoritative structured state.

The model may propose updates; your runtime validates and writes them.

Good scratchpad fields are small, typed, and goal-oriented: goal, steps_done, steps_remaining, blockers, artifacts.

Bad scratchpads are free-form diaries that grow without bound.

Tool-result caching

Cache when:

The tool is expensive (search, browser, heavy SQL).
Results are stable for the session window (product catalog snapshot, repo file content at a commit).
The same args may be requested again after summarization dropped the full observation from history.

Invalidate when the world may have changed (user uploaded a new file, write tool succeeded, TTL expired).

Interaction with the context window

Only a projection of these stores enters each model call.

You might send the last N turns of history, a one-screen scratchpad JSON, and one cached tool snippet relevant to the current step.

The full history can live in Redis or a database while the model sees a budget-safe view.

Advanced Considerations & Applications

Bounded agency and plan-and-execute

Plan-and-execute agents benefit from a scratchpad that holds the plan and step index separately from chat chatter.

ReAct-style loops still need history of observations, but you can collapse early observations into "done: searched X, found Y" on the scratchpad and drop the raw pages.

Multi-agent sessions

Handoffs should pass a package: goal, constraints, scratchpad snapshot, and a short transcript excerpt - not the entire unbounded log.

Each specialist may keep its own scratchpad keyed under the same session id with a role prefix.

Security

Short-term memory often contains PII, secrets from tool output, and other tenants' data if isolation fails.

Encrypt at rest when required, redacted logs, and hard keying by tenant are part of short-term design, not polish.

Approach	Strength	Weakness	Best Fit
All state in the prompt	Simple, no external store	Hits context limits fast; hard multi-instance	Demos, single-process CLIs
History list only	Natural chat continuity	Weak for multi-step plans; noisy	Short Q&A agents
History + scratchpad	Clear task progress	Extra schema to design	Multi-step workflows
History + scratchpad + tool cache	Fewer redundant tool calls	Cache invalidation bugs	Research, support, data agents
Durable DB as only store	Survives process restarts	Latency; must still project into context	Multi-worker production

Common Misconceptions

"Short-term memory is just the chat log." The log is one component. Progress state and caches matter as much for multi-step agents.
"Bigger context windows remove the need for memory design." Larger windows delay the pain. Cost, latency, and distraction still grow with every unused token.
"Vector RAG replaces short-term memory." RAG retrieves durable knowledge. Short-term memory tracks this conversation and this plan.
"If it is in Redis it is long-term memory." Storage technology is not the taxonomy. TTL session keys are short-term; cross-session user profiles are long-term.
"The model will remember if I said it once." The model only "remembers" what you re-send (or summarize) in a later call. Your store is the source of truth.

FAQs

What is the minimum short-term memory for a useful multi-turn agent?

A session id, a message list you append every turn, and a max history policy (count or tokens). Add a scratchpad when tasks span more than a few tool steps.

Should tool results live in the message list or a cache?

Both can: keep a compact observation in history for the next decision, and keep the full payload in a cache or object store if you may need it again.

How is short-term memory different from the system prompt?

The system prompt is relatively stable policy. Short-term memory is session-specific and changes every turn.

Do I need Redis for short-term memory?

No. In-memory dicts work for single-process demos. Redis (or equivalent) helps when multiple workers, restarts, or shared sessions appear.

When should I summarize instead of truncate?

Truncate when early turns are disposable small talk. Summarize when early turns hold goals, constraints, or decisions you still need.

Can scratchpad state replace conversation history?

Not for user-facing chat. Users expect conversational continuity. Scratchpads complement history for machine-readable progress.

Where do "thinking" or reasoning traces go?

Usually not into long-lived user history. Keep them in ephemeral traces for debug, subject to provider and privacy rules.

How do frameworks like LangGraph fit this model?

Graph/checkpoint state is a form of short-term (and sometimes durable) session memory. Map their channels onto history vs scratchpad vs cache deliberately.

What should never enter short-term session state?

Long-lived secrets (prefer vaults), other tenants' data, and unbounded binary blobs without references and size caps.

How do I test short-term memory?

Simulate multi-turn scripts: assert messages append, budgets trim, scratchpad updates, and session A cannot read session B.

Is "one run" the same as one model call?

No. One run is many model and tool calls under one session until a stop condition or user ends the conversation.

When does short-term become long-term?

When you deliberately extract facts meant to survive session end and retrieve them later under a user or org scope.

Short-Term Memory Basics - first multi-turn history agent
Managing Conversation History Within a Context Budget - trim and budget recipes
Scratchpad Patterns - structured working memory
Session Isolation - multi-tenant scoping
Long-Term Memory: What Should Survive Between Agent Sessions - durable memory boundary

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.

It is not a vector store of lifelong facts.

It is the working set: transcript turns, intermediate plans, and reusable tool results that keep one multi-turn run coherent.

Summary

Within one run, agents need three distinct short-term stores: conversation history (what was said), scratchpad / working state (task progress), and tool-result cache (observations you may reuse without re-calling tools).
Insight: Stuffing everything into one growing message list burns tokens, leaks noise into decisions, and makes multi-tenant bugs hard to see. Separating stores makes truncation, security, and debugging intentional.
Key Concepts: session, context window, message transcript, scratchpad, tool observations, TTL, isolation key.
When to Use: Multi-turn chat, multi-step tool loops, or any agent that pauses between HTTP requests and must resume with prior state.
Limitations/Trade-offs: Short-term memory dies with the session (or TTL). Facts that should survive across days belong in long-term memory or durable product databases.
Related Topics: context budgets, Redis session state, summarization, session isolation.

Foundations

A run or session is a bounded unit of work: one user conversation, one support ticket thread, or one autonomous job with a session id.

Everything the model sees on the next model call is working memory.

That working set is finite because the model has a context window (input plus output tokens per call).

Three buckets of short-term data solve different jobs:

Conversation history - user and assistant messages (and often system policy). Needed so the agent does not re-ask what the user already said.
Scratchpad / task state - structured progress: checklist of steps, chosen plan, intermediate variables (ticket_id, files_touched, confidence). Needed so multi-step work does not rely on the model re-deriving status from a messy transcript.
Tool-result cache - recent observations keyed by tool name and arguments (or a content hash). Needed so you avoid redundant expensive calls and can re-inject a compact form of a large result.

These are not three chat roles.

They are three storage concerns that you may serialize into messages when you call the model, but you should manage them separately outside the prompt when possible.

Long-term memory (preferences, CRM facts, embeddings) is a different product surface.

If a fact must matter next week for a different session, design an extraction and retrieval path.

If a fact only matters until the user closes the chat, keep it short-term.

Mechanics & Interactions

How the runtime uses short-term memory

Typical turn flow:

Load session by (tenant_id, user_id, session_id) - never by user message content alone.
Assemble context: system policy + optional long-term snippets + recent history + scratchpad summary + any forced tool results.
Call the model (and tools as needed).
Append new messages; update scratchpad fields; cache tool results with TTL.
Enforce budget: drop, compress, or summarize oldest material before the next call.

# Conceptual session shape (not a framework API)
session = {
    "messages": [],      # conversation history
    "scratchpad": {},    # structured task state
    "tool_cache": {},    # key -> {value, expires_at}
}

Conversation history

History is usually a list of role-tagged messages the chat API accepts.

Include tool-call and tool-result messages when the provider requires them for multi-turn tool use.

Do not treat every log line as history.

Debug traces, raw HTML, and multi-megabyte blobs belong in storage with a short pointer in the prompt, not as full pastes every turn.

Scratchpad

The scratchpad is authoritative structured state.

The model may propose updates; your runtime validates and writes them.

Good scratchpad fields are small, typed, and goal-oriented: goal, steps_done, steps_remaining, blockers, artifacts.

Bad scratchpads are free-form diaries that grow without bound.

Tool-result caching

Cache when:

The tool is expensive (search, browser, heavy SQL).
Results are stable for the session window (product catalog snapshot, repo file content at a commit).
The same args may be requested again after summarization dropped the full observation from history.

Invalidate when the world may have changed (user uploaded a new file, write tool succeeded, TTL expired).

Interaction with the context window

Only a projection of these stores enters each model call.

You might send the last N turns of history, a one-screen scratchpad JSON, and one cached tool snippet relevant to the current step.

The full history can live in Redis or a database while the model sees a budget-safe view.

Advanced Considerations & Applications

Bounded agency and plan-and-execute

Plan-and-execute agents benefit from a scratchpad that holds the plan and step index separately from chat chatter.

ReAct-style loops still need history of observations, but you can collapse early observations into "done: searched X, found Y" on the scratchpad and drop the raw pages.

Multi-agent sessions

Handoffs should pass a package: goal, constraints, scratchpad snapshot, and a short transcript excerpt - not the entire unbounded log.

Each specialist may keep its own scratchpad keyed under the same session id with a role prefix.

Security

Short-term memory often contains PII, secrets from tool output, and other tenants' data if isolation fails.

Encrypt at rest when required, redacted logs, and hard keying by tenant are part of short-term design, not polish.

Approach	Strength	Weakness	Best Fit
All state in the prompt	Simple, no external store	Hits context limits fast; hard multi-instance	Demos, single-process CLIs
History list only	Natural chat continuity	Weak for multi-step plans; noisy	Short Q&A agents
History + scratchpad	Clear task progress	Extra schema to design	Multi-step workflows
History + scratchpad + tool cache	Fewer redundant tool calls	Cache invalidation bugs	Research, support, data agents
Durable DB as only store	Survives process restarts	Latency; must still project into context	Multi-worker production

Common Misconceptions

"Short-term memory is just the chat log." The log is one component. Progress state and caches matter as much for multi-step agents.
"Bigger context windows remove the need for memory design." Larger windows delay the pain. Cost, latency, and distraction still grow with every unused token.
"Vector RAG replaces short-term memory." RAG retrieves durable knowledge. Short-term memory tracks this conversation and this plan.
"If it is in Redis it is long-term memory." Storage technology is not the taxonomy. TTL session keys are short-term; cross-session user profiles are long-term.
"The model will remember if I said it once." The model only "remembers" what you re-send (or summarize) in a later call. Your store is the source of truth.

FAQs

What is the minimum short-term memory for a useful multi-turn agent?

A session id, a message list you append every turn, and a max history policy (count or tokens). Add a scratchpad when tasks span more than a few tool steps.

Should tool results live in the message list or a cache?

Both can: keep a compact observation in history for the next decision, and keep the full payload in a cache or object store if you may need it again.

How is short-term memory different from the system prompt?

The system prompt is relatively stable policy. Short-term memory is session-specific and changes every turn.

Do I need Redis for short-term memory?

No. In-memory dicts work for single-process demos. Redis (or equivalent) helps when multiple workers, restarts, or shared sessions appear.

When should I summarize instead of truncate?

Truncate when early turns are disposable small talk. Summarize when early turns hold goals, constraints, or decisions you still need.

Can scratchpad state replace conversation history?

Not for user-facing chat. Users expect conversational continuity. Scratchpads complement history for machine-readable progress.

Where do "thinking" or reasoning traces go?

Usually not into long-lived user history. Keep them in ephemeral traces for debug, subject to provider and privacy rules.

How do frameworks like LangGraph fit this model?

Graph/checkpoint state is a form of short-term (and sometimes durable) session memory. Map their channels onto history vs scratchpad vs cache deliberately.

What should never enter short-term session state?

Long-lived secrets (prefer vaults), other tenants' data, and unbounded binary blobs without references and size caps.

How do I test short-term memory?

Simulate multi-turn scripts: assert messages append, budgets trim, scratchpad updates, and session A cannot read session B.

Is "one run" the same as one model call?

No. One run is many model and tool calls under one session until a stop condition or user ends the conversation.

When does short-term become long-term?

When you deliberately extract facts meant to survive session end and retrieve them later under a user or org scope.

Short-Term Memory Basics - first multi-turn history agent
Managing Conversation History Within a Context Budget - trim and budget recipes
Scratchpad Patterns - structured working memory
Session Isolation - multi-tenant scoping
Long-Term Memory: What Should Survive Between Agent Sessions - durable memory boundary

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.

Short-Term Memory: What an Agent Actually Needs Within One Run

Search Documentation

Short-Term Memory: What an Agent Actually Needs Within One Run