Redis for Fast Short-Term Agent State

Use Redis as a shared, low-latency store for conversation history and scratchpad fields so any worker can continue a session after the previous request finished.

Summary

Serialize a session document (messages + scratchpad + optional tool cache metadata) under a tenant-scoped key, set a TTL, and load-modify-save on each turn with optimistic concurrency or a lock when concurrent writers exist.

Recipe

1. Run Redis locally or use a managed instance; enable AUTH and TLS in non-dev environments.
2. Choose a key layout: agent:stm:{tenant_id}:{user_id}:{session_id} (adjust separators to your house style).
3. Define a JSON schema for the session payload (version field included).
4. On each turn: GET → deserialize → append messages / update scratchpad → enforce size limits → SET with TTL refresh.
5. Use short TTLs for abandoned chats (hours to days) and explicit delete on user "clear chat."
6. Add metrics: hit rate, payload bytes, trim events, lock contention.

Working Example

from __future__ import annotations
 
import json
import os
import time
from typing import Any
 
import redis  # pip install redis
 
r = redis.Redis.from_url(os.environ.get("REDIS_URL", "redis://localhost:6379/0"), decode_responses=True)
 
SESSION_TTL_SEC = 60 * 60 * 24  # 24h; tune per product
 
 
def session_key(tenant_id: str, user_id: str, session_id: str) -> str:
    return f"agent:stm:{tenant_id}:{user_id}:{session_id}"
 
 
def new_session() -> dict[str, Any]:
    return {
        "v": 1,
        "messages": [
            {"role": "system", "content": "You are a brief, careful assistant."},
        ],
        "scratchpad": {},
        "updated_at": time.time(),
    }
 
 
def load_session(tenant_id: str, user_id: str, session_id: str) -> dict[str, Any]:
    raw = r.get(session_key(tenant_id, user_id, session_id))
    if not raw:
        return new_session()
    return json.loads(raw)
 
 
def save_session(tenant_id: str, user_id: str, session_id: str, session: dict[str, Any]) -> None:
    session["updated_at"] = time.time()
    key = session_key(tenant_id, user_id, session_id)
    r.set(key, json.dumps(session), ex=SESSION_TTL_SEC)
 
 
def append_turn(tenant_id: str, user_id: str, session_id: str, user_text: str, assistant_text: str) -> dict:
    sess = load_session(tenant_id, user_id, session_id)
    sess["messages"].append({"role": "user", "content": user_text})
    sess["messages"].append({"role": "assistant", "content": assistant_text})
    # Optional: call trim_to_budget(sess["messages"]) before save
    save_session(tenant_id, user_id, session_id, sess)
    return sess

For concurrent turns on the same session, wrap the read-modify-write in a Redis lock (SET key NX EX) or store a version and use WATCH/MULTI so two writers do not clobber each other.

Deep Dive

Why Redis fits short-term memory

Agent turns are chatty: every user message may hit a different app instance.

In-process dicts break under horizontal scale and deploys.

Redis gives sub-millisecond reads for multi-kilobyte session blobs and first-class TTL so memory reclaims itself when users leave.

What to store vs not store

Key design and isolation

Keys must include tenant and user (or an equivalent authz scope), not only a client-supplied session uuid.

If the client can guess session ids, missing scope checks become cross-tenant reads.

Authorize that the authenticated principal owns (tenant, user, session) before GET/SET.

TTL and refresh

Refresh TTL on activity so live chats do not expire mid-conversation.

Do not use infinite TTL for chat state unless you have a separate purge job.

Compliance may require shorter retention than product convenience wants - document the choice.

Size limits

Cap serialized session size (for example 256 KB-1 MB) and trim history before save.

Oversized values hurt Redis memory and request latency.

If you need more history, keep cold transcript in Postgres/S3 and only hot-tail in Redis.

Consistency patterns

LangGraph and similar runtimes may offer Redis checkpointers; still apply the same keying, TTL, and size rules underneath.

Gotchas

1. Session id only keys. session:{uuid} without tenant scope is a multi-tenant footgun.
2. No TTL. Redis fills with zombie chats until OOM policies surprise you.
3. Storing API keys in session JSON. Never; inject secrets at runtime from env/secret managers.
4. Unbounded message append. Always trim or summarize before SET.
5. Race on parallel tabs. Two browser tabs can interleave turns; lock or serialize per session.
6. decode_responses and binary. Be consistent; tool payloads with bytes need base64 or external storage.
7. Hot keys. One celebrity session is fine; millions of large sessions need memory monitoring and eviction policy awareness.

Alternatives

FAQs

Related

Related: Short-Term Memory Basics

Related: Session Isolation

Related: Scratchpad Patterns

Related: Managing Conversation History Within a Context Budget

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.