Choosing a Deployment Target for an Agent Workload
Where an agent runs shapes timeouts, cold starts, secret delivery, concurrency, and how you recover from crashes.
Pick the target from workload shape first, not from habit or marketing labels.
Summary
- Agent workloads fall into a few runtime shapes (request/response, long loops, always-on workers). Serverless, containers, and VPS hosts fit those shapes differently.
- Insight: A multi-minute tool loop on a 60-second function hard-fails. An idle always-on box burns cash when traffic is sparse. Wrong placement creates silent cost and reliability debt.
- Key Concepts: stateless handlers, wall-clock timeouts, cold starts, persistent processes, containers as the unit of ship, egress and secret surfaces.
- When to Use: Early product decisions, re-platforming after a timeout incident, or splitting interactive chat from background research.
- Limitations/Trade-offs: Hybrids are common. One agent product can use serverless for the API edge and a worker pool for long jobs.
- Related Topics: Docker packaging, serverless recipes, VPS process managers, cold-start mismatch, secrets across targets.
Foundations
An agent workload is more than "call the model once."
Typical steps: accept a request or event, load context, call one or more models, invoke tools, loop, write state, and return or notify.
That loop has properties that matter for hosting:
| Property | Why it affects hosting |
|---|---|
| Duration | Platform max execution time |
| Burstiness | Scale-to-zero vs always-on cost |
| State | In-memory session vs external store |
| Tools | Local binaries, browsers, network egress |
| Concurrency | Rate limits and memory per replica |
| Restart behavior | Crash recovery and in-flight work |
Three common placement classes:
- Serverless functions (Vercel Functions, AWS Lambda, Cloud Functions, similar) - invoke on demand, bill mostly for runtime, hard timeouts, often cold starts.
- Containers on a orchestrator (ECS, Cloud Run, Kubernetes, Fly, Railway containers) - ship a Docker image, scale replicas, more control over CPU/memory and process lifetime.
- VPS / bare VM / always-on process (systemd, supervisord, PM2 on a droplet or EC2) - you own process restarts, networking, and patching; best fit for sticky long-running workers.
Managed "agent platforms" still sit on one of these underneath. Evaluate them with the same table.
Mechanics & Interactions
Match shape to target
| Workload shape | Prefer | Avoid as primary |
|---|---|---|
| Short chat turn, under platform timeout, little local state | Serverless or scale-to-zero containers | Oversized dedicated VPS |
| Multi-minute research / multi-tool loop | Containers or VPS workers + queue | Pure request-scoped serverless without async handoff |
| Always listening (Slack, Telegram, websocket bots) | VPS or long-lived container | Cold function that only wakes on HTTP if the platform needs a durable socket |
| Heavy tools (Playwright, local code exec) | Container or VPS with sandboxing | Tiny function packages and tight /tmp |
| Spiky public traffic, idle nights | Serverless or Cloud Run-style scale-to-zero | Large always-on fleet |
| Strict private networking / custom NIC rules | VPC-attached containers or VPS | Public-only function with no private path |
Decision sequence
- Measure or bound wall-clock duration of a realistic success path (P95 and hard max).
- List local dependencies (OS packages, browsers, GPU, filesystem).
- Decide where conversation state lives (Redis, DB, object store - never only process memory if replicas can die).
- Pick the lightest host that still meets max duration and dependencies.
- Add a queue when intake must be fast but work is slow (Scaling Agent Systems Basics).
- Harden secrets and egress the same way on every target (Environment and Secrets Management Across Deployment Targets).
Hybrid is normal
A production pattern:
Client → API (serverless or edge)
→ enqueue job id
Worker (container / VPS) → agent loop → tools → model API
→ write result / webhook
Client polls or receives pushThe API stays under timeout limits. The worker owns the long loop.
Cost sketch (qualitative)
| Factor | Serverless | Containers (managed) | VPS |
|---|---|---|---|
| Idle cost | Near zero | Low if scale-to-zero; else baseline | Always on |
| Busy cost | Per-ms + invocations | Per replica-hour | Fixed box + oversize risk |
| Ops load | Low | Medium (images, health) | Higher (OS, TLS, restarts) |
| Failure modes | Timeout, cold start, package limits | OOM, deploy lag, cluster config | Disk full, unattended process death |
Exact prices change constantly - model your cost per successful task, not only host list price.
Advanced Considerations & Applications
Timeouts are product constraints
If your platform caps at N seconds, your agent product must either finish earlier, stream partial progress, or offload work.
Do not "hope" a multi-hop RAG + tools path stays under the cap.
See Cold Starts and Long-Running Agent Loops: A Serverless Mismatch.
Tool sandboxes pull you toward containers
Code execution, browser automation, and untrusted file handling want isolation (Containerized Sandboxes for Code-Executing Agent Tools).
That isolation is natural with Docker images and hard on many pure function runtimes.
Multi-region and multi-tenant
- Stateless handlers + external state scale horizontally more easily than sticky in-memory agents.
- Per-tenant isolation may require separate keys, queues, or even separate worker pools - placement must support that split.
Trade-off table
| Approach | Strength | Weakness | Best when |
|---|---|---|---|
| Serverless function as agent | Fast ship, scale-to-zero | Timeouts, cold starts, package limits | Short turns, light tools |
| Managed container | Portable image, flexible runtime | More moving parts than a function | Medium/long jobs, custom deps |
| VPS process | Full control, long lived | You own restarts and security | Always-on bots, early ops simplicity |
| Queue + worker | Decouples intake from work | Needs job semantics and idempotency | Anything longer than API SLOs |
Common Misconceptions
- "Agents are just APIs, so serverless always wins." Agents often need longer than a single HTTP budget.
- "Docker means Kubernetes." You can run one container on a VM or a simple PaaS without a full cluster.
- "A bigger VPS fixes timeouts." Timeouts are platform policy; VPS helps only if you leave the function model.
- "In-memory conversation is fine in prod." Restarts and multi-instance deploy will drop it.
- "Secrets in env vars on one host means we are done." Targets differ in injection, rotation, and audit (Environment and Secrets Management Across Deployment Targets).
FAQs
Should my first production agent be serverless?
Yes if a realistic path fits under the platform timeout with margin and tools are remote APIs only. Otherwise start with a container worker.
When is a single VPS enough?
Low concurrency, one team owning the box, and a process manager with restart policy. Plan an exit path before traffic multiplies.
Do I need both Docker and a process manager?
Containers define the image. On a VPS you still want systemd or similar so reboots and crashes recover the process. Orchestrators replace that with their own restart policy.
How do cold starts interact with first-token latency SLOs?
Cold starts add hundreds of ms to many seconds before your code runs. Warm capacity, provisioned concurrency, or always-on workers buy latency at cost.
Can I run LangGraph / CrewAI / similar on all three targets?
Framework choice rarely forces the host. Duration, state store, and OS dependencies do. Verify framework server modes against your platform at build.
What about GPU agents?
Treat GPU as a specialized container or VM pool, not a generic function. Queue jobs onto GPU workers.
How does this relate to CI/CD?
Placement decides what you deploy (zip, image, or binary) and how you roll back. Pair with CI/CD and Agent Lifecycle Basics.
Is Cloud Run "serverless" or "container"?
It is a scale-to-zero container platform: Docker image semantics with request-driven scaling. Map it with container packaging rules and serverless-ish timeout habits.
Should chat streaming force a specific host?
Streaming needs a connection long enough for the response. Short function timeouts break streams; use longer-lived handlers or separate stream gateways.
What metrics decide a re-platform?
Timeout rate, cold-start P95, cost per successful task, deploy frequency pain, and tool dependency friction.
Can one Dockerfile serve all environments?
Often yes for app code. Keep env-specific config out of the image and inject at runtime (Containerizing an Agent with Docker).
Where do personal always-on bots fit?
Usually a small VPS or home lab with a process manager, not a commercial function with aggressive idle freezes - see self-hosted personal agent sections if you run that pattern.
Related
Related: Deploying Agents Basics
Related: Containerizing an Agent with Docker
Related: Deploying an Agent to Vercel or AWS Lambda Serverless Functions
Related: Long-Running Agents on a VPS: Process Management and Restarts
Related: Cold Starts and Long-Running Agent Loops: A Serverless Mismatch
Related: Deploying Agents Best Practices
Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana
@solana/web3.js+@solana/spl-token.