Why Testing an Agent Differs from Testing Deterministic Code

Deterministic unit tests assert that the same input always yields the same output. Agent systems break that contract because models sample, tool results drift, and multi-step plans take different paths to the same goal.

You still need tests. You need a layered strategy: hard assertions where code is deterministic, and evals where quality is graded rather than string-matched.

Summary

Agent quality is scored with distributions and rubrics, not only byte-identical snapshots of free-form text.
Insight: Exact-match tests on model text flake, hide real regressions, and train teams to freeze brittle prompts instead of measuring task success.
Key Concepts: non-determinism, tool isolation, trajectory checks, gold sets, LLM-as-judge, pass rates, regression gates.
When to Use: Any multi-step LLM agent with tools, planning, or open-ended answers; also any single-turn product surface where wording can vary.
Limitations/Trade-offs: Evals cost money and time, judges can be biased, and flaky suites erode trust if thresholds and seeds are sloppy.
Related Topics: unit-testing tools, simulated loops, gold-set suites, LLM judges, prompt/model regression checklists.

Foundations

What classic tests assume

Ordinary code is a pure or nearly pure function of inputs and state you control.

assert add(2, 2) == 4
assert parse_invoice(blob)["total"] == Decimal("19.99")

Those assertions stay green forever if the implementation does not change.

What agents actually do

An agent typically:

Interprets a user goal in natural language.
Chooses tools and arguments under uncertainty.
Observes tool results and updates a plan.
Stops when a stopping condition fires or a budget is exhausted.
Emits a final answer whose wording is not unique.

Two healthy runs can call tools in different orders, phrase answers differently, and still complete the task. Two broken runs can look fluent while skipping a required tool or inventing facts.

Layers of "correct"

Layer	Example pass condition	Test style
Tool / pure functions	Schema, math, DB row shape	Deterministic unit tests
Policy and host code	Caps, allowlists, timeouts	Deterministic unit/integration tests
Single step behavior	Valid tool call shape	Contract tests + light mocks
Task success	Ticket created with right fields	Scenario sims + gold checks
Answer quality	Helpful, faithful, on-policy	Rubrics / LLM-as-judge
Safety	No PII leak, no forbidden tools	Red-team suites + hard guards

If you only unit-test tools, you never catch planner failure. If you only snapshot full transcripts, every model bump breaks CI for the wrong reason.

Mechanics & Interactions

Why exact string equality fails

Models draw from a distribution. Temperature, system prompts, provider routing, and even "identical" model versions can change tokens.

Exact equality on free-form text:

Flakes under tiny sampling noise.
Misses semantic regressions that still match a lucky template.
Blocks useful rewrites that improve clarity without changing meaning.

Prefer invariants over full string equality:

Structured fields that must be present.
Required tool names in the trajectory.
Numeric tolerances or set equality on extracted entities.
Rubric scores above a threshold averaged over N runs.

Where determinism still lives

Much of an agent stack is ordinary software.

Test it like ordinary software:

Tool implementations and argument validation.
Idempotency stores and approval gates.
Context assembly and truncation rules.
Stop conditions, retries, and circuit breakers.
Parsers for tool JSON and final structured output.

These tests should be fast, free of live model calls, and mandatory in every PR.

The agent loop as a test surface

A full loop couples three moving parts: model, tools, and orchestrator.

Strategy	What you freeze	What you measure
Unit-test tools	Model out of the picture	Correct side effects for given args
Scripted model / recorded responses	Tool world real or mocked	Orchestrator chooses right next step
Simulated scenarios	Partial mocks for external systems	End-to-end task success
Live model evals	Tools often real or sandboxed	Quality distribution on a gold set

Production confidence needs more than one row of that table.

Metrics that replace `assert equal`

Pass@k / pass rate: fraction of runs that satisfy hard success criteria.
Trajectory score: did required tools run (and forbidden ones not)?
Faithfulness: final answer consistent with tool observations.
Latency and cost per successful task: regressions that quality scores miss.
Judge scores: rubric dimensions (correctness, completeness, tone, safety).

Report scores with sample size and version pins (prompt hash, model slug, tool schema version).

# Sketch: hard success is deterministic; wording is not compared
def task_passed(run) -> bool:
    return (
        run.final_status == "completed"
        and "create_ticket" in run.tools_called
        and run.ticket["priority"] == "P1"
        and run.cost_usd < 0.50
    )

Advanced Considerations & Applications

Flakiness budgets

Non-determinism does not excuse chaos.

Control variance with:

Fixed seeds when the provider supports them (still not a full guarantee across vendors).
Multiple trials per case and a minimum pass rate (for example 4/5) instead of one-shot pass/fail.
Separate smoke (cheap, PR) and nightly (full gold set, multi-model) lanes.
Cached or recorded model responses for pure orchestrator tests.

Cost of truth

Live evals spend money. Budget them like load tests: small on every PR, larger on main and release candidates.

Interaction with CI/CD

A model or prompt change is a behavior change, not a docs-only tweak. Treat eval gates the way you treat integration tests for a payment service: block deploys when pass rate or safety scores drop below agreed thresholds (see CI lifecycle docs in this category).

Trade-off table

Approach	Strength	Weakness
Exact text snapshot	Simple	Flaky / brittle
Structured field asserts	Stable, cheap	Misses open-ended quality
Gold set + hard checks	Grounded task success	Needs curation
LLM-as-judge	Scales soft criteria	Judge bias and cost
Human review sample	Highest trust	Slow, not PR-speed

Healthy pipelines combine hard checks for must-never-fail cases with judges for style and partial credit.

Common Misconceptions

"Agents are untestable." Tools and host policy are fully testable; the model layer is evaluable with rates and rubrics.
"Temperature zero makes it deterministic." It reduces variance but does not freeze providers, tools, or multi-step branching.
"One golden transcript is enough." A single path overfits yesterday's plan and misses alternate valid trajectories.
"If the demo works, ship it." Demos are n=1 marketing; production needs distributional evidence.
"Unit tests replace evals." They complement evals. They do not score planner quality.
"Judges remove the need for humans." Judges need calibration against human labels on a seed set.

FAQs

Can I still use pytest for agents?

Yes. Use pytest (or any unit framework) for tools, parsers, and host policy. Use the same runner to orchestrate eval jobs that assert on pass rates and thresholds, not only on raw strings.

What should fail a PR immediately?

Deterministic regressions (tool bugs, schema breaks, policy bypasses) and hard safety failures. Soft quality dips may warn on PR and block only on larger suites.

How many trials per case do I need?

Start with 1 for pure unit tests, 3-5 for flaky model paths on critical cases, and more for release gates. Raise N until the pass-rate confidence interval is useful for your risk level.

Should I mock the model always?

Mock for orchestrator and tool-wiring tests. Use live or recorded-but-refreshed model calls for quality evals. Mocking the model forever hides real prompt regressions.

How do I test open-ended answers without a judge?

Extract structure (JSON schema, required citations, entity sets), check tool trajectories, and reserve free-form prose for rubric scoring when structure is not enough.

Are snapshot tests ever useful?

Yes for stable structured artifacts (tool schemas, prompt templates, fixture JSON). Avoid snapshotting free-form model prose as a hard gate.

What is the difference between an eval and a test?

A test usually expects a fixed result. An eval measures quality against criteria, often aggregating many cases and runs into scores and thresholds.

How do tool flakiness and model flakiness interact?

External tools add their own nondeterminism. Sandbox or script external systems in CI so failures point at the agent, not at a flaky third-party API.

Do multi-agent systems change the picture?

They add handoff and privilege surfaces. You still unit-test tools, but scenarios must cover wrong handoffs and shared-state races.

When is exact match acceptable on model output?

When the model is constrained to a tiny closed set (enums, yes/no, fixed JSON schema with strict decoding) and you still allow for rare provider failures with retries.

How should product managers read eval reports?

As task success rates, cost per success, and safety incident counts - not as "the prompt looks better."

What is the first step if our suite is only manual demos?

Extract pure tools into unit tests, write ten gold tasks with hard success checks, and run them on every model/prompt change before adding fancy judges.

Related: Testing AI Agents Basics

Related: Unit Testing Individual Agent Tools

Related: Simulating Full Agent Loops in a Test Harness

Related: Building a Gold-Set Eval Suite for Agent Quality

Related: LLM-as-Judge: Scoring Agent Output Automatically

Related: Regression Testing an Agent After a Model or Prompt Change

Related: Testing AI Agents Best Practices

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.

You still need tests. You need a layered strategy: hard assertions where code is deterministic, and evals where quality is graded rather than string-matched.

Summary

Agent quality is scored with distributions and rubrics, not only byte-identical snapshots of free-form text.
Insight: Exact-match tests on model text flake, hide real regressions, and train teams to freeze brittle prompts instead of measuring task success.
Key Concepts: non-determinism, tool isolation, trajectory checks, gold sets, LLM-as-judge, pass rates, regression gates.
When to Use: Any multi-step LLM agent with tools, planning, or open-ended answers; also any single-turn product surface where wording can vary.
Limitations/Trade-offs: Evals cost money and time, judges can be biased, and flaky suites erode trust if thresholds and seeds are sloppy.
Related Topics: unit-testing tools, simulated loops, gold-set suites, LLM judges, prompt/model regression checklists.

Foundations

What classic tests assume

Ordinary code is a pure or nearly pure function of inputs and state you control.

assert add(2, 2) == 4
assert parse_invoice(blob)["total"] == Decimal("19.99")

Those assertions stay green forever if the implementation does not change.

What agents actually do

An agent typically:

Interprets a user goal in natural language.
Chooses tools and arguments under uncertainty.
Observes tool results and updates a plan.
Stops when a stopping condition fires or a budget is exhausted.
Emits a final answer whose wording is not unique.

Two healthy runs can call tools in different orders, phrase answers differently, and still complete the task. Two broken runs can look fluent while skipping a required tool or inventing facts.

Layers of "correct"

Layer	Example pass condition	Test style
Tool / pure functions	Schema, math, DB row shape	Deterministic unit tests
Policy and host code	Caps, allowlists, timeouts	Deterministic unit/integration tests
Single step behavior	Valid tool call shape	Contract tests + light mocks
Task success	Ticket created with right fields	Scenario sims + gold checks
Answer quality	Helpful, faithful, on-policy	Rubrics / LLM-as-judge
Safety	No PII leak, no forbidden tools	Red-team suites + hard guards

If you only unit-test tools, you never catch planner failure. If you only snapshot full transcripts, every model bump breaks CI for the wrong reason.

Mechanics & Interactions

Why exact string equality fails

Models draw from a distribution. Temperature, system prompts, provider routing, and even "identical" model versions can change tokens.

Exact equality on free-form text:

Flakes under tiny sampling noise.
Misses semantic regressions that still match a lucky template.
Blocks useful rewrites that improve clarity without changing meaning.

Prefer invariants over full string equality:

Structured fields that must be present.
Required tool names in the trajectory.
Numeric tolerances or set equality on extracted entities.
Rubric scores above a threshold averaged over N runs.

Where determinism still lives

Much of an agent stack is ordinary software.

Test it like ordinary software:

Tool implementations and argument validation.
Idempotency stores and approval gates.
Context assembly and truncation rules.
Stop conditions, retries, and circuit breakers.
Parsers for tool JSON and final structured output.

These tests should be fast, free of live model calls, and mandatory in every PR.

The agent loop as a test surface

A full loop couples three moving parts: model, tools, and orchestrator.

Strategy	What you freeze	What you measure
Unit-test tools	Model out of the picture	Correct side effects for given args
Scripted model / recorded responses	Tool world real or mocked	Orchestrator chooses right next step
Simulated scenarios	Partial mocks for external systems	End-to-end task success
Live model evals	Tools often real or sandboxed	Quality distribution on a gold set

Production confidence needs more than one row of that table.

Metrics that replace `assert equal`

Pass@k / pass rate: fraction of runs that satisfy hard success criteria.
Trajectory score: did required tools run (and forbidden ones not)?
Faithfulness: final answer consistent with tool observations.
Latency and cost per successful task: regressions that quality scores miss.
Judge scores: rubric dimensions (correctness, completeness, tone, safety).

Report scores with sample size and version pins (prompt hash, model slug, tool schema version).

# Sketch: hard success is deterministic; wording is not compared
def task_passed(run) -> bool:
    return (
        run.final_status == "completed"
        and "create_ticket" in run.tools_called
        and run.ticket["priority"] == "P1"
        and run.cost_usd < 0.50
    )

Advanced Considerations & Applications

Flakiness budgets

Non-determinism does not excuse chaos.

Control variance with:

Fixed seeds when the provider supports them (still not a full guarantee across vendors).
Multiple trials per case and a minimum pass rate (for example 4/5) instead of one-shot pass/fail.
Separate smoke (cheap, PR) and nightly (full gold set, multi-model) lanes.
Cached or recorded model responses for pure orchestrator tests.

Cost of truth

Live evals spend money. Budget them like load tests: small on every PR, larger on main and release candidates.

Interaction with CI/CD

Trade-off table

Approach	Strength	Weakness
Exact text snapshot	Simple	Flaky / brittle
Structured field asserts	Stable, cheap	Misses open-ended quality
Gold set + hard checks	Grounded task success	Needs curation
LLM-as-judge	Scales soft criteria	Judge bias and cost
Human review sample	Highest trust	Slow, not PR-speed

Healthy pipelines combine hard checks for must-never-fail cases with judges for style and partial credit.

Common Misconceptions

"Agents are untestable." Tools and host policy are fully testable; the model layer is evaluable with rates and rubrics.
"Temperature zero makes it deterministic." It reduces variance but does not freeze providers, tools, or multi-step branching.
"One golden transcript is enough." A single path overfits yesterday's plan and misses alternate valid trajectories.
"If the demo works, ship it." Demos are n=1 marketing; production needs distributional evidence.
"Unit tests replace evals." They complement evals. They do not score planner quality.
"Judges remove the need for humans." Judges need calibration against human labels on a seed set.

FAQs

Can I still use pytest for agents?

Yes. Use pytest (or any unit framework) for tools, parsers, and host policy. Use the same runner to orchestrate eval jobs that assert on pass rates and thresholds, not only on raw strings.

What should fail a PR immediately?

Deterministic regressions (tool bugs, schema breaks, policy bypasses) and hard safety failures. Soft quality dips may warn on PR and block only on larger suites.

How many trials per case do I need?

Start with 1 for pure unit tests, 3-5 for flaky model paths on critical cases, and more for release gates. Raise N until the pass-rate confidence interval is useful for your risk level.

Should I mock the model always?

Mock for orchestrator and tool-wiring tests. Use live or recorded-but-refreshed model calls for quality evals. Mocking the model forever hides real prompt regressions.

How do I test open-ended answers without a judge?

Extract structure (JSON schema, required citations, entity sets), check tool trajectories, and reserve free-form prose for rubric scoring when structure is not enough.

Are snapshot tests ever useful?

Yes for stable structured artifacts (tool schemas, prompt templates, fixture JSON). Avoid snapshotting free-form model prose as a hard gate.

What is the difference between an eval and a test?

A test usually expects a fixed result. An eval measures quality against criteria, often aggregating many cases and runs into scores and thresholds.

How do tool flakiness and model flakiness interact?

External tools add their own nondeterminism. Sandbox or script external systems in CI so failures point at the agent, not at a flaky third-party API.

Do multi-agent systems change the picture?

They add handoff and privilege surfaces. You still unit-test tools, but scenarios must cover wrong handoffs and shared-state races.

When is exact match acceptable on model output?

When the model is constrained to a tiny closed set (enums, yes/no, fixed JSON schema with strict decoding) and you still allow for rare provider failures with retries.

How should product managers read eval reports?

As task success rates, cost per success, and safety incident counts - not as "the prompt looks better."

What is the first step if our suite is only manual demos?

Extract pure tools into unit tests, write ten gold tasks with hard success checks, and run them on every model/prompt change before adding fancy judges.

Related: Testing AI Agents Basics

Related: Unit Testing Individual Agent Tools

Related: Simulating Full Agent Loops in a Test Harness

Related: Building a Gold-Set Eval Suite for Agent Quality

Related: LLM-as-Judge: Scoring Agent Output Automatically

Related: Regression Testing an Agent After a Model or Prompt Change

Related: Testing AI Agents Best Practices

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.

Why Testing an Agent Differs from Testing Deterministic Code

Search Documentation

Why Testing an Agent Differs from Testing Deterministic Code