What Makes an Internal API a Good Agent Tool
An internal API is a good agent tool when a model can call it safely, recover from failure, and produce useful next steps without human babysitting on every request.
Raw REST surfaces designed for UI forms often fail that test. Agents retry, invent parameters, and call the same endpoint many times in one run.
Summary
- Tool quality is mostly API design for non-deterministic callers - not model cleverness.
- Insight: Bad tools cause loops, double writes, and silent wrong answers that evals miss until production.
- Key Concepts: idempotency, scoped auth, structured errors, narrow schemas, stable contracts, side-effect ranking.
- When to Use: Before wrapping any internal service; use this as a go/no-go checklist for tool candidates.
- Limitations/Trade-offs: Perfect APIs still need timeouts, rate limits, and human gates for high-impact actions.
- Related Topics: REST wrapping, versioning, database tools, sandboxed code exec.
Foundations
A tool is a model-callable function with a name, description, parameter schema, and host-side implementation.
An internal API is the backend the tool calls (HTTP, RPC, DB driver, or queue publish).
Agents differ from UI clients in four ways:
- They may call the same tool repeatedly with slight argument drift.
- They cannot read OpenAPI prose as well as a typed SDK user; they rely on names and descriptions.
- They treat tool results as ground truth unless errors are explicit.
- They will try "creative" arguments when the schema is loose.
So "good for humans" does not equal "good for agents."
Properties that matter most
| Property | Agent-friendly meaning | Failure if missing |
|---|---|---|
| Idempotency | Safe retries for the same intent | Duplicate charges, double tickets |
| Clear errors | Machine-parseable failure reasons | Model invents success or loops |
| Scoped access | Least privilege credentials | Blast radius on prompt injection |
| Narrow inputs | Few enums, required fields, bounds | Hallucinated IDs and free-text SQL |
| Predictable outputs | Stable JSON shapes, small payloads | Context bloat and parse thrash |
| Observable side effects | Logs, request IDs, audit fields | Undebuggable production runs |
Mechanics & Interactions
How the model uses the API (indirectly)
- The runtime sends tool schemas in the prompt or API tools list.
- The model emits a tool call with JSON arguments.
- Your host validates args, calls the internal API, and returns a string or JSON result.
- The model plans the next step from that result.
The internal API never "talks" to the model. Your wrapper is the only truth filter.
Idempotency in practice
Prefer APIs that accept a client-generated idempotency key or natural keys (external order id, ticket source key).
# Tool args include idempotency_key so retries do not double-create
def create_ticket(title: str, body: str, idempotency_key: str) -> dict:
return http_post(
"/tickets",
json={"title": title, "body": body},
headers={"Idempotency-Key": idempotency_key},
)If the backend cannot be idempotent, the tool layer must enforce once-only semantics (dedupe store, pending-approval queue) rather than exposing raw POST.
Error contracts agents can use
Return structured, non-HTTP-only failures when possible:
{"ok": False, "error_code": "NOT_FOUND", "message": "Order 42 missing", "retryable": False}| Signal | Model behavior you want |
|---|---|
retryable: true + backoff | Try again or use alternate tool |
retryable: false + clear code | Stop, ask user, or escalate |
| Validation errors naming fields | Fix args without guessing |
| Auth/scope errors | Do not invent alternate credentials |
Avoid bare 500 Internal Server Error strings with no code. The model will rephrase and retry forever.
Scoped access
Map each tool to a service identity with only the methods that tool needs.
- Read tools → read-only tokens or DB roles.
- Write tools → separate keys, lower rate limits, optional human approval.
- Never pass the agent's user-facing API key into a general "HTTP proxy" tool.
Schema shape
Good agent schemas are boring:
- Prefer enums over free strings for status, region, and action type.
- Require IDs the agent already retrieved from a list tool.
- Cap string lengths and array sizes.
- Split "search" and "mutate" into different tools so the model cannot mix intents.
Advanced Considerations & Applications
Side-effect ranking
Not every endpoint should be a tool.
| Side effect | Default exposure | Extra control |
|---|---|---|
| Read / search | Tool OK | Result size limits |
| Create with undo | Tool OK | Idempotency key |
| Update critical records | Tool + audit | Approval gate |
| Delete / transfer money | Rarely auto | Human-in-the-loop |
| Shell / arbitrary SQL | Avoid raw | Sandbox + allowlist |
Composition with multi-tool agents
APIs that return stable IDs and short summaries compose well with follow-up tools.
Huge HTML or multi-megabyte JSON dumps waste context and hide the fields the model needs.
Provide pagination tokens and limit parameters rather than "return everything."
Trade-off table
| Approach | Safety | Model success | Ops cost | Best for |
|---|---|---|---|---|
| Thin pass-through of full REST | Low | Medium | Low | Prototypes only |
| Curated tools over domain actions | High | High | Medium | Production agents |
One mega-tool (call_api) | Very low | Low | Low | Almost never |
| MCP server wrapping curated tools | High | High | Medium | Shared multi-client tools |
When an API is not ready
Do not wrap it yet if:
- POST is not idempotent and has no natural key.
- Errors are HTML pages or empty bodies.
- Auth is shared superuser credentials.
- Responses change shape without versioning.
- Destructive actions lack confirmation paths.
Fix the API or put a facade service in front before agent exposure.
Common Misconceptions
- "If OpenAPI exists, the agent can use the API." OpenAPI helps humans and codegen; models still need narrow tools and good descriptions.
- "Idempotency is only for payments." Any create or side-effecting update benefits when agents retry.
- "Returning full objects is always better." Oversize payloads harm context and increase leak risk.
- "Clear English error messages are enough." Models also need stable
error_codeandretryableflags. - "Least privilege is a later hardening step." Scope is a design property; retrofitting after a leak is late.
FAQs
Is every internal microservice a good tool candidate?
No. Prefer domain actions the agent needs for a job ("refund order", "list open P1s") over raw resource CRUD for every table.
Should tools mirror REST resources 1:1?
Usually no. Collapse multi-step UI flows into one tool when the agent should not invent intermediate states.
How important is the tool description versus the schema?
Both matter. Descriptions drive when to call; schemas drive how to call. Weak either side produces wrong tools or wrong args.
Can I expose GraphQL as one flexible tool?
Dangerous. Free-form queries act like shell access to your graph. Prefer fixed operations with typed variables.
What about long-running jobs?
Return a job id and a separate status tool. Do not block the agent loop for minutes without a timeout and poll pattern.
Do I need human approval on every write?
No. Rank by impact. Low-risk, reversible writes can auto-run; irreversible or high-cost writes need gates.
How do rate limits interact with agent loops?
Agents can burst. Put per-run and per-tool rate limits in the host, surface 429 as retryable with delay, and cap loop iterations.
Should error stacks go back to the model?
No. Return sanitized codes and messages. Log stacks server-side with a request id the model may cite.
Is idempotency only HTTP-level?
No. The durable key must live where the side effect happens (DB unique constraint, queue dedupe, payment provider key).
How do I test that an API is agent-ready?
Run scripted tool calls: retries, invalid args, permission denied, empty results, and max payload sizes. Then add agent evals on real tasks.
Where does MCP fit?
MCP standardizes discovery and invocation. It does not fix a bad underlying API. Curate tools first, then expose via MCP if multiple clients need them.
What is the fastest improvement for a messy API?
Add a facade with strict schemas, idempotency keys, structured errors, and read-only credentials for the first agent tools.
Related
Related: Custom Tools Basics
Related: Wrapping a REST API as an Agent Tool
Related: Versioning and Deprecating Agent Tools Without Breaking Agents
Related: Custom Tools Best Practices
Related: The Principle of Least Privilege Applied to Agent Tools
Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana
@solana/web3.js+@solana/spl-token.