Data Analysis Agents: From Spreadsheet to Insight
Data analysis agents turn tables into decisions by pairing sandboxed code execution with narrative insight.
They do not replace statisticians for causal claims.
They do accelerate the path from "here is a CSV" to "here are the patterns, caveats, and charts a human should review."
Summary
- Analysis agents iterate: profile data → propose transforms → run code → read results → explain findings with references to columns, filters, and cells.
- Insight: Pure LLM arithmetic on pasted rows is unreliable. Pure BI dashboards lack flexible question-answering. Agents bridge both when bounded and verified.
- Key Concepts: code interpreter / sandbox, data profiling, reproducible transforms, numeric verification, narrative layer, lineage.
- When to Use: Ad-hoc questions on spreadsheets, warehouse extracts, experiment CSVs, and ops exports where the next query depends on prior results.
- Limitations/Trade-offs: Bad joins, silent NaNs, and overconfident causal language still happen. High-stakes metrics need human sign-off and fixed pipelines.
- Related Topics: research agents, knowledge-work differences, RAG for metric definitions, best practices for accuracy.
Foundations
A spreadsheet question has two hard parts: computing the right number and saying what it means.
Models are weak at longhand arithmetic over large tables and strong at framing hypotheses and explanations.
Code execution flips that: pandas, SQL, or a warehouse query engine computes; the model chooses the next transform and writes the story.
The agent loop looks like knowledge work with a deterministic tool in the middle:
- Load or connect to a table (with schema + sample rows in context).
- Profile: dtypes, null rates, key cardinalities, obvious outliers.
- Plan a calculation or chart.
- Execute code or SQL in a sandbox.
- Observe stdout, returned frames, or errors.
- Iterate or finalize a structured insight report.
"Insight" here means a reviewable claim about the data, not a business strategy decree.
Good outputs name the filter, time range, metric definition, and result.
Bad outputs say "sales are up" without the query that produced the number.
Analysis agents sit between coding agents and research agents.
Like coding agents, they verify with execution.
Like research agents, they must narrate uncertainty and keep an evidence trail (here: code + result snapshots, not web citations).
Mechanics & Interactions
Profiling before analysis prevents fantasy joins.
Feed the model schema, 5-20 sample rows, and basic stats - not a 200k-row paste into the prompt.
Large data stays in the sandbox or warehouse; the model sees summaries and query results only.
Tool design matters more than the model logo.
Minimum useful tools:
list_tables/get_schemarun_sqlorrun_python(sandboxed)savefigorrender_chart(optional)export_resultto a draft artifact store
Return small, typed observations: row counts, head previews, error messages, checksum of output frames.
Do not return multi-megabyte frames into the chat transcript every turn.
Metric definitions are a second knowledge problem.
"Revenue" may mean booked, billed, or cash.
Either bind the agent to a metrics catalog (RAG over certified definitions) or force it to declare the formula it used in code.
Undefined metrics are the top source of executive distrust.
Narrative generation should consume structured results.
Prefer: code computes a table → model writes bullets referencing that table id.
Avoid: model invents percentages, then code is asked to "confirm" a pre-decided story.
Safety for code-executing analysis:
- No network from the sandbox unless explicitly required.
- Read-only warehouse credentials by default.
- CPU, memory, and runtime limits per turn.
- Block shell escapes and file writes outside a workspace.
# Pattern sketch: model proposes code; runtime executes; only summary returns
def analyze_step(propose_code, sandbox, max_rows_preview=20):
code = propose_code()
result = sandbox.exec(code, timeout_s=30)
return {
"ok": result.ok,
"error": result.error,
"preview": result.frame_head(max_rows_preview),
"rowcount": result.rowcount,
"code_hash": result.code_hash,
}Advanced Considerations & Applications
SQL-first vs Python-first.
Warehouse-native agents should push compute to SQL for scale and governance.
Python shines for awkward files, quick plots, and one-off munging.
Hybrid agents choose per step; policy can require SQL for production metrics.
Reproducibility.
Persist the exact code, parameters, data version (file hash or table snapshot id), and result digest.
Without lineage, last week's "insight" cannot be defended in a metrics review.
Multi-table and join risk.
Agents will happily fan-out join on poorly named keys.
Require explicit join keys in the plan, row-count sanity checks pre/post join, and a maximum join complexity before human approval.
Charting and decks.
Charts are tools, not proof.
Every chart artifact should pair with the query and filters that produced it.
Auto-building full slide narratives is possible; auto-sending them to leadership without review is usually not.
When not to use an analysis agent.
- The metric is already certified in a BI tool with a known dashboard.
- The question is pure causal inference needing an experiment design.
- The dataset is too sensitive for the chosen execution environment.
- Latency must be sub-second for the same fixed query (use a prepared query instead).
| Approach | Strength | Weakness | Best fit |
|---|---|---|---|
| Paste table into prompt | Zero infra | Breaks at scale; weak math | Tiny tables, demos |
| Fixed SQL templates | Reliable, cheap | Inflexible questions | Known Q&A set |
| Agent + SQL sandbox | Flexible, governed compute | Needs good schema tools | Warehouse Q&A |
| Agent + Python sandbox | Great for files/plots | Harder governance | CSV/Excel ad-hoc |
| Full auto BI agent | Broad exploration | High wrong-insight risk | Shadow mode only at first |
Common Misconceptions
- "The model can just add the column itself." Prefer executed code for aggregations over multi-thousand-row mental math.
- "If code runs without error, the insight is true." Wrong filters and wrong metric definitions still produce clean code.
- "Analysis agents replace analysts." They accelerate exploration and drafting; analysts still own definitions and decisions.
- "More autonomy means better insights." Unbounded exploration burns cost and invents stories. Bound turns and require structure.
- "Charts prove the claim." Charts illustrate a query result. The query and definition are the proof.
FAQs
What makes a data analysis system an agent?
The model chooses successive transforms based on prior query results, rather than running one fixed SQL string every time.
Should every analysis agent execute code?
If numbers matter, yes - or call a governed metrics API. Freeform numeric generation is a last resort for tiny examples only.
How do I stop the agent from dropping rows silently?
Require row counts before and after filters/joins in the observation schema, and fail the step if counts move unexpectedly without explanation.
Can the agent write back to the spreadsheet?
Only with explicit permission and versioning. Default to export of a new sheet or artifact, not overwrite of source data.
How do analysis agents cite evidence?
Cite code cell / query id, table version, filters, and result snapshot - the data analog of a URL citation.
What model tier should I use?
Often a mid tier for planning plus a capable model for final narrative. Tiny models struggle with multi-step SQL reasoning; verify at build.
How is this different from a coding agent on a repo?
The primary artifact is a correct, explained statistic or chart, not a merged PR. Tests are row-count checks and metric definitions, not unit tests.
Where do metric catalogs fit?
As retrieval tools that return certified formulas before the agent invents one. Prefer catalog hits over free invention.
What belongs in an eval set?
Known answers on fixture CSVs: totals, group-bys, join pitfalls, null handling, and "refuse when definition missing" cases.
Can agents handle streaming or real-time data?
Yes with care: pin a time window per run so results are reproducible, and avoid claiming live accuracy without a watermark.
How do I present uncertainty?
Separate observed numbers from interpretation. Use language like "in this extract" and list data quality issues from profiling.
When should analysis become a fixed pipeline?
When the same question is asked weekly with a stable definition. Promote the agent's successful SQL to a scheduled job and demote the agent to ad-hoc only.
Is pandas required?
No. Any sandboxed compute that returns structured results works: SQL engines, Spark jobs, or warehouse notebooks - with the same observation discipline.
What is a good first production slice?
Read-only SQL on one certified mart, five template questions, structured answers, and human review in Slack before any dashboard publish.
Related
- Business & Research Use Cases Basics - cited summary habits that carry over
- How Knowledge-Work Agents Differ from Coding Agents - where execution vs evidence dominate
- Research Agents: Multi-Source Investigation and Synthesis - qualitative fan-out pattern
- Business & Research Use Cases Best Practices - accuracy and sourcing rules
- Coding & Developer-Tooling Use Cases Basics - sandbox/execution cousin
Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana
@solana/web3.js+@solana/spl-token.