How to debug an AI agent that did the wrong thing three steps ago

Debug backwards from the output. Start at the wrong answer, walk the run's span tree in reverse until you find the last step whose output was still correct, and the step after it is the one that broke. Classify that step as retrieval, tool, reasoning or formatting, because the four look identical in the final answer and have different fixes.

The mistake is upstream of the sentence you are reading

The complaint arrives as a screenshot. The assistant told a customer they had no orders, or quoted a policy that expired in March, or produced a summary that lost the one constraint the user cared about. The final answer is fluent, well formed and wrong, and it returned HTTP 200 with no error anywhere in the stack.

What actually happened sits several steps earlier. A retrieval step matched on the company name instead of the account id and returned three irrelevant documents. A tool returned {"orders": []} because the query was malformed, and the model read the empty array as a fact about the world. A summarisation step compressed a 4,000-token context into 300 tokens and dropped the sentence that said "excluding cancelled orders".

By the time the model writes the last paragraph it is reasoning correctly over bad inputs, which is why "the model got confused" is almost never the diagnosis.

Reading the final answer harder does not recover any of this. The trace does, if it carries per-step inputs and outputs. What a trace needs to contain, and how the spans nest, is covered at AI agent observability; the instrumentation side is in agent tracing.

How do I find out why my AI agent did something wrong

Bisect the run. A run is an ordered sequence of steps where each step's input is derived from earlier outputs, so exactly one step is the first to produce something wrong, and everything after it is a consequence. Find that step, classify it, then fix the class of failure rather than the sentence it produced.

Debug backwards in five steps

1. Find the run before you theorise about it

Filter traces by the end-user identifier and a time window, then narrow by surface. Do not search for the text of the wrong answer: the model phrases it differently each time and the customer is paraphrasing a screenshot. If the trace store has no end-user attribute on the root span, you are reduced to guessing from timestamps, and two overlapping tenants make that guess wrong.

Once you have the run, note three facts before opening anything: the agent version, the trigger (web chat, Slack, an API key, a schedule) and the terminal status. A run that ended on a turn limit and a run that ended on a normal stop are different investigations.

2. Walk the span tree backwards to the last correct output

Read the spans in reverse order and ask one question of each: was this output still correct? Not "does it look reasonable", but "does it contain the fact the final answer got wrong". Walk backwards until the answer flips to yes. The first span after that flip is the failing step.

This is a bisection, so on a long run you can jump. Check the midpoint span first, then halve. A twenty-step run reaches the failing step in about five reads instead of twenty, and the discipline of asking a binary question stops you from stopping early at the first span that looks odd.

3. Diff the step's input against what it should have seen

Open the failing step and read the input it actually received, in full. Not the template, not the prompt file in your repo, the resolved input as sent. This is where most of the surprise lives: a variable that interpolated as an empty string, a system prompt that lost a section when the agent version changed, a context block truncated at a token limit, a tool result serialised as "[object Object]".

Then write down what the step should have seen for the correct answer to be possible. If the required fact is absent from the input, the failure is upstream and you keep walking. If the fact is present and the step still produced the wrong output, you have found it.

{
  "span": "tool_call",
  "name": "search_orders",
  "arguments": { "query": "Northwind Trading", "limit": 10 },
  "result": { "orders": [] },
  "duration_ms": 342,
  "status": "ok"
}

That span is the whole incident. The tool worked, returned in 342 ms and reported success. The model had asked for orders by company name in a system keyed on account id, got an empty array, and told the customer they had no orders. Nothing about it looks like a failure in an aggregate view.

4. Replay the step with the recorded tool results

Re-run the failing step in isolation with two things pinned from the trace: the exact input, and the results the tools returned during the original run. Pinning tool results is what makes the replay deterministic; without it the tool hits live data, returns something different, and the step passes for reasons unrelated to your fix.

{
  "case_id": "run-8f21-step-4",
  "input": {
    "message": "when did my last order ship?",
    "end_user_id": "8812",
    "account_id": "acct_4471"
  },
  "pinned_tool_results": {
    "search_orders": { "orders": [] }
  },
  "expected": "asks for clarification or reports a lookup failure, and never states that the account has no orders"
}

Run it three times. Three failures means the input is wrong and the fix is upstream or in the prompt. One failure in three means sampling, and the fix is an output contract or a lower temperature. Zero failures means you pinned the wrong thing and should go back to step 3.

5. Turn the case into a regression case

Save the replay as an eval case with the pinned tool results attached, and add the assertion you just wrote by hand. The case has to fail before your fix and pass after it, or you have not proven the fix addresses this bug. Promoting real runs into a suite is the cheapest source of cases you will ever have, and the method is in building an eval set from production.

Classify the failure before you pick a fix

Retrieval, tool, reasoning and formatting failures are indistinguishable in the final answer. They diverge completely in the trace, and each has a fix that does nothing for the other three.

The classification is worth writing down in the incident notes, because the same wrong answer recurring under two different classes is two bugs.

Three misdiagnoses that cost the most time

Blaming the model for an empty tool result. A tool that returns [] with a 200 status is reporting success, and the model has no way to tell "nothing matched your query" from "your query was wrong". Make the tool return a distinguishable error for a malformed or unresolvable query instead of an empty collection. This is also the mechanism behind most runaway loops, since the model rephrases and retries a call it believes merely missed, which is worked through in agent infinite loops.

Blaming the prompt for a truncated input. When a context assembly step drops content at a token budget, the prompt template on disk is still correct and the resolved input is not. Compare the token count of the sent input against what you expect, and check whether the dropped region is at the boundary the truncation policy cuts.

Blaming non-determinism for an input that changed. Two runs of "the same" request often differ in the input, because retrieval returned different documents, a tool returned newer data, or a conversation carried different history. Diff the two inputs before concluding the model is inconsistent. Sampling is real, and a replay with pinned tool results is what separates the two.

Where this gets easier

Every step of this procedure depends on being able to read a step's actual input and actual output, which is the part most stacks lose first. Runtype's execution traces record each step's input and output, every tool call with its arguments and result, and latency per step, so "the last step that was still correct" is something you read off the run rather than reconstruct from partial logs, and a recorded execution can be promoted directly into an eval case with its inputs intact.