LLM debugging: why text logs aren’t enough

A typical LLM application is not one request and one response. It may retrieve documents, call tools, retry a provider request, update state, and ask the model to make several decisions. A text log can contain every message and still make the failure difficult to locate.

What text logs usually hide

• Which model call caused the first divergence
• How long each step waited or retried
• Which tool result was passed into the next decision
• Whether a failure began in retrieval, orchestration, or the model response

1. See the run as a hierarchy

A trace gives you a parent run with child spans for model calls, retrieval, tool execution, and application work. That hierarchy makes it possible to move from the overall outcome to the exact step that changed it.

2. Separate latency from correctness

A slow run and an incorrect run can have different causes. Timeline data helps you distinguish a slow provider call from a fast but invalid tool result. Measure the duration of each meaningful step instead of relying on total request time.

3. Preserve the causal sequence

When the agent makes a bad choice, inspect the observation immediately before it. Was the retrieved context empty? Did the tool return an error field? Did a retry change the conversation? The nearest preceding event is often more useful than the final exception.

4. Compare runs instead of guessing

Choose a healthy baseline and compare it with the failure. Differences in span order, prompt version, model, tool arguments, retries, and timing turn a vague incident into a testable hypothesis.

5. Keep sensitive content under control

Prompts, retrieved documents, tool arguments, and outputs can contain sensitive information. Redact according to your policy and use local analysis where it fits your workflow. The goal is better debugging context without creating unnecessary data exposure.