Here is the conversation nobody rehearses for.
Three weeks ago, one of your AI agents did something it shouldn't have. It approved a refund outside policy. It emailed a document to the wrong distribution list. It pulled a record from a system it had access to but no business reason to touch. Nobody noticed at the time, because nothing broke — the agent behaved exactly as an agent behaves, chaining tool calls until it reached what it decided was a reasonable end state.
Today, your biggest client's security team is on a call asking for a timeline. Which records were accessed. In what order. On whose authority. Which model version made the call, and what the tool returned when it did.
And the honest answer — the one your engineering lead gives you privately afterwards — is that nobody can reconstruct it.
That is not a hypothetical failure mode. It is the default state of most agentic deployments running in production right now.
Deployment outpaced governance, and the analysts are saying so out loud
Forrester's 2026 assessment of the space is titled "The State Of Agentic AI In 2026: Companies Are Chasing, Few Are Catching." That framing — chasing versus catching — is the polite version of a blunt observation: organisations moved fast on agent deployment and much slower on everything that has to sit underneath it. Pilots got funded. Evidentiary infrastructure did not.
The discourse is shifting too. CASRAI recently published a piece framed around a "September 2026 AI Safety Incident Cluster" — and the word that matters there is cluster. Once incidents are discussed in the plural and in the recent past tense, the executive question changes shape. It stops being "could this go wrong?" and becomes "when it goes wrong for us, what exactly do we hand to the board, the auditor, the insurer and the client?"
Those are two completely different engineering problems. The first is about controls. The second is about evidence.
Why agent logs fail under scrutiny
The instinct is to assume this is a discipline problem — somebody forgot to turn logging on. It usually isn't. It's an architectural mismatch, and it's worth understanding precisely, because it determines what you have to build.
Traditional application logging was designed for deterministic control flow. A request enters, moves through a known set of branches, and exits. You log the entry, the exit, the exceptions and a few checkpoints in between. That's sufficient, because the code path between checkpoints is fixed and readable. If you know the inputs and you know the version of the code, you can infer everything that happened in the middle.
Agentic systems violate that assumption at every single tool call. The same prompt, the same user, the same data — and the agent may take a different path today than it took yesterday. It may call three tools instead of five. It may re-read a document because the first summarisation left it uncertain. It may decide mid-run that a sub-goal is unnecessary and skip it.
So when you log the request and the response, you have captured the two least interesting facts about the incident. The reasoning, the intermediate tool calls, the data each call returned, and the state that informed the next decision — all of that happened inside a black box you didn't instrument, and it cannot be re-derived after the fact. Re-running the prompt does not reproduce the incident. It produces a new, differently-shaped run that tells you nothing about what actually occurred on the day in question.
This is why "we have logs" and "we can prove what happened" are not the same sentence in an agentic architecture.
Your agent logs are now evidence, not telemetry
There's a second reason this matters more than it did eighteen months ago. Google Cloud Threat Intelligence published research on adversaries leveraging AI for vulnerability exploitation and initial access — meaning the attacker side of the equation is also being automated.
The implication for your log strategy is direct. If an agent in your environment behaves anomalously, you now face a genuinely ambiguous question: was that a model making a poor judgement call, or was that an attacker manipulating the agent's inputs to make it do something useful for them? Prompt injection, poisoned retrieval sources and compromised tool endpoints all produce behaviour that looks, from the outside, exactly like a model having a bad day.
Distinguishing the two requires decision-level detail. Without it, every agent misfire becomes an unresolvable incident that you cannot confidently close out — and unresolvable incidents are the ones that cost you contracts, because the client's security team has no basis on which to accept your assurance.
What the evidentiary layer actually consists of
Building this in is not exotic. It is, however, something that has to be designed into the agent architecture rather than bolted on after an incident. At Trebta, we build it from day one on every agentic AI system we ship, because retrofitting it is materially harder and always happens under pressure.
Decision-level tracing
Every step of the run captured as a first-class record: the state the agent held, the action it selected, the reasoning artefacts available at that moment, and the outcome. Not a text dump — a structured, queryable trace where each decision has an identity you can point a third party at.
Tool-call provenance
For each tool invocation: what was called, with what arguments, under which credential or delegated identity, what the tool returned verbatim, and how long it took. The returned payload matters enormously. If a retrieval step handed the agent poisoned content, that payload is the evidence, and it exists nowhere else once the run completes.
Version pinning across the stack
Model version, system prompt version, tool schema version, policy version, and the configuration hash of the orchestration layer — all bound to the run. "The model was updated at some point that month" is not an answer you want to give an auditor.
Replayable runs
The ability to take a historical run and step through it deterministically, with recorded tool responses substituted for live ones. This is the difference between explaining an incident and demonstrating it. It also turns every past incident into a permanent regression test.
Retention that matches your obligations
Incidents surface weeks after they happen. A seven-day retention window on agent traces means your investigative capability expires before your problems do. Retention needs to be set against your contractual notification windows and your regulatory exposure, not against your storage budget.
Third-party readability
The final test: can the trace be exported into a form that an external security team, auditor or insurer can read without your engineers narrating it? If reconstruction depends on tribal knowledge, you don't have evidence. You have an argument.
The five-question self-assessment
Take an agent you have in production and answer these honestly:
- Can you reconstruct a specific run from 30 days ago — not the inputs and outputs, but the full sequence of decisions and tool calls?
- Can you attribute a single decision to a specific model version, prompt version and tool response?
- Can you distinguish a model error from an injected instruction using data you already hold?
- Can you produce all of that for a third party within the notification window in your largest client contract?
- Can you prove the trace itself wasn't altered after the fact?
If you answered no to two or more, you don't have an agent governance gap. You have an agent forensics gap, and it will surface at the worst possible moment — during someone else's investigation, on someone else's timeline.
Talk to us before the incident, not after
Trebta sits at the intersection of agentic AI engineering and security, which is an uncommon combination — most AI shops build the agent and assume the evidence layer is somebody else's problem. We run a short diagnostic on existing agentic deployments: we take one production agent, attempt to reconstruct a real run from your existing telemetry, and show you exactly where the trail goes cold and what it would take to close it. If you're still at the design stage, we build the evidentiary layer in from the first commit as part of our AI and cyber security work.
Either way, the useful time to find out whether you can prove what your agent did is before someone formally asks. Get in touch with our team and we'll walk through it with you.