Back to the blog
Guardrails·3 min read

Why your agents keep repeating the same mistakes

Samuel Corso
Samuel Corso
Share
Why your agents keep repeating the same mistakes
Guardrails

On Monday, a senior engineer corrects an agent's PR: wrong logger, use the structured one. On Wednesday, a teammate's agent makes the same mistake. By Friday it's back in a third PR. Nobody is incompetent. The agent doesn't remember your team's corrections, and every session starts from zero.

Individually, each fix is trivial, thirty seconds in code review. In aggregate, your most experienced engineers spend the week re-teaching the same lessons to a system that can't retain them, and the cost never shows up on a dashboard because it's spread across dozens of small corrections nobody bothered to count.

Sometimes the same failure mode is not trivial at all. Last month an agent made CI green by deleting the failing test. The diff looked like success, a green check and a clean merge, and the regression it was hiding shipped anyway. That's the wrong-logger problem with a worse blast radius: an instruction the model was given, ignored over a long session, and caught by no one because nothing outside the model was enforcing it.

Frequency, not severity

The instinct is to obsess over the catastrophic case, the agent that runs DROP TABLE against a prod-shaped credential (OWASP's agentic Top 10 files this under tool misuse and unsafe code execution). Those make great post-incident stories, but severity is rare and frequency is constant. It's the low-grade repetition that actually erodes trust in your agents and burns your seniors' time, and it stays invisible because no single instance is worth escalating.

Why the agent can't fix this itself

The obvious answer is a rules file, and you should have one. CLAUDE.md and .cursor/rules are real institutional memory. But they share one weakness: they're advisory. The model reads them, weighs them against everything else in a long context, and can quietly decide not to follow them on turn two hundred. They're hand-written, so they only cover what someone remembered to write down, and they're unmeasured, so you never learn which ones are actually holding.

That's the core idea behind AgentTrail. If a mistake must never recur, the thing that prevents it has to live outside the model, where the agent can't choose to forget it. Enforcement that depends on the agent deciding to comply isn't enforcement.

What AgentTrail does

It watches your real agent sessions and detects when the same correction recurs across developers and agents. The moment a pattern repeats, it surfaces it and offers to turn it into a rule, long before anyone has re-fixed it a dozen times. You're not authoring policy from a blank page; you're confirming the ones your own history already proves you need.

Before you enforce anything, you can replay the rule against your last thirty days of sessions and see exactly what it would have caught, computed on your own data with no model calls. A rule ships once it's proven, not on a hunch. Then AgentTrail enforces it in real time as each action runs: warn, hold it for a human, or block it outright before it touches your systems. Because the check lives outside the model, it holds no matter what the agent decides mid-session. The catastrophic command is handled the same way. It's just one more pattern that must never recur.

And because every decision is recorded, you can watch the repeat rate for each rule fall over time, see which rules are earning their keep, and add up the risk you've avoided. The third correction becomes the last one, and you can prove it.

These failure modes have names. For the systematic version, the OWASP Top 10 for agentic applications maps them out, and our OWASP coverage page shows where each one can be enforced today and where it can't.

Stop babysitting your AI agents.

AgentTrail catches the risky actions your agents take and turns each repeat into an enforced guardrail. Get started with the open-source guard.

Related reading