Back to the blog
Autonomy·6 min read

Your agent got a new model overnight. Should it keep its privileges?

Samuel Corso
Samuel Corso
Share
Your agent got a new model overnight. Should it keep its privileges?
Autonomy

If your team runs Cursor agents on OpenAI's models, they could be running on something else by Friday. OpenAI plans to cut Cursor off on Thursday, November 12, after SpaceX bought Cursor in August. As far as I can tell the date still isn't final. Either way, it raises a question that matters even if you've never opened Cursor: when the model under your agent changes, what happens to everything you trusted it with?

Most model switches don't make the news. Ours never did, and we've had plenty.

Nine models in seven months

Our main repo is seven months old, and this week we went through its history. Nine different Claude models have co-authored commits on our main branch: Opus 4.6, 4.7, 4.8, 5 and 5.5, Sonnet 5 and 5.5, and Fable 5 and 5.1.

If you use Claude Code and never picked a model, the same thing has probably happened to you. Claude Code's docs keep a list of what its default setting has pointed to. On a Max plan it went from Opus 4.8 to Opus 5 to Opus 5.5. On Pro, one update moved it from Sonnet 5 straight to Opus 5.5. The standard install updates itself in the background, so the change shows up the next time you open a session. The opus and sonnet shortcuts move too. The docs say they "point to the recommended version for your provider and update over time."

None of this is a scandal. Models get better and the defaults move with them. Once in a while, two vendors fall out. The trouble is what a switch does to everything you decided about your agent before it happened.

The badge problem

Picture a hospital where privileges belong to the badge instead of the surgeon. One night the badge changes hands. The next morning someone new walks into the operating room with every privilege the last surgeon earned, and nobody stops them, because the badge checks out.

Real hospitals don't work like that. In most US hospitals, every newly granted privilege comes with a period of focused review. The Joint Commission, the nonprofit that accredits hospitals across the US, calls it focused professional practice evaluation. It applies even to a surgeon with twenty years behind them who just moved from the hospital down the road.

Most agent setups are the badge hospital. Think about what your agent now does without asking. Maybe it runs overnight, merges small PRs or reads production logs. You allowed that because of how it behaved. But your allowlist sits in a settings file that has no idea which model is reading it, and your API keys and CI permissions don't know either. Whatever you decided to let your agent do in June, you decided about a model that may not be running anymore.

"But the new model is better"

Usually it is. But the model is the part that makes the calls. It reads your CLAUDE.md and decides how seriously to take it. It sees a failing test and decides whether to fix the code or loosen the test. Vendors measure this, and the numbers move between versions. When Anthropic launched the Claude 4 models, it said both were 65% less likely than Sonnet 3.7 to take shortcuts or exploit loopholes on tasks that invite them.

That's good news if you were upgrading. It also means 3.7's track record would have been the wrong guide to 4 on exactly that behavior. A model can get better at writing code and worse at stopping to ask before it does something drastic. A benchmark can't tell you which of those happened in your repo, so you have to watch the new model work.

What should carry over

So should it keep its privileges? Not automatically. A new model is a new agent wearing the old one's badge. It should earn its way back, just faster than the first time. Here's how we're building this at AgentTrail.

The track record belongs to the model that earned it. When the model changes, the old record stays, labeled with that model, but it stops counting as evidence about the new one.

A model change should never be silent. It should be recorded like any other change to who the agent is, and whoever owns the agent should hear about it the same day.

A person decides what carries over, not some background update. In the first version, the owner would get a notice and pick: keep the agent's privileges, or put it back under supervision until it earns them again. Later we want the default to follow the risk. Something like auto-merging PRs would drop a step. Anything that touches production data would start over.

Re-earning should be quicker the second time, because the rules are already in place and you know what normal looks like. In our current draft, an agent first earns unattended runs in a project after 30 decisions on its held actions, at least 90% of them approvals. After a model change, we think roughly five clean runs should get it back up each step.

One thing we haven't settled is the gap between the switch and the owner's decision. Drop privileges by default, and teams whose new model is fine get annoyed. Keep them by default, and the agent has the most room on the one day you know the least about it. It's one of the open questions we're taking to our first design partners. If you have a view, I want to hear it.

What you can do this week

You don't need us for most of this.

Find out what your agents actually run. If your agents add a co-author line to their commits, your git history already knows. Claude Code adds one, and ours mostly name the model:

git log --format='%(trailers:key=Co-Authored-By,valueonly)' | grep . | sort | uniq -c | sort -rn

Pin the model for anything that runs without you watching. In Claude Code, use the full name, like claude-opus-5-5, not a shortcut. If you manage a team, limit which models people can pick with availableModels in managed settings. You'll still upgrade, just on a day someone chose.

Treat the first week on a new model like a new hire's first week. Put the risky actions back behind a person for that week, and actually read the first sessions. If your Cursor agents run on OpenAI's models, that week starts the day they switch.

Keep a short list of things no model does without a person saying yes, however good its record. Start with four: skipping git hooks with --no-verify, switching the hooks off altogether, editing its own configuration and opening credential files. None of that should depend on which model is on the other end, so put the list where the model can't argue with it, in a hook that checks each action before it runs.

What we're building

AgentTrail OS is our product for teams running coding agents. It already shows which model ran each session, and it's free to start at agenttrail.sh/signup. What it doesn't do yet is tell you when that model changes, so today you only spot a switch if you go looking. We're building that notice, along with privileges that drop when the model changes and come back as the new one proves itself.

Whether or not you try it, run that git command and email me your count at sam@agenttrail.sh. Ours was nine.

FAQ

Does Claude Code change its default model on its own?

Yes. Its default setting and its opus and sonnet aliases move to newer models over time, and the standard install updates itself in the background. To stay on one model, set the full name, such as claude-opus-5-5.

How do I pin the model my team's agents use?

Set the full model name in settings, or pass --model for one session. To enforce it, list the allowed models in availableModels in managed settings. A managed model is only a starting default.

Should an agent keep its privileges after a model change?

Not automatically. Its track record belongs to the model that earned it. Record the change, tell the agent's owner the same day, and let a person decide what carries over.

Stop babysitting your AI agents.

AgentTrail catches the risky actions your agents take and turns each repeat into an enforced guardrail. Sign up free and connect your first agent.