The enthusiasm around autonomous agents continues, and alongside it there is a question that comes up in every conversation with an organisation starting to use them seriously. It is not always phrased the same way, but it is always the same question.
What happens if it does something we did not intend, and we only find out tomorrow.
This is not paranoia. An agent is useful exactly to the extent that it is allowed to act without asking. The moment every action needs approval, what you built is an elaborate form. The moment no action needs approval, what you built is something nobody in the organisation will agree to own. Almost all of the value sits in the range between those two points, and all of the work is knowing where in that range to stand.
The method we developed for ourselves, and later for organisations, rests on three questions. What it is allowed to do, what it costs, and how you know what it did.
Why a permissions list is not enough
The natural way to bound an agent is to write a list of permitted actions. That works at the start and falls apart fairly quickly.
The reason is that the real question is almost never whether an action is permitted, but whether it is permitted now, in this context, after what came before it. Sending an email is entirely legitimate. Sending forty emails in two minutes to a list the agent assembled itself from a file it read is something else, even though every individual step cleared the list.
What we put in place of the list is a ratio. An agent starts with a narrow space to act in and widens it as it accumulates a history of actions that passed review. In the other direction, an action that was rejected or turned out to be wrong narrows it back.
The logic here is not especially technological. It is exactly how a new employee operates. In the first week they show you everything, after six months they show you only the unusual cases, and if they get something badly wrong they are back to the first week.
On budget
The part people are most surprised by is that an agent without a budget is not really an agent.
When there is no limit on how much a given task may consume, there is no pressure to choose between what matters and what is merely possible. The agent will do everything, in whatever order it happened to come up, and stop when something else runs out. That is not only expensive. It means the most important task of the day may simply not get done because something marginal came first.
Once there is a budget, a question appears that has to be answered in advance. What matters more here. And that question, which sounds like a technical one about resource management, is really an organisational one that usually nobody bothered to answer in writing before.
On the log
Every system like this writes a log. That is the easy part, and also the easiest place to fool yourself.
A log nobody reads is not an oversight mechanism. It is after the fact evidence, useful exactly once, during the investigation that follows. What turns a log into something live is that a specific person goes through it on a regular cadence, and that it is short enough for that to be realistic.
So we do not record everything. We record what cannot be undone, what costs money, and what leaves the organisation. The rest is kept but not surfaced, because a log you cannot scan in five minutes becomes, within two weeks, a log nobody scans.
What did not work for us
The first version we built was far stricter. Nearly every meaningful action stopped and asked for human approval, on the assumption that it is better to start on the safe side and loosen gradually.
What happened is that people stopped using it. Not because it got in the way, but because it turned the agent into something that required constant presence, which is precisely what it was meant to save. After a few weeks we discovered that part of the team was simply going around it and running things by hand, which is of course a far worse state than any level of autonomy we would have approved.
The conclusion was that oversight users escape from is not oversight. It only moves the risk somewhere that has no log at all.
Closing
Most of the public conversation about autonomous agents is about capability. What they can do now, and what they will be able to do in six months.
Our experience is that inside a real organisation that is almost never the bottleneck. The bottleneck is trust, and trust is not built from declarations but from history. Which is why our method spends much less time on what the agent is capable of, and much more on how it earns the right to do it alone.