This is part three of a three-part series on how much authority to hand an AI agent. Part one is the agent that only watches. Part two is the agent that asks permission before it acts.
We have an agent that has been running nine times a day, every business day, since the fourth of June. In all that time, it has been allowed to change exactly two things about our store.
It earned both of them separately, after we watched it call each one correctly while a human did the actual work.
That is how we handle AI agent autonomy, and it runs against how most people set these up. The usual approach is to decide up front how much an agent should be allowed to do, wire it that way, and find out later whether that was wise. We do the reverse. Everything launches able to change nothing, and authority gets added one specific action at a time, with a reason attached to each.
Here is how that actually works, including the day it went wrong.
Key takeaways
-
The agent launched on June 4 with zero authority to change anything, and stayed that way for four weeks before its first autonomous action.
-
A written table maps every kind of incoming message to two decisions: reply in the channel, and tell John. Sometimes both. Often neither.
-
It has two approved autonomous actions today, each added only after we had done that exact task by hand enough times to trust the pattern.
-
Each approved action is scoped to the one person with authority over it. The same words from anyone else get a polite decline.
-
The first autonomous build shipped broken, because the agent trusted a number it should not have and quietly skipped a step.
-
Silence is a valid outcome. Most of its nine daily runs end with no message anywhere.
-
Running it costs nothing recurring, which is what makes a slow rollout affordable.
What does the agent actually do all day?
It listens. Nine times a day, every hour from 9 am to 6 pm Eastern except 1 pm, it wakes up and checks whether the team responded to anything it asked.
It exists because our other agents kept starting conversations they could not finish. One posts a daily summary of the product listings it built and asks the team to sort out anything blocked. People reply. Somebody fills in a spreadsheet column. Somebody answers a question in a thread. With nothing listening, all of that sat there until a human went looking.
This one closes the loop. It reads the replies, compares them against what it knew last time, answers what it can answer, and messages me directly when something needs a decision. The 1 pm gap is deliberate. That hour belongs to the build task, and two agents working the same data at the same time is a problem you only have to create once.
Why did it launch with zero authority to change anything?
Because we did not know yet what the team would ask it for.
The first version could reply in one Slack channel and message me. That was the entire permission set. It could not create a product, edit a spreadsheet, change a tag, or touch anything in the store. The instructions said so in plain language, along with the reason: watch the team's patterns for a couple of weeks before granting any write authority.
That waiting period turned out to be the most valuable part of the build. What people actually asked for looked nothing like what I would have guessed. Reading a month of real requests told us which ones repeated, which were one-offs, and which sounded routine but hid a judgment call. Designing permissions for behavior you have not observed yet is just guessing with extra confidence.
One line in the instructions governs the whole thing: if you are tempted to bend any of these rules, stop and put it in the message to John instead. New authority gets written down deliberately, never assumed.
How does the agent decide whether to speak at all?
A written table, covering every case we could think of.
Each row is a kind of incoming message, and there are two columns: reply in the channel, and message John. Every signal resolves to a yes or no in both. Banter and a quick thanks get a short reply and nothing sent to me. An operational acknowledgement gets a tiny reply or silence. A factual question it can answer from its own notes gets answered on the spot. A real decision, somebody saying kill this product or change this spec, gets a brief acknowledgement in the channel and a message to me, because an acknowledgement is not an answer.
The most important row covers requests it is not allowed to fulfill. Somebody asks it to publish something. It replies with a friendly version of "noted, I will get with John at our next catch-up," messages me, and does nothing else. No refusal, no lecture, no silent failure. It closes the loop socially while taking no action.
Writing that table was most of the work. An agent without one improvises, and an agent that improvises about when to speak will eventually improvise about when to act.
What does it mean for an action to be ratified?
It means we did that exact task by hand, repeatedly, watched the agent call it correctly every time, and then wrote the permission down with the trigger and the limits spelled out.
Two actions are on that list today, and they belong to two different people. One is a specific kind of duplicate product build that a teammate on the live selling side needs regularly and that follows a fixed spec every time. The other is building out a batch of listings once the person who runs that program says the batch is approved.
Both share the qualities that make an action safe to hand over. The request has one obvious meaning. The output is a draft, never something live to customers. The person asking holds the actual authority to ask. The work is identical every time, which means there are steps but no judgment.
Everything else is still a polite decline and a message to me. The list is short on purpose. A short list of things an agent may do is easier to hold in your head than a long list of things it may not.
Why is authority tied to a person and not a phrase?
Because "build this batch" is only a safe instruction when it comes from the person responsible for that batch.
Both approved actions are scoped to their own named requester. If anybody else posts the identical request, word for word, the agent gives the standard acknowledgement and messages me instead of acting. That is not a comment on my colleagues. A Slack channel is a place where anyone can type anything, including somebody quoting an old message or pasting a screenshot.
This matters more as agents read more of your workplace. An agent that takes open-ended instructions from any channel a person can post in is a set of keys you have handed out and stopped tracking. Scoping every approved action to a named requester keeps the answer to "who can make this happen" short enough to remember.
What happened the first time it acted on its own?
It shipped a product attached to no sales channels, and a teammate spent an evening backfilling all 24 of them by hand.
Here is the sequence. On July 2, the first autonomous build, the agent reached the step where it attaches a new product to our sales channels. We published to two dozen of them between marketplaces, apps, and storefronts. The agent checked a field reporting how many channels were available, read a zero, concluded there was a permissions problem, and carefully skipped the step rather than doing something it expected to fail.
Nothing was wrong with permissions. That particular count field is unreliable and returns a shop-level number that has nothing to do with the product in front of it. The agent was reading a broken gauge and responding sensibly to a false reading.
The product went out attached to nothing, and looked completely fine from the inside.
What did that failure actually teach us?
Two things, and the second one is the one I still think about.
The narrow lesson is now a binding rule in the instructions: verify channel attachment by counting the actual list of channels, never by reading the count field, and do the attach step every time even if the number looks strange. That is a one-sentence fix to one piece of bad data.
The broader lesson is that careful and safe are different things. The agent did the cautious thing. It saw something that looked wrong and declined to act, which is the behavior everybody says they want from an autonomous system. Skipping a step is still a decision, and a quiet one, which makes it worse than an error. A loud failure gets fixed in ten minutes. A defensive skip ships and looks fine until somebody notices the product has no home.
Since then, when we hand over a new action, we spell out what the agent should do when something looks off: report it and stop, or continue anyway. Leaving that to its judgment is how you get a careful mistake.
How do you know when an agent is ready for more autonomy?
When you have already been doing that specific task by hand, over and over, and the agent has been calling it correctly the whole time without being allowed to act.
That is what the listening period buys you. For four weeks, the agent was saying, in effect, "here is what I would do about this," and a human was doing it. By the time we granted authority, we had a record instead of a prediction.
Anthropic has published research on measuring agent autonomy in practice that is worth reading if you are thinking about this seriously. The failure numbers point the same direction. Gartner predicts over 40 percent of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls. Granting broad authority on day one is an efficient way to collect all three.
The test I use now is simple. If I cannot describe the trigger, the exact steps, and what the agent should do when something looks wrong, it is not ready, and neither am I.
Why does the agent go silent most of the time?
Because it usually has nothing to add.
If nothing changed since the last run and nobody mentioned it, the agent posts nothing and messages nobody. Nine runs a day, and most of them end in silence in both places.
It does still write to its own memory and its own run log on every single pass, including the quiet ones. That is the same distinction from part one of this series: a quiet channel and a failed run are two different things, and the only way to tell them apart is a record the agent keeps for itself.
What does a slow rollout cost?
Patience, and nothing else, because the agent itself has no recurring bill.
Nine runs a day, every business day, on the AI subscription we already pay for. That is worth sitting with when you think about autonomy. If every agent came with a monthly invoice, a cautious rollout would be genuinely expensive, since you would be paying full freight for a month while the thing is deliberately allowed to do almost nothing. Somebody would start asking when it will earn its keep, and that pressure is exactly how premature authority gets granted.
Agents that cost nothing to run are agents you can afford to be slow with.
The whole series in one idea
Three agents, three levels of authority, and the level is the design decision that matters most.
The fraud agent only looks. It reads orders and discount codes every morning and tells a human when something is off. It cannot touch the store, and it never will.
The inventory audit proposes. It finds broken listings, writes them into a sheet with checkboxes, and a separate task acts on whatever a human approved.
The listening agent acts on two narrow things, having earned each one.
Pick the lowest level that solves your problem and stay there until the agent gives you a reason to move up. Almost every agent worth building lives on the first rung and never needs to leave it.
Frequently asked questions
What is AI agent autonomy?
It is how much an agent is allowed to do without a human approving it first. Autonomy works better as a list of specific actions an agent may take, each with its own trigger and limits, rather than a single overall level of trust.
How much autonomy should an AI agent start with?
None. Launch it able to read and report but not to change anything, and leave it there long enough to see what people actually ask it for. Real requests look nothing like the ones you plan for.
How do you decide when to give an agent more authority?
When you have done that exact task by hand many times, the agent has been recommending the right call throughout, and you can write down the trigger, the steps, and what it should do when something looks wrong. Missing any of those three means it is not ready.
Should an AI agent accept instructions from anyone in a Slack channel?
No. Scope each approved action to the specific person with authority over it. Identical wording from anyone else should get an acknowledgement and an escalation, not an action, because a channel is somewhere anyone can type anything.
What is the most common way autonomous agents fail?
Quietly. Ours read an unreliable data field, assumed something was broken, and skipped a step to be safe. A loud error gets fixed the same day. A cautious skip ships and looks fine until somebody notices the work is incomplete.
Should an agent report in even when nothing happened?
No. Silence should be the normal outcome. The agent should still write to its own memory on every run, so that a quiet channel and a failed run are never confused for each other.
Does a slow, cautious agent rollout cost more?
Only in patience. Because there is no recurring fee beyond the AI subscription you already pay for, a month of running an agent that is deliberately allowed to change nothing costs you nothing but time.
Final thoughts
The pitch for agents is that they act on your behalf. The uncomfortable half of that promise is that acting on your behalf includes acting wrongly on your behalf, at whatever speed you handed them.
Giving away authority one action at a time is slower, and it feels overly cautious while you are doing it. The payoff is that months later I can name every single thing our agents are allowed to change without asking, and explain why each one is on the list. That inventory is not a limitation on what we built. That inventory is the thing we built.
Start at zero and make each step get earned.
Interested in learning more?
This is part three of a three-part series on agent authority. Part one is How We Built an AI Agent to Catch Discount Code Fraud Every Morning. Part two is How We Built an AI Agent That Asks Permission Before It Acts.
For more of the operator side of running a store, take a look at Always Delete Your Test Coupon Codes Before a Bot Finds Them and Shop App "Bought in Past Month": Is Shopify Leaking Your Sales?
