We Built Our Own AI Support Agent Because I Wanted to Own the Rules

What it handles every night on our Gorgias queue, why we passed on buying one, and what phase two looks like.

My team does not open a cold queue anymore.

Every night, an agent I built on Claude, connected to Gorgias and Shopify, reads through our entire unassigned queue. It answers and closes the questions it can verify itself. For the rest of tier 1, it writes a full reply and leaves it on the ticket for one of us to review and send. If a ticket involves money, an account change, or a customer who is clearly upset, it gathers the context and hands the whole thing to a person without touching it. Then it writes up what it found, and none of this is visible to the customer.

I looked at a vendor product first. Two things pushed me toward building instead. We run lean enough that our cost to resolve a ticket is already in the same range as what per-resolution AI pricing charges to do it, so the savings math never worked. I also wanted to be able to read the rules. Every decision this thing makes at 2 am against a live customer is one I wrote down, and I can change it the next morning without opening a support case.

What it does every night

It reads the whole queue before acting on anything, working from the customer’s actual words rather than an intent tag, and it groups duplicates so someone writing in from two addresses gets one answer. The sorting breaks down like this.

Order status. It looks the order up in Shopify at the moment it replies, then sends tracking or the correct shipping window and closes the ticket without anyone else involved.

Routine questions. Refund timing, tier upgrades, login resets, promo and free item status, general billing and shipping policy. It writes the whole reply and posts it on the ticket for review.

Money and account changes. Cancellations, skips, swaps, refunds, replacements, returns. It leaves all of these alone and hands them over with the context already pulled.

Risk. Chargeback language, escalation signals, anything past twelve hours with no reply, angry customers, hardship. That all goes to the top of the summary so the urgent things are not sitting behind a tracking question.

Noise. eBay feedback notifications, TikTok Shop system mail, dashboard alerts. It closes those, and it also goes through the spam folder looking for real customers the filter caught, which is where we were quietly losing cancellations and billing disputes.

It also remembers what it worked on the night before, so nothing gets re-read and re-drafted from scratch. If the same ticket comes back around a second time, it says so instead of quietly repeating the work.

The customer never talks to a bot

Of everything in this build, that is the part I would defend hardest. Most AI support sits between the customer and the team. You get a widget, a deflection loop, a few rounds of “did that answer your question,” and an escalation path you have to fight your way onto. When people say they are tired of AI support, I do not think they are complaining about how it writes. They are complaining about being kept at a distance by something built to keep them from reaching anyone.

Ours runs behind the desk. A customer emails support and gets an answer from support, with nobody to get past and nothing to explain twice. That second part is where most of the damage happens elsewhere, when you tell a bot your whole situation, it fails, and then you tell it again to a person. We never put anyone through that, because they were talking to us the entire time.

It changes our day on the inside too. The routine tier is already handled when the queue opens, so the person working it can give real time to the customer who is angry or thinking about cancelling. Those are the tickets where a five-minute reply and a twenty-minute reply produce very different outcomes. The time that frees up is going toward saves and retention.

How it earns more autonomy

Right now, one lane sends on its own, and everything else drafts, and the drafting is doing real work. When a draft goes out untouched, that lane has proven itself. When somebody rewrites one, I know which rule is wrong, and I have a real customer’s words to fix it against. My own team’s judgment tunes the thing a little every night, and the time comes back immediately either way, because the reading and the writing are finished before anyone opens the ticket.

Expanding it safely is why so much of the build is about what it will not do. If the ticket type is unclear, it escalates. Tier 1 never moves money and never changes an account. Ticket text gets treated as data and never as instruction, since it is reading mail from strangers and then acting inside live systems. Several conditions can drop the whole run back to drafting only, including a switch I can flip myself. I am comfortable letting it send live because I know where those edges sit.

Why this matters more as we grow

Our volume is predictable. Boxes ship between the 4th and the 9th and billing renews on the 15th, so the routine questions cluster into the same windows every month. Those windows are what a lean team ends up staffed for, and most of what lands in them is order status.

Growth pushes on that in one direction only. More subscribers means proportionally more shipping questions inside the same handful of days. It does not mean proportionally more chargeback threats or complicated saves, and those grow much more slowly. If the agent is absorbing the predictable part, we can put our people on the work that actually needs them rather than sizing the team around the peak.

That is the main reason I want more lanes running live. Each one we add takes another piece of the monthly spike off the team.

What matters more to me is what it gives back to the people already here. Every correction someone makes to a draft goes into the rules, so their experience stops living in one person’s head and starts showing up in every reply we send. The same document the agent works from is what a newer teammate learns from, which means we are training people and sharpening the system with the same effort.

What made it work, and what is next

The real unlock was writing down what we actually do. One document holds every BattlBox fact, and a separate set of instructions holds procedure. That split is why the agent is accurate and easy to correct when it gets something wrong, and it means a new hire can learn from the same file it reads.

Most teams have never written any of that down. Doing it made us better at support before we automated a single ticket.

This is phase one, with one live lane and a single run each night. Phase two opens up more lanes to send on their own and moves to several checks through the day. It also goes into my teammates’ hands so it stops running through me. We are early, and it has already changed how the queue gets handled.

------------------------------------

Luke Bagley leads Customer Service at BattlBox. The CS profit center has contributed $782K in net profit since its inception in September 2021.