The question that arrives first
Every conversation about an AI support agent lands on the same question, usually before anyone has described an actual customer problem: what is the deflection rate?
Fair question. Wrong one to answer first. The order matters more than most teams believe, because the first metric a team names is the metric its design starts serving. Name containment first and containment is what you will get. You will get it from the behaviors that are cheapest to build, and most of those behaviors make the customer’s day worse.
The number I can report from my own work is 32% ticket deflection across global support volume, from LLM-powered virtual agents running inside live Zendesk-based workflows. It is a true number and a poor benchmark. You do not have my denominator, my product, my customer base or my knowledge base. Anyone who hands you a containment percentage without those 4 things has handed you a ratio with no units. Treat mine the same way.
Containment is a ratio you can move without helping anyone
Deflection measures conversations that ended without a human. That is the whole of what it measures. It does not know whether the customer got an answer or gave up. It does not know who opened a second ticket from a different address, called their account manager or went and posted about it.
Consider the levers that raise it. Make the path to a person harder to find. Require the customer to state the problem twice before the handoff triggers. Keep the agent talking through the exact moment when a specialist would have said “let me get someone.” Close the conversation after silence and count it as contained. Every one of those raises the number this week. Not one of them resolves anything. They are also the fastest changes to ship, which is precisely the danger. When containment is the target, the backlog sorts itself by ease, and the easy items hide the customer rather than serve them.
The levers that genuinely raise containment are slower and far less photogenic. Fixing the knowledge-base articles that went stale at the last release. Rewriting the entitlement lookup so the agent can check what the customer actually bought. Getting Product to change the default that generates the tickets in the first place. None of that work looks like AI, so it rarely gets funded out of an AI budget, and the agent gets tuned instead.
A deflected ticket and a resolved problem are not the same event. One is measured at the company. The other is measured at the customer.
The failure that costs the most is not the one you are watching
A support agent has 2 ways to fail a customer. It can refuse: say it cannot help, hand off, cost the customer a minute. That failure costs you a ticket. Or it can answer confidently and wrongly, and then the customer acts on it. Applies the wrong policy to a production endpoint. Waits out a fix that was never coming. Turns off the control that was protecting them.
Only the first failure shows up in a containment metric, and it shows up as a loss. The second shows up as a win. Contained, no human required, closed, until it comes back a week later as an escalation with a different subject line and an angry account owner attached. Optimizing containment does not merely miss that failure. It rewards it.
Measure these first, and in this order
Resolution, measured at the customer. Of the conversations the agent handled alone, how many produced no further contact from the same customer about the same problem inside a defined window? Pick the window and defend it: short enough to mean something, long enough to catch the boomerang. This metric is unpopular because it is expensive. It needs identity resolution across channels. It needs a human decision about when 2 tickets are the same problem. Teams skip it and report containment precisely because containment needs neither. That is the entire reason containment is the industry’s favorite number.
Handoff fidelity. When the agent gives up, what does the specialist receive? The transcript, the steps already attempted, the account context and the diagnosis the agent had formed? Or a fresh queue item that starts the customer at “Hi, can you describe the issue?” The cheapest available proxy for customer anger in an automated support system is the count of questions a customer has to answer twice. Instrument it. A handoff that drops state converts a mildly annoyed customer into a hostile one at the precise moment a person is finally listening.
Wrong-answer rate on a graded set. Build an evaluation set out of real conversations, labeled by people who have worked the queue. Grade the agent as correct, incomplete or wrong. Track wrong separately from did not answer. They are different diseases with different cures: wrong answers come from retrieval and stale content, refusals come from scope and tooling. A single blended accuracy score averages them into a number nobody can act on.
Then containment. Once those 3 metrics are instrumented and reported next to each other, containment finally becomes useful: a monitoring signal, never a target. Watch it for movement and investigate the movement. Containment rising while the wrong-answer rate rises is not a win. It is an incident with a delay on it.
What changes when you reorder
Reordering the metrics reorders the work, which is the real point. Resolution-first sends the opening weeks of an agent program into the knowledge base, the entitlement data and the escalation path, because that is where resolution actually lives. Handoff fidelity forces a decision about context passing before launch, when it is cheap, instead of after the first bad week. A graded evaluation set produces the artifact you will need anyway, the moment Security or support leadership asks how you know it works.
Containment-first sends those same weeks into prompt tuning and conversation design. Both matter. Both are finishing work, not foundation. A beautifully designed conversation sitting on top of a wrong knowledge base is a confident wrong answer, delivered politely, at scale.
The number is a result, not a goal
Goodhart’s law usually gets quoted at economists, but support operations gets the sharpest version of it: a measure that becomes a target stops being a measure. Containment is the most target-shaped number in this category. It is a single percentage, it moves fast, it fits on a board slide, and it can always be improved by making the customer’s experience worse.
So report it. Report it plainly, once, with its denominator attached and no adjectives stacked on it. Then put resolution, handoff fidelity and wrong-answer rate on the same page, at the same size. Hold the line when someone asks why containment is not higher this quarter. The honest answer is the strongest one: it could be higher by Friday, and here is what we would have to do to our customers to get it there.
Written in a personal capacity. Nothing here represents the position of any employer, past or present.