Everyone guards the wrong artifact

Ask a team what they would lose if their AI program had to start over tomorrow, and most of them point at the prompts. Prompts are the visible part. They get versioned, argued about, and hoarded in private documents.

They are also close to worthless. A prompt is a current best guess written in the dialect of whatever model happens to be in production this quarter. Swap the model and half of it stops meaning what it meant. Change the product and the guess goes stale without anyone noticing. Prompts are cheap to write and cheap to replace, which is the definition of a weak asset.

The evaluation set is the opposite. It is the written record of every case the organization has agreed the system must get right. Why each case is in there. What a correct outcome looks like. None of that depends on a model, a vendor, or a framework. Rewrite every prompt, move platforms entirely, and a good eval set still tells you whether the new thing is better than the old one.

A prompt records what you tried. An evaluation set records what you learned. Only one of those is worth carrying into the next system.

Call it what it is. Organizational memory, with a schema.

It is not a benchmark

Public benchmarks answer a question nobody in a support operation is asking: how does this model do on average, on tasks that are not yours. An evaluation set answers a narrower and far more useful question. On the situations our customers actually bring us, does this system behave the way we said it should.

Which means the cases cannot be invented at a whiteboard. Synthetic cases test the imagination of whoever wrote them. Real conversations test the product.

Where the cases come from

Mine the operation, in roughly this order of value.

Escalations, sorted by reason. Every handoff is a labeled failure or a labeled boundary. Both are worth a case.

Reopens and repeat contacts. The customer came back, which means the first answer was wrong, incomplete, or unusable. All 3 are different cases.

Anything that touched a guardrail. Both directions: the stops that fired, and the situations where a specialist thinks one should have.

Major incidents. Every single one donates a case. If an incident produced a lesson and no case, the lesson is a story, and stories do not survive reorganizations.

The most-read knowledge articles. That is the demand curve, already measured, already sitting in the knowledge base.

The questions the operation is afraid of. Ask specialists directly: what would you not want this thing answering on its own. The answers arrive fast and they are specific.

Cluster rather than cherry-pick. One strange conversation is not a failure pattern. A pattern is a cluster somebody can name in a sentence, and naming it is most of the work.

Anatomy of a case

A case that ages well carries more than an input and an expected output.

  • The input as the customer actually wrote it, including the messy version with the missing detail and the wrong product name. Cleaning it up removes the part that makes it hard.
  • The context available at runtime. An expectation that assumes information the system cannot see is not an expectation, it is a wish.
  • The expected behavior — and “hand off, for this reason” is a first-class correct answer, not a failure. Sets that only reward answers train systems that never stop talking.
  • Provenance. One sentence on why this case exists and what it came from.
  • The owner and the date. Somebody agreed to this expectation on a particular day.

Then the ones that matter most in a review: the must-nots. What the system must never state, never promise, never touch. Security and compliance conversations care about the shape of those must-nots, not about average answer quality. A set without them cannot answer the only question that room asks.

Eval sets decay

They decay quietly, which is what makes them dangerous.

The product ships a change and expectations that were right last quarter are now wrong. Policy changes and a must-not becomes a must. Packaging changes and a whole category of question changes meaning. The customer mix shifts. A model gets swapped and the failure modes move somewhere the set was never pointed.

The failure state is not a suite that goes red. It is a suite that stays green while the operation gets louder. Green suite, angry customers, and a team that trusts the wrong instrument.

Managing decay is unglamorous and cheap. Date every case. Review on a schedule instead of on a scare. Retire cases that no longer describe the product rather than nursing them. Above all, check coverage against the current escalation reasons. If the top reason customers escalated this quarter has no cases behind it, the suite is grading last year’s system.

Who owns it

The most common ownership mistake is handing the set to whoever wrote the prompt. It quietly becomes a collection of cases the prompt already passes.

Split it by who actually knows the answer.

Support owns what a good answer is. They have been producing good answers by hand for years. The eval set is the first time anyone has asked them to write the standard down.

Security and compliance own the must-nots. They hold the authority to say never. Getting that in writing early is what makes a review a conversation instead of a fight.

Product owns the changes. When behavior changes, cases change in the same release, not 2 months later.

The AI team owns the ritual. The schedule, the tooling, the regression run, and the rule that nothing reaches production without it.

The healthiest signal to watch for: a support specialist who has never touched a model can add a case and knows what happens to it after they do.

Closing the loop for real

Manual mining happens twice and then stops. Somebody runs a heroic analysis before launch, repeats it after the first bad week, and then the calendar wins.

Making it continuous is an integration problem. Model Context Protocol server integrations connect the agent platform’s API to the conversation store. Conversations then get read at volume, grouped by failure pattern, and promoted into cases as routine work rather than a project. That is the plumbing behind the phrase “continuous improvement,” which otherwise means somebody intends to look at it again.

The loop closes on 3 habits:

Every rollout runs the suite before it goes out, not after.

Every incident donates a case in the same week, while the details are still exact.

Every new failure pattern gets its case written before the fix. Fix first and the case gets written to match the fix, which is how a regression sneaks back in wearing a clean shirt.

What it actually buys

Permission to change things. Swapping a model stops being an act of faith and becomes a measurement.

An answer in security review. “How do you know it behaves” is a document, not a promise.

Onboarding. New team members learn the operation faster by reading real cases than by reading any handbook, because the cases carry the exceptions and the handbook never does.

And durability. Deflection was the headline on the rollout I own — 32% across global support volume. That number is a result, and results are perishable. The set of cases behind it is what makes the next version better than this one. Then the version after that better again, long after everyone who built the first one has moved on.

Written in a personal capacity. Nothing here represents the position of any employer, past or present.