The demo was never the hard part
A working agent takes weeks. Putting it in front of customers takes something else, and it is not more model work.
Enterprise AI programs die in the same room. The pilot is good, the demo lands, leadership wants it yesterday. Then it goes to Security, to Legal, and in some industries to a risk committee, and it stops. Not with a rejection, which would at least be actionable, but with a request for information nobody on the project can answer in the shape the reviewer needs. Weeks pass. The champions get reassigned.
The artifact that clears that room is a document. Writing it is the hardest and most valuable work in the program, and it is almost always scheduled last, for whoever is free.
Writing it last is the expensive choice
A compliance case written after the build is a description. Written before the build, the same document is a specification.
The decisions it forces are the ones teams defer by default. Which tools the agent may call. What gets logged, and for how long. What it must never do without a human. How it gets turned off, and by whom. Each is nearly free on a whiteboard and expensive in a running system. Teams that write the case last discover that the logging their audit trail requires was never captured. The first months of production behavior cannot be reconstructed. The reviewer’s only honest answer is no.
Worth saying plainly: Security is not the obstacle. Security’s job is to bound blast radius. What gets rejected is an unbounded blast radius described in adjectives. The agent is safe. We have guardrails. There is a human in the loop. Hand that same reviewer a bounded system described in nouns and the conversation changes character.
Security does not reject AI. Security rejects an unbounded blast radius described in adjectives.
The anatomy, in 7 sections
The threat model
Not “is the model safe.” Who is the adversary, what do they want, what can they reach. Name 3 actors. The customer who tries to talk the agent into a commitment it cannot make, or into data that is not theirs. The internal user routing around a policy that exists for a reason. And the content itself as an injection vector: an attachment, a ticket body, a knowledge-base page. The third gets missed because the knowledge base looks like your own data, and once it is retrieved, so does everything in it.
The asset under protection is not the model. It is the data plane the agent can reach. Write down the union of the permissions of every tool it can call, as a list of nouns: these tenants, these fields, these actions. That union is the blast radius. Arriving with it already written is worth several meetings.
The data flow
Every hop, in order: what the customer types, where it lands, which processors see it, whether it crosses a tenancy or a region, who retains what, and whether any of it trains anything. The sentence a reviewer is listening for is contractual: customer content is not used to train third-party models, and here is the clause that says so. If you cannot point at the clause, do not write the sentence. Write the real position and the mitigation instead. Reviewers forgive a limitation they can see. They do not forgive finding one.
Include the boring hops: analytics, log shipping, the evaluation store, the transcript export built for a dashboard. The undocumented hop is the finding.
Retention
Retention is where most cases collapse. They specify one number for “logs” while the system holds at least 4 distinct artifacts with different needs: the conversation transcript, the assembled prompt including retrieved context, the model output and the derived evaluation data.
State the conflict rather than hide it. The evaluation loop wants long history, privacy wants short. Resolve it by separating raw from derived: short retention on raw transcripts, longer on de-identified evaluation artifacts, named access for each. Then write the part nearly everyone forgets. Deletion propagation. A deletion request has to reach the evaluation store and the prompt logs, not just the ticket. Deletion that stops at the ticketing system is a finding with a fuse on it.
The human boundary
The most consequential table in the document. Classify actions by reversibility, not by how risky they feel. Read-only retrieval and explanation: autonomous. Changes confined to the ticket, such as tagging, routing and drafting: autonomous, logged, sampled. Changes to the customer’s environment, entitlements, billing or security posture: never autonomous, with no exceptions written in this quarter’s optimism.
Commitments are out of scope for the agent’s language as well. “That will be fixed in the next release” is a commitment the company has to honor, whether or not a tool call happened.
Name the review mode for each class. A human in the loop approves each action. A human on the loop reviews a sample afterward. Both are legitimate. Conflating them is what gets an otherwise sound design rejected. Reviewers have seen “human oversight” used to mean nothing.
Failure modes
Enumerate your own before anyone asks. Confidently wrong answers. Hallucinated policy or entitlement. Retrieval serving content that went stale at the last release. Tone failure on an angry customer. Silent looping. Cross-tenant leakage through injected content. Upstream model degradation, which shows up as latency before quality. And the one that never makes these lists: the agent was right and the customer needed a person anyway.
Give each one a detector, a signal and an owner by role. A failure mode with no detector is a hope with a bullet point. A document that lists its own failure modes earns more trust than one claiming none. Reviewers are professionally suspicious of clean.
The audit trail
The test is one sentence. Months from now, can you reconstruct why the system said that, to that customer, on that date? Answering it needs the prompt version, the retrieval set with document identifiers and versions, the model version, every tool call with its parameters and results, which boundary rule fired and which human reviewed it. Anything less is logs, and logs are not an audit trail.
Store it so it cannot be quietly edited, and restrict who can read it: by construction the trail is full of customer content. Keep the claim accurate too. Never say a system has been audited unless it has been. Say it is built to survive an audit, then make that true.
Rollback
Rollback converts fear into procedure, in 3 parts. A kill switch a duty manager can pull without an engineer, tested on a schedule, because an untested one is a diagram. A defined fallback state that is not an outage: conversations route to the queue with context preserved and priority set. And version control over the things that actually change behavior, because prompts, retrieval indexes, tool definitions and boundary rules are configuration that behaves like code.
Write the triggers in advance. Which signal, at what level, for how long, pulls the switch. Deciding that during an incident means deciding it in an argument.
What the document does to the build
Writing all 7 sections first is not overhead. It is design. The threat model determines tool scope. The audit trail determines what you log on day 1, the only day logging is cheap. Rollback determines your deployment model. The human boundary table is a rollout plan in disguise: each class of action is a phase, you ship the reversible ones first and earn the rest with evidence.
It changes the room, too. A reviewer handed a real threat model stops behaving like a gate and starts behaving like a co-author. In my own rollout, the security and compliance case is what unlocked enterprise adoption. It was written work, not model work.
The argument
The agent is not the product. For customer-facing AI inside an enterprise, the deliverable is permission, and permission is a written artifact: threat model, data flow, retention, human boundary, failure-mode register, audit trail, rollback. Everything else is a demo.
Most teams have the demo. Very few have the document. That gap is the entire distance between a promising pilot and a system in production, and it is closed by writing, not by a better model.
Written in a personal capacity. Nothing here represents the position of any employer, past or present.