What "Production-Ready" Actually Means for an AI Agent (And the 9 Things Most Vendors Skip)

"Production-ready" has lost all meaning in the AI services industry. Every vendor uses it. Most of them are using it to describe agents that aren't.

I want to define it operationally, because the gap between an agent that works in a demo and an agent that holds up in production for twelve months is enormous, and most mid-market buyers don't know to ask about it.

There are nine things that separate a real production agent from a demo. If your vendor is missing five or more of these, you're about to ship something that won't survive its first quarter.

1. Error handling for every edge case

A demo handles the happy path. A production agent handles the cases where the happy path breaks.

Real production error handling means: every external call has a retry policy with exponential backoff. Every parse failure has a fallback. Every timeout has a defined behavior. Every unexpected input is logged and handled, not crashed on. The agent doesn't break when the upstream API returns malformed data, when a record is missing a required field, when a network call times out, when a rate limit kicks in.

Most demos work because the demo data was clean. Production data isn't.

2. Audit logs on every decision

Every action the agent takes should be loggable, queryable, and explainable after the fact.

This means: the inputs that drove the decision, the model's reasoning trace, the confidence score, the action taken, the human who approved it (if applicable), and the downstream effects. Stored in a way that you can query a month later and answer "why did the agent do this?" in under a minute.

Vendors who skip this ship black boxes. When the agent makes a mistake - and it will - you'll have no way to figure out why, no way to fix the underlying issue, and no way to defend the decision to a regulator, a customer, or a court.

3. Rollback paths the team can trigger

Every production agent should have a "stop" button that the operating team can pull without calling the vendor.

Real rollback means: a way to disable the agent immediately, a way to revert to the manual process, a way to flag and reverse decisions the agent already made (if reversible), a way to put the agent into "human review only" mode without taking it fully offline. The team that operates the agent should be able to do all of this without engineering involvement.

Vendors who skip this lock you in. When something goes wrong, you have to wait for them to fix it. That's not production-ready. That's hostage architecture.

4. Monitoring with operator-readable alerts

A production agent generates a stream of operational signals. Most vendors set up monitoring that engineers can read and operators can't.

Real monitoring means: dashboards in the tools your operating team already uses (Slack, your existing reporting tool, email), alerts in plain English ("the lead routing agent has been retrying for 20 minutes - likely an upstream API issue"), thresholds set to operational reality (not just technical thresholds), and escalation logic that pages the right person at the right time.

When the dispatch board lights up at 4pm on a Friday, the alert needs to go to the dispatcher, not to a Datadog channel nobody checks.

5. Version control for the agent's logic

Production agents need to be versioned the same way production code is.

This means: every change to the agent's logic, prompts, tools, or rules is tracked. Rollback to a previous version is one command, not a rebuild. Diffs between versions are auditable. Deployment to production goes through a defined release process, not a "we changed the prompt" Slack message.

Vendors who skip this can't tell you what changed when something starts behaving differently. That's not maintenance. That's archaeology.

6. Security review pass before deployment

A real production agent has been reviewed against your security policies before it goes live, not after.

This includes: data handling (what does the agent read and write, where is it stored, how long is it retained), authentication (how does the agent identify itself to your systems, how is access scoped), AI provider review (which model, hosted where, what's their data policy), and compliance (does this pass HIPAA, SOC 2, GDPR, or whatever applies to your business).

Most demos skip this entirely because the demo isn't actually accessing production data. When the agent ships for real, the security review either delays you for months or - worse - gets skipped because everyone is in a hurry. Both outcomes kill the deployment.

7. Integration testing under real load

A demo runs against a sandbox with a small data set. A production agent has to handle real volume against real data with real latency requirements.

Real integration testing means: the agent has been tested against production-scale data (not a sample), against the real APIs you'll use (not mocks), at the real volume you'll see (not "a few records"), and through the real failure modes that happen in production (rate limits, timeouts, partial outages).

Most vendors skip the load testing because it's slow and they don't want to delay the demo. Then the agent ships, hits real volume in week two, and falls over. The vendor blames "scaling issues." The buyer blames the vendor. The agent gets turned off.

8. Change management documentation

The team that has to live with the agent needs to know how to use it, what to do when it misbehaves, when to escalate, and what's changing about their workflow.

Real change management documentation means: a runbook the operating team can reference, an FAQ that answers the questions they actually have, a defined escalation path with names and contact methods, and training materials that match how the team actually works.

Most vendors hand off a Notion page that's mostly technical documentation and call it done. The operating team can't read it. They invent their own workarounds. The workarounds become the workflow. The agent ends up doing 30% of what it was supposed to do.

9. A named production owner on the client side

Every production agent needs a specific human at your company whose job description includes operating it.

This isn't a "champion." A champion is someone who's enthusiastic about the project. A production owner is someone whose performance review includes the agent's performance. They monitor it, they triage issues, they make calls about when to escalate, they review the logs, they coordinate with the vendor on improvements.

If your engagement doesn't have a named production owner before the agent ships, the agent will orphan itself. This is the most common cause of post-launch failure, and it's the easiest to fix - but only if you address it during scoping, not after.

What to ask your vendor

Run this list against any AI vendor proposal you're considering. For each of the nine items, ask:

Is this in the engagement scope? Who's responsible for it? When does it happen - at kickoff, mid-build, or pre-launch? What does "done" look like?

A vendor who can answer cleanly on all nine has built production agents before. A vendor who hand-waves on three or more is selling you a demo dressed up as production work.

This isn't a complete list - there's more nuance in each item - but it's enough to filter most vendors. The good ones will appreciate that you're asking. The bad ones will get uncomfortable and try to redirect the conversation. That redirect is the signal you need.

The honest truth about production

Most "shipped" AI agents in mid-market companies right now are missing five or six of the nine items above. They were built by vendors who didn't know to scope for them, sold to operators who didn't know to ask for them.

Those agents are operating in production today. They're brittle, opaque, hard to maintain, and quietly costing more than they save. Some of them have already been turned off. Most of them will be turned off in the next 18 months when they break and nobody can fix them.

This is the next wave of AI failure that the industry isn't talking about yet. The first wave was pilots that never shipped. The second wave is going to be agents that shipped without being production-ready, and quietly died in their first year.

The way to avoid being part of the second wave is to ask harder questions during scoping. Production-ready isn't a label. It's a list. If your vendor can't show you they've scoped for all nine, they haven't built production agents before - no matter what they claim.

That's the only test that matters.