How to deploy an AI agent to production without it falling apart

The demo always works. The problem is everything that happens after the demo, and that gap is where most agent projects quietly die.

The first agent I put in front of real customers looked flawless in the demo. It answered support questions, pulled order status, issued refunds under a threshold. Everyone in the room nodded. Two days after launch it refunded the same order four times because the customer kept rephrasing the request and the agent kept treating each message as new. That is the actual lesson in how to deploy an AI agent to production: the model was never the risk. The system around the model was.

I embed with companies for short engagements and ship agents that have to hold up after I leave. So I care less about clever prompting and more about what breaks at 2am when I am not watching. Here is what I actually do, in the order I do it.

Start by deciding what the agent is allowed to do, not what it can do

An LLM can generate any string. That is the whole problem. Left alone it will confidently invent an order ID, call a tool with garbage arguments, or promise a customer something your business does not offer. The single most useful thing you can do before launch is draw a hard boundary around actions, and enforce that boundary in code, not in the prompt.

I split every agent into two buckets. Read actions are cheap and reversible. Write actions touch money, state, or a human's inbox. Reads I let the agent do freely. Writes go through a gate. The gate is plain code that validates arguments, checks a limit, and sometimes asks a person. The model proposes. The system disposes.

def issue_refund(order_id: str, amount_cents: int, actor: str):
    order = db.get_order(order_id)
    if order is None:
        return {"error": "order_not_found"}
    if order.already_refunded:
        return {"error": "already_refunded"}   # idempotency, saved me many times
    if amount_cents > order.total_cents:
        return {"error": "amount_exceeds_total"}
    if amount_cents > 5000:
        return queue_for_human_review(order_id, amount_cents, actor)
    return payments.refund(order_id, amount_cents)

Notice the model never sees the refund limit. It cannot argue its way past 5000 cents because the limit lives in a function it does not control. If you only take one idea from this post, take that one.

Tools are your real prompt

People spend days polishing a system prompt and thirty seconds writing tool descriptions. Backwards. The model spends most of its decision budget choosing tools and filling arguments, so that is where you should spend yours. Name tools like you would name functions for a junior engineer. Make descriptions say when to use a tool and when not to. Keep arguments typed and few.

A tool that takes a free text query is a tool that will be called with nonsense. A tool that takes an enum and a validated ID is a tool that behaves. I return structured errors from every tool so the model can recover instead of hallucinating a success. When a lookup fails, I want the agent to see order_not_found and ask the user, not paper over it.

If a tool can be called wrong, the model will eventually call it wrong. Design tools so that wrong is hard to express.

Write the system prompt like a runbook, then freeze it

My system prompts are boring on purpose. Role, scope, hard rules, escalation path. No personality essays. The rules that matter are the ones that tell the agent what to refuse and when to hand off to a human. I keep the prompt in version control next to the code, because a prompt change is a deploy, and an untracked prompt change is a bug you cannot reproduce.

You are the support agent for Acme. You help with order status and refunds.

Hard rules:
- Never invent an order ID. If you do not have one, ask.
- Never promise a delivery date. Read it from get_order only.
- Refunds above $50 are not yours to approve. Call escalate_to_human.
- If the user is angry or mentions legal action, call escalate_to_human.

When a tool returns an error, tell the user plainly and offer the next step.
Do not guess. Guessing is worse than saying you are not sure.

That last line does more work than any amount of clever instruction. An agent that admits uncertainty and escalates is a safe agent. An agent that always has an answer is a liability wearing a helpful mask.

You cannot ship without evals, and vibes are not evals

Here is the part teams skip and then regret. Before I put an agent live I build a set of test cases from real transcripts and known edge cases. Fifty to a hundred is enough to start. Each case has an input, the tools that should get called, and a check on the outcome. I run the whole set on every prompt change and every model swap.

Why does this matter so much? Because agents fail silently. You tweak one line of the prompt to fix a rare case and quietly break three common ones, and you will not notice until a customer does. An eval suite turns that invisible regression into a red number in CI. I have caught a model upgrade that improved writing quality and destroyed tool selection accuracy at the same time. Without evals I would have shipped it.

• Golden path cases: the flows that must always work, checked on every commit.
• Adversarial cases: prompt injection, users trying to get free refunds, contradictory instructions.
• Boring failure cases: empty inputs, missing records, the same request sent twice.

Observability, or you are flying blind

The day you go live, someone will ask why the agent did a strange thing, and you need an answer in minutes, not days. So log everything. Every message, every tool call with its arguments and result, the token counts, the latency, and a trace ID that ties one conversation together. I dump this into Postgres for the first few weeks because it is easy to query and I can just write SQL when something looks off.

Tools like LangSmith or Langfuse are worth it once you have volume, but do not let tool shopping delay launch. A trace ID and a structured log table will carry you a long way. What you are really building is the ability to answer one question fast: what did the agent see, and what did it do next?

{
  "trace_id": "conv_9f2a",
  "step": 3,
  "model": "claude-sonnet",
  "tool": "issue_refund",
  "args": {"order_id": "A-1187", "amount_cents": 4200},
  "result": "queued_for_review",
  "latency_ms": 812,
  "tokens_in": 1934,
  "tokens_out": 88
}

Set a hard budget on loops

Agents loop. Tool call, think, tool call, think. Left unbounded, a confused agent will happily burn forty steps and a few dollars in tokens chasing its own tail. I cap steps, usually at eight to ten, and I cap total tokens per conversation. When the cap trips, the agent stops and escalates to a human instead of spiraling. Put that limit in before launch. You will hit it, and you will be glad it was there.

Assume prompt injection, because it is coming

If your agent reads anything a user or a third party can write, a support ticket, a web page, a PDF, then someone will eventually plant instructions in that text telling your agent to ignore its rules. This is not paranoia. It is Tuesday. The defense is the same boundary from the start of this post: the agent's permissions live in code, so a malicious message can make the model say strange things but it cannot make the refund function skip its checks.

I treat all retrieved content as data, never as instructions. I keep untrusted text clearly fenced in the context, and I never let the model's summary of a document decide whether a write action fires. The moment tool permissions depend only on what the model believes, you have handed control to whoever writes the input.

Roll it out like you expect to be wrong

I never flip an agent to a hundred percent of traffic on day one. It starts in shadow mode, generating responses that a human reviews before sending. Then it handles a small slice of easy conversations with a human watching. Then it earns more scope as the eval numbers and the real transcripts hold up. This is slower and it is the reason the agent is still running months later instead of getting switched off after one bad week.

Keep a kill switch that a non engineer can hit. A feature flag that routes everything back to humans, one click, no deploy. When something goes wrong, and it will, you want the fix to be a toggle, not a frantic pull request at midnight.

The part that actually keeps it alive

Shipping the agent is week one. The work that decides whether it survives is the boring maintenance after. I read transcripts every day for the first month. Real conversations tell you what your evals missed, and they always miss something. Each surprising failure becomes a new test case, then a fix. That loop, watch, capture, test, patch, is the whole job. An agent is not a thing you launch. It is a thing you keep honest.

None of this is glamorous. There is no clever trick. The agents that hold up in production are the ones wrapped in dull, unforgiving plumbing: gated writes, typed tools, a real eval suite, full logs, step caps, and a human who reads what happened. Get those right and the model almost becomes the easy part. Get them wrong and it does not matter how good the demo was. Ask me how I know about the four refunds.