Biblioteca de conceptos
Production Patterns for Agentic Systems
The architectural and operational patterns that separate agentic systems that survive in production from prototypes that break under load — covering observability, durability, deployment, evaluation, and incident response.
What it is
Production agentic systems require patterns and disciplines that prototypes do not. The work to take an impressive demo to a system that runs reliably for thousands of users, costs predictably, surfaces incidents quickly, and improves continuously is substantial — and it is mostly architectural and operational, not algorithmic.
The patterns that matter span the system lifecycle: deployment, observability, durability, cost control, evaluation, governance, and incident response. None is glamorous. All compound.
Why it matters
The gap between agentic prototypes and agentic production systems is wider than the gap between traditional ML prototypes and ML production. The reasons are structural — variable latency, variable cost, multi-step traces, judgment-based correctness, evolving model and prompt landscape — and they require deliberate operational design.
Most organizations I work with have built impressive prototypes and stalled at production. The blocker is rarely the model. It is the absence of the operational disciplines that make production work. The teams that close this gap pull ahead; the ones that don’t accumulate prototype debt.
The other consequence: agentic systems in production fail differently from traditional software. Incidents look like quality drift, cost runaways, prompt injection escalations, retrieval misses on entire content categories. The incident response playbook from traditional software is necessary but not sufficient.
The patterns
The eight patterns that separate production from prototype, in rough order of dependence.
1. Trace-level observability. Every agent run produces a trace — a tree of model calls, tool invocations, retrievals, and decisions, with timing, cost, and content captured. Without this, debugging is archaeology. Tools: Langfuse, LangSmith, Braintrust, Helicone, custom on top of OpenTelemetry. Non-negotiable.
2. Structured logging with correlation IDs. Every action in the system carries a goal ID and a step ID. Logs across services can be joined to reconstruct what happened. The agent runtime, tool services, retrieval layer, and downstream systems all share the IDs. This is plumbing; it has to be there from day one.
3. Cost and budget enforcement. Per-run cost tracking, per-user cost allowances, hard ceilings that fail closed. A runaway agent that burns $500 on a single goal should be impossible by construction, not by hope.
4. Deterministic guardrails. Allowlists, output filters, input validation, structured-output schemas. Code, not prompts. See the guardrails entry for depth.
5. Evaluation harness running in CI. A regression suite of representative cases that runs on every change. See the evaluation entry for depth. Without it, every change is a bet.
6. Durable workflows for long runs. For agents that take minutes or hours, durability matters. Worker crashes should not lose progress. Temporal, LangGraph with checkpointing, or custom durable runtime. The pattern: state lives in a durable store, not in process memory.
7. Versioning of everything. Model version, prompt version, tool version, retrieval index version — all tracked, all rollable-back. A change is a deploy; a rollback is an instant revert. The teams that get this right ship faster because they can ship safely.
8. Incident response playbook. Defined escalation paths for quality drift, cost spikes, prompt injection escalation, and downstream system damage. Communication templates. Rollback procedures. Post-mortem requirements. This is operating model; it has to exist before the first incident, not after.
Architecture choices that compound
Separate the agent runtime from business logic. The runtime handles the loop, observability, retries, durability. Business logic — tools, prompts, orchestration — sits on top. This separation makes both layers replaceable independently.
Treat tools as products of the platform. Tools have owners, versions, tests, and a deprecation policy. The model’s tool surface is a designed artifact, not an accumulation.
Prompts in version control with diffs. Prompt changes are PRs. Reviewed. Tested against the eval suite. Deployed via the same mechanism as code. This sounds obvious; it is rarely done.
Async by default. Most agentic work doesn’t need to be synchronous. A queue-and-callback model handles variable latency gracefully and lets the system batch, retry, and prioritize.
Per-user isolation. Memory, context, and tool access scoped per user with explicit boundaries. Cross-user leakage is a security failure, not an inconvenience.
Vendor comparison
The production stack is composed, not bought.
| Layer | Strong options |
|---|---|
| Runtime | LangGraph, Temporal + LLM, custom on direct APIs |
| Observability | Langfuse, LangSmith, Braintrust, Helicone |
| Evaluation | Braintrust, Langfuse, Phoenix, custom on pytest |
| Guardrails | Lakera, NeMo Guardrails, custom |
| Tool definitions | MCP servers, framework-native tools |
| Cost tracking | Provider native, Helicone, custom |
| Workflow durability | Temporal, LangGraph checkpointing |
There is no single vendor that ships a production agentic platform end-to-end. The teams that ship reliably compose from the layers above.
Yoann’s take
My production checklist for any agentic system: trace observability from day one, cost ceilings hard-enforced, deterministic guardrails on tool calls and outputs, evaluation suite of at least 50 cases running on every PR, prompts in version control with PRs, durable workflow runtime if runs exceed a few minutes, per-user isolation, incident response playbook with named on-call. None of this is optional. Skipping any of it produces a system that works in demos and breaks in production. The order I usually deploy them: observability and cost tracking first, evaluation suite next, guardrails before any external launch, durability when run lengths warrant it. The architectural choice that compounds the most: separating the agent runtime from business logic. Once that separation is clean, every subsequent improvement is local. Without it, every change is a system rewrite. The companies winning at agentic AI in 2026 are not the ones with the best models — they are the ones with the strongest operational discipline. The model is a commodity. The system is the moat.
— Yoann
Related reading
- Agent Evaluation Frameworks
- Deterministic Guardrails
- Cost and Latency Budgeting for AI
- AI Governance Frameworks
- Agentic AI Architecture Patterns
External references
- “Building LLMs in Production” — practitioner essays
- Eugene Yan — patterns for LLM systems
- Hamel Husain — evals + production posts