In Q1 2024, Gartner recorded a 1,445% surge in enterprise inquiries about multi-agent AI systems. By Q2 2025, the question had shifted from “should we explore agents?” to “why isn’t ours working?” Gartner projects that 40% of enterprise applications will embed autonomous agents by the end of 2026. The era of isolated LLM calls is over. What’s replacing it is considerably more complex — and considerably more powerful — than most organisations anticipated when they first prototyped a chatbot.
The Monolithic Agent Problem
The first generation of production AI agents followed a familiar pattern: one large model, one system prompt, and a long list of tools. The results were predictably inconsistent. A monolithic agent asked to research a competitor, summarise the findings, draft an outreach email, schedule a follow-up, and log the outcome in CRM is being asked to context-switch across five distinct cognitive modes in a single pass. Attention degrades. Instructions conflict. The context window fills with noise before the task is half complete.
The microservices analogy is imperfect but useful. A monolithic web application works fine at low scale and fails catastrophically under load, not because the code is wrong but because the architecture wasn’t designed for the load profile. Single-purpose services, each owning a bounded domain, compose into systems that scale horizontally and fail gracefully. The same logic applies to agents. Specialisation isn’t a nice-to-have; it’s the prerequisite for reliability. An agent whose entire job is evaluating code quality will outperform a generalist agent given the same task, because its system prompt is tighter, its tool surface is narrower, and its failure modes are easier to detect and handle.
The Eight Design Patterns
The agent design space has consolidated around a small number of patterns that appear repeatedly in production systems:
- ReAct (Reason + Act): The agent interleaves reasoning steps with tool calls, producing a chain of thought before each action. This makes debugging tractable and prevents the model from skipping steps under pressure.
- Reflection: After producing an output, the agent (or a separate critic agent) evaluates it against a rubric and iterates. Reflection loops dramatically improve output quality on complex tasks at the cost of latency.
- Tool Use: The agent selects from a defined catalogue of tools — APIs, database queries, code interpreters, web search — based on the task at hand. Tool catalogues should be narrow; giving an agent 40 tools degrades selection accuracy.
- Planning: Before executing, the agent produces an explicit plan decomposing the goal into subtasks. The plan can be validated before execution begins, enabling human-in-the-loop checkpoints without interrupting every step.
- Multi-Agent Collaboration: Specialised agents work in parallel on different aspects of a problem and hand results to a synthesiser. Effective for research tasks, competitive analysis, and any workflow where breadth matters.
- Sequential Workflows: Output of one agent becomes input to the next. The classic pipeline. Simple to reason about, easy to test, appropriate when tasks have clear linear dependencies.
- Human-in-the-Loop: High-stakes actions — sending emails, executing trades, modifying production databases — require explicit human approval before proceeding. Not a limitation; an architectural feature that makes deployment to conservative organisations possible.
- Hierarchical Orchestration: An orchestrator agent decomposes goals and delegates to worker agents, aggregating results. This mirrors management structures in human organisations and handles task complexity that would exceed any single context window.
State Is the Hard Part
Every production multi-agent system that has failed in an interesting way has failed because of state. The problems are predictable once you know to look for them.
Race conditions emerge when two agents simultaneously update shared state — a task queue, a shared document, a database record — without coordination primitives. The result is inconsistent state that neither agent can recover from cleanly. Distributed locks, optimistic concurrency with version numbers, and event-sourced state management all address this, but none are free.
Deadlocks arise from circular dependencies. Agent A waits for Agent B’s output before proceeding; Agent B is waiting for Agent A’s confirmation. Neither can move. Detection requires timeout mechanisms and explicit dependency graphs surfaced to the orchestrator.
Memory exhaustion in long-running workflows is subtler. An agent that accumulates context across hundreds of steps eventually runs out of usable context window. Episodic compression — summarising earlier steps and evicting raw history — is necessary but lossy.
Production systems separate memory into four layers: episodic memory (what happened in this session), semantic memory (facts about the world, stored in a vector database or knowledge graph), procedural memory (how to do specific tasks, encoded as prompts or fine-tuned weights), and working memory (the live context window). Architectures that conflate these layers into a single undifferentiated blob of context pay for it in reliability.
MCP and A2A: The Protocols That Matter
Two protocol developments in 2024–2025 changed the agent infrastructure landscape more than any model release.
Anthropic’s Model Context Protocol (MCP) standardises how agents access tools and external data sources. Before MCP, every agent framework had its own tool-calling convention, making it impossible to reuse tools across systems without wrapping them in adapter code. MCP provides a uniform interface: a tool server exposes capabilities through a standard schema, and any MCP-compatible agent can consume them without integration work. The effect is a growing ecosystem of interoperable tools that agents can access without bespoke plumbing.
Google’s Agent-to-Agent (A2A) protocol solves a different problem: how do agents from different vendors, running on different infrastructure, coordinate without a central controller that becomes a single point of failure? A2A enables peer-to-peer agent coordination through a standardised message format and discovery mechanism. An agent built on Claude can delegate a subtask to an agent built on Gemini without a human configuring the integration. This is the TCP/IP moment for agent networks — the point at which the infrastructure becomes invisible and the ecosystem above it can compound.
Heterogeneous Model Routing
Running every agent on a frontier model is economically indefensible at scale. A well-architected multi-agent system routes tasks to the appropriate model tier based on the cognitive demand of the task.
Frontier models (GPT-4o, Claude Opus, Gemini Ultra) handle orchestration, complex reasoning, ambiguous instructions, and tasks where errors are expensive. They are expensive per token and slow; use them where the quality delta justifies the cost.
Mid-tier models (Claude Haiku, GPT-4o-mini, Gemini Flash) handle well-defined tasks with clear success criteria — summarisation, classification, structured extraction, code generation within a defined template. They are 10–50x cheaper than frontier models with acceptable quality on these tasks.
Small specialised models handle high-frequency, low-complexity execution: intent classification, routing decisions, entity extraction, simple transformations. Latency matters here more than capability. Cost per operation is often under a hundredth of a cent.
The routing logic itself is an agent — usually a lightweight classifier that evaluates incoming tasks against a routing policy. The economics compound quickly: a system that correctly routes 60% of tasks to the mid-tier and 20% to small models can reduce inference costs by 70–80% with minimal quality degradation on the overall workflow.
What Production Actually Requires
The gap between a working demo and a production-grade multi-agent system is wider than most teams expect. The difference is almost never the model. It’s the operational infrastructure wrapped around the model.
Circuit breakers prevent cascading failures. If a downstream tool or agent is consistently returning errors, the circuit opens and the system falls back to a degraded-but-functional path rather than hammering the failing component with retries.
Exponential backoff with jitter is the correct retry strategy for transient failures. Fixed-interval retries produce thundering herds; exponential backoff with randomised jitter distributes retry load across the failure recovery window.
OpenTelemetry distributed tracing is non-negotiable. A multi-agent workflow spanning 12 agents and 40 tool calls produces traces that are impossible to debug without structured observability. Every agent invocation, tool call, and state transition should emit a span with the relevant context.
Human approval gates for high-stakes actions are not a failure of trust in the AI — they are a compliance requirement and a liability boundary. The gate should be asynchronous, logged, and auditable.
Complete audit trails capture every input, output, and decision point in an immutable log. This is required for debugging, required for compliance, and — increasingly — required by enterprise procurement.
The pattern that works in production is a deterministic backbone — explicit workflows, typed interfaces, observable state transitions — with intelligence applied precisely where the task is genuinely ambiguous and the cost of a wrong decision is acceptable. Agents that are autonomous everywhere fail unpredictably. Agents that are autonomous in the right places, within a well-instrumented deterministic frame, ship to production and stay there.