Multi-Agent Systems: When They Help and When They Fail

A multi-agent system splits one task across several LLM agents, each with its own prompt, tools and context window. The published results point both ways. Anthropic's research system beat a single agent by 90.2% on its internal evaluation. In a Google study of 180 agent setups, every multi-agent design lost 39% to 70% against a single agent on a planning task.
The difference is the shape of the task, and how much extra compute the comparison allowed. This article collects what peer-reviewed papers and the engineering teams at Anthropic, OpenAI, Google and Cognition have reported. It covers the main orchestration patterns, accuracy against token cost, the failure modes that one study found in more than 1,600 annotated runs, and where the two protocols, MCP and A2A, fit.
We build AcruxCore, an open-source LLM-ops platform. The section on how AcruxCore handles this is ours; everything else is drawn from the cited research.
At a glance
| Question | Short answer |
|---|---|
| What is a multi-agent system? | Several LLM agents, each with its own instructions, tools and context, that work on one task and pass results or control between them. |
| When does it help? | When the task splits into independent parts, such as broad research, or needs more context than one window holds. |
| When does it hurt? | When each step depends on the step before, as in sequential planning and most coding. |
| What does it cost? | Anthropic reports about 15 times the tokens of a chat. Several studies find that much of the accuracy gain disappears when a single agent gets the same token budget. |
| Why do these systems fail? | One study of 1,600+ annotated runs groups 14 failure modes into three categories: system design, misalignment between agents, and weak verification. |
| Do I need MCP or A2A? | MCP connects an agent to tools and data. A2A connects agents that run in separate services. Agents in one program call each other directly and need neither. |
What is a multi-agent system?
Anthropic's Building effective agents (December 2024) separates two kinds of LLM system. A workflow runs LLMs and tools "through predefined code paths." An agent lets the LLM "dynamically direct" its own process and tool use. A multi-agent system puts several agents, or a mix of agents and fixed steps, inside one application.
Three research frameworks made the idea popular. AutoGen (Wu et al., COLM 2024) builds applications from agents that talk to each other. MetaGPT (Hong et al., ICLR 2024) and ChatDev (Qian et al., ACL 2024) give agents software-team roles and run them through fixed phases such as design, coding and testing.
Agent orchestration: five common topologies
Agent orchestration is the code that decides which agent runs next, what it receives, and when the run stops. Most systems use one of five topologies, or a mix of them.

| Topology | How control moves | Published example |
|---|---|---|
| Supervisor (orchestrator-worker) | A lead agent splits the task, calls workers, and combines their results | Anthropic's research system: a lead researcher starts 3 to 5 subagents in parallel |
| Hierarchical | Supervisors manage other supervisors, or role agents work in fixed phases under one process | MetaGPT and ChatDev: role agents organised like a software team |
| Handoffs | One agent passes the whole conversation to another agent, which takes over | OpenAI Agents SDK: a triage agent hands off to a billing or support agent |
| Peer debate | Several agents answer, read each other's answers, and revise over rounds | Du et al.: three agents debate for two rounds |
| Shared blackboard | Agents read from and write to one shared store, and a controller picks who acts next | MetaGPT's shared message pool; Han and Zhang's blackboard system |
Supervisor, or "agents as tools"
Anthropic's guide calls this pattern orchestrator-workers: a central LLM "dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results." OpenAI's A practical guide to building agents calls it the manager pattern, where each specialist is exposed to the manager as a tool.
Anthropic's How we built our multi-agent research system (June 2025) is the most detailed public report on this topology. A lead agent saves its plan to memory, starts subagents that search in parallel, and decides whether more research is needed. Our supervisor multi-agent tutorial builds a small version of this pattern: one router call that picks a specialist, which then runs its own tools.
Handoffs
OpenAI's guide calls this the decentralized pattern. In the Agents SDK handoff docs, a handoff is a tool that the model sees with a name such as transfer_to_refund_agent. When the model calls it, the run continues on the new agent. By default, that agent sees the entire previous conversation. An input_filter can remove or rewrite parts of the history first. The OpenAI Agents SDK triage tutorial shows what such a handoff looks like in a trace.
The difference from the supervisor matters in practice:
| Supervisor (agent as tool) | Handoff | |
|---|---|---|
| Who talks to the user after the call | The supervisor | The new agent |
| What the specialist receives | Only the input the supervisor writes for it | The whole conversation, unless filtered |
| Where results are combined | In the supervisor | Nowhere; the last agent answers |
| Typical use | Research, analysis, tasks that need one final answer | Customer service routing, where each specialist owns its part of the dialogue |
Peer debate
In Du et al. (ICML 2024), several copies of a model answer the same question, read each other's answers, and revise over several rounds. Liang et al. (EMNLP 2024) add a judge agent that ends the debate. They found the debate works best when it stops adaptively, and that an LLM judge may not be fair when the debating agents are different LLMs.
Shared blackboard
A blackboard is a shared store that every agent can read and write. MetaGPT uses a shared message pool where agents publish outputs and subscribe only to messages that fit their role. In Han and Zhang (2025, preprint), a control unit picks which agents act from what is on the board, until the board shows consensus. The authors report the best average score among the systems they compared, with fewer tokens.
What the evidence shows: accuracy vs token cost
The headline gains are real, but most of them come with much more compute.
Where multi-agent systems won
- Broad research. Anthropic's system, with Claude Opus 4 as lead and Claude Sonnet 4 subagents, beat a single Claude Opus 4 agent by 90.2% on Anthropic's internal research evaluation.
- Debate on reasoning tasks. In Du et al., three agents debating for two rounds raised GPT-3.5's accuracy on 100 grade-school math problems from 77.0% to 85.0%, and on arithmetic from 67.0% to 81.8%. The authors list the extra compute as the method's main limitation.
- Decomposable work. In Google's Towards a science of scaling agent systems (Google Research blog, January 2026, based on the preprint by Kim et al.), a centralized multi-agent design improved a financial-reasoning benchmark by 80.9% over a single agent.
What those wins cost
Anthropic reports that agents use about 4 times the tokens of a chat, and their multi-agent system about 15 times. On the BrowseComp benchmark, token usage alone explained 80% of the variance in performance. The number of tool calls and the model choice explained most of the rest. Anthropic concludes that multi-agent systems "require tasks where the value of the task is high enough to pay for the increased performance."
What happens at equal compute
Several studies give the single agent the same budget, or a simple sampling method, and compare again.
| Study | Comparison | Result |
|---|---|---|
| Smit et al., ICML 2024 | Several debate protocols vs self-consistency and other ensembling, mostly on GPT-3.5, on seven datasets | Debate, in its original form, did not reliably beat self-consistency or Medprompt. One debate protocol won after its agreement level was tuned. Debate needed more API calls |
| Li et al., TMLR 2024 | Plain sampling and majority voting, with more and more agents | Accuracy rose with the number of agents, with no communication between them |
| Tran and Kiela, 2026, preprint | A single agent vs five multi-agent designs, with reasoning tokens held equal, on multi-hop questions | The single agent matched or beat every multi-agent design |
| Jwalapuram et al., 2026, preprint | Automatically generated multi-agent systems vs chain-of-thought with self-consistency | The generated systems underperformed and cost up to 10 times more |
| Kim et al., 2026, preprint | 180 configurations: one single-agent and four multi-agent designs, three model families, four benchmarks | Large gains on decomposable tasks; 39% to 70% losses for every multi-agent variant on sequential planning |
Two points follow. First, running the same model several times and voting already gives part of the benefit, with no orchestration at all. Second, task shape decides whether coordination helps. The Google study found gains where work could run in parallel, losses where each step depended on the last, and more overhead as the number of tools grew. Its predictive model picked the best architecture for 87% of held-out configurations.

Key point: In a Google study of 180 agent setups, every multi-agent design lost 39% to 70% against a single agent on sequential planning, while it gained on tasks that split into parallel parts. Task shape decides whether more agents help.
Tran and Kiela predict that multi-agent designs win when one agent's context is used badly, or when more total compute is spent. Anthropic's case fits the first condition: the task needs more context than one window holds.
Why multi-agent systems fail: the MAST taxonomy
Why Do Multi-Agent LLM Systems Fail? (Cemri et al., NeurIPS 2025) is the largest study of how these systems break. The authors annotated more than 1,600 traces from 7 open-source frameworks, including ChatDev and AG2. Human annotators agreed with a Cohen's kappa of 0.88, which is high. The studied systems failed on 41% to 86.7% of their tasks.
The resulting taxonomy, called MAST, has 14 failure modes in three categories.

| Category | Share of failures (sum of its modes) | Most common modes |
|---|---|---|
| System design issues | about 44% | Step repetition (15.7%), unaware of termination conditions (12.4%), disobey task specification (11.8%) |
| Inter-agent misalignment | about 32% | Reasoning-action mismatch (13.2%), task derailment (7.4%), fail to ask for clarification (6.8%) |
| Task verification | about 24% | Incorrect verification (9.1%), no or incomplete verification (8.2%), premature termination (6.2%) |
The category totals are our sums of the per-mode shares in the paper. The most common failures are simple: an agent repeats itself, does not know when to stop, or ends the run with a check that is missing or wrong.
Do better prompts and topologies fix it?
The authors tried two fixes on two systems: clearer role prompts, and a changed topology.
| System and benchmark | Baseline | Improved prompts | New topology |
|---|---|---|---|
| AG2 on GSM-Plus, GPT-4 | 84.75% | 89.75% | 85.50% (not significant) |
| ChatDev on ProgramDev (32 tasks) | 25.0% | 34.4% | 40.6% |
| ChatDev on HumanEval | 89.6% | 90.3% | 91.5% |
Both fixes helped, yet the best ChatDev setup still failed most ProgramDev tasks. The authors conclude that these failures need deeper changes in system design.
Failures the engineering reports describe
The labs report the same kinds of failure from production:
- Too many agents. Anthropic's early versions started 50 subagents for simple queries. The fix was effort-scaling rules in the prompt: 1 agent for a simple fact, 2 to 4 for a comparison, more than 10 only for complex research.
- Conflicting decisions. In Cognition's Don't Build Multi-Agents (June 2025), two subagents built parts of a game that did not fit together, because neither saw the other's choices.
- Errors that spread. In the Google study, independent agents that never checked each other amplified errors up to 17.2 times. A central orchestrator reduced this to 4.4 times.
Handoffs and context: what each agent should see
At each handoff, one question decides what the next agent can do: how much of the history does it need?
| Option | What the next agent gets | Risk |
|---|---|---|
| Full history | Every message and tool result so far | The window fills, and the agent inherits earlier mistakes |
| Filtered history | Selected parts, such as the user's messages only | It may lose a decision it needed |
| A written brief | A short task description from the lead | The brief may be vague, and the agent repeats or misreads work |
| Compressed history | A summary of earlier actions and decisions | A wrong summary carries its error forward |
OpenAI's handoffs pass the full history by default, and Cognition argues that agents should share full traces. Anthropic's lead agent writes a brief instead. Its early briefs were short, and in one case two of three subagents duplicated the same work. Anthropic now gives each subagent an objective, an output format, tool guidance and task boundaries.
MCP vs A2A: where each protocol fits
The topologies above need no protocol when every agent runs in one program. A supervisor calls a worker as a function, and a handoff passes the message history to the next agent. Two open protocols cover the connections that leave the program.
| MCP (Model Context Protocol) | A2A (Agent2Agent) | |
|---|---|---|
| Connects | An agent to tools, data and prompts | One agent to another agent |
| Introduced | Anthropic, November 2024 | Google, April 2025 |
| Governance today | Agentic AI Foundation, under the Linux Foundation (December 2025) | Agent2Agent project, under the Linux Foundation (June 2025) |
| Main concepts | Servers that give an agent access to tools and data | An Agent Card that describes what an agent can do, and tasks with a lifecycle whose outputs are artifacts |
MCP is how an agent uses a tool or data source that runs as a separate server. A2A is how an agent asks an agent in another service to do a task, even one built by another vendor. Google's A2A announcement says A2A "complements" MCP, and A2A runs on existing standards: HTTP, server-sent events and JSON-RPC.
A first multi-agent system rarely needs A2A, because its agents usually live in one codebase and call each other directly.
The counterargument: build one agent first
The labs that build multi-agent systems also advise against starting with one.
- OpenAI: "Our general recommendation is to maximize a single agent's capabilities first." Split only when an agent fails to follow complicated instructions or keeps choosing the wrong tool.
- Anthropic: find "the simplest solution possible, and only [increase] complexity when needed." Its research system report also says multi-agent systems fit poorly where agents must share context or depend on each other, which includes most coding tasks.
- Cognition: a single-threaded agent keeps one continuous context, so it follows both of their principles. A separate compression model can extend it when the context gets too long, though they call that hard to get right.
The equal-budget studies support this default, and the extra structure adds failure modes that a single agent does not have, such as derailment between agents. The exceptions follow the task shape:
| Situation | Better default |
|---|---|
| Parts of the task can run in parallel and do not depend on each other | Supervisor with parallel workers |
| The information needed is larger than one context window | Supervisor, with subagents that return short summaries |
| Different parts need different permissions, models or spend limits | Separate agents, each with its own key and limits |
| A conversation must move to a specialist who owns it from then on | Handoff |
| Each step depends on the result of the step before | One agent |
| You want higher accuracy on a short reasoning task | Sampling and voting first; debate only if it beats that at equal cost |
To pick a design, start at the top and stop at the first question you answer yes to.

How to tell whether a multi-agent design is helping
- Run an equal-budget baseline. Give one agent the same total tokens, or sample it several times and vote. If it matches, the extra agents are not the cause of the gain.
- Measure cost per solved task. Record tokens and cost for each agent, not only for the run. A system 15 times more expensive needs a gain that justifies it, and a budget per agent stops a runaway loop from spending it all.
- Read the failing traces with MAST categories. For each failed run, ask whether it was a design problem, a misalignment between agents, or a verification problem. The fix differs for each.
- Look for repeated steps and missing stops. Step repetition was the most common mode in MAST, and unawareness of termination was third. A trace with the same tool call ten times, or a run that hits the step limit, shows them.
- Check every handoff payload. Read exactly what the next agent received. This needs a trace that keeps every agent's calls in one run, with the turns of a conversation grouped into one session. A brief that leaves out a decision is the start of a misalignment failure.
- Use a different model as the judge. Liang et al. found LLM judges may be unfair when the agents are different LLMs, so do not let one of the agents grade its own team. How reliable LLM judges are covers the other biases to check before you trust a judge's score. For multi-turn runs, the judge also needs the earlier turns of the conversation.
- Repeat runs. The AG2 results above carry a spread of about ±1 to ±2 points over six runs, so a small gain from one run proves little.
How AcruxCore handles multi-agent systems
AcruxCore is not an orchestration framework. It has no agent graph, no handoff or supervisor runtime, and no A2A support. You write the orchestration yourself, or use a framework such as the OpenAI Agents SDK, CrewAI or LangChain. AcruxCore records each run, controls what each agent may spend, and evaluates the results.
| Problem from this article | What AcruxCore does today |
|---|---|
| Seeing which agent did what | One trace per run, with nested spans. Span kinds include agent, llm, tool and chain. Gateway calls join a trace through the x-trace-id and x-parent-span-id headers |
| Cost per agent | Every span records its model, tokens and cost, so a trace shows what each agent's calls cost |
| Framework-built agents | An OTLP endpoint reads OpenInference spans. The Python SDK's register(instrument=[...]) covers the OpenAI Agents SDK, CrewAI and LangChain |
| Multi-turn runs | Traces that share a session_id are grouped into a session |
| Different models and limits per agent | Each gateway call names its own model. Give each agent its own virtual key, with allowed models, requests and tokens per minute, and its own budget |
| Testing a change on real conversations | Datasets built from sessions keep the earlier turns, and the evaluation judge sees that history |
Some gaps are worth stating plainly:
- There is no dedicated handoff span kind. A handoff from the OpenAI Agents SDK appears as a
toolspan. - There is no separate "agent name" field. An agent is a span of kind
agent, named after the agent. - Online evaluation rules score single
llmspans. They do not score a whole multi-agent trace or session. - AcruxCore runs an MCP server, so an MCP client can read traces and run prompts. It is not an MCP client and does not speak A2A.
The CrewAI trip-planning tutorial shows a two-agent crew traced this way, with one agent span per agent and their model and tool calls nested inside.
A starting checklist
- Build a single agent with good tools first, and measure it
- Split only when the task has independent parts, or needs more context than one window
- Compare every multi-agent design against a single agent at the same token budget
- Choose the topology from the task: supervisor for parallel work, handoff for routing a conversation
- Write explicit stop conditions, and a limit on how many subagents can start
- Decide what each agent receives at a handoff, and check it in the trace
- Add a verification step before the final answer
- Record tokens and cost per agent, and compare cost per solved task
- Classify failed runs by MAST category before changing prompts
The open question
Almost every published gain for multi-agent systems compares against a single agent that got fewer tokens. The few studies that equalise the budget find much smaller gains, or none, outside tasks that split cleanly. What is still missing is a shared way to report multi-agent results with cost: accuracy at a fixed token budget, against both a single agent and simple sampling. Until benchmarks report that by default, a claimed improvement may only show that the system spent more.
Sources
Research papers
- Cemri et al., Why Do Multi-Agent LLM Systems Fail?, NeurIPS, 2025
- Du et al., Improving Factuality and Reasoning in Language Models through Multiagent Debate, ICML, 2024
- Smit et al., Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs, ICML, 2024
- Liang et al., Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate, EMNLP, 2024
- Li et al., More Agents Is All You Need, TMLR, 2024
- Wu et al., AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation, COLM, 2024
- Hong et al., MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework, ICLR, 2024
- Qian et al., ChatDev: Communicative Agents for Software Development, ACL, 2024
- Kim et al., Towards a Science of Scaling Agent Systems, preprint, 2025
- Tran and Kiela, Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets, preprint, 2026
- Jwalapuram et al., The Illusion of Multi-Agent Advantage, preprint, 2026
- Han and Zhang, Exploring Advanced LLM Multi-Agent Systems Based on Blackboard Architecture, preprint, 2025
Engineering reports and specifications
- Anthropic, Building effective agents, December 2024
- Anthropic, How we built our multi-agent research system, June 2025
- Anthropic, Introducing the Model Context Protocol, November 2024
- Anthropic, Donating the Model Context Protocol and establishing the Agentic AI Foundation, December 2025
- OpenAI, A practical guide to building agents
- OpenAI, Agents SDK: Handoffs
- Google, Announcing the Agent2Agent Protocol (A2A), April 2025
- Google, Google Cloud donates A2A to Linux Foundation, June 2025
- Google Research, Towards a science of scaling agent systems: When and why agent systems work, January 2026
- Cognition, Don't Build Multi-Agents, June 2025