AI Agent Orchestration: Patterns, Cost and Tests for Multi-Agent Systems

Most teams meet orchestration the same way. A single agent works in a demo, then the job grows. Someone adds a second agent, then a third, and a diagram appears with boxes and arrows. The system gets harder to test, costs more per task, and sometimes gives a worse answer than the single agent did. This guide covers what orchestration is, the common patterns, the controls that keep a multi-agent system safe and affordable, and how to tell when one agent is the better answer.

It is written for two readers. Engineers get the patterns, the failure modes and tested code. Business leaders get the cost logic and the questions to ask before approving a multi-agent build.

We organize the guide around a named model, the Four Controls. It also has a cost section built on the numbers Anthropic published, a failure table, a comparison of frameworks and protocols that we checked against their own documentation, and a production checklist. Where a sentence is our engineering judgment and not a finding from a source, we label it as ours.

What agent orchestration is

An agent is a program that uses a language model to choose its own steps and call tools. Our agent harness guide covers the runtime that makes one agent safe. Orchestration starts one level up. It is about what happens when there is more than one agent, or when one agent sits inside a larger flow with other parts.

When we read Microsoft's architecture guide, the first thing we noted was that it frames the choice as a ladder. At the bottom is a direct model call. Next is a single agent with tools. At the top is multi-agent orchestration, where several specialized agents coordinate and an orchestrator or a peer protocol manages the split of work, the sharing of context and the merging of results. Microsoft says each step up adds coordination overhead, latency and cost, and advises using the lowest level of complexity that reliably meets your requirements.

Anthropic draws a related line between workflows and agents. In its words, workflows are systems where predefined code paths direct the model and tools, and agents are systems where the model directs its own process and tool use. That line matters here because orchestration can sit on either side. A fixed chain of model calls is orchestrated by your code. A lead agent that decides how many helpers to start is orchestrated by a model. Most production systems mix the two.

Orchestration is not a harness or a protocol

Three words get mixed up. Here is how we use them.

• Harness. The runtime around one agent: its loop, tools, context, memory and permissions. Each agent in a multi-agent system has one.
• Orchestration. The layer that coordinates several agents or steps. It owns the plan, the handover, the shared state, the limits and the final check.
• Protocol. An agreed message format, such as MCP for tools or A2A for agent-to-agent calls. A protocol moves messages. It does not decide anything.

If you can describe your system as one loop with tools, you have an agent and a harness. If you need a picture with arrows between agents, you have orchestration.

Five shapes of orchestration

Every design we read about in the sources, and every one we sketched ourselves, came down to one of five shapes or a mix of them. The diagram shows each one and who decides the next step.

The names vary by vendor. Anthropic lists prompt chaining, routing, parallelization, orchestrator-workers and evaluator-optimizer. Microsoft lists sequential, concurrent, group chat, handoff and magentic. LangChain lists subagents, handoffs, skills, router and custom workflow. The shapes below are our grouping of those lists. They are not a standard.

Sequential chains

Each step takes the output of the one before. Anthropic calls this prompt chaining and says it fits tasks that split cleanly into fixed subtasks. The trade is latency for accuracy, because each call is an easier task. Microsoft's sequential pattern is the same idea with agents as the steps.

Use it when you can write the steps down. It is the easiest shape to test. Many tasks that people call agentic are chains.

Parallel branches

Several agents or calls run at once and your code merges the results. Anthropic describes two forms: sectioning, where independent subtasks run in parallel, and voting, where the same task runs several times to get diverse outputs or more confidence. Parallel results can disagree, so you need a rule for conflicts.

Use it for independent checks, such as a security review and a cost review of the same document. It saves time, not tokens.

Routers

A first step reads the request and picks one path. Anthropic defines routing as classifying an input and directing it to a specialized follow-up task, and says it works when distinct categories are better handled separately and can be classified accurately. Only one specialist runs, so cost stays close to a single agent. The risk sits in the classifier. A wrong route sends work to the wrong specialist, so a router needs a confidence rule and a fallback.

Supervisor and workers

A lead agent plans, starts workers, and merges what they return. Anthropic calls this orchestrator-workers and says it fits cases where subtasks cannot be predicted ahead of time, because the orchestrator decides them from the input. It is the most expensive shape, and the one most prone to quiet failures.

Handoffs

The active agent passes control to another agent, which handles the rest of the turn. In the OpenAI Agents SDK, a handoff is shown to the model as a tool named transfer_to_<agent_name> by default, and by default the new agent sees the full conversation history, as though it took over the conversation. Microsoft lists infinite handoff loops and unpredictable routing paths among the risks of this pattern.

Use handoffs when the right specialist only becomes clear during the work, and when that specialist should talk to the user directly. Support triage is the usual example.

A comparison of the five shapes

The cost column is our engineering estimate from how each shape calls the model. It is not a measurement. The one measured ratio we cite is Anthropic's for its supervisor system, in the cost section below.

The Four Controls

A pattern tells you the shape. It does not tell you whether the system is safe to run. For that we use a short model, the Four Controls: Decide, Share, Cap and Check. The name and grouping are ours. We built them by listing every place the sources report a multi-agent system breaking and sorting those into four groups.

• Decide. Who chooses the next step: your code, a classifier, or a model? Every unknown here is a place where behavior can drift.
• Share. What does each agent see? A briefing, the full history, a shared state, or only a reference to a file? Too little and agents repeat or contradict each other. Too much and cost and context grow.
• Cap. What is the most the system may do? Workers, concurrent workers, depth, turns, tokens and money. A cap in code is a rule. A cap in a prompt is a hope.
• Check. What decides that the result is good? A test, a verifier or a person, and something that is not the lead agent grading its own work.

The next four sections take one control each. The sections after them cover when to skip orchestration, the failure modes, the tools and a worked example.

Control 1: Decide who picks the next step

The first design choice is whether code or a model chooses the route. OpenAI's SDK docs name the two options plainly. In LLM-driven orchestration, the model plans, reasons and chooses actions, which suits open-ended tasks. In code-driven orchestration, your code sets the flow, which makes speed, cost and performance more predictable. The docs say you can combine them.

Our recommendation: put as much of the route in code as the task allows. A model that picks the route can pick wrongly, and you pay for each wrong pick. Keep the model's freedom for the parts where the path cannot be written down.

Graphs: making the route explicit

A graph writes the route down. Nodes do work, edges say what comes next, and some edges depend on state. LangGraph is built this way. Its overview describes it as a low-level orchestration framework and runtime for long-running, stateful agents, that lists persistence, human-in-the-loop, memory and debugging among its core benefits and names durable execution and streaming as areas of focus. Its workflow docs show the same five patterns as graphs, including routing through a conditional edge and orchestrator-workers through a Send call that creates worker nodes with their own inputs.

What we like about a graph is that the route is data you can read, test and replay. For that reason the shared-state sample later is a tiny graph runner of about 50 lines, written to show the idea without a framework.

Handoffs and agents as tools

Two designs look alike and behave differently. The OpenAI docs call them handoffs and agents as tools.

• Agents as tools. A manager agent keeps control of the conversation and calls specialists as if they were tools. Use it when one agent should own the final answer or combine several specialists' outputs.
• Handoffs. A triage agent routes the conversation to a specialist, who then handles the rest of the turn. Use it when routing is part of the workflow and the specialist should reply directly.

LangChain draws the same line. In its subagents pattern a main agent coordinates subagents as tools and all routing goes through the main agent. In its handoffs pattern agents transfer control through state-changing tool calls. In our judgment the practical difference is who owns the answer. With a manager, there is one place to add a check. With a handoff, each specialist needs its own checks.

Handoffs need a loop guard. Microsoft names infinite handoff loops as a risk and says avoiding excessive bouncing between agents is challenging. The sample below caps hops and spots ping-pong between two agents.

"""Handoffs between agents with a hop cap and a ping-pong check.Each agent is a stub function that returns either an answer or a handoff.Runs offline.""" def triage(ticket):    return {"handoff": "billing"} if "invoice" in ticket else {"handoff": "technical"} def billing(ticket):    if "error 500" in ticket:        return {"handoff": "technical"}    return {"answer": "Billing: invoice corrected."} def technical(ticket):    if "invoice" in ticket:        return {"handoff": "billing"}  # a bad rule: sends it back    return {"answer": "Technical: restarted the service."} AGENTS = {"triage": triage, "billing": billing, "technical": technical}  def run(ticket: str, start: str = "triage", max_hops: int = 4) -> str:    path, current = [start], start    for _ in range(max_hops):        out = AGENTS[current](ticket)        if "answer" in out:            print(f"path {' -> '.join(path)}")            return out["answer"]        nxt = out["handoff"]        if nxt in path[-2:-1] or nxt == current:  # A -> B -> A            print(f"path {' -> '.join(path + [nxt])}")            return f"ESCALATE to human: ping-pong between {current} and {nxt}"        path.append(nxt)        current = nxt    print(f"path {' -> '.join(path)}")    return f"ESCALATE to human: hop cap {max_hops} reached"  if __name__ == "__main__":    print(run("please fix my login"))    print(run("invoice 77 is wrong"))    print(run("invoice 78 shows error 500"))

We ran it with Python 3.13.3 on 2026-10-27. This is the real output:

path triage -> technicalTechnical: restarted the service.path triage -> billingBilling: invoice corrected.path triage -> billing -> technical -> billingESCALATE to human: ping-pong between technical and billing

We gave the third ticket a bad rule on purpose. Billing sends it to technical, technical sends it back, and the guard stopped the run at the third hop and escalated to a person.

Routing with a cheap classifier

We would not use a large model as a router. Its job is to pick one of a few known routes, which is a classification task. A small classifier that returns a probability for each route should cost less than a frontier model, and the probability gives a confidence rule: act when it is high, escalate when it is low.

TypeSafe's Jev is the model we looked at for this job. In the docs we read, TypeSafe describes System One models as built to make fast, structured decisions that software can use directly. Jev is the first of them. It returns typed answers with probabilities. The Choice type picks from a defined set, and the docs' own example is billing, technical or account. It accepts text input, including JSON objects and arrays. It does not write replies, produce code or explain its reasoning. TypeSafe says calibration is measured across groups of predictions and does not guarantee that one answer is correct, and suggests using confidence to decide when to act and when to escalate to a person or a reasoning model.

In the LangChain pull request we read, a langchain-typesafe package at version 0.0.1 added a TypeSafeClassifier runnable, merged on 2026-09-17. The agent middleware that would plug it into an agent loop was still a separate open pull request.

Our read: the docs page gives no latency or price, so we leave those out and would measure them before choosing it. Using Jev as a router in front of agents is our suggestion. TypeSafe's docs describe classification and escalation, not agent routing.

The sample below uses a stub classifier to show the confidence rule that any classifier router needs.

"""A cheap router in front of agents. The classifier is a stub that returns probabilitiesfor each route, the way a typed classifier would. It is not any vendor's model.Low confidence goes to a stronger model or a person instead of a guess. Runs offline.""" ROUTES = ("billing", "technical", "account")  def stub_classifier(text: str) -> dict[str, float]:    t = text.lower()    if "invoice" in t:        return {"billing": 0.93, "technical": 0.04, "account": 0.03}    if "password" in t:        return {"billing": 0.05, "technical": 0.30, "account": 0.65}    return {"billing": 0.34, "technical": 0.33, "account": 0.33}  def route(text: str, threshold: float = 0.80) -> str:    probs = stub_classifier(text)    best = max(probs, key=probs.get)    if probs[best] >= threshold:        return f"{best} (p={probs[best]:.2f}) -> specialist agent"    return f"uncertain (best {best}, p={probs[best]:.2f}) -> escalate to a stronger model or a person"  if __name__ == "__main__":    for msg in ("invoice 12 is wrong", "I forgot my password", "it feels slow today"):        print(f"{msg!r}: {route(msg)}")

We ran it on 2026-10-27. This is the real output:

'invoice 12 is wrong': billing (p=0.93) -> specialist agent'I forgot my password': uncertain (best account, p=0.65) -> escalate to a stronger model or a person'it feels slow today': uncertain (best billing, p=0.34) -> escalate to a stronger model or a person

The 0.80 threshold is a demo value. We recommend setting yours from labeled examples of your own requests and measuring how often an escalation was needed.

Control 2: Share what each agent sees

Agents in a multi-agent system do not share a mind. Each has its own context window. What you pass between them is the whole coordination.

Briefs: what a worker must be told

In the Claude Agent SDK, a subagent starts with a fresh context. The only content passed from parent to subagent is the Agent tool's prompt string, so file paths, error messages and decisions the subagent needs must go in that prompt. The same page says intermediate tool calls and results stay inside the subagent, and only its final message returns to the parent.

What we see in this is a feature and a trap. It keeps the lead's context small. It also means a vague brief produces a vague result. Anthropic's team reports that each subagent needs an objective, an output format, guidance on the tools and sources to use, and clear task boundaries. In their account, when the lead's instructions were short, subagents misinterpreted the task, and some duplicated each other's searches.

We write a brief as a short form. Ours has five lines: the goal, what done looks like, the output format, the tools and sources allowed, and what the worker must not do.

The opposite view: share everything

Cognition argued that multi-agent systems are fragile because agents lack shared context. It gives two principles: share context and full agent traces, not just individual messages, and remember that actions carry implicit decisions, and conflicting decisions carry bad results. Its example splits a game-building task into a background subtask and a bird subtask. The two subagents drift apart in style and the final agent has to reconcile incompatible parts.

The sources partly disagree here. Cognition's first post called a single-threaded linear agent the simplest way to follow its principles, and describes a compression model for tasks that outgrow one context window, which it says is hard to get right. Its April 2026 follow-up says the original observations still hold for parallel writers, and that the patterns working now keep writes single-threaded while extra agents add intelligence: a reviewer that starts with a clean context, a stronger model called as a tool, and a manager that splits a large task across child agents. It also calls unstructured swarms mostly a distraction. Anthropic ships a multi-agent system for research and reports a large gain, and its January 2026 guidance says to split work by context boundaries, because separate planning, implementation and testing agents create a telephone game of lost information. Having read both sides, we think both can hold. Research splits into independent reading tasks. Building one coherent artifact does not.

Shared state and reducers

The second way to share is a state object that every node reads and writes. A graph runtime merges the writes. The merge rule is called a reducer. In LangGraph's orchestrator-worker example, all worker outputs are written to a shared state key, declared with an operator.add reducer so that parallel writes are appended.

An append works for a list of findings. It does not work for a single decision, such as the style of the output. If two parallel workers write different values to a single-value key, something has to notice. The sample below adds that check, and a checkpoint after every step so a crashed run can resume.

"""A tiny state graph: nodes, a reducer per key, a conflict check, a step cap andcheckpoints. Shows how parallel nodes share state and how a run resumes. Runs offline."""import copy APPEND = "append"     # many nodes may add to the listSINGLE = "single"     # one value: two different writes in the same step is a conflictREDUCERS = {"findings": APPEND, "style": SINGLE, "status": SINGLE}  class Conflict(Exception):    pass  def merge(state: dict, updates: list[dict]) -> dict:    new = copy.deepcopy(state)    written: dict[str, object] = {}    for u in updates:        for key, val in u.items():            if REDUCERS[key] == APPEND:                new.setdefault(key, []).extend(val)            else:                if key in written and written[key] != val:                    raise Conflict(f"two nodes wrote different values to '{key}': {written[key]!r} and {val!r}")                written[key] = val                new[key] = val    return new  def run(nodes: dict, steps: list[list[str]], state: dict, checkpoints: list, start_step: int = 0, max_steps: int = 10) -> dict:    """steps is a list of groups. Nodes in one group run 'in parallel' on the same input state."""    for i in range(start_step, min(len(steps), max_steps)):        updates = [nodes[n](state) for n in steps[i]]        state = merge(state, updates)        checkpoints.append((i + 1, copy.deepcopy(state)))   # saved after every step        print(f"step {i + 1} {steps[i]} ok")    return state  def research_a(s): return {"findings": ["fact A"], "style": "formal"}def research_b(s): return {"findings": ["fact B"], "style": "casual"}   # disagrees with Adef research_b_fixed(s): return {"findings": ["fact B"]}def writer(s): return {"status": "drafted from " + ", ".join(s["findings"])}def crash(s): raise RuntimeError("tool timed out")  if __name__ == "__main__":    print("-- conflicting parallel writes")    try:        run({"a": research_a, "b": research_b, "w": writer}, [["a", "b"], ["w"]], {}, [])    except Conflict as e:        print("conflict:", e)     print("-- crash then resume from the last checkpoint")    cps: list = []    nodes = {"a": research_a, "b": research_b_fixed, "w": crash}    try:        run(nodes, [["a", "b"], ["w"]], {}, cps)    except RuntimeError as e:        print("failed:", e, "| last checkpoint is step", cps[-1][0])    nodes["w"] = writer   # the fix ships    step, saved = cps[-1]    final = run(nodes, [["a", "b"], ["w"]], saved, cps, start_step=step)    print("final:", final)

We ran it on 2026-10-27. This is the real output:

-- conflicting parallel writesconflict: two nodes wrote different values to 'style': 'formal' and 'casual'-- crash then resume from the last checkpointstep 1 ['a', 'b'] okfailed: tool timed out | last checkpoint is step 1step 2 ['w'] okfinal: {'findings': ['fact A', 'fact B'], 'style': 'formal', 'status': 'drafted from fact A, fact B'}

In our run the conflict was caught at the merge, before the writer node saw a mixed state. We cannot see every hidden decision, but we can name the keys where disagreement is not allowed. The crashed run resumed from step 1 and did not repeat it. That matches what Anthropic's team reports about long runs: agents are stateful and errors compound, so they built systems that resume from where the agent was and paired model adaptability with deterministic safeguards such as retry logic and regular checkpoints.

Memory and artifacts

For long runs, a state object is not enough. Anthropic's post says the lead agent saves its plan to memory at the start, because the context is truncated once it exceeds 200,000 tokens. It also recommends artifact systems where specialized agents create outputs that persist independently, with subagents passing lightweight references back to the coordinator.

We would copy that idea, because it is simple. Have workers write their full output to a file or a store and return a short summary and a reference. The lead opens the file only when it needs the detail.

Control 3: Cap what the system may spend

What multi-agent costs

The best-known numbers are the ones Anthropic published, and we checked them in the original post. In its multi-agent research system, a lead agent with smaller subagents outperformed a single agent running the lead model by 90.2% on Anthropic's internal research evaluation. The same post, which dates from the previous model generation, says agents typically use about 4 times more tokens than chat interactions, and multi-agent systems about 15 times more.

If the ladder worries you, start with the levers in our post on reducing LLM costs without losing quality, and see the model choice guide for matching a cheaper model to each worker.

When we compared the January 2026 guidance with the earlier post, we found it gives a wider and lower range. From its own testing, Anthropic reports that multi-agent implementations typically use 3 to 10 times more tokens than single-agent approaches for equivalent tasks, because of duplicated context, coordination messages and summaries at handoffs. The same guidance says the main benefit of parallel agents is thoroughness, not speed, and that total run time can still be longer than a single agent.

We found two more 2026 data points that show how far the bill can move. Anthropic's long-running app harness took a retro game maker from a $9, 20-minute solo run to a $200, 6-hour run with a planner, generator and evaluator, over 20 times the cost, with a more functional result. A later run on a newer model cost $124.70 over 3 hours 50 minutes, and the author reports the evaluator became unneeded overhead for work the generator handled alone. Anthropic's compiler project ran 16 parallel Claude instances over nearly 2,000 sessions for just under $20,000 in API costs.

We would not generalize the research-system figures. They describe one system, one task type and an earlier model generation, and the 90.2% comes from Anthropic's own evaluation. Anthropic also reports that token usage alone explained 80% of the variance in performance on its BrowseComp evaluation. Our read: on research tasks, more tokens spent in parallel bought better answers. The numbers do not show that more agents are better in general.

Anthropic is direct about the other side. Its research-system post says multi-agent systems need tasks where the value is high enough to pay for the increased performance, and that domains needing shared context or with many dependencies between agents were not a good fit. Its 2026 guidance says single agents handle most enterprise workflows well and to start with the simplest approach that works.

Anthropic also reports that two parallelization changes (the lead starting several subagents at once, and subagents calling several tools at once) cut research time by up to 90% for complex queries. Parallelism buys time. It does not reduce spend.

LangChain's docs gave us a second view. They count model calls and tokens for each of their patterns in three scenarios. The scenarios are LangChain's own and small, so we use the figures to compare shapes, not to forecast a bill.

Source: LangChain multi-agent docs, read 2026-10-27. Their summary is that stateful patterns (handoffs and skills) save about 40 to 50% of calls on repeat requests, and that for multi-domain tasks the parallel patterns (subagents and router) are the most efficient. What we take from it is that the cheapest shape depends on the request mix. A support desk with repeat questions and a research tool with broad questions want different shapes.

Caps in code

A cap that lives in a prompt can be ignored. Put these in code.

• Worker cap. The most subtasks the lead may create. Anthropic reported that early versions of its system spawned 50 subagents for simple queries, and the team added scaling rules for how much effort each query type deserves: one agent with 3 to 10 tool calls for simple fact-finding, and more than 10 subagents only for complex research. We recommend adding a hard cap in code beside those rules.
• Concurrency cap. The most workers running at once. It protects your rate limits and your bill.
• Depth cap. Whether workers may start workers. The Agent SDK lets a subagent spawn subagents, so one prompt can grow into a tree. Its default is 3 layers of subagents below the main agent, and setting 1 stops subagents from spawning their own.
• Budget cap. A token or dollar limit for the whole run, workers included. In the Agent SDK, the spend limit counts subagent requests, refuses to spawn more subagents once reached, stops background subagents still running and ends the query with an error_max_budget_usd result.

We wrote the sample below to show three of those ideas, a worker cap, a concurrency cap and a budget checked between waves, in plain Python.

"""Supervisor and workers with three hard caps: workers, concurrency, token budget.Runs offline. Every "model" is a stub, and a "token" here is one word, so the numbersshow structure only. They are not a measurement of any real model."""import asyncio MAX_WORKERS = 4        # most subtasks the supervisor may createMAX_CONCURRENT = 2     # most workers running at the same timeTOKEN_BUDGET = 250     # stop spawning when the total reaches this  def tokens(text: str) -> int:    return len(text.split())  def supervisor_plan(task: str) -> list[str]:    """Stub supervisor. A real one asks a model to split the task."""    return [f"{task}: {part}" for part in ("pricing", "security", "support", "roadmap", "legal")]  async def worker(subtask: str, gate: asyncio.Semaphore, ledger: list) -> dict:    async with gate:        await asyncio.sleep(0.01)  # stands in for a model call        topic = subtask.split(": ")[1]        finding = f"Finding on {topic}: three facts gathered, one open question."        used = tokens(subtask) + tokens(finding) + 120  # 120 = stub cost of tool output read        ledger.append((f"worker:{topic}", used))        return {"topic": topic, "finding": finding, "tokens": used}  async def run(task: str) -> str:    ledger: list[tuple[str, int]] = []    plan = supervisor_plan(task)    ledger.append(("supervisor:plan", tokens(task) + sum(tokens(p) for p in plan)))    if len(plan) > MAX_WORKERS:        print(f"plan has {len(plan)} subtasks, cap is {MAX_WORKERS}: dropping {plan[MAX_WORKERS:]}")        plan = plan[:MAX_WORKERS]    gate = asyncio.Semaphore(MAX_CONCURRENT)    results = []    # Spawn in waves so the budget check sees spend before the next wave starts.    for i in range(0, len(plan), MAX_CONCURRENT):        spent = sum(t for _, t in ledger)        if spent >= TOKEN_BUDGET:            print(f"budget {TOKEN_BUDGET} reached at {spent}: not spawning {plan[i:]}")            break        results += await asyncio.gather(*(worker(s, gate, ledger) for s in plan[i:i + MAX_CONCURRENT]))    summary = " | ".join(f"{r['topic']}: {r['finding']}" for r in results)    ledger.append(("supervisor:synthesis", tokens(summary)))    for name, used in ledger:        print(f"{name:<22}{used:>5} tokens")    print(f"{'total':<22}{sum(t for _, t in ledger):>5} tokens, {len(results)} workers")    return summary  if __name__ == "__main__":    asyncio.run(run("Compare vendor X"))

The model is a stub and a token here is one word, so the numbers show structure only. We ran it on 2026-10-27. This is the real output:

plan has 5 subtasks, cap is 4: dropping ['Compare vendor X: legal']budget 250 reached at 289: not spawning ['Compare vendor X: support', 'Compare vendor X: roadmap']supervisor:plan          23 tokensworker:pricing          133 tokensworker:security         133 tokenssupervisor:synthesis     21 tokenstotal                   310 tokens, 2 workers

What we see in this output: the plan asked for five workers and the cap kept four. The first wave of two workers passed the budget of 250, at 289, so the second wave never started. A budget is checked between waves, so a wave can push spend over it, and we would leave headroom. The output also lists every agent and its cost, which is the start of a trace.

The Agent SDK's own caps

If you use the Claude Agent SDK, the same ideas are options. The subagent page we read on 2026-10-27 lists these defaults, and says the limits need TypeScript SDK v0.3.219 or Python SDK v0.2.127 and later.

The spend default is the one to watch. With no budget set, there is no cap. The same page says Claude decides on its own when to spawn a subagent and how many, and that Claude Opus 5 delegates more readily than earlier models, so the limits matter most for queries that run it.

Is it worth the extra cost?

A multi-agent run that costs more and succeeds more often is worth it when the extra successes are worth more than the extra spend. We turned that into a function and checked it with asserts.

"""Is a multi-agent run worth its extra cost? A simple break-even model.Inputs are placeholders you replace with your own measurements. Runs offline.""" def run_cost(chat_units: float, price_per_chat_unit: float) -> float:    """One 'chat unit' is the tokens of a plain chat turn. Price it from your own logs."""    return chat_units * price_per_chat_unit  def worth_it(single_success: float, multi_success: float, value_per_task: float,             single_units: float, multi_units: float, price_per_chat_unit: float) -> dict:    extra_cost = run_cost(multi_units, price_per_chat_unit) - run_cost(single_units, price_per_chat_unit)    extra_value = (multi_success - single_success) * value_per_task    uplift_needed = extra_cost / value_per_task if value_per_task else float("inf")    return {"extra_cost": round(extra_cost, 4), "extra_value": round(extra_value, 4),            "uplift_needed": round(uplift_needed, 4), "worth_it": extra_value > extra_cost}  if __name__ == "__main__":    # Multiples from Anthropic's note: agents about 4x a chat, multi-agent about 15x.    # Price per chat unit and task value are made-up placeholders.    a = worth_it(0.70, 0.85, value_per_task=20.0, single_units=4, multi_units=15, price_per_chat_unit=0.05)    b = worth_it(0.70, 0.72, value_per_task=20.0, single_units=4, multi_units=15, price_per_chat_unit=0.05)    c = worth_it(0.70, 0.85, value_per_task=1.0, single_units=4, multi_units=15, price_per_chat_unit=0.05)    for name, r in (("high-value task, big gain", a), ("high-value task, tiny gain", b), ("low-value task, big gain", c)):        print(f"{name}: {r}")    assert a["worth_it"] and not b["worth_it"] and not c["worth_it"]    assert a["extra_cost"] == 0.55    print("asserts passed")

The 4 and 15 multiples come from Anthropic's research-system post, and the newer guidance gives 3 to 10 times, so we use them as placeholders. The price per chat unit and the value per task are placeholders too. Replace all of them with your own measurements. We ran it on 2026-10-27. This is the real output:

high-value task, big gain: {'extra_cost': 0.55, 'extra_value': 3.0, 'uplift_needed': 0.0275, 'worth_it': True}high-value task, tiny gain: {'extra_cost': 0.55, 'extra_value': 0.4, 'uplift_needed': 0.0275, 'worth_it': False}low-value task, big gain: {'extra_cost': 0.55, 'extra_value': 0.15, 'uplift_needed': 0.55, 'worth_it': False}asserts passed

What we see in the three cases: a big gain on a high-value task pays, and a tiny gain does not. A big gain on a cheap task does not pay either, because the extra spend is larger than the value. That is Anthropic's point about task value, computed as arithmetic.

Ways to cut orchestration cost

Control 4: Check the result and the path

For the wider method of building test sets and judges, see our LLM evaluation guide.

In our reading of the sources, a multi-agent system can reach the right answer by a bad path. It can use four workers where one would do, ignore a worker's result, or loop twice before it gets there. A check that only reads the final answer misses all of that, and the bad path is what shows up on the bill and in the next failure.

Score the outcome and the path

Anthropic's team describes an evaluation approach that started small. By their account they began with about 20 queries that reflected real usage. They used a language model as a judge, in a single call that outputs a score from 0.0 to 1.0 and a pass or fail grade. They also kept human testing, which caught cases automation missed, such as the agents choosing search-optimised content farms over authoritative sources. The judge criteria they list include factual accuracy, citation accuracy, completeness, source quality and tool efficiency.

We recommend adding path checks that need no model at all, because they come straight from the trace. Our sample scores three hand-made traces on outcome, tokens, duplicate calls, agents used and whether the lead used every worker's result.

"""Score multi-agent runs from their traces. Checks the outcome, and also the path:duplicate work, handoff loops, budget, and whether the lead used all worker results.Traces are hand-made fixtures. Runs offline.""" BUDGET = 1000 TRACES = {    "good": {"passed": True, "events": [        ("lead", "plan", 80), ("w1", "search:pricing", 150), ("w2", "search:security", 160),        ("lead", "synthesize", 120)], "worker_results_used": 2, "worker_results": 2},    "wasteful": {"passed": True, "events": [        ("lead", "plan", 80), ("w1", "search:pricing", 150), ("w2", "search:pricing", 150),        ("w3", "search:pricing", 150), ("w4", "search:security", 160), ("lead", "synthesize", 120)],        "worker_results_used": 4, "worker_results": 4},    "lost-work": {"passed": False, "events": [        ("lead", "plan", 80), ("w1", "search:pricing", 150), ("w2", "search:security", 160),        ("lead", "synthesize", 700)], "worker_results_used": 1, "worker_results": 2},}  def score(trace: dict) -> dict:    calls = [(a, c) for a, c, _ in trace["events"] if a != "lead"]    dupes = len(calls) - len({c for _, c in calls})    total = sum(t for _, _, t in trace["events"])    return {        "outcome": "pass" if trace["passed"] else "FAIL",        "tokens": total,        "over_budget": total > BUDGET,        "duplicate_calls": dupes,        "agents": len({a for a, _, _ in trace["events"]}),        "results_used": f'{trace["worker_results_used"]}/{trace["worker_results"]}',    }  if __name__ == "__main__":    print(f'{"run":<10}{"outcome":<9}{"tokens":<8}{"dupes":<7}{"agents":<8}{"used":<6}flags')    for name, tr in TRACES.items():        s = score(tr)        flags = []        if s["duplicate_calls"]:            flags.append("duplicate work")        if s["over_budget"]:            flags.append("over budget")        if tr["worker_results_used"] < tr["worker_results"]:            flags.append("lead ignored a worker result")        print(f'{name:<10}{s["outcome"]:<9}{s["tokens"]:<8}{s["duplicate_calls"]:<7}{s["agents"]:<8}{s["results_used"]:<6}{", ".join(flags) or "-"}')

The traces are fixtures we wrote. We ran it on 2026-10-27. This is the real output:

run       outcome  tokens  dupes  agents  used  flagsgood      pass     510     0      3       2/2   -wasteful  pass     810     2      5       4/4   duplicate worklost-work FAIL     1090    0      3       1/2   over budget, lead ignored a worker result

In our run the wasteful trace passed, so a check on the answer alone would call it fine. The path check found two duplicate searches and a cost 59% higher than the good run (810 against 510). The lost-work trace shows a second failure that outcome checks tend to miss: a worker found something and the lead did not use it.

What to measure

Two notes from our reading. MCP's 2026-07-28 revision documents OpenTelemetry trace context propagation conventions for its _meta keys (traceparent, tracestate and baggage), which gives you a standard place to carry a run ID across tool calls. Anthropic also used rainbow deployments, which shift traffic gradually from old to new versions while both run, because agents are stateful and a deploy can break runs in flight.

When one agent beats many

The most useful call we can make on an orchestration project is to skip it. For a plain-language view of what agents do well and badly, read our reality check on AI agents. The diagram is the order of questions we ask.

Here is what we found in the sources behind each stop.

• One model call. If a prompt, a schema check and a retry are enough, more structure adds risk with no gain. Anthropic's advice is to find the simplest solution possible and only increase complexity when needed, and to measure whether the added complexity improves outcomes.
• A fixed workflow. If the steps are known, code them. Microsoft says a single agent with tools is often the right default for enterprise use cases, and that multi-agent designs add overhead, latency and extra failure modes.
• One agent with tools. Try it first and measure it. LangChain says not every complex task requires multi-agent, and that a single agent with the right tools and prompt can often do similar work.
• Multi-agent. Reach for it when the parts are independent and large, when one agent fails on a test set despite good tools, or when you need separate security boundaries for each agent. Microsoft lists cross-domain problems, distinct security boundaries and parallel specialization as the cases that justify it. LangChain lists context management, distributed development by separate teams and parallelization.

Signs you added agents too early

• You cannot say what each agent does that the others cannot.
• Two agents share most of their tools and context.
• Quality did not improve on your test set after the split.
• The agents were split by job title, such as researcher and writer, rather than by a real limit.

That last point is our judgment. Splitting by role looks like a team. The sources we read split by real limits: context size, tool count, security boundary or independence of parts.

Coding and coupled work

Coding is where the sources split most. Cognition's 2026 post says parallel agents that write code still tend to fail because each makes implicit choices that clash, and that the readonly subagent, such as a search helper, is still the most common multi-agent setup and acts more like a tool call. Anthropic's compiler project shows the other end: 16 Claude instances, each in its own container with a clone of a shared git repo, claimed tasks by writing lock files and used no orchestrating agent. The result was a roughly 100,000-line C compiler with about a 99% pass rate on most test suites, but Anthropic also lists limits, including that new features often broke existing functionality. That task had a clear automatic test suite, which most real software work lacks. Our read is that the rule extends beyond code to any task that produces one coherent artifact, such as a contract or a report with a single voice.

Failure modes to design for

From the sources and our own samples we collected ten repeating failures. Each has a symptom, a guard in the orchestration layer, and a test. The last column says where the row comes from. "Our reasoning" means we derived it from how the code works and no source states it.

O9 deserves a note. Our agent harness guide covers prompt injection and least privilege for a single agent. In a multi-agent system the lead often treats a worker's return as trusted text, and a worker that read a hostile page can pass the instruction upward. The subagent page says the Agent SDK scans a subagent's final message for instruction-shaped patterns before the parent reads it, as of Claude Code v2.1.210 and later. We take that as a sign the vendor sees the same risk. We would not rely on a scan alone. Give each worker only the tools it needs.

Frameworks and protocols, checked in October 2026

You do not have to build orchestration from scratch. If MCP is new to you, our business explainer on the Model Context Protocol is a short start. The table lists what each project's own documentation says, as we read it on 2026-10-27.

LangGraph

LangGraph's overview describes a low-level orchestration framework and runtime, and says it can be used without LangChain. We would choose it when the route should be an explicit graph, with saved state and resume. LangChain's docs also say a custom LangGraph workflow can mix deterministic logic and agentic behavior, and that the other patterns can be embedded as nodes.

OpenAI Agents SDK

The SDK lists agents, agents as tools or handoffs, and guardrails as its core primitives, with sessions and tracing among its main features. Two details stood out to us for orchestration. An input_filter can change what the receiving agent sees, and a beta nest_handoff_history option compacts earlier history into a summary that the docs warn does not redact sensitive data. Guardrails are also scoped. Input guardrails apply only to the first agent in a run and output guardrails only to the agent that produces the final output. If you chain agents by handoff, a check you assumed ran at every step may not, so we would read that scoping before relying on a guardrail.

Claude Agent SDK subagents

Subagents are separate agent instances your main agent can spawn. The docs list four benefits: context isolation, parallelization, specialized instructions and tool restrictions. Each is defined with a description that tells the model when to use it, a prompt, and optional tools, model and turn limit. In our sample we set a cheap model and a read-only toolset for one subagent and a stronger model for another, plus the three caps.

"""Claude Agent SDK: a lead agent with two read-only subagents and caps on cost, depthand concurrency. Needs an API key and the claude-agent-sdk package. We did not run thisfile; it is checked for syntax only."""import asyncio from claude_agent_sdk import AgentDefinition, ClaudeAgentOptions, ResultMessage, query  async def main() -> None:    options = ClaudeAgentOptions(        allowed_tools=["Read", "Grep", "Glob", "Agent"],        agents={            "pricing-researcher": AgentDefinition(                description="Reads pricing pages and notes in the repo. Use for price questions only.",                prompt="Find prices and terms. Return a short list with the file for each fact.",                tools=["Read", "Grep", "Glob"],                model="haiku",                maxTurns=8,            ),            "security-researcher": AgentDefinition(                description="Reads security notes and policies. Use for security questions only.",                prompt="Find security claims and the evidence for each. Say when evidence is missing.",                tools=["Read", "Grep", "Glob"],                model="sonnet",                maxTurns=8,            ),        },        env={            "CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH": "1",            "CLAUDE_CODE_MAX_CONCURRENT_SUBAGENTS": "2",        },        max_budget_usd=2.0,    )    async for message in query(prompt="Compare vendor X on price and security.", options=options):        if isinstance(message, ResultMessage):            print(message.subtype, message.total_cost_usd)  asyncio.run(main())

Per the docs, a tool you leave out of a subagent's tools field is not in its session at all, and by default a spawned subagent runs in the background. For runs with dozens to hundreds of agents, the same page points to a Workflow tool that moves the orchestration into a script the runtime runs outside the conversation context.

MCP and A2A

MCP and A2A are often named together as orchestration standards. They are plumbing. A2A's own site says MCP and A2A are not competitors and are complementary, with MCP for agent-to-tool communication and A2A for agent-to-agent communication. The same site says A2A was originally developed by Google and donated to the Linux Foundation; its Technical Steering Committee includes representatives from AWS, Cisco, Google, IBM Research, Microsoft, Salesforce, SAP and ServiceNow; and the license is Apache 2.0. A banner on the site says A2A joins the Agentic AI Foundation. The page does not explain how that relates to the Linux Foundation, so we state only what the page says.

For MCP, the 2026-07-28 revision removed protocol-level sessions and the Mcp-Session-Id header and made the protocol stateless. Experimental tasks moved out of the core into an official extension. The revision also deprecates the Roots, Sampling and Logging features, and suggests integrating directly with LLM provider APIs instead of Sampling. For orchestration, our takeaway is that a tool server no longer holds a session for you, so state that must last across calls belongs in your orchestration layer or in explicit handles the server returns.

Do you need A2A? Most single-company systems do not. If all your agents run in one codebase, a function call is simpler than a protocol. That judgment is ours.

A worked example

This example is a worked scenario we constructed. It uses our stub samples and made-up business numbers, and it is not a client result.

The job. An analyst needs a vendor comparison on price, security, support and roadmap, with sources, each week. Today one person does it in half a day.

Step 1: Decide. Walk the four questions. One model call cannot do it, because it needs many sources. The steps are not fixed, because sources differ by vendor. So test a single agent with search tools first, on ten past comparisons as a test set. Suppose it misses security evidence too often, because its context fills with pricing pages first. That is a real limit, and the parts are independent, so a supervisor with workers is justified.

Step 2: Share. The lead writes a five-line brief for each worker: the goal, what done looks like, the output format, the allowed sources and what not to do. Workers write full notes to files and return a short summary and the file name. The state has one list key for findings, appended by all workers, and no shared single-value keys, so there is nothing to conflict. A final writer node produces the single voice.

Step 3: Cap. Four workers at most, two at a time, and a token budget for the run. In our stub run the cap dropped a fifth topic and the budget stopped the second wave, which is the behavior we would want confirmed before launch.

Step 4: Check. Run the ten past comparisons and score success, tokens, duplicate calls, worker results used and accurate finish. Compare the supervisor against the single agent on the same ten. Use the cost function with your measured numbers. If the supervisor does not win on value, keep the single agent.

What EdgeFirm has published

We cite EdgeFirm work only as our published case study states it. The AI code security analysis platform page describes an agent-driven platform built on the Claude Agent SDK, with graph-based system modeling, runtime validation and adaptive failure handling. It publishes no worker count, cost ratio or multi-agent benchmark, so we draw no orchestration numbers from it. The patterns in this guide come from the sources above and from our samples.

Common mistakes

These are the mistakes we would watch for first, based on the failure table and the sources. They are our engineering judgment, not survey results.

• Splitting by job title. A researcher agent and a writer agent look like a team. Split by a real limit, such as context size, tool count, a security boundary or independence of parts.
• Letting the model pick the route for known steps. If you can write the steps, write them. Each model-chosen route is a place to drift.
• Vague briefs. A one-line task to a worker gets a vague result and duplicated searches. Give an objective, a format, allowed sources and boundaries.
• No caps. Without a budget the Agent SDK default is no limit, and a lead can start many workers. Cap workers, concurrency, depth and spend in code.
• Trusting the lead's opinion of its own result. Check with something that is not the lead, such as a test, a verifier or a person.
• Scoring only the final answer. A run can pass and waste three workers, or ignore a result. Score the path.
• Parallel agents on coupled work. If parts must agree on a style, a schema or a plan, parallel writers will drift. Use one agent or a conflict check.
• Skipping the single-agent baseline. Without a one-agent run on the same test set, you cannot say the extra agents helped.

Production checklist

Use this before a multi-agent system touches real data. Each row names its control and the failure it guards against.

Glossary

Next step

If you are weighing one agent against several for a real process, our AI agent development service covers how we scope and test agent builds. For the single-agent runtime, read the agent harness guide. For the business view of where agents fit and what they cost, read AI agents for business. For the executive view, read agentic AI for enterprise, and before you commit budget, see our comparison of an AI proof of concept and a production build. A good first step is to run the single-agent baseline and the four decision questions on one real task, and bring us the result.