Most systems that call themselves multi agent are a single agent with a routing problem and a larger bill. Splitting one agent into three does not make it smarter, it makes it talk to itself, and every one of those internal conversations is billed at the same rate as a real user question.

Coordination cost, measured

Anthropic published numbers from their Research feature that everyone quotes for the wrong reason. Headline figure is that a lead agent on Opus 4 with Sonnet 4 subagents beat a single Opus 4 agent by 90.2 percent on their internal research eval. Read three paragraphs further and you find the honest version: token usage by itself explained 80 percent of the performance variance, and multi agent systems in their data burned roughly 15 times the tokens of a chat interaction against 4 times for a single agent. That is not an architecture beating another architecture. That is one architecture being allowed to spend more.

My own version of that lesson cost me a fortnight. Our support assistant had grown to fourteen tools across two obvious domains, product documentation and the ticket archive, and it kept calling the ticket search when someone asked a configuration question. Splitting it into a docs agent and a tickets agent behind a supervisor felt like the textbook fix, and it worked in the sense that tool selection errors on my 40 question eval set dropped from 9 to 2. Cost per resolved question went from 1.2 cents to 5.8 cents. Answer accuracy sat at 0.83 before and 0.83 after. Latency at p95 went from 3.1 seconds to 7.4 seconds. I had paid a 380 percent cost increase and 4.3 seconds to fix seven tool selection errors, and I reverted the whole thing and fixed it with a better tool description instead.

LangChain publishes a comparison across three request shapes that is worth internalising, because it shows the ranking flipping depending on what a user asks. Numbers below combine their published call and token counts with what I measured on the support assistant corpus.

Notice what happens to handoffs. Cheapest option on a single request, cheapest again when the user repeats themselves, and then worst in the class the moment a question spans two domains, because a handoff chain runs sequentially and cannot fan out. A support assistant gets multi domain questions constantly, since half of what people ask is some variant of whether a documented behaviour matches what their ticket says happened.

Four patterns and what each one buys you

Vocabulary here is a mess across frameworks, so pin the four shapes down before arguing about which to use. Same word means different things in LangChain, the OpenAI Agents SDK and whatever your team read on a blog last week, and most design arguments I have sat through were two people describing different architectures with one noun. Subagents expose each specialist as a tool on one coordinating agent, so control always returns to the centre. Handoffs transfer control outright, so the specialist talks to the user directly until it hands off again. Router classifies the request once, dispatches, and synthesises. Skills keeps one agent and loads specialist instructions into its context on demand, which is not really multi agent at all and is frequently the correct answer.

Keep this next table somewhere you can reach it during a design review. Columns are the four properties people actually argue about, and every row is a real constraint rather than a preference.

Column three is the one people underweight. LangChain documents subagents as costing an extra model call on every request compared with handoffs, and presents that as pure overhead. In production I would pay that call every time, because it is a fixed cost of one round trip against a variable cost that grows with how much specialist context each domain carries. A docs specialist with a 3,000 token instruction block poisons every downstream call in a handoff chain and costs nothing extra in a subagent, since its context dies when the tool returns.

Adding a ticket specialist to the assistant

Last part our assistant gained a checkpointed thread, so it remembers a conversation and a few durable facts about whoever is asking. Now we split it, deliberately choosing the pattern that suits a staged support flow rather than the one that suits our traffic, so that the failure is visible later. Handoffs are the right shape when a specialist needs to own the conversation, which is exactly what happens when a docs question turns into an open ticket.

# pip install langchain==1.3.14 langgraph==1.2.9
# tested on Python 3.12; export OPENAI_API_KEY before running
import os
from typing import Literal
from typing_extensions import NotRequired
from langchain.agents import AgentState, create_agent
from langchain.messages import AIMessage, ToolMessage
from langchain.tools import tool, ToolRuntime
from langgraph.graph import StateGraph, START, END
from langgraph.types import Command

assert os.environ.get('OPENAI_API_KEY'), 'set OPENAI_API_KEY in the environment'

class SupportState(AgentState):
    active_agent: NotRequired[str]

@tool
def transfer_to_tickets(runtime: ToolRuntime) -> Command:
    '''Hand the conversation to the ticket specialist.'''
    last_ai = next(m for m in reversed(runtime.state['messages'])
                   if isinstance(m, AIMessage))
    ack = ToolMessage(content='Transferred to tickets agent',
                      tool_call_id=runtime.tool_call_id)
    return Command(
        goto='tickets_agent',
        update={'active_agent': 'tickets_agent',
                'messages': [last_ai, ack]},
        graph=Command.PARENT,
    )

docs_agent = create_agent(
    model='openai:gpt-4.1-mini',
    tools=[transfer_to_tickets],
    system_prompt=('You answer from the product documentation. '
                   'If the user reports a live incident or an open ticket, '
                   'transfer to the tickets agent.'),
)
tickets_agent = create_agent(
    model='openai:gpt-4.1-mini',
    tools=[],
    system_prompt='You answer from the support ticket archive and changelog.',
)

def route_after(state: SupportState) -> Literal['tickets_agent', '__end__']:
    last = state['messages'][-1]
    if isinstance(last, AIMessage) and not last.tool_calls:
        return '__end__'
    return state.get('active_agent') or '__end__'

b = StateGraph(SupportState)
b.add_node('docs_agent', lambda s: docs_agent.invoke(s))
b.add_node('tickets_agent', lambda s: tickets_agent.invoke(s))
b.add_edge(START, 'docs_agent')
b.add_conditional_edges('docs_agent', route_after, ['tickets_agent', END])
b.add_conditional_edges('tickets_agent', route_after, ['tickets_agent', END])
graph = b.compile()

out = graph.invoke({'messages': [{'role': 'user',
    'content': 'Docs say retention is 30 days but my ticket 4471 shows 7. Which is right?'}]})
for m in out['messages']:
    print(type(m).__name__, str(m.content)[:70])

Printed output on that question, with three model calls and 2,940 prompt tokens:

HumanMessage Docs say retention is 30 days but my ticket 4471 shows 7. Whic
AIMessage 
ToolMessage Transferred to tickets agent
AIMessage Ticket 4471 is on the legacy retention policy, which the changel

Second AIMessage is empty because that turn was nothing but the transfer tool call. Note the shape of the answer: the tickets agent resolved a question that needed both the documented policy and the ticket record, and it only managed it because the handoff carried the user text forward. Had this been two genuinely independent lookups, the sequential chain would have cost double what a parallel subagent fan out would.

Malformed conversation history after a handoff

Look again at that handoff tool and the two messages it forwards. Dropping either one produces the single most common multi agent bug, and it took me about forty minutes to see it the first time because the error names a call id and nothing else. If you forward only the acknowledgement and not the AIMessage that requested it, or forward only the request and not the acknowledgement, the receiving agent sends the provider an unpaired tool call.

# the broken version: forwarding the request without its response
return Command(
    goto='tickets_agent',
    update={'active_agent': 'tickets_agent',
            'messages': [last_ai]},          # ack dropped
    graph=Command.PARENT,
)

# what the provider says
Traceback (most recent call last):
  File 'support_graph.py', line 61, in <module>
    out = graph.invoke({'messages': [...]})
openai.BadRequestError: Error code: 400 - {'error': {'message':
 "An assistant message with 'tool_calls' must be followed by tool
 messages responding to each tool_call_id. The following tool_call_ids
 did not have response messages: call_wR3aQ7pXn2", 'type':
 'invalid_request_error', 'param': 'messages', 'code': None}}

Fix is the pairing rule, and it is worth stating as a rule because it survives every framework: any assistant message carrying a tool call must be immediately followed by a tool message quoting the same tool_call_id. Forward both or forward neither. Anthropic returns a differently worded 400 for the same condition and Gemini tends to fail later with a confusing content ordering error rather than at the boundary, which is why the habit matters more than memorising one provider message.

Parallel subagents and token isolation

Where a question genuinely splits, subagents win and they win on tokens rather than on calls. Each specialist runs against its own context window, does its work, and returns a compressed result, so the coordinator never sees the specialist scratch work. Anthropic describe this as compression and it is the correct framing: a subagent is a device for turning 40,000 tokens of searching into 400 tokens of finding.

Same idea in the OpenAI Agents SDK, where handoffs are first class and history filtering is a named argument. Showing it in a second SDK is deliberate, since the pattern is portable and the API is not.

# pip install openai-agents==0.18.3
import asyncio, os
from pydantic import BaseModel
from agents import Agent, Runner, handoff, RunContextWrapper
from agents.extensions import handoff_filters

assert os.environ.get('OPENAI_API_KEY'), 'set OPENAI_API_KEY in the environment'

class Escalation(BaseModel):
    reason: str

async def on_escalate(ctx: RunContextWrapper[None], data: Escalation):
    print('escalation logged:', data.reason)

tickets = Agent(name='Tickets agent',
                handoff_description='Reads the ticket archive and changelog.')
escalation = Agent(name='Escalation agent')

triage = Agent(
    name='Triage agent',
    instructions='Route documentation questions yourself; transfer anything else.',
    handoffs=[
        handoff(agent=tickets, input_filter=handoff_filters.remove_all_tools),
        handoff(agent=escalation, on_handoff=on_escalate, input_type=Escalation),
    ],
)

async def main():
    r = await Runner.run(triage, 'Ticket 4471 has been open 9 days with no reply.')
    print('final agent:', r.last_agent.name)
    print('input tokens:', r.context_wrapper.usage.input_tokens)
    print('requests:', r.context_wrapper.usage.requests)

asyncio.run(main())

Output on that run:

escalation logged: ticket open 9 days without a response
final agent: Escalation agent
input tokens: 1614
requests: 3

Two details in there earn their keep. handoff_filters.remove_all_tools strips prior tool calls out of what the receiving agent sees, which is the ready made version of the summarisation advice above. And input_type makes the model produce structured metadata at transfer time, so on_escalate can write a row to your ticket system before the specialist starts, rather than parsing a reason out of prose afterwards. Note the SDK exposes cumulative usage on context_wrapper.usage, which is the cheapest coordination meter you will find.

Delegation instructions and duplicated work

Once specialists run in parallel, a new failure appears that has no single agent equivalent: two workers doing the same job and charging you twice. Anthropic hit this directly. Their lead agent was handing out instructions as vague as research the semiconductor shortage, and what came back was one subagent investigating the 2021 automotive chip crisis while two others duplicated each other on current supply chains. Nothing errored. Nothing looked wrong in a trace. Bill was triple and coverage had a hole in it.

Our version was smaller and dumber. I gave the coordinator two specialists whose descriptions both mentioned the changelog, and on questions about recent behaviour changes it called both, every time, for the same information. Cost per question on that slice was 2.1 times the median for three weeks before a colleague noticed while reading traces for something else. Fix was one sentence added to each description drawing a boundary: the docs agent owns current documented behaviour, the tickets agent owns what happened to a specific customer. Overlap went to zero on the next eval run.

Delegation is a writing problem, so treat it like one. A subagent task description that avoids duplication and gaps carries four things, and I now refuse to merge a new specialist without all four present.

Scale the brief to the question too. Anthropic ended up embedding explicit effort rules in the lead agent prompt after early versions spawned 50 subagents for simple queries: one agent and three to ten tool calls for fact finding, two to four subagents at ten to fifteen calls each for a direct comparison, more than ten only for genuinely open research. Without a rule like that, an agent has no way to judge how hard a question is, and the failure is always in the expensive direction. Our assistant now caps parallel specialists at three, and the cap has never once been the reason an answer was wrong.

One more failure worth naming before you ship: handoff ping pong. Two specialists that can each transfer to the other will, given an ambiguous question, pass it back and forth until your step limit fires. Mine did it four times on a question about a billing field documented in one place and disputed in a ticket, because each agent correctly judged the question belonged to the other. Set a hard transfer ceiling, count transfers in state, and above two force the current agent to answer with what it has and say what it could not confirm. A hedged answer beats a loop that costs eleven model calls and returns nothing.

Measuring overhead before you commit to a split

Every argument above collapses into one number you can measure in an afternoon: how many tokens does your system spend deciding, as opposed to answering. Run your eval set through both shapes and compute it. Anything above about a third is a system that is mostly talking to itself.

from langchain.messages import AIMessage

ROUTING_MARKERS = ('transfer_to_', 'delegate_to_')

def coordination_ratio(messages, count_tokens):
    '''Share of assistant tokens spent on routing rather than answering.'''
    routing = content = 0
    for m in messages:
        if not isinstance(m, AIMessage):
            continue
        n = count_tokens(m)
        is_routing = any(
            tc['name'].startswith(ROUTING_MARKERS)
            for tc in (m.tool_calls or [])
        ) or not str(m.content).strip()
        if is_routing:
            routing += n
        else:
            content += n
    total = routing + content
    return round(routing / total, 3) if total else 0.0

# run across the eval set, one thread per question
ratios = [coordination_ratio(run['messages'], tok) for run in eval_runs]
print('median coordination ratio:', sorted(ratios)[len(ratios) // 2])
print('worst case:', max(ratios))

median coordination ratio: 0.412
worst case: 0.667

Forty one percent was our median on the supervisor build, and two thirds in the worst case, which is the number that finally made the revert an easy conversation with my manager. Single agent with better tool descriptions came in at 0.06, since a well described tool call is a handful of tokens rather than a whole model turn. Treat 0.30 as the line: below it coordination is a reasonable overhead, above it you are funding a committee. Wrapping this measurement into your CI is the same discipline as the Data Science Series piece on CI/CD for ML pipelines, and the broader token pricing model sits in the GenAI Series piece on GenAI cost.

Stay single agent until a domain earns its own context window

My rule after the revert is narrow and it has held for a year. Split when a domain carries specialist context large enough that dragging it through unrelated turns is wasteful, roughly above 2,000 tokens of instructions and examples, or when questions genuinely fan out into independent lookups that can run at the same time. Split for those two reasons and subagents is my pick, because context isolation is the thing you are buying and the extra coordinator call is cheap next to it.

What I would avoid: a handoff ring as the default architecture. It reads beautifully in a diagram, it is the cheapest pattern on the two scenarios vendors benchmark, and it is the worst one on the multi domain traffic a real support assistant actually receives. I would also avoid splitting to fix tool selection, which is a prompt problem wearing an architecture costume.

Our assistant can now delegate, and it can also spend your budget arguing with itself, so the next thing it needs is a person in the path. Next part covers human in the loop and approval gates: where to interrupt an agent, how to resume it without replaying work, and which actions should never execute without a signature.

References

• LangChain multi agent patterns, with model call and token comparisons across scenarios
• LangChain handoffs, Command.PARENT and the message pairing rule
• OpenAI Agents SDK handoffs, input_filter and input_type
• Anthropic on building a multi agent research system, including token multiples

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts