Multi-agent patterns

Why split into agents at all

A single agent with twelve tools has twelve descriptions competing in one prompt, and it chooses badly. Splitting into three agents with four tools each gives every decision less to confuse it, and gives each agent a shorter, more focused instruction.

The other reason is ownership. A billing agent with its own prompt, tools and evaluation set can be changed and measured independently. The alternative is one prompt that four teams edit.

A worked comparison: one agent with twelve tools picks the wrong one often enough that the trace is unreadable. Three agents with four tools each choose correctly more often, not because the model is better but because each prompt has fewer plausible wrong answers. The cost is a router.

The supervisor is a router with a memory

It reads the state — usually the conversation and a list of what has been tried — and returns the name of the next worker, or finish. It has no tools of its own and does no domain work.

The workers see the same state. Each one appends its contribution and returns, and the supervisor decides whether another worker is needed. That is the whole architecture; everything else is prompt design inside the workers.

A worked state: messages, tried, and done. The supervisor reads the question and tried, picks a worker not yet tried, and returns its name. The worker appends an answer and marks itself tried. The supervisor runs again and either picks another worker or finishes. The loop is the same conditional edge as any other. The supervisor should not answer the question itself; if it does, the workers are decoration.

A supervisor in LangGraph

from langgraph_supervisor import create_supervisor

workflow = create_supervisor(
    [billing_agent, support_agent, docs_agent],
    model=model,
    prompt="Route to the specialist who can answer. Finish when it is answered.",
)

graph = workflow.compile()
print(graph.invoke({"messages": [("user", "Where is my refund?")]}))

Two failure modes

The first is a supervisor that keeps delegating: billing sends it to support, support sends it back, and the loop consumes the budget. The counter belongs in the state, along with the set of workers already tried.

The second is a supervisor that never delegates, because the routing prompt is vague and it decides to answer itself. The fix is the same as for tools: describe when to route, not just to whom.

A worked bounce: billing sees “refund” and hands to support, support sees “policy” and hands back to billing. Neither records that it already ran, so each handoff looks new. A tried set turns the second handoff into a stop, and a counter caps the total. The symptom is a bill that grows while the answer does not. The supervisor needs the same discipline: a budget for total steps, not only per worker.

The handoff is a state update

There is no hidden channel between agents. A worker that wants to hand off writes the next worker’s name into the state; the supervisor or the router reads it. That is why the handoff is visible in the trace and testable without the model.

The same rule makes the agents replaceable. A worker is a function of the state that returns a partial update, exactly like any other node. If it needs a private scratchpad, that is a local variable, not a field the next agent has to understand.

When one agent is enough

Three agents with two tools each is worse than one agent with six tools. The split earns its cost when the domains are genuinely separate — different tools, different instructions, different people maintaining them — or when one prompt has grown too long to follow.

Start with one agent and split when a specific confusion appears. “The billing questions keep getting answered with support policy” is a reason. “Multi-agent is the pattern” is not.

Split when the domains have different evaluation sets. A billing agent can be measured on billing cases and a support agent on support cases; one agent with both hides which half regressed. The split is justified by the measurement, not the diagram.

Coordination has a cost: every handoff is another model call, and the latency adds up. If the question needs one domain, the router is overhead you pay for every request.