The Single-Agent Ceiling: Why Your AI System Broke at Four Use Cases

You built an agent. It answers questions, uses tools, maintains a conversation. It works. Then someone asks you to make it do more.

Add code review. Now add data analysis. Now add customer support. Now add scheduling. Each new capability means a longer system prompt, more tools jammed into a single function-calling context, and a model that has to figure out which hat to wear for every incoming message.

Here is what happens in practice. The system prompt balloons past 1,000 tokens. The tool list grows to fifteen, then twenty. The model starts choosing the wrong tool because it is parsing instructions for four domains when it only needs one. Editing the system prompt for code review breaks customer support behavior. You cannot test one domain in isolation because every test exercises the entire system.

This is the single-agent ceiling. Every production team hits it. The question is not whether you will hit it, but what you do when you get there.

The answer is not a bigger agent. It is more agents -- specialized, focused, and coordinated by a router that knows which one to call and when.

The Real Cost of One Agent Doing Everything

To understand why single agents break, you need to see the economics. Consider a system prompt that covers four domains: code review, creative writing, data analysis, and customer support. That prompt is at minimum 700 tokens. Every single request -- regardless of domain -- pays the full token cost of the entire prompt.

A code review request has to read through the creative writing instructions, the data analysis guidelines, and the customer support policy before it gets to the code review section. The model's attention is diluted across context it does not need. At scale, this adds up to real money -- you are paying for tokens the model should never have to read.

But the cost problem is the least of it. The real damage is quality degradation.

When a model has five tools for code review and five tools for data analysis and five tools for customer support, it has to choose from fifteen tools for every request. More options means worse choices. The model picks the data analysis tool when it should have picked the code review tool because the request mentioned "analyzing performance" -- and the model pattern-matched to "analysis" instead of understanding the context.

Prompt fragility is the other killer. You update the customer support section to handle a new product, and suddenly the creative writing agent starts generating output that sounds like a support FAQ. Changes are no longer local. Every modification to any domain risks breaking every other domain. This is the equivalent of a monolithic codebase where changing one function breaks an unrelated feature three modules away.

The single-agent ceiling is not a prompting problem. You cannot fix it with better prompt engineering. It is an architectural problem, and it needs an architectural solution.

The Router Agent: Intent Classification as Architecture

The core of a multi-agent system is the router. It does one thing: look at an incoming request and decide which specialist agent should handle it. Get routing right and the system feels intelligent. Get it wrong and it does not matter how good your specialists are.

Most tutorials hand-wave this. "The planner agent decides which agent to call." How? What prompt? What happens when confidence is low? What if the request spans multiple domains?

Here is how it actually works. The router is itself an LLM call -- a fast, cheap one with a low token budget. You send it the user's message along with a list of available agents and their descriptions. It returns a JSON object: the name of the best-matching agent and a one-sentence explanation of why.

The implementation has several design decisions that matter in production.

First, the agent list is built dynamically from the registered agents. Add a new specialist and the router automatically knows about it. Remove one and it disappears from the prompt. No manual prompt editing when you add capabilities.

Second, the router has a very low max_tokens setting -- around 150. It only needs to return a short JSON object. This keeps it fast. The router should be the fastest call in the system, not a bottleneck.

Third, and this is the part most implementations get wrong: the router needs a fallback chain. JSON parsing fails more often than you think. Models wrap responses in markdown fences. They add explanation text around the JSON. They hallucinate agent names that do not exist. A production router strips markdown fences, tries regex extraction if JSON.parse fails, validates agent names against the registry, and defaults to a safe fallback if everything else fails. The router never crashes. It always returns a result, even if it is a bad one.

The confidence score is what separates a demo router from a production one. The router does not just pick an agent -- it reports how confident it is in the choice. Low confidence triggers different behavior: asking the user for clarification, routing to a general-purpose agent, or running the request through multiple specialists and merging results.

This pattern -- intent classification via LLM with confidence scoring and fallback chains -- is the architectural foundation that makes everything else work. The specialist agents can be simple because they only handle one domain. The system prompt for a code review agent only contains code review instructions. Its tools are only code review tools. It never sees customer support data. It never has to figure out which hat to wear.

When You Actually Need Multiple Agents

Not every system needs this architecture. A single agent is the right choice when the domain is narrow, the tool set is small (under five tools), the system prompt fits in a few hundred tokens, and response quality is consistent across all use cases.

Multi-agent systems make sense when you have distinct domains with different expertise requirements, when different tasks need different model configurations (temperature, model selection), when you need to test and iterate on domains independently, and when you want to add new capabilities without risking existing behavior.

The decision framework is simple: if you find yourself writing section headers in your system prompt, you probably need separate agents.

The transition from single-agent to multi-agent is not a rewrite. It is a decomposition. You take the sections of your bloated system prompt and turn each one into a standalone agent. You add a router in front. You wire them together with a shared memory layer so context is not lost when a conversation spans multiple specialists.

The system is more complex in terms of moving parts. But each individual part is simpler, testable, and independently deployable. That is the same tradeoff that made microservices win over monoliths in web architecture -- and the same tradeoff that makes multi-agent systems win over single agents in AI architecture.