Scaling laws for agents will show if orchestrating dozens of LLMs is a dead end

Lambert wants hard data for multi-agent swarms

Researcher Nathan Lambert from the Allen Institute for AI publicly highlights a gap in current discourse: we lack an understanding of exactly how scaling laws behave when we scale the number of independent agents working on a single problem, rather than the parameters of a single model. While inference-time compute (as demonstrated by OpenAI with o1 models) shows clear performance gains with additional "thinking" time, the behavior of multi-agent systems (agent swarms) remains uncharted territory in terms of hard metrics.

For enterprise, this changes how compute budgets are spent

If adding ten more agents to a swarm predictably increases reliability or output quality (similarly to training on more data or using longer inference-time CoT (Chain of Thought)) it changes the math of enterprise architecture. Development teams wouldn't have to wait for GPT-5 or Claude 4. Instead, they could achieve comparable performance by cleverly orchestrating cheaper models (like Llama 3.1 8B) in a massive parallel swarm. The question is the shape of the diminishing returns curve.

Agent communication overhead will eat the scaling benefits

This is the point many agent-framework startups ignore. Adding another agent to the system doesn't just add compute power; it adds exponentially more overhead for orchestration, context passing, and managing hallucinations. If one agent fails, the error can cascade through the swarm. Without isolation and strict input typing, the benefits of scaling will drown in the noise of mutual communication.

Academic studies on swarm communication will be the turning point

Watch for major papers from labs like Google DeepMind or AI2 in the coming months attempting to quantify swarm performance on hard benchmarks (like SWE-bench or MATH). The proof that this works won't be startup demos of "an agent team writing your app," but boring charts showing the asymptotic limits of adding X instances to a system.

Lilith's verdict

The real bottleneck of agentic systems won't be inference, but managing the communication chaos. We are waiting for the researcher who draws the graph showing the exact number of agents at which a swarm collapses into its own hallucinations.