Why an immediate intelligence explosion won't happen even with thousands of agents

Agents will lower operational costs but won't break the understanding barrier

AI labs like OpenAI and Anthropic will soon use thousands of parallel agents for internal processes, sparking a new wave of concern about runaway development and recursive self-improvement (RSI). However, Nathan Lambert, a researcher with a history at HuggingFace and DeepMind, tempers this narrative. While faster and cheaper inference will allow multi-agent systems to run more efficiently, automating routine tasks does not automatically mean an exponential growth in intelligence itself.

The fundamental scaling laws do not change: exponentially larger computational resources are still required for a linear improvement in intelligence. Agent swarms will more easily solve measurable problems with clear outcomes but will struggle to propose entirely new research directions or solve open scientific problems where a clear definition of success is lacking.

Measuring intelligence hits the limits of definitions

The debate about when AI will surpass human experts runs into the fact that the "intelligence" of models has a completely different shape than human intelligence. LLM models do not cross milestones like "virtual assistant" or "AI researcher" all at once; rather, they asymmetrically master 80% of a given role, while the remaining 20% remains an insurmountable obstacle for a long time.

According to industry experts, including John Schulman, we can expect a functional "remote worker" for routine administrative tasks within a year. However, increasing the productivity of top researchers will take longer because while models can design and test experiments more quickly, they lack intuition and the ability to generate breakthrough hypotheses.

Post-training remains a craft for humans

RSI is most often associated with the idea that models will start training themselves. But the post-training phase, where the model learns specific behavior, is still a surprisingly manual process. Creating new and meaningful environments for reinforcement learning (RL) requires human insight, and AI models will soon begin to lack these complex assignments as high-quality training signals.

Automation in labs is currently accelerating engineering (writing code, monitoring) rather than the actual research pushing the boundaries of artificial intelligence. A fundamental increase in capabilities has not yet arrived, and models still need humans to define what the correct outcome actually is.

Commoditization is the reality, not singularity

Instead of an uncontrolled intelligence explosion, we will likely see a drastic drop in the price of current capabilities. Models will be exponentially cheaper at the same level of intelligence. This is great news for business and the spread of tools, but a disappointment for those expecting an artificial superintelligence (ASI) tomorrow.

The Jevons paradox will likely be in full effect here: cheaper models will lead to higher usage volume, creating robust businesses, not a magical transition to a world without human labor. For now, the true singularity remains postponed in a waiting room full of iterative tuning and cost optimization.

Lilith's verdict

Instead of AGI around the corner, we'll get thousands of desperately diligent clerks. The revolution won't come from a lab like a bolt from the blue, but from every boring company suddenly saving a third of its costs on processes you didn't even know existed.