Lilith.
⌕
Editorial illustration: Agents finish 46% of research tasks, but stumble on infrastructure
Lilith illustration · editorial remix

OpenAI has published a second set of internal measurements saying coding agents complete 46% of research tasks estimated at two to four hours of human work without intervention. The figure was 23% in March. The headline is striking, but the report also shows a sharply uneven capability frontier.

Success on multi hour tasks doubled in seven months

For tasks estimated at four to eight hours, zero intervention success rose from 13% to 35%. Average agent workdays per researcher per day increased from 0.5 to 5.7. OpenAI defines one agent workday as eight hours of runtime.

Usage is both intensive and expensive. Median daily use per researcher reached $2,400 at API list prices in October, 62% above September. By mid October, more than two thirds of included researchers reached a daily peak of at least four concurrent agents.

Monitoring advances faster than work that can break systems

Differences between tasks are larger than the overall average. Agents succeeded without intervention on 79% of training run monitoring tasks and 72% of technical questions. They reached 35% on launching training runs, 29% on building evaluation and reinforcement learning environments and only 25% on writing inference and serving infrastructure code.

For research leads, this is a practical delegation map. Agents can scan logs, find anomalies and prepare analyses. Human review remains costly but sensible when a change spends compute or touches production infrastructure.

The company is measuring its own product on its own staff

This is not an independent benchmark. OpenAI measures its own researchers, internal models and workflows, all of which changed during the period. The company notes that researchers may have delegated different kinds of work within each duration bucket, so the increase mixes better models with better usage.

More changed lines of code or more experiments do not automatically mean better research. Interviews in the report describe low quality pull requests, wasted steps and invalid analytical assumptions. People still set priorities and judge which ideas deserve more work.

Research output matters more than the number of open sessions

OpenAI plans to track whether capability research and safety research accelerate at different rates. The useful signal will be the relationship between agent use, development time and the quality of new models, rather than runtime alone.

Comparable measurements from other labs and methods that separate agent capability from task selection would make the claim stronger. Only then will 5.7 agent workdays look like a shorter route to results rather than simply more occupied screens.

Lilith's verdict

A researcher can receive 5.7 agent workdays in one day, yet the agent succeeds on inference code only a quarter of the time. The factory is moving faster, but a person still holds its most sensitive controls.