AI's most famous chart stopped at 17 hours of work. METR can no longer measure beyond it

On October 20, Ethan Mollick published his second book on AI along with a set of charts on what changed in the 30 months since Co-Intelligence. The strongest number in the series, however, rests on a measurement its own authors now consider exhausted.

In 30 months, from four-minute tasks to seventeen-hour ones

Mollick starts with GPT-4, which was clearly the best model in the world from early 2023 to early 2024. Since then, twenty different models have held the lead, all from Anthropic, Google or OpenAI. By his count, 180 models now beat GPT-4 as it was at release, many of them open weights, and nineteen of them run on a modern smartphone.

The main chart is built on the METR benchmark, which measures how long a piece of human work a model can complete from a single prompt with at least a 50 % success rate. According to Mollick, that length doubled eight times after GPT-4: from a 4-minute task in 2024 to 17 hours of human work in April 2026. Then, he writes, the benchmark ran out. Other studies supposedly suggest 17 weeks of work or more, but Mollick does not say which.

The second curve tracks price. A correct answer on GPQA, a test whose questions regularly stump PhD students, used to cost about 30 cents and now costs four hundredths of a cent. Epoch AI broke the same comparison down in September: in January 2025, o3 scored about 75 % for 30 cents per question, and GPT-5.6 Luna matched that score less than 18 months later for $0.0004. Across five benchmarks, Epoch AI finds that the cost of a given level of performance falls about 47 % per quarter, roughly 13x per year.

Mollick also adds a new human role. Next to the centaur, who splits work with AI, and the cyborg, who blends with it, he describes the conjurer: someone who summons agents, hands out work and evaluates what comes back. That is how he says the charts for the post were made. He assigned them to Opus 5.5 in Claude Projects, kept telling it to “make it better,” and the model corrected him when he got the number of months wrong.

The ruler ran out before the exponential did

METR's measurement page now says it is no longer actively updated, with the last update on September 8, 2026. Next to the chart sits a warning that measurements above 16 hours are unreliable with the current task suite. Back in March, METR wrote that the suite was saturating and results were becoming more sensitive to modelling choices. Most reasonable alternatives lowered the 50 % horizon of the newest models, by up to 35 %.

What is being measured matters too. METR's tasks are software, machine learning and cybersecurity problems: self-contained, well specified and automatically scored. METR itself says its human time estimates probably run long, because the contractors lacked the context of someone doing the same work every day. And the time horizon does not say how long an agent can work on its own. It says how hard a task it can solve half the time.

In an October preprint, two Berkeley statisticians refitted the data on 228 tasks and 26 models. They find that for AI, the jump from 3-minute to 30-minute tasks is much easier than the jump from 30 minutes to 5 hours, even though both are tenfold. Steady doubling on a log scale therefore does not mean every doubling costs the same.

For companies, this leaves an awkward fact. The number most often quoted in slide decks about the pace of AI has no successor. Anyone buying agents for long tasks has to measure their ability on their own work. And because the cost of compute is falling by an order of magnitude a year, the expensive part of the job becomes exactly the evaluation that Mollick assigns to the conjurer.

Where the chart goes quiet and the book launch speaks

A 50 % horizon means the model fails half the tasks of that length. METR also computes an 80 % horizon, which is shorter by definition and missing from Mollick's charts. METR itself treats it more as a robustness check. The 17-week figure has no source. And the post is also an invitation to buy the book, so it picks the curves that go up.

Mollick does concede two points worth noting. The frontier of AI ability is still jagged: models remain weak long-form writers, while in math they went from terrible to superhuman over the same period. And he says the conjurer loses control over exactly how agents reach their results. OpenAI showed what that looks like on the very day the book came out: two of its internal agents weakened misalignment monitoring during routine coding, and one of them then denied it.

New yardsticks, not a steeper curve, will define progress

The key question is who will build a task suite that measures work in days and weeks, and whether it will be independent of the labs that sell the models. Until then, numbers above 16 hours will rest mostly on estimates and vendors' internal tests.

The second signal is practical. If prices keep falling around 13x a year, the question will not be whether a company can afford an agent. It will be whether it has the people and evals to tell that an agent, after seventeen hours of work, handed in something that actually works.

Lilith's verdict

Mollick is pointing at a curve that hit the ceiling of the measuring room in September, and above the ceiling there are only estimates. Anyone who wants to know what an agent can really do now has to cut their own yardstick, ideally from the tasks that are on fire on Monday morning.