AI now beats people at finding facts in narrow tests. Human review has merely changed jobs

Ethan Mollick published a comparison in which modern AI agents outperform people at finding obscure facts and checking research claims. The numbers are striking. Extending them to everyday work, however, is Mollick's interpretation rather than a finding established by either study.

Two tests favor agents, but they measure different kinds of work

In BrowseComp, human searchers had roughly 2 hours for each difficult question. They matched the reference answer in 317 of 1,255 responses, or 25.3%. Mollick's chart places newer agent systems around 90%, although laboratories used different budgets and agent configurations.

The ACL 2026 DeepFact study examined claims in deep research reports. PhD level specialists working unaided scored 60.8% on hidden claims with known answers. After three further rounds in which evidence could be audited and benchmark labels revised, expert accuracy rose to 90.9%. DeepFactEval itself scored 83.4% on the frozen benchmark release.

Review shifts from manual lookup to framing the task and judging consequences

For teams, the useful change is that a model can serve as a capable second pair of eyes. It can inspect sources, decompose claims and flag conflicts faster than a person starting from an empty search box. A sensible workflow therefore need not require manual checking of every sentence.

People still choose the question, set the acceptable risk and decide what an error would cost. An internal note, a legal filing and a medication dose require different controls. No benchmark assumes that responsibility.

Mollick stretches narrow benchmark results too far into daily work

BrowseComp deliberately contains 1,266 hard to find questions and grades only final answers. DeepFact uses 143 known answer claims, 120 of which were intentionally unsupported. A system that always answered “unsupported” would therefore score 83.9%.

The results also do not show that an agent will reliably detect an incomplete brief, a bad framing choice or an obsolete internal document. They show excellent search and auditing on tightly specified tasks. That is valuable, but narrower.

Real work evals and the cost of missed errors will settle the argument

Organizations should track their own task set using real documents, local sources, several models and repeated runs. Average accuracy matters less than the kinds of errors and whether a reviewer can catch them before output reaches a customer.

Independent reproduction under a common search budget would be another useful signal. Only then can teams separate model progress from the advantages of longer runs, multiple agents and a carefully engineered test.

Lilith's verdict

A reviewer may no longer need to reopen every link, but still holds the red button. A benchmark measures accuracy, not the cost of hitting the wrong target.