Codex is no longer just writing. It can prove it works

Devin gets a new model straight for testing

Cognition, the company behind the autonomous engineer Devin, has announced the deployment of OpenAI's new model, GPT-6 Astra. Astra reportedly achieves similar results to Fable 5, but at a 64% lower cost. On Cognition's internal benchmark, the model set a new record generating more comprehensive tests, cleaner reports, and better video evidence.

Teams face a shift in who bears the review burden

Until now, the problem with AI agents was that even if they wrote working code, a human had to review it carefully. The more code the agent generated, the more work a senior engineer had reviewing it. Shifting from "writing code" to "proving the code works" is a critical step for scaling agents. It means Devin won't just spit out PRs, but will back them up with evidence an engineer can easily check.

Demo tests aren't tests in the production jungle

There's a difference between an agent testing code in an isolated sandbox and writing a test for an existing monolith with dozens of undocumented dependencies. Astra might be great on a benchmark, but its ability to actually raise coverage in legacy projects hasn't been proven yet. A low cost per token is great, but if the agent spends hours debugging a bad test, costs will rise.

The rate of accepted evidence will decide

We will watch how the ratio of blindly merged pull requests to those a human sends back for a rewrite changes. If Astra truly shortens the time spent on review, it will be a clear signal that the era of mere autocomplete is definitively over.

Lilith's verdict

It is easy to generate tests; the hard part is generating ones a human will trust without having to read them end to end.