BioBasicsBench puts agents on raw lab data. Astra earns 66.7%

OpenAI has introduced BioBasicsBench, a set of 100 tasks built with biotechnology company Ligo. Instead of textbook questions, agents receive raw material from routine wet-lab work.

One hundred tasks test whether an agent can read an experiment

The tasks span five areas, from protein methods through microscopy and cytometry to nucleic acids and cloning. A model must work with gel images, colony photographs, instrument exports, protocols or video, then return a conclusion, measurement or recommended next step.

The authors created rubrics that award partial credit. Nineteen tasks also test whether an agent recognizes that the evidence is insufficient for a conclusion. OpenAI reports that GPT-6 Astra earned an average 66.7% at the highest shared reasoning setting. GPT-5.6 Sol reached 43.8%.

For laboratories, justified hesitation can be more valuable than confidence

The practical value lies in messy evidence. A scientist often needs help deciding whether a control failed, how many colonies actually grew or whether a measurement supports any claim at all, rather than another definition from a biology textbook.

That should also change how research teams buy AI. They need evaluations built around their own instruments, protocols and failure modes, not just a general science leaderboard. BioBasicsBench provides a more useful pattern for that evaluation, even though its ranking cannot replace local validation.

Vendor-run scores cannot establish safe autonomy

OpenAI published results for its own models. The announcement does not provide a public task set and grading implementation, and an independent team has not reproduced the scores. There is also no measured human-scientist baseline, so 66.7% cannot be translated into a share of laboratory work that is safe to automate.

One hundred tasks across five domains are a useful sample but remain a narrow slice of laboratory practice. An average can also hide an error that costs little on one rubric yet ruins a week of real experimental work.

Public tasks and blind trials will show whether the score travels

The next useful signal is whether outside teams can run the same tasks against competing models and experienced laboratory staff. Blind tests on data from different instruments, protocols and sites would be stronger still.

Deployment decisions will depend on the error breakdown. Laboratories need to know when an agent misreads an image, misses a failed control or invents a conclusion from weak evidence. A single percentage cannot serve as that safety barrier.

Lilith's verdict

An agent at the microscope may save hours, but one missed failed control can send a scientist back to the start of the experiment. A 66.7% score is an invitation to a blind trial, not a pass for an unsupervised shift.