OpenAI's BioBasicsBench tests whether AI agents can read real wet-lab data

OpenAI argues that existing benchmarks lean on charts and images selected for scientific papers, which miss the failed experiments, ambiguous images and conflicting observations of real lab work[1]. BioBasicsBench covers five areas: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning[1].

Scientists wrote each task from their own experiments, paired with an expert-written grading rubric, and each went through review by a second Ligo scientist, an external PhD-level wet-lab scientist and a final internal check[1]. Inputs include 31 tasks with gels and blots, 14 with microscopy, 12 with sequencing files and 7 with protocol videos, among others[1].

OpenAI evaluated seven of its own models across 37 model and reasoning effort settings, with five attempts per task, for 18,500 results in total[1]. At the xhigh setting, the results were[1]:

Mean reward is partial rubric credit, not a task success rate. OpenAI says GPT-6 Astra beats GPT-5.6 Sol even at low reasoning effort while using about half the output tokens at xhigh, and puts the gain down largely to stronger native multimodal handling[1]. In one example Astra noticed a negative control plate was labelled with the wrong antibiotic and rejected it, where GPT-5.6 Sol took it at face value[1].

OpenAI is clear that higher scores do not yet mean models can make reliable judgements in the lab[1]. Its published failure cases show Astra picking the wrong colonies because it assumed a larger PCR product meant successful cloning, discounting microscopy evidence that contradicted its preferred interpretation, and overcounting colonies on a plate by roughly 12%[1].

Everyday lab analysis is where AI science assistants either save researchers time or quietly lead them down the wrong path. A benchmark built from messy, real data is a more honest test than curated figures, though for now it only covers OpenAI's models and one partner lab's experiments. OpenAI also says tracking these capabilities will help it understand their implications for biosafety and biosecurity[1].