New benchmark shows AI models still struggle with full research tasks

A group calling itself AItonomy, working with the Gen-Verse project, released Science's Last Exam, a benchmark built by researchers from Harvard, MIT, Stanford, and other institutions. It contains 113 tasks spanning nine scientific disciplines, designed to test whether AI systems can go beyond answering questions and actually carry out research, from forming a hypothesis to running the scientific computing needed to test it. Early results reported alongside the benchmark show that even top AI models perform poorly on complete, end-to-end research tasks, scoring around 37.5 out of 100 on average, with some models scoring zero when asked to handle an entire research pipeline from start to finish. The benchmark is meant to be a harder, more realistic test than typical question-answering evaluations, since it requires sustained multi-step reasoning and practical execution rather than a single correct answer. It's a useful reality check on how far current systems are from independent scientific work.

Why it matters

A sobering, realistic measure of AI's actual research ability, in contrast to more optimistic benchmark claims.

Source: AItonomy / Gen-Verse