Artificial Analysis launches Intelligence Index v5, mostly on private tests

Artificial Analysis rolled out version 5 of its Intelligence Index, its composite score for comparing AI model capability, on October 28. The firm says this is its biggest overhaul since v4 in January, and for the first time most of the index, 60%, now runs on private test sets or held-out answers rather than public benchmarks, which makes it harder for models to have been trained on the answers. The update adds two new benchmarks, including a private coding set from Cognition graded on whether a human maintainer would actually merge the change, plus a science-focused terminal benchmark, and it upgrades three existing tests while dropping others.

At launch, Artificial Analysis had Claude Opus 5.5 leading at a score of 58, just ahead of OpenAI's GPT-6 Astra and GPT-6.1 Sol, with Anthropic and OpenAI holding the top five spots. The gap between the best proprietary models and the best open-weights models widened to 12 points, with Kimi K3.1 Pro the top open-weights model at 46. Cost per task varied by more than 13x across leading models. Later the same day, Artificial Analysis updated its results after Google's Gemini 4 Carbon launched, making Carbon the new number 1 on the index at 59.

The firm also launched a separate Progress Index that tracks model capability over time on one continuous scale going back to November 2022.

Why it matters

Shifting benchmarks toward private, unseen tests makes model leaderboards harder to game and more useful for developers choosing which model to build on.

Source: Artificial Analysis on X