github.com · Benchmarks
Graduate level, Google proof science Q&A: 448 expert written biology, physics and chemistry questions, with a 198 question Diamond subset widely reported in model launches.
GPQA is one of 13 benchmark tools on TrueIQ. See all benchmark tools or how we check facts.
// guides
How to use MMLU benchmark runs: pick the right variant, run it with lm-evaluation-harness or Inspect, smoke test the format, and read the score honestly.
MMLU vs GPQA explained: broad exam recall versus expert written science questions, why GPQA scores run lower, and which benchmark still means anything in 2026.
MMLU vs SWE-bench explained: what each measures, why the scores are not comparable, and which benchmark to trust for knowledge vs coding agents in 2026.
MMLU vs MMLU Pro explained: how the 4 option and 10 option benchmarks differ, why their scores aren't comparable, and which one to trust for LLMs in 2026.
HLE Diamond leaderboard and scores for October 2026: the 1,000 question Humanity's Last Exam subset, Scale and lastexam.ai results, with and without tools.
LLM benchmarks explained: what MMLU, GPQA Diamond, Humanity's Last Exam, SWE-bench and ARC-AGI measure, why scores saturate, and how to spot contamination.
// related
Humanity's Last Exam (HLE)lastexam.ai2,500 expert written questions from CAIS and Scale AI, plus HLE Diamond, a cleaned 1,000 question subset (500 reasoning, 500 knowledge) released September 2026.BenchmarksFree★ 1.7k
ARC-AGIarcprize.orgAbstraction and Reasoning Corpus series from Francois Chollet and the ARC Prize Foundation that tests skill acquisition on novel tasks; ARC-AGI-3 adds interactive games.BenchmarksFree★ 4.9kBIG-benchgithub.comBeyond the Imitation Game: a collaborative benchmark of 200+ diverse tasks from 450 authors at 132 institutions, built to probe and extrapolate language model capabilities.BenchmarksFree★ 3.2kMMLU-Progithub.comHarder successor to MMLU from TIGER-Lab: about 12,000 reasoning heavy questions in 14 disciplines with ten answer options, published at NeurIPS 2024.BenchmarksFree★ 433
SWE-benchswebench.comOriginal Princeton SWE-bench of real GitHub issues where a fix counts only if the repo's tests pass; includes Lite, Verified and related subsets. For Scale AI's harder Pro benchmark, see SWE-Bench Pro.BenchmarksFree★ 6.1kMMLUgithub.comMassive Multitask Language Understanding: a multiple choice benchmark spanning 57 subjects from math to law, introduced in 2020 and now largely saturated by frontier models.BenchmarksFree★ 1.6k