labs.scale.com · Benchmarks
Scale AI coding agent benchmark of long horizon GitHub style tasks with public, private and held out splits; V2 refreshes the public set to 642 validated tasks plus a HARD-51 subset.
SWE-Bench Pro is one of 13 benchmark tools on TrueIQ. See all benchmark tools or how we check facts.
// guides
The FrontierCode benchmark explained: how Cognition grades whether maintainers would merge AI code, what Main and Diamond mean, and how to read the leaderboards.
Coding agent benchmark rankings from SWE-Bench Pro, FrontierCode, Terminal-Bench and the Artificial Analysis index, with sourced scores, dates and why they differ.
The GDPval benchmark tests AI on real work from 44 occupations. How tasks and grading work, how to read the GDPval-AA leaderboard, and what it misses.
MMLU vs SWE-bench explained: what each measures, why the scores are not comparable, and which benchmark to trust for knowledge vs coding agents in 2026.
SWE-Bench Pro explained: how the Scale AI coding benchmark differs from Verified, what changed in V2, and how to read public and private leaderboard scores.
// related
SWE-benchswebench.comOriginal Princeton SWE-bench of real GitHub issues where a fix counts only if the repo's tests pass; includes Lite, Verified and related subsets. For Scale AI's harder Pro benchmark, see SWE-Bench Pro.BenchmarksFree★ 6.1k
NoveltyBenchnovelty-bench.github.ioCarnegie Mellon benchmark for output diversity: 1,100 prompts, each answered 10 times, scored on how many answers are meaningfully different and how useful the new ones are.BenchmarksFree★ 35
GDPvalopenai.comOpenAI benchmark of real professional work: 1,320 expert written tasks across 44 occupations, with a public gold set of 220 tasks graded against human expert deliverables.BenchmarksFree
RAIDraid-bench.xyzOpen benchmark and leaderboard for AI text detectors from the University of Pennsylvania: over 6 million texts across 11 generators, 8 domains and 11 attacks, scored at a fixed false positive rate.BenchmarksFree
FrontierCodecognition.comCognition's private coding agent benchmark that grades whether maintainers of real open source repositories would merge an agent's change, scoring scope, tests and code quality as well as correctness.BenchmarksSee site
Humanity's Last Exam (HLE)lastexam.ai2,500 expert written questions from CAIS and Scale AI, plus HLE Diamond, a cleaned 1,000 question subset (500 reasoning, 500 knowledge) released September 2026.BenchmarksFree★ 1.7k