novelty-bench.github.io · Benchmarks
Carnegie Mellon benchmark for output diversity: 1,100 prompts, each answered 10 times, scored on how many answers are meaningfully different and how useful the new ones are.
NoveltyBench is one of 13 benchmark tools on TrueIQ. See all benchmark tools or how we check facts.
// guides
The Artificial Hivemind paper explained: why AI models give the same answers, what Infinity-Chat and NoveltyBench measure, and how to test output diversity yourself.
// related
SWE-benchswebench.comOriginal Princeton SWE-bench of real GitHub issues where a fix counts only if the repo's tests pass; includes Lite, Verified and related subsets. For Scale AI's harder Pro benchmark, see SWE-Bench Pro.BenchmarksFree★ 6.1k
SWE-Bench Prolabs.scale.comScale AI coding agent benchmark of long horizon GitHub style tasks with public, private and held out splits; V2 refreshes the public set to 642 validated tasks plus a HARD-51 subset.BenchmarksFree★ 555
RAIDraid-bench.xyzOpen benchmark and leaderboard for AI text detectors from the University of Pennsylvania: over 6 million texts across 11 generators, 8 domains and 11 attacks, scored at a fixed false positive rate.BenchmarksFree
Humanity's Last Exam (HLE)lastexam.ai2,500 expert written questions from CAIS and Scale AI, plus HLE Diamond, a cleaned 1,000 question subset (500 reasoning, 500 knowledge) released September 2026.BenchmarksFree★ 1.7k
ARC-AGIarcprize.orgAbstraction and Reasoning Corpus series from Francois Chollet and the ARC Prize Foundation that tests skill acquisition on novel tasks; ARC-AGI-3 adds interactive games.BenchmarksFree★ 4.9k
BIG-benchgithub.comBeyond the Imitation Game: a collaborative benchmark of 200+ diverse tasks from 450 authors at 132 institutions, built to probe and extrapolate language model capabilities.BenchmarksFree★ 3.2k