// benchmark explainer
MMLU-Pro, explained
By the TrueIQ editorial team at Ahzy Capital · Last reviewed Oct 15, 2026 · How we work
MMLU-Pro is a multiple choice test of broad knowledge and reasoning, built in 2024 because the original MMLU had become too easy for top models.
| Made by | TIGER-Lab and collaborators (2024) |
|---|---|
| Format | Multiple choice, 10 options per question |
| Size | About 12,000 questions across 14 subjects |
| Paper | arXiv 2406.01574 |
What it measures
Questions cover areas such as maths, physics, law, economics, psychology and engineering. Compared with MMLU, more questions need multistep reasoning, and going from 4 to 10 answer options makes lucky guesses much less likely (10% instead of 25%).
Where it falls short
- Still multiple choice, so it tests picking an answer, not explaining or doing a task.
- It's public, so questions can leak into training data over time.
- Top models now score high, so small differences between them mean little.
How to read a score
Compare scores only when they were run with the same prompting setup (for example, chain of thought on or off). A few points apart near the top is usually noise.
Where to see current scores
We don't republish scores. Leaderboards change weekly, and numbers depend on how the test was run. Check these instead:
- Artificial Analysis: MMLU-Pro results (independently run)
- MMLU-Pro paper