HLE Diamond (written HLE-Diamond by its authors) is a frozen 1,000 question subset of Humanity's Last Exam, released September 22, 2026 by the Center for AI Safety and Scale AI. It splits into 500 reasoning and 500 knowledge questions. GPT-6 Astra led the launch table at 59.9 percent, and it also leads Scale's leaderboard at 60.6 percent (HLE-Diamond announcement; Scale leaderboard).
Most people searching for HLE Diamond want the numbers, so the score tables come first. After them you will find what the subset is, how the two halves differ, why tools change everything, why two official sources print slightly different scores, and how to get the dataset. Every figure below links to the page it came from, rechecked on October 28, 2026.
HLE Diamond scores right now
There are two official HLE Diamond tables, and they do not match exactly. Both are closed book (no tools) results.
Table 1: the authors' launch post at lastexam.ai (September 22, 2026; every model at "high" reasoning):
| Model | HLE-Diamond (no tools) | Reasoning half | Knowledge half |
|---|---|---|---|
| GPT-6 Astra | 59.9% | 74.2% | 45.6% |
| Gemini 4 Argon | 56.9% | 61.8% | 52.0% |
| Claude Opus 5.5 | 54.6% | 62.4% | 46.8% |
| GPT-6.1 Sol | 53.2% | 67.4% | 39.0% |
| Claude Fable 5.1 | 51.6% | 61.8% | 41.4% |
| Muse Wonder | 34.7% | 43.2% | 26.2% |
| Claude Sonnet 5.5 | 33.1% | 42.4% | 23.8% |
| Kimi K3.1 Pro | 29.5% | 33.6% | 25.4% |
| Grok 4.7 | 22.8% | 31.2% | 14.4% |
Table 2: Scale's HLE Diamond leaderboard (17 models listed when we rechecked on October 28, 2026; top ten shown, with Scale's stated uncertainty and RMS calibration error):
| Rank | Model | Overall | Uncertainty | Calibration error |
|---|---|---|---|---|
| 1 | GPT-6 Astra | 60.6% | ±3.00 | 28.1 |
| 2 | Claude Opus 5.5 | 55.0% | ±3.10 | 7.1 |
| 3 | Gemini 4 Argon | 53.2% | ±3.07 | 53.3 |
| 4 | Claude Fable 5.1 | 51.3% | ±3.10 | 13.8 |
| 5 | Claude Opus 5 | 38.6% | ±3.00 | 21.5 |
| 6 | Claude Sonnet 5.5 | 37.5% | ±3.00 | 36.8 |
| 7 | Muse Wonder 1.0 | 37.3% | ±3.04 | 31.1 |
| 8 | Gemini 3.8 Flash | 34.3% | ±2.90 | 57.2 |
| 9 | GPT-6 Sol | 33.8% | ±2.90 | 38.8 |
| 10 | GPT-5.6 Sol | 31.2% | ±2.90 | 53.8 |
Scale's own main results table gives the split for GPT-6 Astra as 75.6 percent on reasoning and 45.6 percent on knowledge. Further down its leaderboard, Grok 4.7 sits at 23.4 percent and Kimi K3 at 22.2 percent. Note the version: the launch post scores Kimi K3.1 Pro (29.5 percent), while Scale lists the older Kimi K3, so those two rows are not the same model.
The token cost differs a lot too. At high reasoning, the launch post lists average use from about 4,500 tokens per question for GPT-6.1 Sol and 5,200 for GPT-6 Astra up to about 43,200 for Grok 4.7 (Kimi K3.1 Pro used about 9,800), which matters if you pay per token.
What HLE Diamond is
Humanity's Last Exam (HLE) is a benchmark of 2,500 expert written questions with single, checkable answers, built to test models at the edge of human knowledge after older exams such as MMLU stopped separating top models (arXiv paper; Nature article). HLE Diamond is the refined subset that came out of a year of cleaning. What the official pages confirm:
- Size: 1,000 questions, drawn from the existing HLE pool and its held out reserve, according to Scale.
- Frozen: the set is fixed with stable question IDs, so Scale says a score from today is comparable to one from next quarter.
- Headroom: Scale expects the set "to carry signal for the next 6 to 12 months."
- Scope: Scale writes that high accuracy would show expert level performance on closed ended scientific knowledge, and would not on its own suggest autonomous research ability or "artificial general intelligence."
What is not published, as of October 24, 2026: per question criteria for why an item was kept or dropped, or how many items were repaired rather than removed.
Reasoning vs knowledge partitions
Each question carries a partition label, reasoning or knowledge, and the final score is aggregated across both (dataset card). The split is the most useful part of the release. In the launch table every model does better on reasoning than on knowledge, and the order can flip by half: Gemini 4 Argon trails GPT-6 Astra overall but posts the best knowledge score, 52.0 percent. Scale notes a strong correlation between the two halves across models, so the flips are the exception, but a single headline number still hides two abilities.
Without tools vs with tools
HLE Diamond questions are meant to be answered closed book. Because labs also use HLE to test agents that can browse and run code, the authors publish recommended settings for tool use and a second table:
| Model | Without tools | With tools (web and code) |
|---|---|---|
| GPT-6 Astra | 59.9% | 82.9% |
| Claude Opus 5.5 | 54.6% | 73.9% |
| Claude Fable 5.1 | 51.6% | 72.3% |
| GPT-6.1 Sol | 53.2% | 69.1% |
| Claude Opus 5 | 38.1% | 68.9% |
| GPT-6 Sol | 32.8% | 65.0% |
| Muse Spark 1.3 | 24.6% | 55.4% |
Tools add between about 16 and 32 points in this table. A "with tools" number and a "no tools" number are different tests, even for the same model.
Why leaderboards show different numbers
Scale and lastexam.ai are both official sources, yet GPT-6 Astra shows 60.6 percent on one and 59.9 percent on the other, and Claude Fable 5.1 shows 51.3 against 51.6. Neither page explains the gap. Our reading of the two pages:
- They are separate runs. Scale grades with an LLM judge that extracts and checks each final answer, with prompts that follow Wei et al. (2024). Model outputs vary from run to run, and so can judge decisions.
- The gaps are mostly inside the error bars. Scale prints an uncertainty of about plus or minus 3 points for each model. GPT-6 Astra, Claude Opus 5.5, Claude Fable 5.1 and Grok 4.7 differ by less than a point between the two tables. Gemini 4 Argon (56.9 vs 53.2) and Claude Sonnet 5.5 (33.1 vs 37.5) differ by more, so treat their exact rank as uncertain.
- The model lists differ. The launch post names GPT-6.1 Sol and Kimi K3.1 Pro, which Scale's board does not list, while Scale adds models such as Gemini 3.8 Flash, Kimi K3 and DeepSeek V4 Pro.
- Scale ranks with confidence intervals. Its rank counts how many models are significantly better, so two models a point apart can share a rank.
Third party sites that mirror these tables add a further step of copying. When you quote an HLE Diamond score, name the source, the date and whether tools were on. Our guide to LLM leaderboards compared covers the same problem across other boards, LiveBench vs Artificial Analysis shows how two composite indexes can rank the same models differently, and Scale's SEAL leaderboards page lists its other evaluations.
HLE Diamond vs the full HLE and HLE Rolling
There are several versions of "the HLE score" in circulation:
- Full HLE: 2,500 questions, finalized April 3, 2025, multimodal, on Hugging Face as
cais/hle(GitHub). Scale's full HLE board is a separate page with its own numbers; the HLE benchmark page covers the full exam. - HLE Rolling: a dynamic fork launched in October 2025 and updated over time, with a public change log. Scale says most of the fixes that became HLE Diamond landed first in HLE Rolling.
- HLE Verified: an independent February 2026 cleanup (arXiv 2602.13964). Of 2,500 items, 668 passed as is, 1,143 were repaired and 689 set aside, and average accuracy for eight frontier models rose by 7 to 10 points on the verified set.
- HLE Diamond: the authors' own frozen 1,000 question release.
Scores on different versions are not interchangeable. HLE Diamond is also unrelated to GPQA Diamond apart from the name: GPQA Diamond is a 198 question slice of a separate graduate science benchmark, so the two scores cannot be compared. For how benchmark families like these evolve, see LLM benchmarks explained and MMLU vs MMLU Pro.
How to access the dataset
HLE Diamond is a gated dataset on Hugging Face, cais/hle-diamond (test split). You request access, and the maintainers ask users not to share, reupload or redistribute it. It carries a new canary string, which the card describes as a superset of the BIG-bench canary, so model builders can filter it out of training data. Questions go to agibenchmark@safe.ai.
To run it yourself, the official HLE repository includes an evaluation script that uses a judge model, and harnesses such as Inspect AI let you run it next to your own tests (our Inspect AI vs DeepEval guide explains when a model eval harness fits better than an app testing library). The HLE maintainers advise against setting the completion limit below 8,192 tokens, because reasoning models can fail when they run out of output. If you are new to judge based grading, see LLM as a judge, and for what the calibration column measures, see LLM confidence scores.
FAQ
What is HLE Diamond?
HLE Diamond (HLE-Diamond) is a frozen 1,000 question subset of Humanity's Last Exam, released September 22, 2026 by the Center for AI Safety and Scale AI after a year of cleaning. It has 500 reasoning and 500 knowledge questions.
Which model has the highest HLE Diamond score?
GPT-6 Astra. It scored 59.9 percent without tools in the authors' launch table and 60.6 percent on Scale's leaderboard, with Claude Opus 5.5 second on Scale at 55.0 percent. With web and code tools, GPT-6 Astra reached 82.9 percent in the authors' tool table.
How many questions are in HLE Diamond?
1,000, split evenly between a reasoning partition and a knowledge partition. The full Humanity's Last Exam has 2,500 questions.
Why do Scale and lastexam.ai show different scores?
They report separate runs graded by an LLM judge, and model outputs vary between runs. Most gaps are within the roughly 3 point uncertainty Scale prints, and the two tables also list different models. Always quote the source and date.
Where can I see the HLE Diamond leaderboard?
Two official places: Scale's HLE Diamond leaderboard, which ranks 17 models with error bars and calibration error, and the authors' launch post, which adds reasoning and knowledge splits, token use and a with tools table.
Can I download HLE Diamond?
Yes, after requesting access to the gated cais/hle-diamond dataset on Hugging Face. The authors ask users not to share, reupload or redistribute it, and it carries a canary string so it can be filtered out of training data.
