HELM

crfm.stanford.edu · LLM Evals

Open source

#16 in LLM Evals

Stanford's Holistic Evaluation of Language Models, measuring accuracy, robustness, bias and efficiency across many scenarios.

#multimetric#stanford#leaderboard#transparency

HELM is one of 20 LLM eval tools on TrueIQ. See all LLM eval tools or how we check facts.

// guides

Guides that cover HELM

How to Use the MMLU Benchmark, Step by Step
How to use MMLU benchmark runs: pick the right variant, run it with lm-evaluation-harness or Inspect, smoke test the format, and read the score honestly.
LLM Confidence Scores: Can You Trust Them?
Can you trust an LLM confidence score? What verbalized scores, logprobs and sampling measure, what 2026 calibration research shows, and how to set safe thresholds.
How to Evaluate LLM Model Performance
How to evaluate LLM model performance before you pick one: build a task test set, fix settings, add error bars, test prompt sensitivity, then weigh cost.
LLM Benchmarks Explained: What the Scores Mean
LLM benchmarks explained: what MMLU, GPQA Diamond, Humanity's Last Exam, SWE-bench and ARC-AGI measure, why scores saturate, and how to spot contamination.
Best LLM Evaluation Frameworks in 2026, Compared
The best LLM evaluation frameworks in 2026: Promptfoo, DeepEval, Ragas, Inspect and TruLens for apps, plus lm-evaluation-harness and HELM for model benchmarks.
LLM Evaluation Metrics: What to Measure and How
The LLM evaluation metrics that matter in 2026: exact match, BLEU, ROUGE, BERTScore, pass@k, LLM judge rubrics, RAG and safety metrics, and when to use each.

// related