opencompass.org.cn · LLM Evals
Comprehensive evaluation platform from Shanghai AI Lab supporting many models, datasets and a public leaderboard.
OpenCompass is one of 20 LLM eval tools on TrueIQ. See all LLM eval tools or how we check facts.
// guides
The best LLM evaluation frameworks in 2026: Promptfoo, DeepEval, Ragas, Inspect and TruLens for apps, plus lm-evaluation-harness and HELM for model benchmarks.
// related
OpenAI Evalsgithub.comFramework and open registry of evaluations for testing language models and the systems built on them.LLM EvalsOpen source★ 19.7klm-evaluation-harnessgithub.comEleutherAI's unified harness for running hundreds of academic benchmarks against any model with consistent prompts.LLM EvalsOpen source★ 14.3klightevalgithub.comHugging Face toolkit for evaluating LLMs across many backends with reusable tasks and detailed sample level results.LLM EvalsOpen source★ 2.6ksimple-evalsgithub.comOpenAI's lightweight reference code for zero shot benchmark runs. No longer updated for new models since July 2025; it now mainly hosts HealthBench, BrowseComp and SimpleQA.LLM EvalsOpen source★ 4.7k
HELMcrfm.stanford.eduStanford's Holistic Evaluation of Language Models, measuring accuracy, robustness, bias and efficiency across many scenarios.LLM EvalsOpen source★ 3k
DeepEvaldeepeval.comPytest style framework for unit testing LLM apps with metrics for hallucination, relevance, RAG quality and more.LLM EvalsOpen source★ 19.2k