// category · 20 tools · 17 open source
LLM Evaluation Tools
Frameworks and platforms for testing language models and LLM applications, from academic benchmark harnesses to CI friendly unit tests, LLM as judge scoring and red teaming kits.
LLM evaluation tools test how well a language model application does its job before and after it ships. This category covers open source frameworks, hosted evaluation platforms and red teaming kits. They differ from public benchmarks: instead of ranking models on a shared exam, they score your own prompts, your own retrieval pipeline and your own agent against test cases you define.
Evaluation frameworks. DeepEval, Ragas, promptfoo, OpenAI Evals, Inspect AI and the LM Evaluation Harness let you write tests in code and run them in CI. Some focus on retrieval quality, with metrics such as faithfulness and context precision; others are built for running academic benchmarks against many models.
Hosted platforms. Braintrust, Patronus AI and similar services add datasets, experiment tracking and dashboards on top of scoring, which helps teams compare prompt and model changes over time.
Red teaming. garak, PyRIT, DeepTeam and Giskard probe models for jailbreaks, prompt injection, data leaks and other failure modes. They complement normal quality tests rather than replacing them.
Most modern evals rely on three kinds of scorer. Code checks test exact or structured outputs. Reference metrics compare an answer with a known good one. LLM as a judge uses a second model to grade answers against a rubric. Judges scale well but show position and verbosity biases, so validate them against human labels on a sample first. Narrow pass or fail questions tend to be easier to validate than vague one to five scales.
A sound process looks like this: collect real examples, write clear criteria, build a test set that includes known failures, run it on every change, and add production failures back into the set.
Start with our playbook on how to evaluate LLM applications, then read LLM as a judge, RAG evaluation metrics and best LLM evaluation frameworks. For security testing, see LLM red teaming tools.
DeepEvaldeepeval.comPytest style framework for unit testing LLM apps with metrics for hallucination, relevance, RAG quality and more.LLM EvalsOpen source★ 19.2k
Inspect AIinspect.aisi.org.ukMIT licensed framework from the UK AI Security Institute for rigorous LLM and agent evaluations, with tool use, sandboxed tasks and 200+ prebuilt evals.LLM EvalsOpen source★ 3.1k
Promptfoopromptfoo.devOpen source CLI for testing prompts, models and RAG apps with assertions, side by side comparisons and automated red teaming. Part of OpenAI since 2026 and still MIT licensed.LLM EvalsOpen source★ 26.5k
garakgarak.aiNVIDIA's open source (Apache 2.0) LLM vulnerability scanner that probes models and endpoints for jailbreaks, prompt injection, data leakage and toxic output.LLM EvalsOpen source★ 9.7k
OpenCompassopencompass.org.cnComprehensive evaluation platform from Shanghai AI Lab supporting many models, datasets and a public leaderboard.LLM EvalsOpen source★ 7.6k
Giskardgiskard.aiOpen source Python library for evaluating and red teaming AI agents and RAG apps. Version 3 is a modular rewrite for multiturn testing; v2 is no longer maintained.LLM EvalsOpen source★ 5.9k
PyRITmicrosoft.github.ioMicrosoft's open source (MIT) Python Risk Identification Tool for red teaming generative AI, with composable targets, converters, scorers and multiturn attacks.LLM EvalsOpen source★ 4.7k
OpenAI Evalsgithub.comFramework and open registry of evaluations for testing language models and the systems built on them.LLM EvalsOpen source★ 19.7k
Ragasragas.ioOpen source library of RAG and LLM app evaluation metrics with synthetic test set generation, maintained by Vibrant Labs, the company founded by the creators of Ragas.LLM EvalsOpen source★ 16.2k
TruLenstrulens.orgInstrumentation and feedback functions for evaluating and tracking LLM apps, known for the RAG triad metrics.LLM EvalsOpen source★ 3.6klm-evaluation-harnessgithub.comEleutherAI's unified harness for running hundreds of academic benchmarks against any model with consistent prompts.LLM EvalsOpen source★ 14.3k
DeepTeamtrydeepteam.comOpen source (Apache 2.0) LLM red teaming framework from Confident AI, built on DeepEval, covering 50+ vulnerability types with OWASP, NIST and MITRE presets.LLM EvalsOpen source★ 3.1klightevalgithub.comHugging Face toolkit for evaluating LLMs across many backends with reusable tasks and detailed sample level results.LLM EvalsOpen source★ 2.6k
Evidentlyevidentlyai.comOpen source library for evaluating, testing and monitoring ML and LLM systems, from data drift to response quality.LLM EvalsOpen source★ 8ksimple-evalsgithub.comOpenAI's lightweight reference code for zero shot benchmark runs. No longer updated for new models since July 2025; it now mainly hosts HealthBench, BrowseComp and SimpleQA.LLM EvalsOpen source★ 4.7k
HELMcrfm.stanford.eduStanford's Holistic Evaluation of Language Models, measuring accuracy, robustness, bias and efficiency across many scenarios.LLM EvalsOpen source★ 3k
Patronus AIpatronus.aiAutomated evaluation platform with research backed judge models for catching hallucinations and unsafe outputs.LLM EvalsSee siteHN 1mlprintsgithub.comOpen source AGPL-3.0 framework from Sentient for implementing, attacking and verifying LLM fingerprinting methods in a shared evaluation environment.LLM EvalsOpen source★ 3
Braintrustbraintrust.devEnd to end platform for evaluating AI products, with datasets, scorers, prompt playgrounds and production logging.LLM EvalsFreemium
Galileogalileo.aiEvaluation, observability and guardrail platform for AI agents and LLM apps. Cisco completed its acquisition in May 2026 and is folding it into Splunk Observability.LLM EvalsFreemium// guides
Guides to LLM evaluation tools
Best LLM Evaluation Frameworks in 2026, Compared
The best LLM evaluation frameworks in 2026: Promptfoo, DeepEval, Ragas, Inspect and TruLens for apps, plus lm-evaluation-harness and HELM for model benchmarks.
LLM Red Teaming Tools: Promptfoo, garak, PyRIT
The best LLM red teaming tools in 2026 compared: Promptfoo, garak, PyRIT, DeepTeam and Giskard, with licenses, first commands and OWASP LLM Top 10 coverage.
Inspect AI vs DeepEval: Which Should You Use?
Inspect AI vs DeepEval in 2026: how the two open source eval frameworks differ on who runs the model, scoring, agents, sandboxes, CI and which team each one suits.
OpenAI Evals vs DeepEval: Which to Use?
OpenAI Evals vs DeepEval in 2026: platform shutdown timeline, DeepEval's pytest CI metrics, multiprovider support, and Promptfoo as the migration path.
LLM Evaluation Metrics: What to Measure and How
The LLM evaluation metrics that matter in 2026: exact match, BLEU, ROUGE, BERTScore, pass@k, LLM judge rubrics, RAG and safety metrics, and when to use each.
How to Evaluate LLM Model Performance
How to evaluate LLM model performance before you pick one: build a task test set, fix settings, add error bars, test prompt sensitivity, then weigh cost.
Prompt Injection Attacks: Examples and How to Test
What a prompt injection attack is, direct and indirect examples, why there is no complete fix, and how to test your LLM app with Promptfoo, garak and PyRIT.
LLM Hallucination Detection: Methods and Tools
How LLM hallucination detection works in 2026: faithfulness checks, detector models like HHEM, self consistency and semantic entropy, and the tools to use.
How to Evaluate LLM Applications (2026 Guide)
How to evaluate LLM applications step by step: error analysis, code checks, calibrated LLM judges, CI gates and production sampling, with tools for each stage.
LLM as a Judge: How It Works and When to Trust It
LLM as a judge explained: how model graders work, the biases that skew them, how to validate one against human labels, and which open source tools to use.
What Is Answer Completeness in RAG Evaluation?
What is answer completeness in RAG evaluation? The share of key facts from a reference answer that a response covers, and how Azure, Bedrock and Ragas score it.
How to Use the MMLU Benchmark, Step by Step
How to use MMLU benchmark runs: pick the right variant, run it with lm-evaluation-harness or Inspect, smoke test the format, and read the score honestly.
What Is Faithfulness in RAG Evaluation?
What is faithfulness in RAG evaluation? The share of an answer's claims backed by the retrieved context, and how Ragas, DeepEval, TruLens and HHEM score it.
RAG Evaluation Metrics: Ragas vs DeepEval vs TruLens
RAG evaluation metrics explained: faithfulness, relevancy, context precision and recall, which need a reference, and how Ragas, DeepEval and TruLens differ.
Artificial Hivemind: Why AI Models Sound Alike
The Artificial Hivemind paper explained: why AI models give the same answers, what Infinity-Chat and NoveltyBench measure, and how to test output diversity yourself.
LLM Confidence Scores: Can You Trust Them?
Can you trust an LLM confidence score? What verbalized scores, logprobs and sampling measure, what 2026 calibration research shows, and how to set safe thresholds.
LLM Benchmarks Explained: What the Scores Mean
LLM benchmarks explained: what MMLU, GPQA Diamond, Humanity's Last Exam, SWE-bench and ARC-AGI measure, why scores saturate, and how to spot contamination.
MMLU vs GPQA: Key Differences and Which to Use
MMLU vs GPQA explained: broad exam recall versus expert written science questions, why GPQA scores run lower, and which benchmark still means anything in 2026.
MMLU vs SWE-bench: Which Should You Use?
MMLU vs SWE-bench explained: what each measures, why the scores are not comparable, and which benchmark to trust for knowledge vs coding agents in 2026.
MMLU vs MMLU Pro: Key Differences and Which to Use
MMLU vs MMLU Pro explained: how the 4 option and 10 option benchmarks differ, why their scores aren't comparable, and which one to trust for LLMs in 2026.
AI Model Fingerprinting Explained
AI model fingerprinting embeds ownership signals in LLMs. How the Fingerprinting Leaderboard works, SaTML 2026 findings, and how it differs from SynthID.
AI Agent Observability: What It Is and Top Tools
AI agent observability explained: trace tool loops, permissions and memory, the six PAI telemetry gaps, and which tools fit, from Langfuse to Datadog.
Best LLM Observability Tools in 2026, Compared
The best LLM observability tools in 2026 compared: Langfuse, LangSmith, Phoenix, Opik, Weave and more, with licenses, self hosting and who now owns what.