ragas.io · LLM Evals
Open source library of RAG and LLM app evaluation metrics with synthetic test set generation, maintained by Vibrant Labs, the company founded by the creators of Ragas.
Ragas is one of 20 LLM eval tools on TrueIQ. See all LLM eval tools or how we check facts.
// guides
What is answer completeness in RAG evaluation? The share of key facts from a reference answer that a response covers, and how Azure, Bedrock and Ragas score it.
Inspect AI vs DeepEval in 2026: how the two open source eval frameworks differ on who runs the model, scoring, agents, sandboxes, CI and which team each one suits.
What is faithfulness in RAG evaluation? The share of an answer's claims backed by the retrieved context, and how Ragas, DeepEval, TruLens and HHEM score it.
OpenAI Evals vs DeepEval in 2026: platform shutdown timeline, DeepEval's pytest CI metrics, multiprovider support, and Promptfoo as the migration path.
RAG evaluation metrics explained: faithfulness, relevancy, context precision and recall, which need a reference, and how Ragas, DeepEval and TruLens differ.
How LLM hallucination detection works in 2026: faithfulness checks, detector models like HHEM, self consistency and semantic entropy, and the tools to use.
// related
DeepEvaldeepeval.comPytest style framework for unit testing LLM apps with metrics for hallucination, relevance, RAG quality and more.LLM EvalsOpen source★ 19.2k
mlprintsgithub.comOpen source AGPL-3.0 framework from Sentient for implementing, attacking and verifying LLM fingerprinting methods in a shared evaluation environment.LLM EvalsOpen source★ 3
Galileogalileo.aiEvaluation, observability and guardrail platform for AI agents and LLM apps. Cisco completed its acquisition in May 2026 and is folding it into Splunk Observability.LLM EvalsFreemium
Inspect AIinspect.aisi.org.ukMIT licensed framework from the UK AI Security Institute for rigorous LLM and agent evaluations, with tool use, sandboxed tasks and 200+ prebuilt evals.LLM EvalsOpen source★ 3.1k
Promptfoopromptfoo.devOpen source CLI for testing prompts, models and RAG apps with assertions, side by side comparisons and automated red teaming. Part of OpenAI since 2026 and still MIT licensed.LLM EvalsOpen source★ 26.5k
garakgarak.aiNVIDIA's open source (Apache 2.0) LLM vulnerability scanner that probes models and endpoints for jailbreaks, prompt injection, data leakage and toxic output.LLM EvalsOpen source★ 9.7k