Braintrust
braintrust.dev · LLM Evals
End to end platform for evaluating AI products, with datasets, scorers, prompt playgrounds and production logging.
Braintrust is one of 20 LLM eval tools on TrueIQ. See all LLM eval tools or how we check facts.
Freemium
// guides
Guides that cover Braintrust
Artificial Hivemind: Why AI Models Sound Alike
The Artificial Hivemind paper explained: why AI models give the same answers, what Infinity-Chat and NoveltyBench measure, and how to test output diversity yourself.
LLM Confidence Scores: Can You Trust Them?
Can you trust an LLM confidence score? What verbalized scores, logprobs and sampling measure, what 2026 calibration research shows, and how to set safe thresholds.
AI Agent Observability: What It Is and Top Tools
AI agent observability explained: trace tool loops, permissions and memory, the six PAI telemetry gaps, and which tools fit, from Langfuse to Datadog.
OpenAI Evals vs DeepEval: Which to Use?
OpenAI Evals vs DeepEval in 2026: platform shutdown timeline, DeepEval's pytest CI metrics, multiprovider support, and Promptfoo as the migration path.
Best LLM Observability Tools in 2026, Compared
The best LLM observability tools in 2026 compared: Langfuse, LangSmith, Phoenix, Opik, Weave and more, with licenses, self hosting and who now owns what.
Best LLM Evaluation Frameworks in 2026, Compared
The best LLM evaluation frameworks in 2026: Promptfoo, DeepEval, Ragas, Inspect and TruLens for apps, plus lm-evaluation-harness and HELM for model benchmarks.
// related
Braintrust alternatives and related tools
DeepEvaldeepeval.comPytest style framework for unit testing LLM apps with metrics for hallucination, relevance, RAG quality and more.LLM EvalsOpen source★ 19.2k
Inspect AIinspect.aisi.org.ukMIT licensed framework from the UK AI Security Institute for rigorous LLM and agent evaluations, with tool use, sandboxed tasks and 200+ prebuilt evals.LLM EvalsOpen source★ 3.1k
Promptfoopromptfoo.devOpen source CLI for testing prompts, models and RAG apps with assertions, side by side comparisons and automated red teaming. Part of OpenAI since 2026 and still MIT licensed.LLM EvalsOpen source★ 26.5k
garakgarak.aiNVIDIA's open source (Apache 2.0) LLM vulnerability scanner that probes models and endpoints for jailbreaks, prompt injection, data leakage and toxic output.LLM EvalsOpen source★ 9.7k
OpenCompassopencompass.org.cnComprehensive evaluation platform from Shanghai AI Lab supporting many models, datasets and a public leaderboard.LLM EvalsOpen source★ 7.6k
Giskardgiskard.aiOpen source Python library for evaluating and red teaming AI agents and RAG apps. Version 3 is a modular rewrite for multiturn testing; v2 is no longer maintained.LLM EvalsOpen source★ 5.9k