To evaluate LLM model performance, test two or more candidate models on the same set of examples drawn from your real task, with the prompt, settings and scoring held fixed, then report each score with a standard error and compare the models question by question. Use public leaderboards only to build the shortlist. Finish by checking how much the result moves when you reword the prompt, and weigh the quality gap against latency and price.

This guide is about choosing and comparing models: which base or hosted model should sit behind your product, and whether a new release is really better than the one you use today. If you already have an app in production and want to find out why it fails, that is a different job; follow our playbook on how to evaluate LLM applications, which starts from traces and error analysis.

Why a leaderboard number is not an evaluation

A public score tells you how a model did on someone else's questions, prompt and settings. That is useful for a first cut (our guide to LLM leaderboards explains which ones to read), but it is not the same as knowing how the model will do on your work.

Three things make a single headline number unreliable:

  • The task may not match yours. A knowledge quiz says little about a support bot, and a coding benchmark says little about contract summaries. Our explainer on LLM benchmarks covers what each famous test actually measures.
  • The settings are rarely the same. Prompt format, few shot examples, chain of thought and temperature all change scores, sometimes by more than the gap between two models.
  • There is no error bar. Evan Miller of Anthropic notes in Adding Error Bars to Evals that evals are usually reported with a "highest number is best" mentality and without any test of statistical significance. A two point lead can be pure noise.

How to evaluate LLM model performance in seven steps

1. Write down the decision

Start with one sentence: "We are choosing between model A and model B for task X, and we care most about Y." Y might be accuracy on a classification, the share of support answers a reviewer accepts, or the pass rate of generated code. Also note the hard limits, such as a latency ceiling, a monthly budget or a need to self host.

2. Shortlist with public data, then stop reading leaderboards

Use one or two aggregators to cut the field to three or four candidates. Artificial Analysis combines an intelligence index with measured speed and price; LMArena ranks models by human preference votes. After that, your own test set decides.

3. Build a test set from your real task

Collect examples that look like real traffic (past tickets, documents, queries or code tasks), each with an answer or grading rule. Include the awkward cases. Keep a held out slice that you never use while tuning prompts, so you are not quietly fitting the prompt to the test.

Size matters more than most teams expect. Miller's paper works through a power analysis: with reasonable (fictional) parameters, reliably detecting a three point difference between two models takes roughly 1,000 independent questions, and he suggests new evals aim for at least that many. You can start smaller, but know that a 100 question set can only detect large gaps.

4. Fix the settings and record them

Use the same prompt template, system message, few shot examples, output limit and scoring code for every model. Record the exact model version, the date and the provider. If you change any of these later, rerun every model.

Be careful with temperature. Setting it to zero feels like it removes randomness, but Miller advises against changing temperature just to reduce variance, because it changes the model's behaviour and can shift noise somewhere you can't remove it. Evaluate at the temperature you will actually use in production, and sample each question more than once if answers vary.

5. Run the models with a real harness

Pick a harness that logs every prompt, response and score:

  • lm-evaluation-harness from EleutherAI is the standard for running open weight models on public benchmarks. Its README shows the basic pattern: install a backend such as lm_eval[hf] or lm_eval[vllm], then call lm_eval --model hf --model_args pretrained=<model> --tasks <task>.
  • Inspect, from the UK AI Security Institute, suits custom tasks and agent evaluations, and Miller's paper points out that its built in stderr() metric computes the standard error correctly.
  • DeepEval and OpenAI Evals are better known for application testing, but they work for model bake offs on your own dataset too. See our LLM evaluation framework comparison for the tradeoffs.

6. Score with the right metric

Match the metric to the output. Exact match or accuracy works for labels and multiple choice. Execution tests work for code. Rubrics scored by people, or by an LLM judge that you have checked against human labels, work for open ended text. Avoid overlap metrics such as BLEU for anything a person will read; our guide to LLM evaluation metrics explains why.

7. Add error bars and compare question by question

This is the step most comparisons skip. For each model, report the mean score and its standard error, and give the 95% confidence interval as the mean plus or minus 1.96 standard errors. Then, because both models answered the same questions, compute the difference on each question and take the standard error of those differences. Miller calls this paired analysis and shows it is a "free" reduction in variance, because models tend to agree on which questions are easy and which are hard.

Two more rules from the same paper:

  • If your questions come in groups (several questions about one document, or one question translated into many languages), use clustered standard errors. On the reading comprehension set DROP, the clustered error for Anthropic's own models was about three times larger than the naive one.
  • If a score is the average of several samples per question, compute the error across the per question averages, not across every individual sample.

In Miller's worked example, a model that looks better on two of three evals has a statistically significant lead only on the third, which belongs to its rival.

How to check LLM prompt sensitivity

Before you trust a winner, change the prompt a little and run again. Sclar and colleagues, in Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design, changed only formatting (separators, casing, spacing, how options are numbered) and found accuracy differences of up to 76 points for Llama 2 13B in few shot classification. Across 320 formats and 53 tasks, GPT-3.5 showed a median spread of 6.4 points and a maximum of 56. The sensitivity did not disappear with larger models, more examples or instruction tuning.

Worse for model comparisons, the best format for one model was often not the best for another, and which of two models looked better could flip just by changing the format. The authors recommend reporting a range of scores across several plausible formats rather than one number from one template.

A practical version for a model bake off:

  1. Write three to five versions of your prompt that mean the same thing.
  2. Run every candidate on every version.
  3. Compare models on their worst and median scores, not only their best.

If the ranking flips between versions, treat the models as tied and let cost, speed or licensing decide.

How to evaluate LLM generated code

For code, run it. The HumanEval paper from OpenAI (Chen et al., 2021) scored functional correctness with unit tests and introduced pass@k: the chance that at least one of k samples passes. In that paper Codex solved 28.8% of problems with one sample, and repeated sampling raised that sharply, which is why you should always state k.

Two cautions. HumanEval has only 164 problems, so in Miller's illustrative table its scores carry a standard error of about three points. And short function puzzles say little about multifile changes. For agent style coding work, use your own repository tasks, or see our guides on SWE-bench Pro and MMLU vs SWE-bench.

Measure what quality costs: latency, throughput and price

A model that wins by two points but doubles your bill or response time may not be the right pick. Measure the same things the independent benchmarkers measure. Artificial Analysis's methodology defines time to first token (the wait before the first token arrives; for reasoning models that is the first reasoning token), output speed in tokens per second after the first token, and end to end response time. It reports medians over the past 72 hours, and since March 2026 its default speed figure uses prompts of about 10,000 input tokens, because longer prompts slow both the first token and the output.

Do the same on your own prompts and from your own region, since network distance affects time to first token. For reasoning models, count the hidden reasoning tokens too: they add both time and cost. Then work out cost per accepted answer, not cost per million tokens, because a cheaper model that needs more retries can cost more.

Look beyond a single accuracy number

Stanford's HELM paper measures seven metrics per core scenario (accuracy, calibration, robustness, fairness, bias, toxicity and efficiency) so tradeoffs stay visible. You don't need all seven, but most decisions need at least quality, robustness to rewording, safety on your sensitive cases and cost.

A checklist before you switch models

  • The test set reflects real traffic and has a held out slice.
  • Every model ran with identical prompts, settings and scoring code, and you recorded versions and dates.
  • Scores carry standard errors, and the comparison uses paired, question level differences.
  • The winner still wins across several prompt wordings.
  • Latency, throughput and cost per accepted answer fit your limits.
  • You saved the logs, so you can rerun the same eval when the next model ships.

FAQ

How do you measure the performance of an LLM model?

Run the candidate models on the same task specific test set with fixed prompts and settings, score each answer with a metric that fits the output (accuracy, execution tests or a rubric), and report the mean with a standard error. Then compare models on paired, per question differences and check speed and cost.

Is a higher benchmark score enough to choose a model?

No. Public benchmarks use other people's tasks and settings, often without error bars, and prompt formatting alone can move scores a lot. Use them to shortlist, then test on your own data.

How many test examples do I need to compare two LLMs?

It depends on how small a difference you need to detect. In Anthropic researcher Evan Miller's worked example, detecting a three point gap reliably took roughly 1,000 independent questions. Smaller sets are fine for catching large gaps, especially if you sample each question several times.

What temperature should I use when evaluating an LLM?

The temperature you will use in production. Lowering it only to reduce noise changes the model's behaviour and can hide variance rather than remove it. If answers vary, sample each question several times and average per question.

How do you evaluate LLM generated code?

Execute it against unit tests and report pass@k with k stated, as the HumanEval paper did. For realistic work, use multifile tasks from your own codebase or agent benchmarks such as SWE-bench Pro rather than short function puzzles.

What is the difference between evaluating an LLM model and an LLM application?

Model evaluation compares models on a fixed task so you can choose one. Application evaluation tests your whole system (prompts, retrieval, tools) on real traces to find and fix failures. Most teams need both.