The RAID benchmark is the largest shared test for AI text detectors: a dataset from the University of Pennsylvania and partners with over 6 million machine written texts across 11 language models, 8 writing domains, 4 decoding settings and 11 adversarial attacks. Detector makers run their tool on a hidden test set and the RAID leaderboard ranks them by how much AI text they catch while wrongly flagging only a fixed share of human writing. A top RAID score tells you a detector does well on that dataset. It does not tell you how it will do on your students' essays or on text from a 2026 model.
That gap matters right now. The leaderboard has filled up fast in 2026: the RAID GitHub repository holds 60 submission folders, new detectors were merged as recently as October 20, 2026, and several commercial tools now post scores above 99 percent. Vendors quote those numbers in marketing. This guide explains what RAID actually measures, how to read the board without being misled, and how it fits with the accuracy research in our guide are AI detectors accurate.
What the RAID benchmark measures
RAID is an acronym for its goal: AI detection that holds up under pressure. It was introduced in an ACL 2024 paper (arXiv:2405.07940) by Liam Dugan and colleagues at the University of Pennsylvania, University College London, King's College London and Carnegie Mellon. The authors wrote it because detector companies claimed 99 percent accuracy on test sets they built themselves.
The design is a grid. The team took about 2,000 human documents in each of 8 domains (news, Wikipedia, paper abstracts, recipes, Reddit posts, poetry, book summaries and movie reviews), wrote a prompt from each title, and generated text with every combination of:
- 11 generators: GPT-2, GPT-3, ChatGPT, GPT-4, Mistral 7B and its chat version, MPT 30B and its chat version, Llama 2 70B Chat, and Cohere's command model in both modes.
- 4 decoding settings: greedy or random sampling, each with or without a repetition penalty.
- 11 attacks: British spelling, deleted articles, extra paragraph breaks, swapped letter case, zero width spaces, extra whitespace, homoglyphs (lookalike characters from other alphabets), shuffled digits, misspellings, paraphrasing and synonym swaps.
| Item | RAID |
|---|---|
| Paper | ACL 2024, arXiv:2405.07940 |
| Core size | About 6.2 million texts (509,014 generations plus attacked copies, and 14,971 human documents) |
| Extra set | RAID extra: Python code, Czech news, German news (2.3 million more) |
| Hidden test split | 10 percent of the data, released without labels |
| Headline metric | Share of AI text caught at a fixed false positive rate (5 percent or 1 percent) |
| License | MIT, installable with pip install raid-bench |
How RAID scores a detector
Every detector outputs a score for each text. To turn that into a yes or no, you need a threshold. RAID fixes the threshold so that the detector flags exactly a set share of the human texts (5 percent or 1 percent), and tunes it per domain so a detector cannot hide a bad false positive rate on poetry behind a good one on news. Then it reports the share of AI texts caught at that threshold. In statistics terms that is the true positive rate at a fixed false positive rate, written as TPR at 1% FPR.
This is the most useful idea in the whole benchmark. The paper showed why: at naive default thresholds, some open source detectors flagged most human text, and ZeroGPT could not get its false positive rate below 16.9 percent at all. Accuracy without a stated false positive rate is close to meaningless, and RAID forces every entry onto the same footing.
Submitting is open. A team downloads the unlabeled test set, runs its detector, and opens a pull request with its predictions. A bot on the repository scores the file against the hidden labels and commits the results, and a maintainer merges it.
What the original RAID results found
The 2024 paper tested 12 detectors (open source models such as RoBERTa, RADAR, GLTR, Fast DetectGPT and Binoculars, plus GPTZero, Originality.ai, Winston and ZeroGPT) and the findings were sobering:
- Settings fool detectors. Adding a repetition penalty cut accuracy by up to 32 points, and random sampling was harder to catch than greedy output across every detector class.
- Each detector has its own weak spot. A homoglyph attack dropped Originality.ai from 85.0 to 9.3 percent at 5 percent FPR, while GPTZero lost only 0.3 points to the same attack. Synonym swaps cut Binoculars by 36.1 points.
- Training data shows. Detectors did best on models and domains like the ones they were trained on.
The authors also took a clear position in the paper's ethics statement: they oppose using detectors in disciplinary or punitive settings until evaluation standards improve, citing the known bias against non native English writers.
How to read the RAID leaderboard in 2026
The leaderboard lets you choose the domain, generator, decoding, repetition penalty and attack, then sorts detectors by TPR at your chosen FPR. We read the result files in the repository on October 21, 2026. On the default view with no adversarial attack, at 1 percent FPR:
| Detector (submission date) | AI text caught at 1% FPR, no attack |
|---|---|
| Winston AI (Sep 2026) | 99.998% |
| Grammarly (Feb 2026) | 99.69% |
| QuillBot (Jan 2025) | 99.24% |
| GPTZero (Oct 2025) | 95.72% |
| Binoculars (2024 paper baseline) | 69.54% |
Those numbers are real, but read them with five questions in mind.
1. Which FPR and which attack setting?
A detector can look perfect at 5 percent FPR and much weaker at 1 percent, and very different with attacks on or off. Always note both settings when you quote a score. One percent is the more honest setting for any use where a false accusation hurts someone.
2. Was it trained on RAID?
The RAID training set is public, so anyone can train on it. The paper says the leaderboard splits detectors that report training on RAID from those that do not, and some entries say so in their names (one 2026 entry is labelled "RAID tuned"). A detector tuned to this data is not proving it generalizes.
3. How old are the generators?
The newest model in RAID is GPT-4 from mid 2023. None of the text comes from 2025 or 2026 models, which write differently. A 99 percent score on GPT-4 and Llama 2 era text says little about detecting today's chat output.
4. Is the benchmark saturated?
When many entries sit between 99 and 100 percent, the top of the board stops separating good detectors from great ones. The authors predicted this in their limitations section: as teams optimize for the benchmark, scores overstate real world accuracy.
5. Who is missing?
Submission is voluntary. Several detectors teachers use are not on the board as of October 21, 2026, including Pangram and Turnitin, so absence is not a bad score, and presence is not an endorsement.
What RAID does not test
RAID measures one narrow thing well: whether a detector separates fully AI written passages from fully human ones under simple, query free attacks. It does not cover:
- Mixed authorship. Real essays are often part human, part AI, then edited.
- Humanizer tools. RAID's paraphrase attack uses a 2023 research model, not the commercial rewriting tools built to beat detectors.
- Non native writers as a group. Domains are drawn from public datasets, not from second language students.
- Watermarks. Provenance signals such as SynthID or OpenAI's text watermark are a different method; see our ChatGPT watermark guide.
- Images, audio and video. For those, see deepfake detection.
How to use RAID when choosing a detector
- Check the board, not the brochure. If a vendor claims a RAID score, find its entry and the exact settings behind it.
- Compare at 1 percent FPR with attacks on. That view is closest to a hostile, high stakes setting.
- Ignore entries trained on RAID unless you only care about this data.
- Run your own test. Collect 50 or more known human texts from your setting and some AI texts from current models, and count the false positives yourself. Our best free AI detector guide lists tools you can try at no cost.
- Never act on a score alone. Treat a flag as a reason to talk, not as proof, which is the same advice in our what is AI detection explainer.
FAQ
What is RAID in AI detection?
It is a benchmark dataset and public leaderboard for AI text detectors from researchers at the University of Pennsylvania and partner universities, published at ACL 2024.
Which AI detector ranks first on the RAID leaderboard?
On the default view with no attack at 1 percent FPR, Winston AI's September 2026 submission scored 99.998 percent when we checked the result files on October 21, 2026. The order changes when you switch the attack, domain or FPR setting, and new entries arrive often.
Is a 99 percent RAID score the same as 99 percent accuracy?
No. It means the detector caught 99 percent of RAID's AI texts while flagging a fixed 1 or 5 percent of RAID's human texts. On your own writers, newer models or edited text, both numbers can be very different.
Can I run the RAID benchmark myself?
Yes. Install the raid-bench package, load the data, wrap your detector in a function that returns a score per text, and run the evaluation. The data is also on Hugging Face under liamdugan/raid. Leaderboard entries need predictions on the hidden test split submitted by pull request.
Is Pangram on the RAID leaderboard?
Not as of October 21, 2026. Pangram publishes its own evaluations; compare them with care, since they use different data. Our Pangram vs GPTZero guide covers both tools.
