Can AI Detectors Tell If You Used AI? Not Reliably — and the Errors Aren’t Random

Picture the email. A student, or a junior employee, or a freelance writer opens their inbox to find that something they wrote — wrote themselves, from scratch, over hours — has been run through an “AI detector” and come back flagged. The tool has assigned a number: 92% AI, say. There is no appeal box, no explanation, and no way to show the working, because the person did the work in their own head and the evidence of that is gone. Now they have to prove a negative to a piece of software that will not say how it reached its verdict.

This is happening constantly, and it rests on a premise that does not hold. AI-writing detectors are marketed as a kind of lie detector for prose: paste text in, find out whether a machine wrote it. But they cannot reliably do that, their mistakes are frequent enough to matter, and — the part that turns an accuracy problem into a fairness one — their mistakes are not evenly distributed. They fall hardest on people who write in plain, predictable English: non-native speakers, neurodivergent writers, anyone trained to keep it simple. This is a guide to why the tools fail, why the failures are skewed, and what to do when one points at you.

They don’t detect AI. They detect predictability.

Start with what the detector actually measures, because it is not what the marketing implies. These tools do not have a secret channel into whether a language model produced a given sentence. They look at statistical features of the text itself. The two you will hear named most often are perplexity — roughly, how surprising each next word is to a language model, where lower means more predictable — and burstiness — how much the length and shape of sentences vary. The working theory is that AI writing is smoother, flatter and more predictable than human writing, so low perplexity and low burstiness get read as machine-made.

The flaw is not in the implementation; it is in the premise. Plenty of humans write in exactly the plain, even, low-variance style the detector treats as a fingerprint of AI. A student writing carefully in a second language does. A person taught the five-paragraph essay does. Someone who values clarity over flourish does. There is no ground truth here — the tool never learns who wrote anything — only a guess that “predictable” equals “artificial.” When a human writes predictably, the guess is wrong, and it is wrong in a way that looks identical to a correct catch.

The accuracy problem, in the vendors’ own numbers

You do not have to take a critic’s word for how unreliable this is; the most telling admission came from the company with the most reason to want it to work. In July 2023, OpenAI quietly withdrew its own AI Text Classifier, citing a “low rate of accuracy.” Its own published figures were stark: the tool correctly identified just 26% of AI-written text as “likely AI-written,” while incorrectly flagging 9% of genuinely human text as AI. It was also unreliable on anything shorter than about 1,000 characters. The maker of ChatGPT built a detector for ChatGPT’s output and could not make it good enough to keep online.

Commercial detectors that stayed on the market have fared no better under independent testing, which repeatedly turns up the same two failure modes at once: false positives, where a person’s own writing is called AI, and trivial evasion, where AI text lightly paraphrased sails straight through. A tool that both accuses the innocent and misses the guilty is not performing detection in any meaningful sense. It is generating a confident-looking percentage that has a weak and unstable relationship to the truth — and then handing that percentage to someone who will treat it as a verdict.

Why one percent is not a small number

Defenders of these tools point to low advertised error rates, and this is where intuition fails people. A 1% false-positive rate sounds harmless. Run it across a population and it stops being harmless. Vanderbilt University, explaining why it disabled Turnitin’s AI detector in August 2023, did the arithmetic in public: it had submitted 75,000 papers to Turnitin in 2022, and a 1% false-positive rate across that volume would have meant roughly 750 student papers wrongly flagged as containing AI writing. Seven hundred and fifty accusations, each landing on a specific person who did nothing wrong, from a single institution in a single year.

This is the base-rate trap that makes low error rates so misleading in the real world. When the thing you are screening for is relatively rare and the population is large, even a small false-positive rate produces a steady stream of false accusations — and because each one attaches to an individual with a grade, a job or a reputation on the line, the human cost is nothing like 1%. It is the same statistical mistake that makes any imperfect screening test dangerous when it is treated as a diagnosis, a problem we have written about in the context of AI trying to guess your age: a classifier that is “usually right” is still catastrophic for the specific people it is usually wrong about.

The bias: the errors aren’t spread evenly

If the false positives were scattered randomly, the tools would merely be unreliable. They are worse than that, because the errors track a demographic line. In 2023 a Stanford research team led by Weixin Liang tested seven widely used GPT detectors on two sets of genuinely human writing: essays by US eighth-graders, and TOEFL essays written by non-native English speakers. The detectors classified the native-speaker essays almost perfectly. On the non-native essays, more than half were wrongly flagged as AI-generated. Same task, same “is this human?” question, wildly different error rates depending on who the human was.

The mechanism is exactly the predictability problem from earlier, viewed through a demographic lens. Writing in a second language often means a smaller working vocabulary and simpler, more regular sentence construction — which is precisely what these tools score as “artificial.” The detector is not detecting AI; it is detecting a constrained command of English and calling it a machine. The study also found that trivial prompting tricks let real AI text evade the detectors, so the tools simultaneously punish honest non-native writers and wave through the actual automated text they exist to catch. The same dynamic catches neurodivergent writers and anyone whose training pushed them towards a plain, formulaic style. This is not a glitch to be patched; it is what the method does, and it is the reason we treat algorithmic bias as a design property, not an accident.

The arms race nobody wins

There is a further absurdity that guarantees the tools punish the wrong people. Because detectors key on predictability, the fix for a genuinely AI-using cheat is trivial: run the output through a “humaniser” that shuffles the vocabulary and varies the sentence lengths, and the score collapses. The Stanford team found exactly this — simple prompting made real AI text sail past the same detectors that were flagging honest non-native writers. So the arms race sorts people precisely backwards. The motivated cheat, who will happily paste their essay into a paraphraser, gets a clean pass. The honest writer, who submits their own plain prose untouched, gets flagged. Every escalation on the detector side is met by a cheaper counter on the evasion side, and the only group that never adopts the counter is the group with nothing to hide.

This is not a hypothetical harm. In 2023 The Markup documented international students at US universities being accused of cheating on the strength of detector scores — the exact population the Stanford study predicted would be flagged. The tool did not catch cheats; it caught people writing carefully in a second language, and handed an authority figure a number that felt like proof. The evasion tools, meanwhile, were a web search away for anyone who actually wanted them.

Is there a version of this that works?

In principle, yes — but not the version being sold. The only technically sound way to know a particular model produced a piece of text is for the model to mark its own output as it generates it, a technique called watermarking: nudging the word choices in a statistical pattern that a matching detector can later recognise. That is a fundamentally different thing from guessing after the fact, because it does not rest on the false premise that predictable writing is artificial writing. But watermarking has hard limits of its own: it only works on text from a model that chose to watermark, the signal can be weakened by editing or paraphrasing, and there is no cross-industry standard forcing every provider to do it — so a determined user simply switches to a model that doesn’t. It is telling that the genuinely responsible research direction is “mark the output at the source,” not “scan the output later and hope” — and that when OpenAI retired its classifier, it said it was turning its attention to exactly these provenance techniques instead.

The harm is asymmetric, and so is the burden

Put the pieces together and the injustice sharpens. The tool produces a confident number. The number is wrong a meaningful fraction of the time. The wrongness concentrates on people who are already at a disadvantage in an English-speaking classroom or workplace. And when the accusation lands, the burden of proof inverts: the accused has to demonstrate that they did not use AI, to a system that will not explain itself, often to an authority figure inclined to trust the software over the student.

Proving a negative is hard for anyone; it is hardest for exactly the people the detectors are most likely to flag — those with less institutional standing, less confidence arguing in English, and less benefit of the doubt to begin with. A tool sold as an instrument of academic integrity ends up manufacturing a specific, repeatable kind of unfairness, and doing it with the unearned authority of a percentage. That the underlying models are also confidently wrong in other settings — the same overconfidence we catalogue in the hallucination problem that is still not solved — should make anyone slower to treat a machine’s certainty as truth, not faster.

What actually works

None of this means cheating isn’t real or that nobody should care who wrote what. It means the detector is the wrong instrument, and there are better ones:

• Treat the score as a weak signal, never as evidence. A detector output is, at best, a reason to look more closely — not a finding. Any process that lets a percentage stand in for a judgement is broken before it starts.
• Keep your receipts. If you write, keep your drafts, notes and version history. Google Docs and Word both retain edit histories that show a document being built over time — far better evidence of authorship than any detector is of the opposite.
• Ask the tool to account for itself. If someone confronts you with a detector result, it is fair to ask what its documented false-positive rate is, whether it has known bias against non-native writers, and what corroborating evidence exists beyond the score.
• Prefer process to forensics. The institutions handling this best have moved back towards judging the work directly — drafts, conversations, the ability to explain your own argument — which is slower than a percentage but does not fabricate accusations.

The honest summary is uncomfortable for anyone hoping for an easy answer: there is currently no reliable way to look at a finished piece of text and know whether a machine helped write it. The tools that claim otherwise are selling confidence they do not have, and charging the cost of that confidence to the people least equipped to dispute it. Until that changes, the safest thing to distrust is not the student or the writer in front of you. It is the detector.