Do AI Detectors Actually Work? The Honest, Data-Backed Answer

Do AI Detectors Actually Work?

No, not reliably. AI detectors and AI checkers can catch obvious, unedited AI text some of the time, but their accuracy drops sharply against paraphrased writing, and they routinely mislabel real human writing as AI-generated, especially from non-native English speakers. A single score from one tool is not proof of anything.

"AI detector" and "AI checker" cover a crowded field: Turnitin's AI writing indicator, GPTZero, Copyleaks, Originality.ai, QuillBot, and dozens of smaller tools. Nearly all of them run the same trick. They measure statistical properties of a text, mainly perplexity (how predictable each word is given the words before it) and burstiness (how much sentence length and structure varies), and use those signals to guess whether a machine or a person wrote it.

That trick worked reasonably well against raw, unedited output from early models. It works much less well now, against newer models and against anyone who runs the text through a paraphraser first, which takes about ten minutes. The rest of this piece walks through the actual numbers: how wrong these tools are, in which direction, and what to do instead of trusting a percentage.

How Accurate Are AI Detectors, According to the Companies That Built Them?

Even the company that built GPT walked away from its own detector. OpenAI launched an AI Text Classifier in January 2023 and retired it six months later, citing a 'low rate of accuracy.' In one independent test just before the shutdown, it correctly flagged AI text only once out of every seven tries.

OpenAI is not a neutral party here. It builds several of the models detectors are trying to catch, and it still could not make its own classifier reliable enough to keep online. The company shut the tool down on July 20, 2023, less than six months after launch, citing its low rate of accuracy. In TechCrunch's own side-by-side test run shortly before the retirement, OpenAI's classifier correctly identified only one of seven AI-generated passages, while a free competing tool caught five of seven the same day.

A University of Maryland study cited around the same time described detector performance in blunter terms: not much better than a coin toss. If the lab that trained the underlying models can't build a dependable detector for its own output, that is a reasonable floor for how skeptical to be about everyone else's.

Do AI Checkers Give False Positives?

Yes, and it is the more damaging failure mode in academic and hiring settings. Turnitin has publicly claimed a false-positive rate near 1 percent for its AI writing indicator. Vanderbilt University ran that number against its own submission volume and concluded it could mean roughly 750 students wrongly flagged in a single year, then disabled the tool.

A 2023 academic study, Testing of Detection Tools for AI-Generated Text (Weber-Wulff et al.), evaluated 14 detection tools, including Turnitin, on human-written and AI-written samples plus adversarially paraphrased versions of the same text. It concluded the tools tested were "neither accurate nor reliable," with a bias toward calling AI-written text human rather than the reverse. In that study, the more common failure was AI text slipping through undetected, not human text getting falsely flagged, and simple paraphrasing made every tool meaningfully worse.

Turnitin's own public position is a false-positive rate near 1 percent for its AI writing indicator. Vanderbilt University took that figure and ran it against its own numbers: roughly 75,000 papers submitted through Turnitin per year, meaning even a genuine 1 percent error rate could mean about 750 students wrongly flagged for AI use annually. Vanderbilt disabled the feature in August 2023 rather than accept that risk, citing both the false-positive exposure and the lack of transparency into how the tool actually decides.

Can Teachers Tell If You Used AI?

Not reliably, and detectors make the risk worse for one group specifically. A Stanford study tested seven popular GPT detectors on 100% human-written essays and found they flagged 61.22 percent of essays by non-native English speakers as AI-generated, versus 5.19 percent of essays by native speakers, roughly a twelvefold gap.

The study, GPT detectors are biased against non-native English writers (Liang, Yuksekgonul, Mao, Wu & Zou, Stanford, published in Patterns), compared 91 real TOEFL essays written by non-native English speakers against 88 essays written by US eighth-graders, all pulled from datasets that predate ChatGPT, so every sample was confirmed human-written. Seven widely used GPT detectors were near-perfect on the native-English essays, averaging a 5.19 percent false-positive rate. On the TOEFL essays, the same seven detectors averaged 61.22 percent, and all seven tools unanimously agreed on flagging 19.78 percent of the non-native essays outright.

The likely cause isn't that detectors can read intent. Non-native writers tend to use a narrower range of vocabulary and simpler sentence structure than native speakers, which is the same statistical signature detectors use to spot AI text. The researchers tested this directly: prompting ChatGPT to enrich the TOEFL essays' word choice, without changing a single fact, dropped the false-positive rate from 61.22 percent to 11.77 percent. Run the experiment backward and simplify a native speaker's essay, and its false-positive rate climbs too. The tools are measuring word choice, not authorship, which means a teacher trusting a detector score is, in part, grading vocabulary range and calling it academic integrity.

Why Is Detecting AI Text So Hard?

Detection is a statistics problem, not a fingerprint check, and paraphrasing defeats it easily. A 2025 study found three detectors that scored 98 to 100 percent accuracy on unmodified AI text fell to 52 to 71 percent on the same text after a single pass through a humanizing paraphraser.

Most detectors score perplexity and burstiness, the same two signals mentioned above. Machine output used to be smoother and more statistically predictable than human writing, which is why the trick worked at all. It stops working the moment someone, or another AI, paraphrases the text, because paraphrasing changes exactly the statistical pattern the detector is reading without changing the meaning underneath it.

A 2025 paper, Evaluating the Performance of AI Text Detectors, Few-Shot and Chain-of-Thought Prompting Using DeepSeek Generated Text (Alshammari & Rao, University of Missouri), tested six detectors against text generated by DeepSeek. On the original, unedited AI output, GPTZero, Copyleaks, and QuillBot scored 100, 98.8, and 98.4 percent accuracy. Against the same text after one pass through a humanizing paraphraser, accuracy fell to 52, 71, and 58 percent respectively, shown in the chart at the top of this page. The Stanford researchers behind the non-native-writer study found something similar from a different angle: a single self-edit prompt asking ChatGPT to "elevate the text by employing literary language" cut detection rates on AI-generated college essays from 100 percent to 13 percent, and on AI-generated academic abstracts, detectors already missed up to 68 percent of unedited fakes before any rewriting at all.

Can Watermarking Fix This?

Watermarking is the more promising long-term direction, but it is not a solved problem for text yet. Google's SynthID is embedded across Google's own AI output and, as of May 2026, OpenAI's image output too. SynthID's text watermark is documented as vulnerable to paraphrasing, the same attack that already breaks statistical detectors.

Watermarking works differently from detection. Instead of guessing after the fact, the model embeds an invisible statistical pattern into its own output at generation time, and a verifier checks for that pattern later. Google's SynthID does this across its image, audio, video, and text generation, and OpenAI announced in May 2026 that it would embed SynthID watermarks into images from ChatGPT, Codex, and its API, alongside C2PA content-provenance metadata. That is real progress for images.

For text specifically, a 2025 robustness study found SynthID's text watermark is "vulnerable to meaning-preserving attacks, such as paraphrasing, copy-paste modifications, and back-translation," the same category of edit that guts perplexity-based detectors. Watermarking also only ever covers text from a model that chose to embed a mark in the first place. It says nothing about AI text generated before watermarking existed, from a model that skips it, or run through a paraphraser after the fact.

What Should You Do Instead of Trusting a Detector Score?

Don't treat a detector percentage as a verdict, in a classroom or a business. Teachers get better results asking about the writing process directly. Employers and business owners get better results judging the deliverable itself, on quality, sourcing, and whether it holds up under a follow-up question, not a single automated score.

If you're a teacher, a detector percentage is not evidence on its own. Pair it with a conversation: ask the student to explain a specific choice in the piece, show an earlier draft, or walk through their research process. That approach catches real academic dishonesty without risking the kind of false accusation that pushed Vanderbilt, and a growing list of other universities since, to turn detectors off entirely.

If you're an employer or a business owner evaluating a freelancer's, vendor's, or agency's work, the same logic applies. Judge the output, not a probability score. Does the writing hold up under a specific follow-up question? Does the code run and pass its tests? Do the cited sources actually say what the piece claims they say?

We build and evaluate AI systems for local businesses and larger companies, and we do not run our own output, or anyone else's, through an AI detector and call the review finished. We judge text and code the way we'd judge a human contractor's work: on whether the claims check out, the sources are real, and the result holds up when someone tries to break it. That standard is slower than a percentage score. It is also the one an AI detector, with its documented false positives and its collapse against basic paraphrasing, cannot fake for you.