Opinion.
AI detection tools are now embedded in universities, newsrooms, and hiring pipelines worldwide. GPTZero, ZeroGPT, Originality.ai, and their competitors promise to distinguish human writing from machine output with scientific precision. They cannot. What they actually detect is a specific kind of statistical regularity that correlates more strongly with clear, structured prose than with artificial intelligence. The result is a system that punishes good writing and disproportionately flags non-native English speakers, while offering no peer-reviewed methodology and no honest accounting of its failure rate.
The core claim of these tools rests on two metrics: perplexity and burstinessIn text analysis, the variation in predictability across a document. Human writing tends to alternate between surprising and expected passages (high burstiness); AI-generated text tends to be uniformly predictable throughout (low burstiness).. Perplexity measures how predictable a text is to a language model. Low perplexity means the model finds the text unsurprising; high perplexity means the text contains unexpected word choices. Burstiness measures variation in that predictability across a document: surprising words and phrases interspersed throughout indicate high burstiness. The theory is that AI generates text with consistently low perplexity and low burstiness (smooth, predictable prose), while human writing is messier, more varied, more surprising.
This sounds reasonable until you consider what it actually measures. A text with low perplexity is not necessarily machine-generated. It is a text that uses common vocabulary, conventional grammar, and straightforward sentence structures. In practice, it is well-organized, clearly written prose. An academic paper with a logical argument and precise language will score as more “AI-like” than a rambling forum post full of typos and sentence fragments. The detector is not identifying a machine. It is identifying a pattern that humans invented long before AI existed.
The Bias Against Non-Native English Speakers
In 2023, researchers at Stanford University published a study in the journal Patterns that should have ended the conversation about AI detection tools in educational settings. Weixin Liang and colleagues tested seven popular GPT detectors on essays written by native and non-native English speakers. The detectors correctly identified native-speaker essays as human-written nearly every time. But they classified over 61% of TOEFL essays, written by real non-native English students, as AI-generated. Nearly all of the 91 TOEFL essays (97%) were flagged by at least one detector. Eighteen essays were unanimously identified as AI-generated by all seven tools.
The mechanism is straightforward. Non-native English writers tend to use simpler vocabulary, shorter sentences, and more conventional grammatical structures. They avoid idioms and complex clause embedding because those are the hardest features of a language to acquire. This produces exactly the kind of low-perplexity, low-burstiness text that detectors interpret as machine output. The detector does not know or care whether a human being sat at a desk and chose those words carefully in a second language. It sees statistical regularity and calls it artificial.
This is not a minor calibrationThe alignment between self-assessed and actual performance or knowledge. Well-calibrated people accurately estimate their own abilities; poorly calibrated people misestimate. issue. In universities that use AI detection tools to screen student submissions, a non-native English speaker writing carefully and correctly in their second language is statistically more likely to be accused of cheating than a native speaker submitting sloppy, unedited work. The tool does not detect dishonesty. It detects the linguistic fingerprint of someone who learned English from textbooks rather than from birth.
The US Constitution Is Apparently AI-Generated
ZeroGPT, one of the most widely used free detection tools, flagged 92% of the United States Constitution as AI-generated. The Bible has triggered similar results. Legal case summaries from the 1990s, decades before large language models existed, have been classified as likely machine-written. These are not edge cases or adversarial inputs designed to break the system. They are straightforward texts, written in formal English, with the kind of structural clarity that detectors interpret as non-human.
This should be disqualifying. A tool that cannot distinguish between the founding document of a democracy and a ChatGPT output is not detecting AI. It is detecting formality. It is detecting coherence. It is detecting a compression artifact of its own methodology, not a property of the text’s origin.
No Peer-Reviewed Methodology, No Disclosed Error Rates
The companies behind AI detection tools publish accuracy claims that sound impressive in marketing materials: 99% accuracy, 98% precision. These numbers come from internal benchmarks, tested on datasets the companies select and control. Independent evaluations tell a different story. A 2024 Stanford analysis found false positive rates for GPTZero ranging from 5% to 15% depending on genre. Studies of ZeroGPT report false positive rates exceeding 20%, meaning the tool incorrectly flags human text as AI-written more than one time in five.
No major AI detection tool has submitted its core methodology to independent peer review. The proprietary models are black boxes. Users cannot inspect the decision boundary, cannot understand why a specific text was flagged, and cannot meaningfully appeal a result. When a university uses these tools to adjudicate academic integrity cases, it is outsourcing a consequential judgment to a commercial product whose internal logic is secret and whose error rate is, at best, poorly characterized.
Compare this to other forensic tools used in high-stakes decisions. DNA analysis has published error rates, standardized protocols, and decades of peer-reviewed validation. Even fingerprint analysis, despite its known limitations, operates under published standards. AI detection tools ask to be treated with the same epistemic authority while providing none of the same accountability.
Laundering Bias Into an Objective Number
The deepest problem with AI detection tools is not that they are inaccurate, though they are. It is that they convert a subjective, culturally loaded judgment (“this writing feels too clean to be human”) into a numerical score that looks objective. A professor who refused to grade a student’s essay because it “sounded too polished” would face reasonable pushback. But a professor who points to a 94% AI probability score from GPTZero can frame the same gut feeling as data-driven analysis.
This is bias laundering. The tool inherits every assumption baked into its training data and detection threshold, then presents the output as a neutral measurement. The assumptions include: that human writing is inherently messy, that predictability signals artificiality, that linguistic sophistication in a second language is suspicious. None of these assumptions are stated. None are defensible. But they are encoded in the model and delivered to the user as a percentage.
The pattern is familiar. Credit scoring algorithms that penalize zip codes as a proxy for race. Hiring tools that downgrade résumés with names associated with particular ethnicities. Recidivism prediction models that flag poverty indicators as risk factors. In each case, the mechanism is the same: a system that transforms structural bias into a number, then hides behind the number’s apparent objectivity.
The Logical Endpoint: Write Worse to Prove You Are Human
If AI detection tools become the standard by which writing is evaluated, the incentive structure inverts. Students learn to write less clearly, use more colloquialisms, introduce deliberate imperfections, and avoid the kind of structured argumentation that triggers a flag. Professionals learn to roughen their prose. Non-native speakers learn that their careful, hard-won command of English is itself evidence against them.
The logical endpoint of this system is absurd and worth stating plainly: the worse you write, the more human you are. Clarity becomes suspicious. Coherence becomes evidence of fraud. The highest compliment a detection tool can pay your writing is to accuse you of not having written it.
This is not a hypothetical. Students have already reported being accused of AI use for submitting well-written work. Academics have had papers flagged. The Center for Democracy and Technology published a brief in 2024 documenting the disproportionate effects on English learners, warning that these tools risk creating a two-tier system where linguistic fluency itself becomes a liability.
The SteelmanA rhetorical technique where you present the strongest possible version of an opponent's argument before refuting it. The opposite of a straw man. and Why It Fails
The strongest argument for these detection products is that something is better than nothing. Universities face a genuine problem: students can now generate passable essays with a few keystrokes, and traditional plagiarism detection (which checks against a database of existing texts) cannot catch original AI output. Institutions need some mechanism to maintain academic integrity, and imperfect detectors, the argument goes, at least provide a starting point for investigation.
This argument fails for a specific reason. A tool with a 10 to 20% false positive rate, deployed at scale across thousands of students, does not provide a starting point for investigation. It provides a starting point for accusation. In practice, the score becomes the verdict. Students flagged by these tools face the burden of proving they wrote their own work, an inversion of the presumption of innocence that would be unacceptable in any other context. And the students most likely to be flagged are those whose writing patterns, whether because of language background or stylistic discipline, happen to resemble the statistical profile the tool associates with machines.
The better approach is also the harder one: designing assessments that AI cannot easily replicate (in-class writing, oral defense, process documentation), investing in pedagogy that makes AI assistance a tool rather than a shortcut, and accepting that the writing landscape has changed in ways that a commercial detection product cannot undo.
What AI Detection Tools Actually Are
AI detection tools are confidence products. They sell certainty to institutions that are anxious about a technology they do not fully understand. The 94% probability score is not a measurement of reality. It is a measurement of the model’s own internal state, reflecting the statistical properties of a text against a training distribution that conflates clarity with artificiality.
The tools do not know who wrote a text. They cannot know. They can only report how surprised a language model would be by the text’s word choices. And surprise, it turns out, correlates poorly with authorship. A human who writes clearly surprises the model no more than a machine does. A machine prompted to write erratically surprises it no less than a human.
Until these tools submit to independent validation, publish honest error rates across demographic groups, and demonstrate that their methodology can survive peer review, they should not be used in any decision that affects a person’s academic standing, career, or reputation. The current state of AI detection is not a technology problem waiting for a better algorithm. It is an epistemological failure: the belief that a statistical proxy for writing style can substitute for knowledge of authorship. It cannot.
Sources
- Liang, W. et al. (2023). “GPT Detectors Are Biased Against Non-Native English Writers.” Patterns, 4(7).
- Stanford HAI: “AI-Detectors Biased Against Non-Native English Writers” (2023)
- Center for Democracy and Technology: “Disproportionate Effects of Generative AI-Detectors on English Learners” (2024)
- Sensei Enterprises: “AI Detector Believes the U.S. Constitution Was Written by AI” (2023)
- The Conversation: “We Pitted ChatGPT Against Tools for Detecting AI-Written Text, and the Results Are Troubling” (2023)
