Heads We Win, Tails You Lose: Why AI Detection Cannot Be the Answer to Academic Integrity
THIS IS RELEVANT TO YOU BECAUSE:
- You're under pressure to respond to generative AI in assessment, and an AI detector that returns a percentage looks like the fast, scalable answer to a problem that suddenly feels unmanageable.
- You're already running a detection tool, or being asked to buy one, and you want to know whether its headline accuracy and false positive figures actually hold up outside the vendor's own testing conditions.
- You're responsible for misconduct processes that have to meet a real standard of proof, and you need to understand what counts as evidence an assessor can genuinely verify, and what does not.
- You're aware that a detector flag falls hardest on the students least able to absorb it, including non-native English writers and international students, regardless of whether they did anything wrong.
- You're accountable for GDPR and, increasingly, EU AI Act compliance, and you suspect that staff may already be pasting student work into free detection sites with no oversight, no data processing agreement, and no institutional visibility.
A FIRST LOOK AT WHAT'S INSIDE...
What does an AI detector actually detect? It estimates how closely a piece of text resembles machine-generated writing. It does not establish whether a machine wrote it, and that gap is where this white paper begins.
Drawing on a peer-reviewed analysis published in the Journal of Higher Education Policy and Management alongside independent testing of the leading commercial tools, the paper sets out why AI detection fails on its own terms. There is no ground truth against which a flagged script can be checked. The base rate fallacy means the very same detector can be reliable in one cohort and worse than a coin toss in another, depending on a figure no institution can ever measure: how much AI use is really happening. And independent research keeps surfacing what vendor marketing tends to leave out, including false positive rates that climb sharply for second-language writers and accuracy that deteriorates further once text is lightly edited or paraphrased.
The paper then traces the consequences institutions rarely sign up for deliberately. The burden of proof quietly inverts, leaving honestly written work to be defended against an accusation that, by the researchers' own analysis, can be neither proven nor disproven. Staff running scripts through free detection sites expose the institution to data protection and intellectual property risk it never authorised. And under the EU AI Act, a tool that cannot explain its own output sits uneasily against the transparency and explainability duties the Act places on the institution as deployer, not the vendor.
Rather than argue for a better detector, the report explains why UNIwise has deliberately chosen not to build generative AI detection into WISEflow Originality, and why verifiable similarity detection against identifiable sources meets an evidentiary standard a probabilistic score cannot. It also asks where the far more common problem actually sits, since UNIwise's own research points to peer-to-peer copying rather than machine generation as the dominant real-world pattern. The paper closes on a question of responsibility: whether adapting to generative AI is the institution's task to carry through assessment design, clear guidance and sound academic judgement, or a burden being passed quietly down to the students it exists to educate.