And you thought actual AI writing was the problem…

You might see that headline and think I’m click-baiting you. I swear I’m not — I don’t make a habit of trying to scare people, and I generally don’t think framing anything as terrifying is beneficial. But there’s something a lot of writers might not know about, or at the very least something I didn’t know about, that could have truly terrifying implications for writers everywhere.

We all need to be wary of AI detectors.

You’ve probably heard about AI writing and how it’s a potential threat to writers everywhere, but I’ve recently come to the conclusion that AI generators are not the real threat. Sure, they’re a threat, but I’m more concerned with the many AI detectors created to determine whether a person’s writing is written by AI or all human-generated. That’s the real threat because, in my experience, these AI detectors are startlingly inconsistent.

What are AI detectors?

If you’re like me, at least like how I was up until a week ago, you probably have never heard much about AI writing tools, let alone AI detector tools! I’ve dabbled in AI only very recently when I tried out ChatGPT to write a form letter to send to my state senators, and I didn’t even like or use the letter it generated and found it wasn’t very helpful.

AI detectors, though, are a whole different animal altogether. With the rise of AI writing generators, there have been increasing concerns, specifically in academic settings, about people using ChatGPT to essentially plagiarize their work. After doing some research, it seems like high school and college students have been trying to use AI writing tools like ChatGPT to write their school essays, and teachers and professors have been trying to find ways to combat this trend.

That’s where AI detectors have entered the scene.

There are now numerous different websites created with one sole aim: to analyze text and determine whether or not it was written by a human being or a computer. There are tons of these detectors out there — zeroGPT, Originality.AI, GLTR, GPT Zero, and a new swathe of built-in AI detectors in academic platforms like Turn It In and Blackboard, just to name a few.

How are AI detectors used?

The goal of these AI detectors is to hold people accountable and make sure their writing is theirs. In academic settings, which seems to be where many of these detectors have started to pop up, this ensures students are doing the work required to actually pass their class.

Outside of academia, many writers, myself included now after learning all of this, have started to use these detectors to check their own work. It’s a way to make sure that someone can’t run your writing through one of these detectors and falsely say you didn’t write it. It’s more work, but it’s likely the way of the future.

What is the threat to writers?

Well, to put it simply: these AI detectors are far from 100% accurate. You can find loads of information online about how wrong these AI detectors can be. There are Reddit threads of concerned students who were called into the dean’s office because their essay was written with too much AI, and had to screen-record themselves typing to prove themselves innocent.

Essentially, these AI detectors can take it too far and, consequently, can harm writers.

I’ve recently run into this where my writing gave a so-called “false positive” on one of these detectors. The trickiest part was that even after rewriting virtually every paragraph, it still showed a high percentage of AI content.

And the even scarier part is that when I ran my writing through an AI detector after copying and pasting it from different files full of the same exact text, I got drastically different results. I’ve found that whether you are pasting your text from a Google Doc, a Word Doc, a PDF, or a different word-processing document can alter your result in these AI detectors.

I don’t know about you, but I find that pretty messed up and, quite frankly, terrifying.

What can we, as writers, do about it?

To be honest, I’m not sure. For now, I’m going to periodically test my writing in these AI detectors, because I don’t see another way to prevent this from happening. It’s incredibly frustrating and makes for extra work, but it seems like the best solution.

The other thing I’m going to do is spread the word. That’s why I’m sharing what I’ve found here so that other writers can be aware of these AI detectors and take action when necessary. The only way we can beat AI is by being smarter than it and continuing to produce excellent, original work.

Most importantly, why does all this matter?

Remember how I said I didn’t know about all this a week ago? Well, the reason I learned about AI detectors is because a client brought this to my attention. They were concerned when the writing I submitted for a project showed that it was over 50% AI. We’ve since resolved the issue, but that’s a problem.

And I’m not the only freelancer who has experienced this — it’s a concern for freelancers everywhere. Just check out Reddit to see what I’m talking about. Here’s one example, and here’s another. Most freelancers build up trust and a great working relationship with their clients, but what happens when someone new comes along and runs your writing through one of these detectors? I’m lucky that I haven’t lost a client over this, but others already have.

Whether you’re a copywriter, freelance journalist, novelist, or something in between, this should concern you. As AI is developed further, and consequently AI detectors become more commonplace, more and more people who work with writers will likely start testing written work. This should be concerning to us all — the mere fact that a website can discredit our writing is nothing short of problematic. It’s terrifying.

And finally, here are some AI detector “false positive” examples from ZeroGPT

Let’s get the worst example out of the way, shall we? I used to write blogs for a tattoo parlor, and one of those blogs, that I wrote entirely myself and is titled “Tattoo Pain: What to Expect When You Get Your Tattoo,” scored as virtually all AI.

Here are the results from a Medium article I wrote just a few months ago all about my experience coming back to writing on Medium after taking almost a year-long break — a topic that I doubt AI could have written about unless it’s also a writer who took a long break from Medium.

Another blog that I wrote for a cleaning company and is titled “Chemicals to Avoid in Household Cleaning Products” scored not quite as high as the one about tattoo pain, but apparently contains almost 50% AI-generated content (again, I wrote this entirely myself.)

Yet another example is this one from an article I wrote almost eight years ago and published in an online magazine, yet somehow contains almost 30% AI-generated text. I don’t think most of the AI tools we know about now were even around eight years ago, much less easily accessible by the average blogger.

And finally, for kicks and giggles, here’s the score of this very article all about AI detectors — I guess I am a human after all, though there is somehow nearly 20% of this article bashing AI detectors written by AI itself…

PS: apparently these AI detectors also scored the constitution as AI-generated, so you’re not alone if your writing is flagged as AI. In fact, you’re in good company if the founding fathers of the United States of America can’t even pass an AI detector.

PPS: I tested it myself just to make sure and can confirm ZeroGPT said the US constitution was written by an AI robot.

Previously published in The Startup