As artificial intelligence (AI) continues to make its way into the digital mainstream, several concerns about the new technology have been raised in response. The rapid advancement of ChatGPT is perhaps the most notable and controversial of a long list of AI tools impacting the world today. When prompted, the tool is capable of answering questions, performing research tasks, coding, and more. That said, the tool's ability to produce generated work carries the issues of inauthenticity and plagiarism. But can ChatGPT be detected?
The Rise of AI-Generated Content Online
Across the internet, AI-generated articles -- marketing materials, blogs, and more -- largely made by ChatGPT, have come to occupy a massive portion of content online. Roughly 57% of all web-based text has been AI-generated or translated through an AI algorithm. While this has proven troublesome for AI itself, the average internet user is faced with the challenge of identifying AI-generated content every time they visit a new website, get a new advertisement, or read a new article.
Understanding AI Detection Tools
Amid so much AI-generated content, an amount which is only increasing in size and scale, are there any ways to determine what content is AI-generated and what is human-made? In short, there are effective tools for detecting text generated by AI tools like ChatGPT, but it is important to understand how and why they work. This sort of detector is not foolproof, after all, but its already high accuracy is heightened when a user understands its functionality.
Advancements in Tools Like ChatGPT Create a Need
When OpenAI released its first ChatGPT model to the public, the results were fairly impressive, but still easily detected as machine writing at a glance. The patterns used and the tool's apparent implementation of human writing were not yet powerful enough to fool the average user. With the rise of ChatGPT-3.5, the tool became effective at holding conversations, writing essays, and helping with technical tasks. It still had limitations, but the signs of progress were clear.
The release of ChatGPT-4 in 2023 and GPT-4o in 2024 marked a major enhancement in the technology, spending more time thinking and outputting higher-quality and more coherent responses to prompts. At this point, even the most knowledgeable internet users have difficulty identifying AI-generated text, or simply cannot tell the difference anymore. While ChatGPT-5 is said to be far in the future, the company's rapid progress has made it next to impossible to identify AI-generated text on one's own.
How Detectors Use AI Against Itself
Due to the advancements in AI technology, the need for AI detectors has become apparent. That said, how does another machine perform better at detecting AI-generated text than a human observer? Simply put, an AI detector uses the same technology as tools like ChatGPT to detect AI-generated content: the large language model (LLM). While each LLM is different, the way the technology works to create distinct functionalities is fundamentally the same.
Rather than being trained solely on human writing, as with ChatGPT, an AI detector is trained on both human writing and AI-generated text. This way, an AI detector can compare differences between human writing, what text it would produce, and what AI-generators have produced. Through this comparison process, an AI detector is trained to identify detectable patterns in AI-generated text as opposed to human writing reliably and effectively.
Perplexity and Burstiness
These detectable patterns between AI-generated text and human writing essentially boil down to two factors: burstiness and perplexity. Perplexity refers to how predictable the next word of a sentence is. It tends to be the case that an AI like ChatGPT will opt for the most likely next word of a sentence, while a human writer might choose a less likely word or something overly complicated. As such, AI generates text with low perplexity and humans write with high perplexity, at least overall.
Burstiness follows a similar concept but focuses more on grammar and sentence structure. AI tools like ChatGPT tend toward sentences of a similar length in order to maintain easy readability, using simple grammar structure for the same reason. On the other hand, a human writer is much more likely to vary sentence structures and adapt grammar to suit their needs, rather than following it as a rigid guideline. As such, AI generates text with low burstiness and humans write with high burstiness, as a general rule.
Not Foolproof, but a Reliable Starting Point
While AI detectors are largely accurate and reliable, they are not entirely foolproof. This makes sense, seeing as AI detectors rely on the same LLM technology as AI generators. Any failings in the model they were trained on will translate to their overall effectiveness. ChatGPT is a powerful tool capable of mimicking natural human speech, making it possible for the AI generator to occasionally create an output that is closer to human writing than what the AI detector might compare it to.
In addition, any updates to a model like ChatGPT put AI detectors behind. While AI generators only rely on existing human writing, AI detectors rely on both existing human writing and the output of the most recent AI generator, meaning that the detector has less time to train than its generator counterpart. An AI detector is less effective following the release of a new AI generator model, and the retraining process must be repeated to keep up.
Why Everyone Can Use an AI Detector
From the average internet user to educational institutions and businesses, the need to detect AI-generated content is only on the rise. For the academic, turning in AI-generated writing as one's own is still dishonest at best and plagiarism at worst, despite the prompt being "theirs." Professors and institutions can turn to AI detectors as a starting point for ensuring academic honesty. For the internet user, navigating through potentially false AI-generated articles and information is made far easier with the aid of an AI detector. For business owners, an AI detector can help ensure that their employees aren't cutting corners and turning in work that isn't their own. Regardless of your position, the ability to identify AI-generated content is a valuable one.