ChatGPT creator OpenAI has launched a tool capable of detecting content written by AI amid concern among experts that the technology could be exploited for academic dishonesty.
The tool, which the research group called an “AI Text Classifier,” is trained to distinguish between text written by a human and text written by a variety of AI engines – not just ChatGPT.
The classifier is a “fine-tuned GPT model that predicts how likely it is that a piece of text was generated by AI from a variety of sources," OpenAI said in a blog post on Tuesday.
"Our intended use for the AI Text Classifier is to foster conversation about the distinction between human-written and AI-generated content," they continued.
The tool labels text as “very likely,” “unlikely,” “unclear if it is,” “possibly” or “likely” AI-generated, and uses data taken from a Wikipedia dataset, a 2019 Webset, and human demonstrations from the InstructGPT language model to train the tool.
Although OpenAI’s AI classifier is reportedly more accurate than others currently available online, OpenAI warns that the new tool – like many others already available – is still in development and is yet to accurately identify AI-authored content.
“The results may help, but should not be the sole piece of evidence when deciding whether a document was generated with AI,” OpenAI admitted, adding that that the classifier “is not fully reliable” and can currently only identify 26 per cent of AI-written texts in its tests.
The new tool is also unable to analyse texts of fewer than 1,000 characters, and OpenAI doesn’t recommend using it in any other languages other than English or content written by children.
AI plagiarism
The tool arrives in arrives amid controversy and speculation surrounding AI’s impact on the future of education and content creation.
Generative AI chatbots like ChatGPT scrape data from to internet to respond to almost any prompt or query. Many can write original essays, stories, and articles without of human input.
Schools and universities were quick to prohibit ChatGPT when it launched in November last year over concerns the software could lead to ‘AI plagiarism’ – where students use the AI to cheat on exams and write essays for them.
Between plummeting college enrollments, hostile state governments, book bans, campus carry, voucher programs, and ChatGPT, I'm really struggling with the fear that it's the end of days for humanities education.
— Dr. Bryan Edward Stone (@bestoneTX) February 1, 2023
Last week, researchers from Pennsylvania's prestigious Wharton School revealed that the chatbot successfully passed the final exam for its MBA program, writing “excellent” explanations for every one of the problems it solved.
In response to the education sector’s concerns surrounding its chatbot, OpenAI admitted that identifying AI-written text has been an important point of discussion among educators,” in the blog post.
OpenAI’s own CEO, Sam Altman, said that education has changed in the past with the emergence of technologies like calculators, and that AI could help spot synthetic text as well as write it to prevent students from exploiting the technology.
The research group said it is engaging with educators in the US” to find a way to ethically implement AI into education without putting the sector at risk.
Detectable AI
OpenAI is not the first to come up with a tool for detecting ChatGPT-authored text. In December last year Edward Tian, a 22-year-old student at Princeton University, created a website called GPTZero that can detect if a piece of writing has been created using ChatGPT.
Plagiarism detection tool Copyleaks also launched its own AI Content Detector this month for educational institutions and publishing.
Such detection tools will likely require several years of development before they reach a point of being accurately able to spot all AI-authored content, but it’s possible.
A recent study by Search Logistics found that the tools could detect 87.2 per cent of AI-generated content from tech publication CNET, which began using AI to write articles as part of an experiment to test the technology.
AI-written content is notably difficult to automatically detect, especially if it is edited by a human. But OpenAI said it would continue pursuing the challenge following the launch of its new tool.
“Our work on the detection of AI-generated text will continue, and we hope to share improved methods in the future,” the tool’s developers explained.
To read more about generative AI, visit our dedicated AI in the Enterprise Page.
