GPTZero: The latest volley in the AI writing wars

[sue_frantz]

In December 2022, I wrote about the new AI tool for generating writing, ChatGPT. Since then, the technology behemoths Microsoft and Google have rushed their own chatbots to public release. Unfortunately, neither were ready for primetime as both were reported to have delivered inaccurate information in their debut.

In the meantime, discussion within the academy has exploded about the impact AI writing tools will have on education. Such discussions are often accompanied by much hand-wringing. Some instructors insist that their assignment prompts are not ChatGPT-able because, for example, the prompt asks for personal examples or personal opinion. Other instructors have embraced ChatGPT as a learning tool where they ask students to start with ChatGPT text, critique said text, and then edit it.

Perhaps the biggest tell that a particular text was written by ChatGPT is that the references are bogus. The references look legitimate, but a quick Google search will reveal that the AI made them up. Making up references is academic dishonesty. A cloze test would provide further evidence that the student did not write the text. In a cloze test, the instructor removes every, say, fifth word from the text in question, and the student is asked to supply the missing words. A student who wrote the text will have an easier time supplying the missing words than a student who didn’t. This online cloze test generator will create a cloze test based on the supplied text and your parameters.

The latest participant in the AI-generated text wars is Edward Tian, a Princeton grad student. During his winter break in Toronto, he spent time in a coffee shop writing code for a computer program that could detect AI-written text. He called it GPTZero (Kidson, 2023), and he has made it freely available. Paste in your text or upload a file, check the box saying you agree to the (pretty generic) terms of service, and click the “get results” button. Tian’s rationale was that since ChatGPT uses an algorithm to write text, code that is based on that same algorithm can detect that same text. For example, ChatGPT writes text by using what the next word in a sentence is most likely to be. Humans, however, tend to be less predictable in our writing. GPTZero uses a similar algorithm to ChatGPT’s to detect the predictably of each word in a sentence. The more predictable the words are, the greater the likelihood the text was writing by AI. For example, this human-written sentence has words that are, well, less predictable: The deliciousness of a Cosmic Crisp apple is to the fruit world what a fine Swiss chocolate is to the confectionary world.

Tian and his colleagues are working a new version of their software called GPTZeroX (Kidson, 2023). This version is made for educators and will include a plagiarism score, highlighted sentences that were likely generated by AI, and the ability to upload multiple files (say, from the same class) at once. While they don’t say it, I fully expect learning management system integration is coming. On the GPTZero website, click the “join the product waitlist” button and fill out their form.

Now, how long until we see the first case of academic dishonesty where a student used AI to generate their master’s thesis or doctoral dissertation? I predict it will be this calendar year.

Reference

Kidson, R. (2023, February 17). Princeton student creates ChatGPT detector. GHacks Technology News. https://www.ghacks.net/2023/02/17/princeton-student-develops-gptzero-software-to-detect-plagiarism-b...