Can You Distinguish Between AI and Human-Generated Text?

[andrea_lunsford]

Writing in Counter Arts and recently reprinted in Medium, journalist and digital media expert Jeff Hayward stumbled on an AI “test” offered by the Globe and Mail. Fourteen questions in all, asking a test taker to distinguish between images, voice/video clips, and texts created by humans and generative AI. Hayward was relieved, and just a little surprised, that he got all the questions in the first two sets correct: the AI generated images were somehow “just too perfect,” he said, and the voice/video cliffs felt somehow “off” to him—and his hunches paid off. 

What really did surprise him, however, was the fact that he got all three of the questions about written passages wrong, thinking that the AI-generated texts sounded “more human” than those generated by humans. So an experienced working journalist, who spends time every day writing and studying writing by others, was taken in by AI. Yikes!

The Globe and Mail quiz is behind a paywall, but I was able to take a look at it without being able to take it. My guess is that I would have been very lucky to get a third of the answers right—and be relatively unable to distinguish material generated by humans from those generated by generative AI. Double yikes!

I’m wondering how our students fare at such tasks. And while the Globe and Mail quiz is available only to subscribers, here are a couple of other similar online quizzes that are accessible, the first from Buzzfeed and the second from the Statistics and Tech Data Library. You might want to try then out with your students. I, meanwhile, am trying to learn to get better at this kind of detection!