British University Conducts Real-World Turing Test to Evaluate AI
University of Reading’s Study Reveals AI’s Ability to Outperform Human Students in Academic Assessments
A British university has turned its microscope on itself, conducting a real-world Turing test.
Alan Turing, the renowned mathematician and computer scientist, proposed the Turing test in 1950. He suggested that a machine capable of deceiving humans into believing it was human should be considered at least as intelligent as a human. Turing predicted that by the year 2000, artificial intelligence would be able to deceive 30% of human interrogators after five minutes of conversation.
In a recent study by the University of Reading, published in the journal "PLOS One," this concept was put to the test.
The university's technology team created 33 fake student profiles. These "students" were enrolled in Bachelor of Science programs in psychology at various stages of their studies. The researchers assisted these fake students in completing a series of online assignments and exams using unedited responses generated by OpenAI's GPT-4 chatbot.
This study, considered the largest of its kind, revealed that 94% of the AI-generated submissions went unnoticed by examiners. Furthermore, the AI students performed better than their human counterparts. Their grades were, on average, about half a grade higher than the real students', with an 83.4% chance of outperforming randomly selected real student submissions.
Professor Peter Scarfe, one of the study's authors, stated that their findings should be a "wake-up call" for educators. A recent UNESCO survey found that less than 10% of schools worldwide had policies or guidelines for the use of generative AI, which has become prominent in the technology and AI industries, overshadowing the machine learning AI sector that dominated before the emergence of ChatGPT and its competitors.
"Many institutions have moved away from traditional exams to make assessments more inclusive. Our research highlights the global importance of understanding how AI will impact the integrity of educational assessments. We may not necessarily revert entirely to handwritten exams, but the global education sector must evolve in response to artificial intelligence," Scarfe noted.