GoKawiilFacepalm: The endemic scale of AI-assisted cheating among students over the past few years is well documented, but few visual representations illustrate the problem as well as a recent chart from a Brown University professor. As concerning as the graph is, the university's muted response to it is potentially even more worrying.
When Brown University economics professor Robert Serrano suspected that most of his students had used AI to cheat on a take-home midterm exam, he decided to compare their scores with those from an in-person final. The result was perhaps one of the clearest examples of generative AI's effect on academia.
The reason Serrano allowed the take-home midterms is both reasonable and a sobering sign of the world American students currently inhabit: Some students were uncomfortable taking exams in class after a gunman killed two students and injured nine others at the university last December.
However, Serrano soon noticed signs that something was amiss. When 86 students signed up for his welfare economics course that semester, compared to the usual 30, he suspected that many had taken it because he allowed take-home exams, providing them the opportunity to use ChatGPT or similar AI tools.
The professor's suspicions were just about confirmed when the midterm scores averaged 96%. This was far above the typical range of 65% to 80% despite Serrano's efforts to make the test more difficult to account for the students' unlimited time.
While many professors have attempted to counter AI cheating with AI-based tools, Serrano understood AI-proofing often produces false positives and negatives. Students at other universities have sued their schools after being falsely accused of using AI, and a study from last year indicated that the practice simply trains students to write against AI detectors, often by using AI.
Instead, Serrano employed a more straightforward solution – he ran his midterm through ChatGPT. Unsurprisingly, the chatbot's answers closely resembled the ones his students submitted, employing similar language and reasoning.
When the professor announced that the final exam would be in person, 18 students dropped the course, and another nine did not show up for the exam. The remaining 59 students' scores averaged 48.6%. Prior averages had never fallen below 65%. Three students scored 0%, and charting the results revealed that most fell more than 30 points behind their midterm scores.
The chart suggests that only two students probably took the tests without AI, one of whom was likely the top scorer with 95.5% on the midterm and 95% on the final. Interestingly, the other student who appears innocent scored just 55% on the midterm and 59% on the final. They posted the lowest midterm score and were the only student whose score improved on the final.
Serrano subsequently dropped the midterm scores and made the final count for 80% of the students' final grade. Normally, he would have passed everyone who scored 50% or higher, but in this case, the professor lowered the threshold to 40%.
... continue reading