New on the Arxiv.org open repository of academic papers, “Simulating H.P. Lovecraft horror literature with the ChatGPT large language model”. Using the current…

GPT-4 architecture [and] advanced prompt engineering methods. [texts were generated and undergraduates asked to “distinguish between genuine Lovecraft works and those generated by our model. … the participants were unable to reliably differentiate between the two

The sample was 301 students. But I’m always sceptical of studies involving a group of adolescents many of whom are likely to be sleep-deprived, malnourished, hungover or perhaps all at the same time. One would have hoped that, due to the ongoing questioning of Psychology as a respectable discipline, such student survey-experiments would have been consigned to the dustbin of history.

Admittedly, these students appear (it’s not stated) to be from the Universidad Pontificia Comillas, a private Catholic university in Madrid said to be run on Jesuit lines. So, they were perhaps not as hungover as in other universities.

But the authors of the paper might at least have addressed one other obvious concern. That these religious students, on being openly told that the comparison text was from Lovecraft, might have had some animosity toward him. Due to things they recalled about the author (an ‘occultist’ etc). Putting 2 + 2 together, many might have realised that this was a Lovecraft vs. ChatGPT test, and thus been inclined to deliberately find the ChatGPT extracts indistinguishable from the “140-word excerpt from “The Call of Cthulhu”” (which was the comparison text).