ChatGPT as a Tool for Serious Academic Enquiry: The Undergraduate Student Response

Steven James GREEN
Dept of English, Linguistics, and Theatre Studies,
Faculty of Arts and Social Sciences (FASS)

Steven discusses his experience of applying AI in his film course, specifically getting his students to critically evaluate ChatGPT’s responses to course-related questions. He summarises their views, where they list the strengths and weaknesses of these AI-generated responses and its viability as a tool for serious academic study.

drobotdream-freepik — Photo attributed to drobotdean on Freepik.

Green, S. J. (2025, January 23). ChatGPT as a tool for serious academic enquiry: The undergraduate student response. CTLT Teaching Connections. https://blog.nus.edu.sg/teachingconnections/2025/01/23/2025-sj-green/

In terms of applying artificial intelligence (AI) in higher education teaching and learning, colleagues teaching within NUS have already received several forms of official guidance regarding the use of ChatGPT and other AI resources in student research (Centre for Teaching, Learning, and Technology, n.d.). In addition, some of us have signed up and tried out ChatGPT for ourselves, and I have previously shared with students my findings regarding its ability to critically analyse ancient Roman poetry (in class, as part of my course, EN4269 “Ovid the Innovator”).

Finally, and in the spirit of the popular strategy of securing ‘360-degree feedback’, I recently asked the upper-level students in my film course EN4252HM “The Ancient Greek and Roman Worlds in Cinema” for their own views, in the form of an assignment. A summary of their views is presented below (see the “Acknowledgements” for the list of student participants). It is hoped that this piece will complement existing pedagogical resources, and that it will be of particular interest to students to hear the reflections of their peers, as we all continue to navigate the fast-moving landscape of AI and consider its meaningful integration into our own scholarly practices.

For the final assignment of the course, I entered into ChatGPT two cognate questions related to one of our set films, and asked students to assess the strengths and weaknesses of the AI responses as a tool for serious academic study. For example, as a way into investigating the issue of the political leanings of Kubrick’s 1960 film, Spartacus (Kubrick & Mann, 1960), I entered the following questions into ChatGPT: ‘How Marxist is Spartacus (1960)?’ and ‘What are the Marxist elements in Spartacus (1960)?’

Students noted several positive features of the AI-generated responses: clear structure; focused, digestible information delivered in uncluttered paragraphs and/ or via bullet-points; a surface attempt at balance. Some lauded ChatGPT’s role in ‘democratising’ knowledge by distilling at speed, from its vast databases, useful information targeted towards a lay audience. As for the academic user, the responses provide a springboard for further exploration and, more subtly, can indicate a general consensus against which one can measure the level of innovation in one’s own ideas. What’s more, a few students felt that, by limiting the enquiry to just two cognate questions, I may not have maximised the full potential of ChatGPT. Indeed, some advocated for ‘prompt engineering’, whereby one engages in a process of ever more targeted prompts in an effort to reach the desired level of detail, rigour, and academic tone.

As the name suggests, Chat GPT sees itself primarily as part of a casual conversation with its user, a role that naturally affects its style, register, and level of detail. For serious academic enquiry, then, students consistently noted a number of drawbacks: brevity of response; lack of detail and specific examples to back up general points; and inability to cite and engage properly with primary sources and secondary scholarship¹. There was also an occasional tendency towards subjective language that calls for qualification (such as referring to a character as ‘likeable’). This combines with ChatGPT’s own acknowledgment of its capacity for error, which should make one wary of accepting wholesale even its more general points.

Students detected a particular weakness within ChatGPT when it came to answering questions from the humanities, which often call for a considered opinion on a non-binary issue, based on weighing up different perspectives. ChatGPT’s general quest for ‘the answer’ misses the fundamental (humanities-) disciplinary desire for open-ended enquiry, the sort of serendipitous risk-taking that is, at the moment at least, confined to the domain of human imagination.

Finally, students noted some particular deficiencies of ChatGPT in enquiries about film. First, its analysis tends to be restricted to considerations of the narrative plot, with little attention paid to the socio-political context of the film’s production, or to cinematic features that enhance storyline, such as camera shots, mises-en-scène, and music/sound. Secondly, and especially for a film such as Spartacus (Kubrick & Mann, 1960), which revolves around the issue of Marxism, ChatGPT seems to be programmed to eschew or remain coy about topics deemed to be socially or politically controversial, or else it will approach such issues either in a neutral fashion or with a bias that betrays the largely western, US-centric databases from which it draws its information.

To end, I offer a personal reflection on the exercise as instructor. When I read the even-handed but largely non-committal ChatGPT responses to the prompts I had set, I found myself commenting in the margins, ‘Did you actually watch this film?’. The answer, of course, is no, and that is the key: what is missing is human interaction and the personal choices, preferences, judgments, and intellectual risk-taking that comes with it. My reflection on this student AI assignment strengthens my view that the better written assignment prompts are those that require a relational assessment (assess X alongside Y) and/or interaction with a particular scholarly viewpoint (assess the merits of viewpoint X in the study of Y). Such prompts help to foreground essentially human qualities.

Acknowledgements

The author would like to acknowledge the students who contributed their views for this learning activity. They are:

Claire LEE, Darcel ANTHONY, Dawn LOH, Elijah WOO, Hanee Rasdeen, Jasmine CHEW, Jefi JOHN, Joshua WOO, Nicole WANG, Nicolette KUM, Sean HOH, See Chung HO, Sheryl POON, Stephanie LEONG, Timothy WAN, Yu Kai GOH, Yueqi VOON, Zachary LIM, Zoe CHIOW.

Exploratory Endnote

Some of these deficiencies might be improved by the students inputting more specific prompts, such as asking ChatGPT to take on the role of a university professor, or asking for in-depth analysis. Even so, the effect may be limited: when I qualified the prompts myself, asking for specific engagement with primary and secondary scholarship, ChatGPT offered a few random quotes that were not integrated meaningfully into the discussion.

References

Centre for Teaching, Learning, and Technology (CTLT) (n. d.). NUS AI Community-of-Practice. https://ctlt.nus.edu.sg/ai-community-of-practice/

Kubrick, S., & Mann, T. (Directors). (1960). Spartacus [Film]. Universal International.

Steven James GREEN is an Associate Professor at the Dept of English, Linguistics, and Theatre Studies in the Faculty of Arts & Social Sciences. He has taught a range of classical Greek and Roman courses at Universities in England, Scotland, and the Republic of Ireland, and his most substantive post was as Senior Lecturer at the University of Leeds from 2004-2013 and Head of Department from 2010-2013.

Steven specialises in Roman literature and culture in the late republic and early empire (first centuries BC and AD) and is particularly interested in those texts that are typically overlooked, unread, or unappreciated by modern readers. For this reason, his research has moved from the conventional world of Ovid to more marginal poems, such as the astrological treatise of Manilius, the hunting manual of Grattius, and now a Latin poetic version of Homer’s Iliad, on which he has completed a detailed commentary, with text and translation (Oxford University Press, 2025 in press). His research has been recognised by the Hannah and Joseph Lees Fellowship (Manchester, UK), the Margo Tytus Fellowship (Cincinnati, US), the Tan Chin Tuan Chinese Culture and Civilisation Programme (Singapore) and most recently, the NUS Humanities and Social Sciences Faculty Research Scholarship.

Steven can be reached at steven.green@nus.edu.sg.