Muzzammil Yassin
Centre for Language Studies (CLS), Faculty of Arts and Social Sciences (FASS)
Muzzammil Yassin (2024). Whisper AI: Enhancing feedback on oral assessments and facilitating research and analysis [Paper presentation]. In Higher Education Conference in Singapore (HECS) 2024, 3 December, National University of Singapore. https://blog.nus.edu.sg/hecs/hecs2024-m-yassin/
SUB-THEME
Opportunities from Generative AI
KEYWORDS
Whisper AI, feedback, speaking assessments, speech corpus, transcribing
CATEGORY
Paper Presentation
INTRODUCTION
Feedback in its various forms plays an integral role in learning a foreign language. In the Arabic Studies Programme at the Centre for Language Studies (CLS), oral assessments have traditionally involved either face-to-face interviews or presentations delivered by learners. The latter may be done “live” or submitted as a recording. Feedback provided on these assessments in the Arabic Studies Programme has usually been scarce, and when provided upon request by the student, is usually general and without much detail or correction. Furthermore, unlike a written assignment, the data from oral assignments might usually not be collected or stored. Thus, when students are provided with feedback, they do not usually get the opportunity to hear what they have said, or how they said it, or visualise it.
This presentation, based on action research, explores the potential of using Whisper from Open AI to enhance the feedback mechanism for oral assignments in language classrooms. The usage of AI discussed here falls within the detect-diagnose-act framework (mentioned in Molenaar, 2022). Over the period of AY 2023/24, Whisper AI was used to transcribe a relatively large amount of spoken data in a bid to provide enhanced feedback (Figures 1 and 2). This facilitated the analysis of students’ linguistic output in order to identify areas for improvement in pronunciation, grammatical structures, idiomaticity, and vocabulary choice. Research has demonstrated the usefulness of transcripts for students in the feedback process (Lynch, 2001, 2007). In addition to facilitating enhanced feedback, the transcription of the spoken data allows for it to be stored and compiled into a corpus. This will subsequently allow for reflection, error analysis, and data-driven design of activities. Thus, utilising Whisper on data collected from oral assignments can help facilitate research in the long run.
OUTCOMES
The outcome of this intervention has an impact on two aspects: feedback provided to the learner, and data compiled for further reflection and research.
Regarding the first, students are provided with a feedback table containing the transcript of their presentation or oral interview. This is usually done in the second half of the semester, after the oral assessment test has been conducted. Written comments are provided alongside the transcript. Students are presented with this feedback table during a consultation session, held face-to-face or over Zoom. The feedback contained within the table is discussed and suggestions are provided for students on how to improve their speaking proficiency. If required, excerpts from the recording can be played. This allows students to hear what they said and visualise it as well. Students take note of the errors made and seek further clarification if necessary. The written comments and the transcript are theirs to keep for future reference.
Student feedback from AY 2023/24 shows positive comments regarding the feedback provided throughout the semester, a large part of which included feedback on oral assessments which were enhanced with the help of Whisper AI (Figures 3 and 4). Such feedback also plays a role in further developing students’ speaking skills (Al Jahromi, 2020).
The other area which benefits from this intervention using Whisper AI relates to collecting and compiling spoken data. This would be extremely useful for reflection and research purposes, especially data that is collected on a longitudinal basis; from the beginner course—LAR1201 “Arabic 1” to the highest advanced course—LAR4202 “Arabic 6”. Such a corpus would allow teachers to document part of the development of learners’ speaking abilities. Such rich and varied transcribed data, along with the audio recordings, has the potential to contribute to a better understanding of areas related to language acquisition and a data-driven design of teaching material.
The following table provides a brief ‘before’ and ‘after’ overview of my teaching practice with regardS to providing feedback on oral assessments.
Table 1
Overview of my teaching practice and the quality of feedback given to oral assessments ‘before’ and ‘after’ the application of Whisper AI
In summary, WhisperAI has helped to fill a gap relating to providing more detailed feedback to students of the CLS Arabic Studies Programme for oral assessments. In addition to the benefit of helping students ‘notice’ and ‘visualise’ (Lynch, 2001) their speech, the data from such interventions can be used to create activities that aim towards correcting common errors in learners’ speech/written production. Technology is a force multiplier which enhances the learning experience when used appropriately. The augmentative approach to using AI demonstrated above seeks to benefit both the learner and the teacher. This is part of what the detect-diagnose-act framework advocates (Molenaar, 2022).
REFERENCES
Al Jahromi, D. (2020). Can teacher and peer formative feedback enhance L2 university students’ oral presentation skills? In Hidri, S. (eds) Changing Language Assessment. Palgrave Macmillan. https://doi.org/10.1007/978-3-030-42269-1_5
Lynch, T. (2001). Seeing what they meant: transcribing as a route to noticing. ELT Journal, 55(2), 124–132. https://doi.org/10.1093/elt/55.2.124
Lynch, T. (2007). Learning from the transcripts of an oral communication task. ELT Journal, 61(4), 311–320. https://doi.org/10.1093/elt/ccm050
Molenaar, I. (2022). Towards hybrid human-AI learning technologies. European Journal of Education, 57, 632–645. https://doi.org/10.1111/ejed.12527
Stillwell, C., Curabba, B., Alexander, K., Kidd, A., Kim, E., Stone, P., & Wyle, C. (2010). Students transcribing tasks: noticing fluency, accuracy, and complexity, ELT Journal, 64(4), 445–455. https://doi.org/10.1093/elt/ccp081