Jodie LUU Tran Huynh Loan and Happy GOH
Centre for English Language Communication (CELC)

Jodie and Happy discuss the strategies and processes they implemented in order to prepare students for a large-scale digital examination.

As digital assessments become part and parcel of the new normal brought on by the coronavirus COVID-19 pandemic, the question of how to conduct large-scale examinations effectively needs to be addressed.

While the Centre for English Language Communication (CELC) has had prior experience with digital assessment, having adopted the digital assessment software Examplify for its Qualifying English Test (QET) for the first time in July 2019, it is only until the upcoming QET in July 2020 that remote proctoring via Zoom will be introduced to the examination process.

In this blog post, we share our experience in trialling an e-examination conducted under remote proctoring environment for the final examination for 92 Senior Middle School 2 (SM2) students doing a one-year bridging course at NUS. This is in preparation for the upcoming QET, which usually has more than 2,000 candidates. 

We share three key lessons learned:

1. Have an approach that guides the process

We adopt an iterative approach which consists of briefings, trial runs, reflections, and enhancements (Figure 1).

 

2. Break and stagger the main trial run

Given that we were to deploy Examplify for completion of the test paper and Zoom for invigilation, the team decided to stagger the trial run into two sub-trial runs (Figure 2). The first sub-trial run focused on only familiarising students with the Zoom invigilation process, and the second was on both Zoom and Examplify.

We found that in addition to easing the students into the assessment process, the staggering approach also allowed the assessment team enough turnaround time to gather feedback from tutors and students, reflect and make necessary enhancements.

3. Anticipate and plan for scale-up

The SM2 English final exam was successfully conducted with minor technical issues reported, and zero cases of academic dishonesty documented based on Turnitin plagiarism check results. Nevertheless, our reflections on the trial runs indicate enhancements must be made for the QET in July 2020 due to the larger number of test takers (up to 2,000 freshmen) who might not have prior experience with digital assessment.

With an estimate of 300 students per QET briefing session, the usual one-way dissemination of information may not be effective to engage the participants. In order to enhance student engagement, we intend to use the flipped classroom approach by requiring students to access asynchronous materials prior to the synchronous sessions. This would allow for flexibility and mobility of learning (Nouri, 2016), ease of access to online resources (Kirkwood & Price, 2013), ability to manage cognitive overload (Abeysekera & Dawson, 2015), and active learning (McLaughlin et al., 2014; Roehl, Reddy, & Shannon, 2013).

In this light, before attending a briefing session, students must first read two sets of briefing documents on Examplify and Zoom invigilation protocol, and then complete a quiz that consolidates the key points they have read from these documents. The synchronous virtual briefing session will focus more on hands-on practice by simulating the full Zoom invigilation procedures, specifically on mobile. After walking students through the Zoom invigilation process (Figure 3), we will allow them to practice a sample QET paper using Examplify on their laptop/desktop. We hope that the active learning strategies adopted for the briefing sessions will help students fully understand the digital examination process and know what to do on the test day itself.

All in all, while it is almost impossible to anticipate all scenarios that can happen during a virtual exam, we believe that we could become more agile and adaptive by applying the iterative process of briefing, trial runs, reflections, and enhancements to get ready for any new normal yet to come.

 

Jodie LUU is an instructor at the Centre for English Language Communication. She is passionate about harnessing the power of digital technologies to create an inclusive, interactive and learner-centred classroom. Having taught a flipped module and co-developed online courses for both the NUS community and global learners (edX), she is also interested in research on pedagogy for MOOCs and blended learning. Jodie can be reached at jodieluu@nus.edu.sg.

Happy GOH is a senior lecturer with the Centre for English Language Communication. She has taught various communication courses designed for different types of learners and purposes, including critical thinking and writing for community and engineering leadership, and oral skills. Her interests include assessment and blended learning. Happy can be reached at happy@nus.edu.sg.

References

Abeysekera, L., & Dawson, P. (2015). Motivation and cognitive load in the flipped classroom: Definition, rationale and a call for research. Higher Education Research & Development, 34, 1–14. https://doi.org/10.1080/07294360.2014.934336

Kirkwood, A., & Price, L. (2013). Examining some assumptions and limitations of research on the effects of emerging technologies for teaching and learning in higher education. British Journal of Educational Technology, 44, 536–543. https://doi.org/10.1111/bjet.12049

McLaughlin, J. E., Roth, M. T., Glatt, D. M., Gharkholonarehe, N., Davidson, C. A., Griffin, L. M., & Mumper, R. J. (2014). The flipped classroom: a course redesign to foster learning and engagement in a health professions school. Academic Medicine, 89, 236–243. https://doi.org/10.1097/acm.0000000000000086

Nouri, J. (2016). The flipped classroom: For active, effective and increased learning – Especially for low achievers. International Journal of Educational Technology in Higher Education, 13, 1–10. https://doi.org/10.1186/s41239-016-0032-z

Roehl, A., Reddy, S. L., Shannon, G. J. (2013). The flipped classroom: An opportunity to engage millennial students through active learning strategies. Journal of Family and Consumer Sciences, 105(2), 44-49. Retrieved from https://www.learntechlib.org/p/154467/.