CHAN Taizan
Office of the Senior Deputy President and Provost

Taizan shares three key findings regarding students’ experiences of going fully online for their courses, and their perceptions of how colleagues have fared in terms of managing the transition to e-learning while maintaining learning continuity. Taizan was one of the panellists of the session on “Student Experience” at the e-learning Symposium 2020.

The COVID-19 pandemic was the catalyst for NUS to quicken, very suddenly, its full adoption of e-learning in the middle of Semester 2, AY2019/2020. What were the student experiences and how well have our academics risen to the challenge? In this post, I share three key findings from the student evaluations conducted at the end of the semester.

Key Findings

Firstly, perceptions of modules in Semester 2, AY2019/2020 were on average more positive than those of the same modules in the past semesters. By comparing the module rating of the modules in this semester against the average rating of the same modules in up to the previous three semesters taught by the same academic team, it was found that the average module rating increased from 4.077 to 4.112. The difference of 0.034, while small, is statistically significant (p < 0.01), suggesting that the difference is not due to chance but to underlying change between this semester and previous semesters.

There are two observations from this comparison:

1. The reason the small difference is significant is because most of the modules tend to score in the narrow range to begin with (between 3.5 and 4.5).
2. While the difference is positive, it does not mean that all the modules scored better this semester. Figure 1 shows the distribution of modules by differences in their scores in this semester compared to previous semesters.

Secondly, while students seemed to experience more challenges than benefits due to the sudden transition to e-learning, their perception of NUS academics’ management and handling of the transition was generally more positive than negative. We used a combination of text analytics, semantic analytics, and manual coding to code and aggregate the issues and compliments from students’ comments into 53 sub-themes and two core themes. Of the 53 sub-themes, ranging from topics such as quality of the lecture, execution of the e-exam, to the handling of safety concerns, 11 (21%) of them pertain to the core theme of student experience regarding the challenges and benefits of e-learning, and the remaining 42 (79%) pertain to students’ perception concerning academic management of COVID-19 and e-learning. While 64% of the topics relating to student experience were negative, a larger proportion of the themes relating to students’ perception of academics handling of the pandemic was positive (60%) (see Figure 2).

 

Thirdly, students were generally understanding of adjustments that had to be made because of the pandemic. However, there were certain aspects of the transition that could have been better managed, in addition to the areas where our academics have managed well. Table 1 provides some interesting examples of the sub-themes identified.

Table 1

Themes categorised by sentiments and the extent to which they impact students’ module ratings

The themes, in addition to being categorised by whether they are negative or positive (along the vertical axis), were also categorised by the extent to which the theme appears to impact students’ rating of a module. The upper-left quadrant in Table 1 indicates the areas which students generally found to be “Negative–Less Acceptable” (i.e. associated with negative sentiments in the comments and low module ratings of 3 and below), while the upper-right quadrant indicates areas which students were more understanding and considered “Negative–More Acceptable” (i.e. associated with negative sentiments but still relatively high module ratings of 4 or 5). The lower-left quadrant includes areas which students found to be “Positive–Sufficiently Satisfied” (i.e. associated with positive sentiments but low module ratings) and the lower-right comprises areas considered “Positive–Beyond Satisfied“ (i.e. associated with positive sentiments and high module ratings).

Key Takeaway

A key takeaway from this is that the themes listed on the left side of Table 1 (“Negative–Less Acceptable” and “Positive–Sufficiently Satisfied”) appear to be basic aspects of e-learning and student welfare that students come to expect from the university and these are the areas that colleagues and staff at NUS have to manage well. The themes in the upper-right quadrant (“Negative–More Acceptable”) suggest that while there were changes that students did not like, they appeared to accept them. Finally, the quadrant “Positive–Beyond Satisfied” showed that many of our colleagues care about our students’ welfare—not just in their academic performance but also their well-being; and many have taken the opportunity to make their classes even better by appropriately adapting to e-learning and using technology effectively to make students’ learning experiences fun and engaging.

Conclusion

In conclusion, while some academic staff and students experienced difficulties with the sudden transition to complete e-learning, students were generally understanding of the challenges imposed by the transition. Our academics, however, have not only taken the opportunity to adopt new technologies and presented new ways to teach to enhance learning, they have also shown understanding and care for our students during the COVID-19 pandemic.

 

Taizan CHAN is the Director of Research and Education Analytics at the National University of Singapore (NUS). He leads a team of data scientists and analysts to provide reporting, analytics, and analytical solutions to senior leaders and other stakeholders at NUS across various domains, including research performance, HR, finance, university endowments, and admissions. He has published papers on technology education, computer science, business and technology, and consulted for several educational and private organizations, including University of Pennsylvania, Caltech, UC Berkeley, Autodesk, and Johnson & Johnson in software development and analytics. Taizan can be reached at pvotchan@nus.edu.sg.