by Anita Toh and Tetyana Smotrova
My colleagues, Misty Cook, Tetyana Smotrova and I created an eLearning module called “What’s your point? Strategies for Clarity and Conciseness” to help students work on common writing errors. These errors were observed sporadically in different students’ work but were difficult to address in a whole-class context. The module consisted of 28 videos ranging from 3 to 30 minutes in length, covering topics such as Improving Clarity with Punctuation, Common Errors with Linking Words, and Avoiding Excessive Nominalisation. The videos were designed in three formats:
- animation with narration (using Powtoon)
- animation without narration (using Powtoon)
- screencast (using Camtasia Studio)
We then conducted a study to examine students’ perception of these different types of instructional video formats. 22 students from the School of Computing were recruited for the focus group study. They were asked to view 2 videos of each format, and complete a pre-viewing quiz, a post-viewing quiz, and a delayed (after one month) viewing quiz. They were also asked to complete an online survey and to participate in a small-group interview.
The findings somewhat contradicted what we had expected and revealed important considerations for future instructional video design:
- How did the students watch the videos?
Research shows that the “net generation” prefers mobile learning to “desk-bound” eLearning (Walinski, 2014; Hutchison, Tin, & Cao, 2008). In addition, many researchers have noted that there is currently a preference for mobile learning and bite-sized learning on-the-go (Kukulska-Hulme & Traxler, 2007). Given these trends, we expected that most of the students would have watched the videos on their mobile phones or tablets while on-the-go, such as during their daily commute. Additionally, the eLearning module we had created was not a credit-bearing module, and watching the videos had no implications on the participants’ university grades. Therefore, we expected that the students would not watch the videos too “seriously”.
Surprisingly, only 1 of the 22 participants watched the videos on a mobile phone while commuting to and from university. The other students watched the videos on their laptops either in study areas around campus or at their study desks at home. Many said that they wanted to learn something useful from the videos, and “it’s hard to take notes seriously when watching in the trains and buses.”
- Which format was most engaging?
Research shows that 85% of all Facebook videos are consumed without sound (Patel, 2016), and most Facebook videos tend to be 2-5 minutes long, featuring images and text with optional background music. Therefore, we expected that the animation without narration format would be most appealing to our net generation participants.
The participants indeed rated the animation without sound videos as the most engaging. Their reasons were that the videos were short (<5 min), the format was new to them and so had a novelty factor (they were used to webcasts), the videos had click-baity titles, the content was new to them, and therefore informative.
Almost half of the participants (9) noted that animation and music are nice features to have, but not necessary.
- Which format was least engaging?
According to Guo, Kim, and Ruby (2014), eLearning videos should be less than 6 minutes in length, show slides interspersed with presenter’s talking head to add a personal touch to the videos, and feature instructors that speak quickly and enthusiastically. Therefore, we expected that the participants would deem the 30-minute screencast as the least engaging.
Indeed, they rated it as the least engaging video and gave such reasons as: it was too long, the audio quality was inconsistent, and the speaker was unexciting. Nevertheless, they acknowledged that the video was informative and offered valuable content.
Surprisingly, more than half of the participants (11) revealed that their preferred format is the screencast because they are used to this format. They just didn’t like the ones we made primarily because of the inconsistent audio quality and unexciting speaker, and to some of them, the content was not new to them.
- Which format was most effective for learning?
Cognitive Load Theory states that information in visual and auditory modalities (text/image plus narration in screencasts and animated explainer videos) can enhance learning and retention (Sweller, 1988). In addition, Cognitive Theory of Multimedia Learning (CTML) states that information via animation and narration simultaneously can improve performance on retention tests (compared to information presented via one medium) (Mayer, 2001). Therefore, we expected that the screencast and the animation with narration formats would be most effective.
Contrary to our expectation, the non-parametric Friedman test did not reveal any statistically significant differences in learning outcomes. There could have been many reasons, for example, this was a non-compulsory module, so there was not much incentive for them to retain the information. This needs further investigation.
- An interesting finding was that the participants had different expectations for videos in credit-bearing modules and non-credit bearing modules.
- The participants admitted that they were more likely to watch the videos seriously if those were part of a credit-bearing eLearning module, or if the students knew they had to complete a quiz after watching the videos.
- They expect videos for credit-bearing modules to be more serious in nature, and they are more likely to watch such videos in “proper” study places instead of on-the-go. In contrast, they expect videos for non-credit bearing modules to be more fun and suitable for watching on-the-go.
- They noted that longer videos are acceptable in credit-bearing modules as long as the content is perceived to be relevant, but videos under 10 mins are preferable. In non-credit bearing modules, they expect the videos to be no longer than 5 mins.
Based on the findings above, we generated a number of recommendations for instructional video design.
- Content needs to be clearly relevant, new, and informative to the students.
- Videos should preferably be no longer than 5 minutes.
- Each video should cover only one topic.
- There should be clear learning outcomes at the start of each video and a summary of key learning points at the end of each video.
- Pre- and post-learning quizzes with specific feedback are necessary to help students assess their learning.
- In-video quizzes are necessary because they are highly engaging and help students check understanding as they view.
- Students sometimes watch the videos at 1.5 to 2 times the normal speed, so subtitles and transcripts are helpful.
- Presenter’s voice needs to be interesting and engaging.
- Video and audio quality need to be consistent and clear.
- There should be an option to watch the videos online or to download them for later viewing.
- Screencasts are ok to be used for instruction. Format, animation, music, presenter’s talking head are not important. What is more important is the relevance and newness of the content.
- If the iMOOC includes self-access independent learning videos, such as those covering grammar, writing or speaking skills, the students do not have to take the entire module. It may be preferable to make a collection of resources available and have the students take a diagnostic quiz with instructions to proceed to view the relevant videos.
In conclusion, what is most important when designing eLearning content is the relevance of the content to the students’ needs, and to clearly communicate this relevance to the students at the start of the module. Poopola’s (2012) assertion sums up the lessons learned from this study aptly: Beware of the “tendency to make eLearning appealing or even entertaining, at the expense of effective learning”. The novelty of various eLearning features will fade over time. The emphasis of eLearning is not on its appeal, but on its effectiveness for learning (Mishra & Koehler, 2006).
References
Guo, P. J., Kim, J., & Rubin, R. (2014, March). How video production affects student engagement: An empirical study of MOOC videos. In Proceedings of the first ACM conference on Learning@scale conference (pp. 41-50).
Hutchinson, M., Tin, T., & Cao, Y. (2008). “In-your-pocket” and “on-the-fly:” Meeting the needs of today’s new generation of online learners with mobile learning technology. In T. Anderson (Ed.), The theory and practice of online learning (2nd ed., pp. 201-220). Edmonton, AB: AU Press.
Kukulska-Hulme, A., & Traxler, J. (2007). Learning design with mobile and wireless technologies. In H. Beetham, & R. Sharpe (Eds.), Rethinking pedagogy for the digital age: Designing and delivering e-learning (pp. 180–192). London, UK: Routledge
Mayer, R. E. (2001). Multimedia learning. New York: Cambridge University Press.
Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers college record, 108(6), 1017-1054.
Patel, S. (2016). 85 percent of Facebook video is watched without sound. Digiday. Retrieved from: https://digiday.com/media/silent-world-facebook-video/
Popoola, O. (2012). E-learning and motivation: a multi-faceted investigation of eleven to fourteen year olds’ attraction to computer-based learning, and their motivational responses to the novelty and nature of a selection of self-study computer-based learning activities. Doctoral dissertation, University of East Anglia.
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257–285.
Waliński, J. T. (2014). Implementing Linguistic Landscape investigations with M-learning for Intercultural Competence Development. International Journal of Mobile and Blended Learning (IJMBL), 6(2), 15-25.