Technology in Pedagogy, No. 13, November 2012
Written by Kiruthika Ragupathi

Gamification has recently been proposed as a means to improve student engagement in the classroom. Gamification is the application of game elements or game mechanics to motivate and engage an audience, in our case, to engage students in the learning process.  With this new approach, students are awarded experience points when they submit assignments and participate in various class activities. When they earn sufficient experience points, they “level-up” over the course of a semester.

Effective teaching has to go beyond the transmission of information and knowledge to the students. As instructors, we should attempt to inspire and motivate students to want to learn and to develop the right attitudes towards learning says Dr Ben Leong, an Assistant Professor of Computer Science at the National University of Singapore. To do this effectively, he believes that there is a need to “connect” with the students and modern technologies now offers greater opportunities to do much more.

How it started

Jane McGonigal, a game designer, gave a TED Talk in Mar 2010 entitled Gaming can make a better world (McGonigal, 2010). Jane highlighted that people around the world invest some 3 billion hours weekly playing online games and she claimed that it would possible to change the world if this time and energy could be channeled into meaningful activities. To prove her point, she designed and built a game, Urgent Evoke (http://www.urgentevoke.com), that encouraged players around the world to work on social projects.  Dr Leong was quite skeptical when he first watched the talk, but then wondered if indeed gamification could really be exploited in some way to do good.

Later that year, Dr Leong was slated to teach an introductory programming methodology to first year undergraduates at the School of Computing. He had taught the class four times before and was wondering what he could do differently to improve the teaching of the class. He observed that many students have very poor time management skills and there was a tendency for them to start on their assignments very close to the deadline. In previous years, students were typically expected to complete seven problem sets over the course of a semester, or an average of one problem set every two weeks. These problem sets were relatively large assignments that can take up to 20 hours to complete.

Because the problem sets were relatively large, some students often failed to complete their assignments on time. There were some students who would make a submission “just on time”. In most cases, students who finished their homework in a rush would typically not have learnt optimally and neither would they have absorbed the material well. In fact, upon checking past records, it was found that on an average, students generally submitted their solutions about 15.5 hours before the deadline.

The game approach

It seemed natural to try to improve the situation by breaking the problem sets into small “chunks”, but there was a concern that even though the total amount of homework would remain the same, such an approach would be perceived to be overly onerous by the students. This was when the idea of applying gamification came up as a plausible solution to make it seem more natural to have smaller assignments and also to try to improve the motivation of the students to finish their assignments in a timely manner.

Dr Leong gathered a team of undergraduate tutors and artists and developed a sci-fi adventure storyline set in a Star-wars-like universe for the course. A series of comics was used to narrate the story. The seven problem sets in the course were broken up into 22 smaller assignments framed around the storyline and were termed as missions. An online learning platform called the JFDI Academy (JFDI Academy, 2010) was developed to support the narration of the story (via the comics) and the dissemination of the missions.

 The following are some of the key features of the gamification platform:

• Experience points and levels: Students were awarded experience points (EXPs) for undertaking various tasks as they proceeded through the course. They gained EXPs from completing missions depending on how correct their submissions are, and also for participation in tutorials and discussion forum. With sufficient accumulation of EXPs, students would to level-up. A student’s level determined the marks obtained for the continuation assessment portion of the final course grade.
• Side Quests: These are optional assignments that allow students to take up for further practice or for regaining some of the EXP lost in the missions.
• Leaderboards:  The leaderboard is a list where students are ranked according to the number of experience points that they earned.
• Achievements: Students are awarded with unlocked achievements and badges when they complete certain tasks.
• Facebook Integration: Feeds are posted on Facebook when students level up and when they unlock achievements.
• Facebook-feed-like Discussion Threads: Students can seek clarification on their missions and discuss with their tutors by posting comments within the platform. Tutors will receive email notification to allow them to quickly respond to the students’ posts and comments in a timely manner.
• Auto-Graded Exercises: Students are supposed to complete the auto-graded exercises within one day after a lecture. The purpose of these exercises is to ensure that the students understand the basic concepts covered. The platform supports both MCQ and coding questions. 

The use of game mechanics was effective in motivating students to complete and submit their assignments earlier. This is evident from the fact that with the introduction of the new game platform, the average time that the students submitted their assignments increased from 15.5 hours before the deadline to 51.2 hours before the deadline. This new system also drastically reduced the feedback turnaround time with each of the student’s mission attempts being graded typically within 24 hours of submission.  

How to do it right

Dr Leong highlighted that the key to gamification is an understanding of the psychology of the students. Only with a firm understanding of the students’ psychology, will we be able to apply the appropriate game mechanics to effectively improve motivation. There is also an underlying goal of making the process of learning fun. The following are some of the key ideas in the application of game mechanics in the gamification of a course:

1. Instantaneous gratification: One key element of games is that when players accomplish tasks, they are immediately rewarded. In the new gamification platform, the students are rewarded with EXP for submitting their assignments. In addition, because the platform is linked to their Facebook, the achievements are simultaneously posted on their Facebook status feeds.  
2. Sense of progress: Games achieve engagement by providing players with a sense of progress. By building clear progress markers along the way, i.e. by allowing students to level up, the gamification platform provides students with a sense of achievement.  Constant and timely feedback from faculty on how the students are progressing gives students a sense of affirmation that they are going in the right direction.
3. Small Intermediate Goals: By breaking up the original large problem sets into smaller and many more missions, it again helps the students to focus on smaller intermediate goals.
4. Rewards: In addition to EXP, the gamification platform also supports the unlocking of achievements and badges. Most of these achievements are merely vanity constructs, but a small number are actually useful. For example, there is a “+1 Day” badge that can be expended by a student to get a one day extension on an assignment.
5. Competition. The top 15 students will be shown on the course leaderboard. The remaining students are not ranked to avoid embarrassment for students at the bottom. As a result, the top 30 students will typically compete to be on the leaderboard.  On the other hand, the weaker students are also encouraged to work harder because they will know that they are falling behind since it is evident how far the best students in the class have done.

Potential pitfalls and challenges

It is clear that gamification if employed in the right manner is effective in improving student engagement. However, there are certain challenges and pitfalls as follows:

1. Game balancing: Because EXP is awarded for a large number of activities; there is a need to decide on how much EXP to award for each activity in order to ensure that students exhibit the right behaviour. If the EXP awarded for some component is too much, it might cause excessive stress and if too little, the students might just neglect it. It is not often easy to determine the sweet spot. Game companies spend a lot of effort balancing the rewards in real games.
2. Significant cost: To execute the gamification of a course well involves significant amounts of resources. The building of a gamification platform is expensive because it involves a lot of development manpower. That said, in Dr Leong’s experience, some 70% of the manpower was spent not on the building of the platform, but in the development of the lesson material. In other words, to break the original 7 problem sets into smaller missions, there was a significant amount of editing and development of some new assignments.  
3. Novelty factor: It is also clear that some of the interest and excitement generated by the course was due to a novelty factor. If all classes were structured as games, then it might become tiresome for the students.

Dr Leong indicated that his greatest concern about gamification, is that if not done correctly, there is a danger that it might in fact cause students to be focused on leveling up instead of on the learning process. Also, a key element in effective learning is the development of intrinsic motivation in the students. We want students to learn because they want to learn. The application of gamification is a form of external motivation that makes students do what we want them to do because we are to some extent, exploiting their psychology. In this light, it might be detrimental to the goals of developing intrinsic motivation. As with all things, there is always the risk of “too much of a good thing” and Dr Leong believes that educators who sincerely care about the learning of their students and who are mindful of the pitfalls should be able to avoid them.

It is also helpful to highlight that a key idea in the pioneered gamification approach is to move away from a fixed-weightage continual assessment system, where different components are worth fixed amounts, to a more amorphous everything-feeds-into-EXP assessment system, where only a final level matters. This new system offers significantly more flexibility and is something that our modern students can easily identify with since many of them play games and games are structured in a similar way. This new approach to continual assessment is broadly applicable to all classes even if game mechanics and elements were not incorporated.  For example, this has already been tested in CS2010 and was found to be effective.

What can technology do for education

Dr Leong explained that his work could be construed not only as an experiment in applying gamification to in classroom teaching, but as a “clean slate” experiment to explore how technology can be applied to improve teaching. It turns out that he had opportunity to build a new learning platform from scratch and the following are some of his achievements.  

1. Enhance student engagement:  It was clear that the application of game mechanics was effective in improving student engagement and the learning experience. The new system was well-received and there was a significant improvement in the module feedback rating compared to past years.
   
2. Allow students to learn at their own pace: By structuring assignments in the form of missions, side quests and contests, we can ensure that all students cover a certain basic amount of material and yet allow some students to do more if they wish without making the process stressful for those who do not. Side quests and contests offer some EXP, but only a small amount compared to the main missions and they integrate quite seamlessly.
   
3. Significantly reduced grading/feedback turnaround time: A key innovation that was achieved was the significant reduction in grading turnaround time to within 2 to 13 hours. This was possible because the system will automatically generate email notifications to the tutors whenever students submitted their assignments. The tutors were asked to try to grade assignments within 24 hours, but all of them were able to do much better than that.
4. Improve student teacher interaction: Each homework assignment had an associated Facebook-style conversational feed that allowed students to discuss the grading/assignments with the tutor. Also, the system supported line-by-line annotations so that tutors can pinpoint the exact mistakes that were made and discuss them with students in a separate conversational feed. 
5. Effective & efficient student monitoring: The automatically graded exercise system (Paths) provides the teaching staff with a means to quickly identify students who are struggling with the course material and also see where the students get stuck. The EXP overview screen also allows the teaching staff to identify struggling students within the first 3 to 4 weeks of the semester. Previously, such students could only be identified after the midterm exams in Week 7.
6. Quality assurance for teaching staff. The platform has many built in metrics to measure and enforce performance standards on the tutors who grade the assignments. The lecturer is able to check on how promptly and well the assignments are graded and the students’ queries are answered.  The system tracks the students’ queries so that none will be missed and all queries will typically be responded to within a day.

In conclusion, Dr Leong ended the session emphasizing that game mechanics is not a silver bullet which can automatically improve teaching. Teaching is fundamentally the business of people. Software cannot and will probably never be able to replace human teachers. We should view game mechanics as yet another tool that if employed correctly has the potential to improve teaching. However, the tool will only be effective if the course is already well-designed with effective assignments and supporting lectures and tutorials. He also indicated that he currently has a final year student who was developing an open-source version of a gamification platform to support teaching that should be completed in the next semester.

Q & A Session

Following the presentation by Dr Leong, a lively discussion ensued and listed below are some questions from the subsequent Q & A session.

References

1. JFDI Academy (2010)
   http://jedi.ddns.comp.nus.edu.sg
2. McGonigal, J. (2010). Gaming can make a better world. TED: Ideas Worth Spreading.
   http://www.ted.com/talks/jane_mcgonigal_gaming_can_make_a_better_world.html