Technology in Pedagogy, No. 18, February 2014
Written by Kiruthika Ragupathi

Using peer feedback among students is a useful tool in education. Feedback from peers is usually available more speedily than instructor feedback and is given in a language that they can easily relate to; effectively conveyed, it helps them review and question their own personal beliefs. This process of giving peer feedback requires active engagement as well as a critical understanding of what an assessment task demands and the criteria used to assess and grade work.

Peer feedback can communicate peer expectations in team assignments says Damith Rajapakse, a senior lecturer with the Department of Computer Science from the School of Computing at the National University of Singapore. He says, instructors can also use peer evaluations to reward or penalize team members based on students’ contribution levels and can easily supplement instructor feedback on student performance. In this session, Damith introduced a tool that he and his team developed to manage peer feedback and peer evaluations in class.

Reason for using Peer Feedback

Damith started the sharing session by highlighting a study Norman Vaughan from the Mount Royal University, Canada on the perceived value of peer assessment feedback. The students in that study were asked to rate the value of peer assessment feedback before and after a course, and then it was followed up with getting student’s perceptions on the value of teacher assessment feedback. It was reported that an emphasis on formative peer feedback impacted students’ perceptions on the value of instructor assessment. The study results highlights that students’ participation in a peer-feedback activity creates a win-win situation for both instructors and students with students valuing their teacher feedback.

When Damith started getting his students from the large classes to work in teams, he felt the importance and usefulness of using peer feedback. That is because, when working in teams, students who generally slack tend to rarely participate and usually contribute very little to the project work, yet these students are not penalized enough and ride on the other members’ work to attain grades higher than what they deserve. To give a grade that a student really deserves, Damith felt the necessity for a system that can allow students to easily give peer feedback and enable teachers to access the information effortlessly. Secondly, when there are a number of deliverables from students every week, instructors would not have enough time to give feedback immediately, particularly on student presentations.

These two situations prompted his team to conceptualize on developing an online system, TEAMMATES (http://teammatesOnline.info) that can be used for managing peer feedback. An online system makes the maintenance and collection of feedback easy and effective. He shared that the system is currently being used by over 50 universities with over 7000 users and can be freely accessed.

A video tour of the TEAMMATES system can be accessed from the TEAMMATES home page.

Damith detailed four main features of the TEAMMATES system, and how it was useful for his classes:

1.      Peer evaluation / feedback for student teams

When students work on team projects, he found TEAMMATES to be particularly useful for collecting feedback from peers working on the same project. Students first estimate the performance of their peers’ participation and contribution to the project, and provide anonymous feedback to their team members. Second, they complete a self-evaluation of their performance using the system. This allows them to compare their own evaluation with the final score assigned by one’s own team’s perception. Finally, they also provide confidential feedback and comments on their peers to the instructor. This facilitates the instructor to easily identify the problem teams and be able to moderate the scores. It also gives an opportunity for instructors to intervene at an early stage while students get the time needed to amend their behaviour/problem before it is too late.Since the comments from peers are made transparent and open to all team members, students take ownership and responsibility. All of these, enable the instructors to penalise any under-performing student with more conviction and reward the deserving students with confidence.

For participants to get a sense of the system, Damith then went to demonstrate the usage of the system both from an instructor’s perspective and a student’s perspective. Once logged in to the TEAMMATES system, an instructor can easily create a course and enroll his students by copying from an already available spreadsheet. Instructors can then assign time periods at which students can give feedback, and the system follows up with an automatic email reminder to students. The students are then required to login to the system through the link provided in the email and get to key in the feedback based on their own participation and contribution, peer’s participation and contribution taking into consideration the team dynamics.

2.      Flexibility to create other feedback paths

As an instructor, you will be allowed the flexibility to determine your own set of questions, feedback paths, and visibility levels from the system. The real flexibility lies in allowing the instructor to pick any question from the set of questions for his students to give feedback on their peers.

The feedback paths can be chosen so as to allow feedback to be provided:
(a) between students in the same course,
(b) for various instructors in the course,
(c) amongst other teams within the course

The visibility level is so flexible that the instructor can choose from various options like:  students can see the feedback or comment; students can see the author of the comment; etc.

3.      Closed-loop feedback

A closed-loop feedback system is planned in such a way that it not only allows instructors to receive anonymous feedback from the students but also enables them to respond to the particular student, but anonymously as well. Thus, students are able to receive personalized quick responses. Instructors are also able to easily track if their students are reading the comments, and appropriate intervention is then possible when necessary. Depending on the questions chosen, instructors would be able to better understand student misconceptions and unclear areas.

4.      A repository of student information

The last feature was not so much related to peer feedback, but allows for easy maintenance of one’s own students that an instructor has taught thus far. This will provide an easy access for instructors to collect feedback from past students, and also be able to contact them for guest lectures.

Pedagogical advantages that Peer Feedback offers

Damith highlighted the following features that he liked and prompted him to start the development of a student peer evaluation tool:

1.      Provide early and frequent feedback

Peer Feedback enables students to gain initial feedback on their work at an early stage, not only in a timely manner but more frequently as well allowing them to respond to the feedback in future assignments. Providing early, frequent, and incremental feedback in a non-threatening environment can play an effective formative role in students’ personal development.

2.      Formative first, summative second

Peer feedback is about students providing constructive comments on a peer’s work; it does not involve awarding of marks but is a formative step prior to submission of a piece of work. Such feedback can help students to recognize and rectify gaps between peer/instructor expectations and their own performance. TEAMMATES is designed for both formative and summative purposes, but places greater emphasis on the formative.

3.      Shifts responsibility from the teacher to the students

This way, students are more involved in their assessment process when they have a larger responsibility. Not only do students take a closer look at the performance of their peers, but they are also constantly reminded of their own performance and are likely to use that as their frame of reference for improving.

4.      Develop self-reflection skills in our students

It enables the development of critical reflection skills and the ability to give constructive feedback to peers. Students can better engage with assessment criteria and internalise them for application in their own work. It enables the development of skills like making informed judgments, self-evaluation, critical reflection skills, critical thinking, analyzing learning outcomes and formulating constructive feedback to the peers.

5.      Introduce diversity in teams

The instructor has the ability to better understand the student strengths and weaknesses in terms of team dynamics, knowledge, skills and attitude based on the system scores. This enables faculty to introduce diversity in the teams based on student capabilities and contribution in team projects. The more the diversity in a team, the higher the benefits for each student as peers would learn to depend on each other in a positive way for a variety of learning tasks with the diverse groups that they work with.

Summary of Feedback/ Suggestions from the Discussion

Following the presentation by Damith, participants got into a lively discussion and asked Damith questions on how they could start using the system. Listed below are some questions from the subsequent Q & A session.

Q:	How does the system quantify the scores – the positives (+) and the   negatives (-)?
DR:	The   system does not quantify the scores. This is scaled down internally to address   the quantification. The values are taken to make a comparison, and acts more   of a red flag to faculty. Therefore, the values displayed are relative, and as   instructors you will need to look for mismatch (i.e.) the relative   proportions.
Q:	Are students honest or do they overplay their own contributions?
DR:	Usually students underplay the   team members’ performance, but overplays their own contributions and   participation. Therefore, it is normal to have a gap between the contribution   level ‘claimed’ by a student and the level ‘perceived’ by peers. Hence this   needs to be taken into consideration when grading.
Q:	Are there unintended consequences? Can this lead to unhappiness   amongst team members?
DR:	The   situation is not as lenient as before. The students who do more work   generally like the system, as they have their own voice and are able to   report the actual scenario back to the instructors. Since the comments are   visible to all, students will need to take on ownership and responsibility.
Q:	What if slackers don’t give feedback?
DR:	Sometimes, there is a   possibility for this to happen. A student who doesn’t give feedback would get   a reminder from the system. As an instructor, you could also give a gentle   nudge or reminder to the students. And if the students still don’t and is   marked down by other team members for his participation, then this will act   as a sort of confirmation.
Q:	Does the system provide criteria to the students that they can refer   to before giving score?
DR:	Yes, these can be done. The   rubrics can be included as part of the question itself i.e. at the question   level.
Q:	Do you plan to integrate it with the NUS learning management system?
DR:	Since the system caters to many   other universities and schools, there is no plans to integrate it with the   NUS learning management system.
Q:	Is the use of TEAMMATES Free?
DR:	Yes, you can register for an   account for TEAMMATES at http://teammatesOnline.info.   No installation is required, just get an account, and you can start using the   system right away! However, students would need to use their google accounts   to use the TEAMMATES system.

 

References:

Morrow, L. I.  (2006) An application of peer feedback to undergraduates’ writing of critical literature reviews, Practice and Evidence of Scholarship of Teaching and Learning in Higher Education, 1(2), 61-72.

Draper, S. & Cutts, C. (2006). Targeted remediation for a computer programming course using student facilitators, Practice and Evidence of Scholarship of Teaching and Learning in Higher Education 1(2), October 2006, 117-128.