A number of instructors at McGill have implemented peer assessment (PA) in their courses and have generously shared some of their reflections on the topic.
Professor Marta Cerruti teaches Materials Surfaces: A Biomimetic Approach (MIME 515/CHEE 515) to undergraduate and graduate students in the Faculty of Engineering. In a conversation about her experience with PA, she shares how she implemented PA in her course both within and among student teams, reflects on students’ PA experience, and gives advice for instructors thinking about trying PA in their courses.
What was the major assignment in this course?
Over the semester, students had to work in teams to develop a presentation and an accompanying report on an engineering topic of their choosing.
How were the teams formed?
Inspired by the work of Barbara Oakley, I decided to assign students to teams, rather than having the students choose their own. This allowed me to maximize diversity in the teams, as Oakley recommends. Specifically, I considered the students’ past performance, their disciplinary focus—whether they were in chemical or materials engineering—and whether they were undergraduate or graduate students. There were just over 30 students in the course, so in total, there were 10 teams of three to four students.
How did the assignment play out?
Students received feedback from peers on other teams on their presentation assignment twice – once mid-course on a short presentation and once at the end of the course, on a longer presentation. As well, they assessed their teammates twice on how well they were working together within the teams – once mid-course as a practice round, and once at the end of the course, which could affect their grade. In all, there were six main steps:
First, students submitted a two-paragraph summary of their project proposal via myCourses. That was ungraded, and only the co-instructor and I saw it. We gave feedback on the summary.
Second, teams gave a mid-course team presentation for which they prepared outside of class. The team presentations were three minutes each. In a speed-dating format, the teams presented to other teams during one class period:
- One team presented to another team. The listening team took notes on the presenting team’s message and performance, in preparation for giving feedback.
- The teams switched roles, so that the other team also had a chance to present. The listening team took notes on the presenting team’s message and performance.
- The teams had 10 minutes to provide oral feedback about the presentations to one another, based on their notes.
Then each team repeated steps A, B, and C with three more teams.
After class, students individually gave constructive written feedback using Office Forms to each team whose presentation they had seen. The Office Forms tool was easy to use for this feedback among teams, for the students and for me. (screenshot below; full feedback form here:
The teams then gave feedback on the feedback they received [editor’s note: such feedback is also referred to as “back evaluation”] – that is, they indicated how useful the feedback was, both in words and with a number on a scale of 0 (not useful) to 2 (very useful). Then the TA went through the written feedback and numbers, and transformed the 0, 1 or 2 into a grade. We had already explained to the students in the course outline how these numbers would be transformed into grades.
An advantage of these short presentations was that just from walking around the classroom – even though I did not listen to or provide feedback on the presentations at this stage – I was able to see who was engaged and who wasn’t. In certain teams, some people were visibly participating more than others. So it was immediate feedback for me to realize if there were teamwork issues or not.
Third, the student teams revised their presentations based on the in-class discussions among teams and the written peer feedback, and then they uploaded a revised version of their 3-minute presentation to myCourses. These were PowerPoint slides with a recorded voiceover which had to include all team members’ voices. I graded these uploaded mid-course presentations and gave feedback.
Fourth, the students practiced rating and commenting on the cooperation of their team members with a mid-course peer evaluation (p. 29), where students filled in a hard-copy form outside of class describing each team member’s contributions. The next class period, the students brought their forms to class. The students met in their teams and discussed these forms for 20 minutes. By filling out and then sharing the evaluation form, each team member could learn what the others thought about their work within the team. This was meant as a wake-up call in case anything was going wrong in the teams. During that time I walked around and asked a few questions to hear how things were going overall.
Fifth, each team presented for 10 minutes to the entire class. This occurred during three class periods at the end of term. They also submitted a final report. Each student was assigned to give peer feedback, again using Office Forms, on three presentations – one per class period. One of the feedback questions was “What do you think was the take-home message?” This question let students see if what they wanted to convey was actually understood. A few days later, each team had to write a one-page addendum to the final report on what they would do differently based on the feedback from peers.
Sixth, after the final presentation, each student rated and commented on their own cooperation within their team, and the cooperation of their team members, in an Office Form. This peer rating was also based on Oakley’s work (p. 30). These peer ratings contributed to students’ final grade. Then all their team assignment grades were multiplied by a factor depending on their peers’ assessment of their individual contributions, based on a formula that Oakley describes (p. 31). Because of this formula, each student’s ultimate assignment grade varied somewhat from that of the other students on their team. Students saw the results of the formula but did not see the comments of their peers for this round of evaluation.
What did students think of the experience?
They took the PA seriously and seemed to find it helpful. I gathered some feedback with a short, anonymous survey at the end of the course. Here are a couple of their comments: “I learned how to structure critiques in a soft manner” and “I learned how to make use of my critical evaluation skills.” The comments were quite positive.
And students mostly seemed happy with their teammates’ contributions: with one exception, all the average peer ratings were “excellent” or “very good”.
In the case of that one exception, one team had two students who had contributed to the mid-course presentation and two who hadn’t. The team asked to meet with me because they weren’t getting along well and wanted to change their team membership. In the meeting, it seemed that two of the students had higher expectations for the work quality than the other two.
I thought a lot about what to do. Since the students didn’t seem to be working together at all, I offered to split the team of four into pairs, such that the two students with higher expectations would work together, and the two students with lower expectations would work together. I allowed the pair of students who hadn’t contributed to the presentation to create a new presentation. Ultimately, the two students who seemed to have lower expectations got fairly low grades. If team members hadn’t assessed peers’ contributions to teamwork, these students probably would’ve continued as in other classes, where some team members don’t do much work, yet get the same grade as the team members who do more work.
What advice do you have for instructors considering having their students do PA of teamwork?
I can suggest a few things:
- Have students do PA mid-course and at the end. It’s helpful because students become familiar with the task and understand ahead of time what the expectations are.
- Explain to students in advance how assessment of their peers can impact their grades. Show specific examples, saying “Here’s an example of what happens to someone’s assignment grade if you give them a satisfactory, or an excellent, for their contribution to the team.”
- Put a couple of team graded assignments earlier in the term. Students were instructed to only include the names of team members who contributed to the assignments. By having a couple of smaller assignments earlier on, students can safely remove someone’s name from an assignment to send a message without dramatically decreasing their grade. For instance, next time I plan to have the initial two-paragraph summary submission as a graded assignment for a few points. That way, if a student does not contribute to the assignment, that student’s name may not appear on the assignment and this effectively signals that there is an problem.
Do you use PA to have students provide feedback on one another’s contributions to teamwork? What questions do you ask students to respond to when they provide this feedback to one another?
0 comments on “Peer assessment: Within and among teams”