A number of instructors at McGill University have implemented peer assessment (PA) of teamwork—asking students to assess peers’ contributions to teamwork as well as their behaviour throughout the completion of the assignment—in their courses and have generously shared some of their reflections on the topic. In the current remote teaching context, PA and PA of teamwork can support students’ engagement with one another, even when they are physically at a distance.
Jodi Tuck, Faculty Lecturer in McGill’s Ingram School of Nursing, shared how she and her colleagues have implemented PA of teamwork in two undergraduate courses: Individual and Family Development across Lifespans 1 and 2 (NUR1-224 and NUR1-225). In a conversation about students’ experience with PA of teamwork, Jodi addresses their ability to work together in teams and provide each other with feedback. She also emphasizes the importance of using PA to highlight student contributions to teamwork.
This post was co-authored by Carolyn Samuel and Jennie Ferris.
What inspired you to have students do PA of teamwork?
Being professional is an important skill in nursing. Also, nurses work a lot in teams, so, we want students to develop their skill set by working in teams and giving each other feedback. It’s part of our course objectives.
What was students’ PA task?
Using CATME (description below), students assessed their peers’ contribution to teamwork on 4 different scenarios over the term. Students assessed each other according to the 5 CATME rating categories (called “domains”):
- contributing to the team’s work
- interacting with team members
- keeping the team on track
- expecting quality
- having relevant KSAs (knowledge, skills, and abilities)
The CATME domains were each marked out of 5 for a total of 25 marks. Students got a score for each scenario, and that score was entered into myCourses manually. Students were penalized by 1 mark if they didn’t assess their peers. They still got their marks from their peers, but since they didn’t contribute to their self-evaluation, they lost 1 mark for not having done that. It was a small penalty. The scores were submitted through CATME. We exported them to an Excel sheet and made a cumulative score out of 25.
CATME (Comprehensive Assessment of Team Member Effectiveness) is a web-based tool that enables instructors to implement and manage peer assessment of teamwork. From the CATME website: “The CATME system enhances team learning by teaching students how to contribute effectively to teamwork and creating accountability for team-member contributions. The system facilitates communication among teammates and between students and their instructor.” Read the research behind CATME: Ohland, M. W., Loughry, M. L., Woehr, D. J., Bullard, L. G., Felder, R. M., Finelli, C. J., Layton, R. A., Pomeranz, H. R., & Schmucker, D. G. (2012). The comprehensive assessment of team member effectiveness: Development of a behaviorally anchored rating scale for self and peer evaluation. Academy of Management Learning & Education, 11(4), 609-630. https://doi.org/10.5465/amle.2010.0177
How did you go about forming the student teams?
We have learners from many different backgrounds and we wanted diverse teams. Using CATME, we asked students questions, for example, on language, ethnicity, scheduling, and, academic experience, and then used their responses to create teams. Students could skip questions they didn’t feel comfortable answering. In a class of 30 students, we usually have 5 teams of 6 students. The teams stay the same the whole term.
What are your thoughts on students’ ability to give peers meaningful and appropriate feedback?
I gave students “tips” documents (see here and here) on how to give effective, constructive feedback, and we talked about it a bit in class. Also, students were required to practice assessing fictitious peers in example scenarios that are part of the CATME tool. Some students did 8 or 10 practices. But we noticed that students scored everyone 5 or everyone 4, which was undiscerning. Many students were concerned that they’d hurt other people’s feelings, and so would only put positive feedback even if they didn’t mean it. We addressed this issue more directly in the second term: First, they lost a mark if they didn’t provide peers with comments, and second, they lost half a mark if they didn’t write reflective comments that included both one strength and one thing to improve.
Sometimes, students don’t work together well in teams. How well did your students work together?
Before I released students’ feedback, I skimmed everything. I could see from the comments which teams were working well, with students giving each other constructive feedback. For teams that were not working well, it was more challenging. You could see a conflict coming out in how teams were interacting in the classroom setting, and it’s nice that CATME highlights these conflicts. So, I was able to tune in to those comments and realized from students’ feedback that some teams were dysfunctional. Sometimes, this was due to students’ realities or personal experience. For example, single parents tended to have less time to contribute to team work, and they were penalized for it. I understand why a mother of 2 can’t meet on a Saturday morning for teamwork when the children have swimming, but the 18-21 year-olds in the cohort didn’t. So that’s a little thing to take into account.
And written English turned out to be among the most important criteria for determining a team’s success and satisfaction. About 30% of our student body is francophone and we also have a lot of international students. Teams that functioned well supported those students well, and gave them voice and empowered them. But teams that functioned poorly did the opposite. In comments to the instructor, some students felt almost taken advantage of in their team, with comments like, “I have a disproportionate amount of responsibility in terms of …” I really like that in CATME students can write confidential comments to their instructor. Both positive and negative comments were helpful because any student could talk to me about how they were feeling about the team process. Anyway, I tried to spread our international students across the different teams because they tended to need more support in terms of their language abilities. We’ve also considered making a French team, where students could actually submit their work in French. But then how would we address international students across teams [whose first language is not English or French]? It’s difficult to get the balance right.
How did you deal with the teams that were not working well?
Some students were offended by comments they got, so I had meetings with the teams that were struggling and made suggestions for how they could approach the different issues. I was able to mentor them in terms of how to address conflicts among peers. We never switched someone to another team due to conflict.
What are your thoughts on the technology you used?
CATME is not integrated into myCourses. This is just an administrative piece, but it was probably one of the bigger barriers to usage. You have to be patient and comfortable with technology to use CATME. It’s cumbersome and not that user-friendly. For example, there are set questions you can ask students for setting up teams, but the questions didn’t fit our needs. You can create questions and put them in a community bank, but if you want to change one thing in the question, you have to re-save a whole new question with a different title. Now that I’ve used CATME and have my own ideas about the questions I want, it’ll be easier.
What would you like to say to instructors who might be interested in implementing PA of teamwork in their courses?
I don’t think students can do teamwork effectively without some form of PA. Weak students might hide behind stronger students and CATME helps highlight that, but you should still have enough individual assignments to be able to discern among individuals. My students do an individual assignment and multiple team assignments, and sometimes, when I see the individual assignments, I’m like, “Wow, how did that student get this far?” They’re going to fail their individual assignment and still end up getting an A in the course. CATME, to an extent, highlights this issue so that I have an idea where some of the weaker students are.
Blog readers: How do you teach students to give and receive constructive feedback? What strategies do you use to address student team conflicts?
Check out TLS’ peer assessment and peer assessment of teamwork resources.
0 comments on “Peer assessment of teamwork: Developing a professional competency”