Faculty Today

Online tools for assessment and engagement in large classes

Using mouse

This post featuring Prof. Ken Ragan is the latest installment in our ongoing series about assessment tools for large classes. On October 7 from 8:30-10:00 a.m., Prof. Ragan will be the guest speaker at a breakfast workshop on “Evaluation and Feedback for Large Classes”. For details and to register, go here.
“Experiment. I used to feel like I couldn’t experiment.”

One might imagine that experimentation would occur naturally in an undergraduate physics course – and indeed, the biweekly laboratory sections of Physics 101 are abuzz with students engaged in active discovery. But what about in a lecture hall filled to the brim with nearly 700 students: is there a place here for experimentation as well? Professor Ken Ragan thinks so, especially when it comes to trying out new ways of engaging, giving feedback, and assessing his students.

Like many instructors, Ken wants his students to leave his course able to demonstrate well-developed analytical and critical thinking skills, both crucial for success in the sciences: “Students need to be able to make simplifying models and apply fundamental concepts – going back to basics, problem-solving … Students are more concerned with getting to the answers, and I want them to go back to basics and understand the problem.”

The challenge was how to design his course in such a way that his students could develop these important skills, give his students adequate feedback on their development throughout the term, and not completely overwhelm him (or the TAs) with loads of extra marking. In addition to a mid-term and final exam, Ken also uses several online tools that allow students to practice the concepts brought up in class and develop their problem-solving skills.

Ken Ragan

Using online tools for feedback and assessment

Ken’s students are given weekly online assignments (worth 10% of the final grade) that give them a chance to work through additional problems framed as real-world situations with familiar objects, like an elevator or a scale. The weekly problem sets are based on the material presented during the week’s lectures; they’re released to students the night before the lecture, and are due the day after. The assignments are set up and graded using an online system called LON-CAPA (more information on the system can be found here: http://www.lon-capa.org/), and students access the problems through myCourses.

CAPA personalizes each student’s problems – no two students’ questions are exactly alike, so some independent effort is required – and students get to try their hand at each question up to six times, without penalty, until they get the question right. This iterative process allows students to receive instant feedback on their problem-solving methods, and allows them to troubleshoot and experiment with alternative approaches.

The CAPA interface can take some getting used to, Ken says, but ultimately it pays off in the form of deeper understanding, and better problem-solving and teamwork skills: “Students work together on these assignments and have CAPA parties in the resident halls, so they really enjoy [the assignments] even if at first they don’t like CAPA so much.” As Ken explains to his students, “the heart of physics is problem solving. Used correctly, the assignments allow you to hone your problem-solving skills”.

In addition to the practice problems, students are assigned two short quizzes every week, on the evenings before lectures: the five questions are based on assigned pre-lecture readings. While the quizzes don’t count towards the final grade per se, students can receive bonus points (up to 5%) for achieving at least 60% on the term’s quizzes (and also meeting a minimum standard of in-class attendance).

This strategy gives students an additional opportunity to engage with the course material and receive feedback on their comprehension prior to the lectures. The added incentive of bonus marks (however small) improves the level of participation in the quizzes, and subsequently his students’ preparedness.

The fifth question on each quiz asks students to name something they didn’t understand about the readings, or something they found particularly interesting. Ken spends about an hour reading a third of the students’ answers to question five – about 250 of them. Armed with the information gleaned from the responses, “it gives me an idea of what I need to focus on during the lecture…I don’t really change the lecture [content] but it tells me how to pace [it]”. This last-minute approach is sometimes called “just-in-time teaching”, and it can take a little bit of practice. Says Ken: “It’s taken me some time to get used to just-in-time teaching, and I’m not quite where I want to be yet. The lecture is still largely prepared, but I’m more conscious of how it’s going and where students are.”

He has some advice for others thinking about giving it a try: “It’s easier to do just-in-time teaching in lower-level courses. Don’t do it the first time you teach a course. It takes time to develop an understanding of students and the course … you need to really know the material and be willing to change quickly how you present or order material.”

It’s clear that Ken’s use of these online tools benefits both students and instructor. Students get feedback on their performance early and very often, are given many opportunities to put conceptual and theoretical material into practice, and also are learning good study habits that are transferrable to other courses: having so many little tasks to complete on an ongoing basis means they can’t “cram” at the last minute. By the same token, Ken receives continual input and feedback from his students, and is able to tailor his teaching to better facilitate their learning.

Ken’s willingness to try out these online tools may seem like a bold move: many students have come to expect a certain assessment style in very large classes and getting them to buy in to other formats can be challenging. But, says Ken, “I tell them I’m experimenting and that I think what I did before didn’t work as well.”
Would you like to see your students better prepared for class? Would you like to have a better sense of their understanding of the materials? Join Prof. Ken Ragan at a discussion to learn more about using online assignments to motivate students to read material before coming to class, on October 7

Original publication date: September 30, 2014

+ posts

Hakai postdoctoral scholar at the University of British Columbia and Simon Fraser University (B.C.)

0 comments on “Online tools for assessment and engagement in large classes

%d bloggers like this: