Evaluating Student Peer Review Feedback -- Putting Eli's Advice into Action

What is Quality Peer Review?

Today a question came up on POD-L seeking resources for evaluating student peer review, with a request for books, possible rubrics and other tools. Happily, two of the respondents pointed to one of my favorite peer review resources -- Eli (http://elireview.com), in particular this professional development module, written by Eli's team for teachers called "Feedback and Revision: The Key Components of Powerful Writing Pedagogy" at http://elireview.com/content/td/feedback/.

In the module, there's a section, part 3, called "What Feedback Is and How to Teach It." The team writes about peer review:

So So what, then, are the qualities of helpful feedback?

1. It is formative — it helps learners get better at a task or increases their understanding.
2. It is timely — it happens at a moment when it's possible to learn and change (e.g., revise).
3. It is descriptive, goal referenced and directed. 

As teachers, our goal should be to prepare students to give feedback that helps a writer understand: 

1. What they accomplished (descriptive).
2. What they were asked to accomplish (goal referenced).
3. What they must do next (goal directed). 

Of course, we help our students when our own feedback has these characteristics. But how can we help our students learn to provide better feedback? Three specific things we can do include: 

1. Modeling effective feedback.
2. Providing ample opportunities for deliberate practice in giving feedback.
3. Constructing effective review prompts.

A Peer Review Assignment is a Writing Assignment

If you look at the outline above from Eli's team, you see in it all the necessary elements of a writing assignment.  The first three items gesture toward the writing situation -- touching on purpose and audience. The second three name explicit goals and features of the review written -- moving into the elements of the genre Eli's team.  And the third three elements get the kinds of things that go around writing assignments -- models, practice, and good prompts to elicit the kind of writing one hopes to see.

You don't have to agree with the nine ideas above for every peer review activity you plan. And if you're going to grade the peer review, or evaluate it in some way, you might add a tenth item that describes how you will assess the reviews given.

With that in mind, then the question of how to evaluate peer reviews written by students is in inextricably linked to the design of the peer review assignment and its prompt. Eli's middle three items --- features that reviews should have present and should address -- can, for example, be recast as rubrics for a quick scoring tool,  or used as the basis of formative feedback on the review given by not only the professor but also the writers who received the feedback.

Teach Students What They Need to Know to Be Able to Write the Kind of Reviews You Want Them to Write

A writing assignment works better if the student has some instruction and guidance in how to write the kind of writing being assigned. So too for peer review feedback, which is a form of assigned writing. Consider the idea above from the Eli team that good peer review describes what a writer is doing well. Students might be readily able to deem something good, mimicking teachers past who maybe wrote "nice" or "good job" in the margins of papers those students had written. But that passive use of adjectives is not the same thing as describing why a passage is good or nice. The art of describing why writing works can be taught; it can be practiced.

So it wouldn't be enough to say to students "describe why and how the writing works" if their review is going to be judged on that criteria. One has to take a few minutes to teach active description, something like -- "This passage really works well because the details ('the crisp crust of cheese snapped on first bite' and 'I could hear in each bite of pasta the music my mother always played when she made her sauce from scratch') made me pause and painted a picture of what your meal looked like and tasted like. Great verbs helped it to move." And how that active description differs from passive description in a comment like this -- "I liked the dining room passage; it was good." One has to teach why the active and detailed description better helps a writer, and why the review activity, to succeed, requires that kind of writing from the reviewer.

And very often, in peer review, one has to give writers the chance to return to a review type -- the description of what is working -- again and again, so that writers can practice the technique, make mistakes in applying it, revise even.

So setting examples and criteria for the kind of comments one hopes to see, and then teaching students how to write to meet those criteria, and letting them try it more than once -- weekly or biweekly peer reviews is what Eli's team recommends -- builds the ability of students to become good reviewers.

Just as any writing assignment, done well and done often, makes any writer, over time, better at writing the kind of thing the assignment calls for.¹

If assessment of student performance is tied to clear articulation of what is expected in peer review comments, and if that expectation is taught so that writers understand and can find a path to meeting the criteria, with practice, then assessment serves a formative and celebratory purpose. Which is always a cool place for assessment to be.

__

1. Yes, there be a fragment there.