#worthassigning: Effective Peer Review Assignment Design from the Eli Review Team

The team at Eli Peer Review (http://elireview.com) has a new professional development piece up for faculty on designing effective peer review assignments at http://elireview.com/content/td/reviews/. If you do faculty development, tutor training, like using digital tools to teach, offer a writing and the teaching of writing course, you'll find this invaluable. 

The full module includes video pieces from professors and useful illustrative graphics, but here's a humble text excerpt from the piece that speaks to its quality and smarts:

Reviews from which writers receive helpful feedback that will drive revision rarely happen without coaching, especially with novice reviewers. Teachers in feedback-rich classrooms must give as much attention to designing reviews as they do to designing writing prompts.  

Review prompts shape how reviewers talk to writers, influencing the details reviewers notice and ignore. Prompts are not just words instructors use, but also the various forms of response they choose to help reviewers read a draft carefully and respond to it thoughtfully. 

Unhelpful feedback is often the result of reviewer insecurities, caused by many factors:

•     They don’t know how to talk about writing, generally.
•     They aren’t aware of the learning goals of a project, specifically.
•     They aren’t comfortable providing feedback to peers, especially friends.

When designing a review, there are three important factors we can take into account that will help overcome these obstacles and result in better feedback: we can consider the cognitive load of our reviews, start with pedagogical goals and design reviews backwards, and be detailed and specific in how we prompt students. 

I really like how this starts -- students can give good peer review feedback (the piece cites relevant empirical research showing as much) with coaching and guidance. And often that's not provided in peer review assignment design. And the second point --- review activities, to succeed, require as much attention, as writing assignments. 

Eli Review is a Web-based software platform for writing workshop pedagogy; it purposefully puts teachers in the role of guide on the side, with the only teacher-centered commenting space being in their revision plan tool. In that tool, students choose which written comments they'll follow in their revision; after choosing the comments and saying how they'll use them, the professor can comment on the revision plans. Professors do not comment on drafts, nor do they use Eli to grade (there's no grade book), the emphasis is on designing good reviews, coaching good feedback, and for professors, commenting on revision plans based on writers' decisions about classmates review comments. It's a dramatic shift.

But here's the thing, you do not have to teach or have colleagues teach with Eli Review to benefit from its advice on assigning good peer review assignments. The advice is simply smart no matter what tools you use for peer review. Here's another excerpt, this one elaborates on their observations about considering cognitive load:

 One mistake we often make is giving students too much to do. Asking reviewers to read too much text and address too many questions can often mean that they don’t have time to respond thoughtfully. Module 1 discussed the issue of time and feedback loops, but some specific strategies for reducing cognitive load on reviewers include. 

Review smaller texts - consider smaller, focused reviews as a text develops, rather than asking reviewers to digest and respond to a large text. In an example like this, writers get feedback early, on small pieces, helping make sure that the larger draft they’re building toward is on the right track, with the added benefit of making plagiarism much harder (since you can watch as a text evolves from earliest kernels to a full draft):
[image not shown]

Multiple reviews of the same text - Divide reviews to conquer cognitive load. Design smaller, swifter reviews that are focused on specific, granular goals. This will let reviewers focus carefully for discreet moves:  
[image not shown]

The advice to review smaller pieces of writing, smaller pieces of larger texts, or smaller goals in a larger text make incredible sense no matter how one does peer review.

Finally, because the writing is so clear, and the case for the approach so compelling, if you put into practice some of the strategies the piece suggests, no matter the technology you use, this is a piece worth assigning to your students before they begin doing peer review. It will help them to better understand your peer review approach, why you're asking them to do peer review, and it demonstrates convincingly that students can become reliably good peer reviewers.