Testing, One, Two, Three

The New York Times's (free registration required) Tamar Lewin describes learning how to grade papers for the SAT II writing test. Any of us who've done placement test reading on our own campuses or for our own programs will be able to relate to the strange experience of being "calibrated," wherein readers are trained to apply the reading rubrics and ratings the same way.

Read the piece closely, and you'll hear Lewin expressing something any large placement or evaluation test reader must come to grips with: dealing with writing that's not really written to be read, but written to be evaluated. And reading writing not to hear what a writer has to say, but to see if a writer can write at some level of competency. So reading is as regimented as the conditions for writing for the test are.

Students who wrote the exams Lewin read had only 20 minutes to write; they had to "respond to the statement 'The world is getting better all the time.'"

Meanwhile, Lewin and other raters were told:

Ignore the handwriting. Read holistically, not analytically. Do not reread. Read supportively, and grade what's there, not what's missing. If the paper is absolutely illegible, or completely off topic, give it to your table leader. Read the whole thing before making any judgment, since some papers improve greatly once the student gets going. Remember these are children, and they had only 20 minutes to write. And remember, even if this is the 90th paper you're reading on the war in Iraq or Sept. 11, it's the first for the writer.

But that's the nature of this kind of testing: artificial writing for artificial reading. Still, it's better than relying on multiple choice.

The one advantage that writing programs have when they design their own essay placement procedures -- the test question or writing prompt(s); test writing conditions and time limits; rubrics; and reader training -- is that, unlike the SAT II, writing programs can shape the testing to match their courses and goals for first year writing. They can allow students, if they want, to use dictionaries and handbooks during the writing. They can devise questions that might involve responding to readings of the kinds students will get in college courses.

But even with all that, a lot of what Lewin describes will ring true for what many of us do locally when we run our placement programs.