Sunday, November 23, 2014

#worthassigning: Laura Hudson on Twine, the Video-Game Technology for All

If you go to to learn about Twine, you get this:
Twine is an open-source tool for telling interactive, nonlinear stories. 
You don't need to write any code to create a simple story with Twine, but you can extend your stories with variables, conditional logic, images, CSS, and JavaScript when you're ready. 
Twine publishes directly to HTML, so you can post your work nearly anywhere. Anything you create with it is completely free to use any way you like, including for commercial purposes. 
Twine was originally created by Chris Klimas in 2009 and is now maintained by a whole bunch of people at several different repositories. 
That's intriguing -- its open source, don't need to write any code, simple stories possible -- but it doesn't tell a story about what Twine can do.

Happily and wonderfully, very compassionately -- yes, a compassionate look at software -- Laura Hudson offers a wonderful profile of Twine by profiling a game developer who creates emotional powerful and compelling games, a woman named Porpentine.

Here are some excerpts from Hudson's piece, just as a sampler of what makes her writing so good.
A free program that you can learn in one sitting, Twine also allows you to instantly publish your game so that anyone with a web browser can access it. The egalitarian ease of Twine has made it particularly popular among people who have never written a line of code — people who might not even consider themselves video-game fans, let alone developers.  
Chris Klimas, the web developer who created Twine as an open-source tool in 2009, points out that games made on it “provide experiences that graphical games would struggle to portray, in the same way books can offer vastly different experiences than movies do. It’s easy to tell a personal story with words.” 
Twine games look and feel profoundly different from other games, not just because they’re made with different tools but also because they’re made by different people — including people who don’t have any calcified notions about what video games are supposed to be or how they’re supposed to work. While roughly 75 percent of developers at traditional video-game companies are male, many of the most prominent Twine developers are women, making games whose purpose is to explore personal perspectives and issues of identity, sexuality and trauma that mainstream games rarely touch on.
And on Porpentine:
One of the most prominent and critically acclaimed Twine games has been Howling Dogs, a haunting meditation about trauma and escapism produced in 2012 by a woman named Porpentine. The gameplay begins in a claustrophobic metal room bathed in fluorescent light. Although you can’t leave, you can “escape” once a day by donning a pair of virtual-­reality goggles. Each time, you’re launched into a strange and lavishly described new world where you play a different role: a doomed young empress learning the art of dying; a scribe trying to capture the beauty of a garden in words; a Joan of Arc-like figure waiting to be burned on a pyre. And each time you return to the metal room, it’s a little dirtier and a little more dilapidated — the world around you slowly decomposing as you try to disappear into a virtual one. 
“When you have trauma,” Porpentine says, “everything shrinks to this little dark room.” While the immersive glow of a digital screen can offer a temporary balm, “you can’t stay stuck on the things that help you deal with trauma when it’s happening. You have to move on. You have to leave the dark room, or you’ll stay stunted.” 
When I first met Porpentine outside a coffee shop in Oakland, Calif., she was wearing a skirt and patterned knee socks, her strawberry blond hair pulled back in a small plastic barrette. We decided to head to a nearby park, and as we walked across the grass, she pivoted on one foot — an instant, unconscious gesture — and did a quick little spin in the sunshine. When we arrived at a park bench, one of the first things we talked about was trash, because her Twine games teem with it: garbage, slime and sludge, pooling and oozing through dystopian landscapes peopled by cyborgs, insectoid empresses and deadly angels. In Howling Dogs, the trash piles up sticky and slow; in other games, like All I Want Is for All of My Friends to Become Insanely Powerful, tar floods the room suddenly from an indistinct source. Forget pretty things, valuable things: Porpentine’s games are far more interested in what society discards as worthless. 
“Trash has very positive connotations in my world,” she said, trying to smooth the wild ends of her hair as the wind off a nearby lake kept bringing them to life. “A lot of my work is reclaiming that which has been debased.” A transgender woman who has faced harassment for much of her life, Porpentine referred to herself as “trash-bodied” several times as we talked. It’s not an insult, she explained: “Me and my friends, we hide in the trash. People call us trash, but we glorify in it.” At 14, she was kicked out of her home. It’s not an unusual story — an estimated 20 to 40 percent of homeless teenagers are gay, bisexual or transgender. When I asked her what she was doing before she made Twine games, she said, “Just surviving.”
What makes Hudson's piece compelling to read, and useful for thinking about as a teacher of writing, is that it accomplishes three distinct things, all signaled in her opening lines:
Perhaps the most surprising thing about “GamerGate,” the culture war that continues to rage within the world of video games, is the game that touched it off. Depression Quest, created by the developers Zoe Quinn, Patrick Lindsey and Isaac Schankler, isn’t what most people think of as a video game at all. For starters, it isn’t very fun. Its real value is as an educational tool, or an exercise in empathy.
Her piece touches on GamerGate, uses it as a reference and reminder. So that's the first thing it does, offers a commentary on the issues surrounding GamerGate. And she gets at that not by focusing on GamerGate per se, but on the platform and games women are writing. And it's that -- focusing the story on women writing games unique to gaming, where women make up 50% of gamers -- which comments on GamerGate. So the second accomplishment of her work is providing a deft understanding of Twine as a gaming platform. Why it's easy to use, how the ease-of-use opened up gaming to non-computer code trained developers. She doesn't write a tech manual on using Twine, but her descriptions of the games she's played, including many detailed scenes from Porpentine's games, illustrate how compelling and powerful Twine can be in creative hands. And third, her piece is about women who make games, and the kinds of games they are making, brought to life by her portrait of Porpentine.

Hudson's piece is an exercise in empathy, and for that alone it's worth assigning. But it's also, in its way, a piece about the power of story. The games Hudson highlights, her portrait of Porpentine, reminded me of two works I love about the power of story. After reading about Porpentine, I couldn't help but think of this line from Leslie Marmon Silko's Ceremony: “I will tell you something about stories . . . They aren't just entertainment. Don't be fooled. They are all we have, you see, all we have to fight off illness and death.”

Hudson's description of Porpentine's games, and Porpentine's own words, also put me in mind if Adrienne Rich's "Diving into the Wreck," which to me has always been about the point where words end and feeling/sensing begins, words carrying one to the point of immersion and deeper knowing. So the developers Hudson writes, "explore personal perspectives and issues of identity, sexuality and trauma." And the diver in Rich's poem explores as well:
I came to explore the wreck.
The words are purposes.
The words are maps.

Writing in Twine

Hudson's piece is a great read -- lots to think about in English and writing classes, technology and culture courses, and can stand as course reading. But Twine is also a writing space, Hudson explains, "Twine games, they’re essentially nothing but words and hyperlinks; imagine a digital “Choose Your Own Adventure” book, with a dash of retro text adventures like Zork."

And so her piece is also an invitation to writing in Twine, an encouragement, even if never explicitly made.

Twine would be a great platform for having students write in.

I just came from a session at NCTE where the panelist spoke of the transitions teachers make from being students in teaching programs to being teachers. They spoke of the fear and anxiety new teachers face, the doubts, the challenges. One speaker, Audrey Lensmire, described group she formed, a kind of teacher support group for 6 women in her program who were out teaching. It began because the women would appear at her office, shut the door, and share. The teacher ed program lacked a course or space where teachers could talk about the mental and emotional strains of teaching. Many of the women in Audrey's group also carried silently mental health issues around depression, anxiety, some were being treated. But until they got in the group and could talk about this, carried that burden alone.

I am also thinking of Twine and Lensmire's work with the work Margaret Price did in her book, Mad at School: Rhetorics of Mental Disability and Academic Life,  The book, to quote from a review by Stephanie Kerschbaum,  "reveals how individuals with what she terms mental disabilities—which she defines broadly to include not only intellectual and developmental disabilities, but also autism, learning disabilities, and psychosocial disabilities and mental illnesses—are persistently excluded, ignored, and targeted in higher education."

Playing Twine games composed by artists such as Porpentine can be a powerful empathetic experience, just as hearing Audrey Lensmire and her colleagues today was powerful. Or just as reading Margaret Price, Melanie Yergeau, Adrienne Rich, Leslie Marmon Silko, and today, Laura Hudson, is powerful.

I wonder: what would a game in Twine about teaching look like, if Audrey Lensmire's group were to write one, and how would the writing of the game change their view of  themselves as teachers? What would Melanie Yergeau, who does great work digital media and disability studies, who writes compellingly and unflinchingly about autism, compose?

It's not that these people -- Melanie Yergeau, the women in Audrey Lensmire's group, Margaret Price -- need to or have to use Twine. But their work looks at the kinds of issues that Twine developers explore, and so I cannot help wonder what composing in Twine might bring to those whom their work supports. They are all women scholars who, while they ground their work in theory and the relevant literature, give voice to what's often unsaid, through the personal stories that also permeate their work.

And it's not that the Twine pieces we ask students to write, or maybe write ourselves, have to explore the inner emotional and cognitive worlds Margaret, Audrey, and Melanie share explore and share. Twine simply looks to be an intriguing platform for digital composition, where many courses use games or explore their creation. Twine can be a tool where different writing assignments, different writers, will bring to it their own purposes, their own maps.

Thursday, November 20, 2014

What David Wiley Gets Wrong About Textbook Publishers and How it Can Hurt OER's Growth

I just tweeted this observation in response to remarks David Wiley made in a quick interview with Steve Kolowich for the Chronicle of Higher Education's Wired Campus Blog (a good read).
Let me explain. But first . .

some caveats . . 

a reminder:
I work, happily and proudly, for the Bedford/St. Martin's imprint of Macmillan Education, a college textbook publisher. From my point of view, and I would say many of my colleagues, Open Educational Resources are a good thing; OER is no more the enemy of Macmillan Education than Pearson is. They are competitors, part of a changing industry. A professor who assigns an OER resource instead of ours is no different, from my point of view, than a professor assigning a Norton textbook instead of ours.

an acknowledgment:
OER content has improved. Early OER was disorganized, hard to find, not well supported, often poorly edited. I know because I wrote some of it and read a lot of it. It was more a collection of class handouts and assignments scattered hither and yon in varied OER projects or personal collections. But now the content and structures around OER are getting better, with crucial support coming from state legislative initiatives, foundations, and projects like David Wiley's Lumen Learning.

a belief:
I've contributed a bit to OER projects, advising on projects like Joe Moxley's Writing Commons, recommending Mike Palmquist's Writing Studio, reviewing free online tools like bibliography builders, doing peer review for an open access journal,  and regularly pointing faculty to free content I would find #worthassingning were I teaching. I think OER is a good thing, a valuable movement. It has helped motivate commercial textbook publishers to do more with e-commerce and e-books that help bring down costs. Competition does that.

How David Wiley Sees Textbook Companies Abilities to Reach Faculty -- Not Nicely, Not Accurately.

Steve Kolowich's discussion with David Wiley focused on the difficulty of making faculty aware of OER resources, knowing how to find them, choose them, and assign them. Wiley has this take on publishers' advantages:
Traditional textbook companies “have a budget to send out dead trees—big marketing budgets,” he said. “That’s what you’re really competing with: (a) with that marketing budget and (b) with a very, very lean-back experience for the faculty member where content just magically appears, for them, on their desk, and they say, ‘Oh, this table of contents looks good,’ or ‘I like the cover art on that one.’ It’s not generally a very proactive experience for many of them.” 
“I don’t think you can market OER the same way you can market textbooks traditionally,” he said. “The way textbooks get marketed is there are these huge collections of databases of which faculty members are teaching which courses at which institutions and how many sections and how many students are estimated to be in each section—tracking all of that data and figuring out which of those are worth mailing dead trees to, and which can we only do in email, and—just huge amounts of money. And it’s the kind of money you can only spend when you charge $250 for a textbook.”
I love the dead trees line. Textbooks, of course, are not dead trees, though some trees, or recycled products from earlier dead trees, do die in their making. Textbooks are pedagogical tools, whether printed or online.  But still, Wiley's disdainful evocation of dead trees, which is linked to the price of textbooks, wraps a factual observation about what publishers do. Publishers do send exam copies. And publishers do try to keep track (Though not as efficiently as Wiley’s evocation might suggest.) of which professors teach which courses, how many students are enrolled those courses, which prior books, if any were used in those courses, and other things it really helps to know.

But publishers do try to be smart about which books to send; postage is a cost; exam copy books finding their way to the used book market is a cost; and professors peeved by getting too many unwanted and asked for books, and deciding never to order from that publisher is a cost.

But here’s the thing, sending books in the mail works. Here’s the other thing, the real important thing that David Wiley gets wrong -- by omission: sending books matters, and data helps, but the strength college textbook publishers have is not in databases and a book sampling budget, but in people, two groups especially: the sales force who knock on faculty doors, and the editors who develop books with faculty input and by faculty authors.

People Matter More Than Data and Mailing Books

Look, good information matters a lot, and it can be recorded, stored, and turned into data. But to get good information, you need good people. A good publisher sales representative will learn which faculty teach which courses and how they like to teach the course. Good sales representatives filter pedagogical need, helping faculty find the book or digital media that will most likely be useful. A good sales rep is also an advocate, conveying to editors and publishers faculty feedback, requests, and critiques.

So if a professor gets a book on her desk and likes the table of contents and decides to order the book, very often it's because the process to that point -- the meeting with reps, looking at other books, talking to editors -- was very proactive. It's a myth to think that sending books and targeting faculty based on what's in a data base makes the difference. Faculty are more sophisticated than Wiley implies. The decision on which book to use, even if not made after consulting a sales rep or editor from a publishing company, is made on more than what's in the table of contents or what's on the cover.

Choosing what content to use in a course is hard when one is choosing something new. The decision to choose is proactive. The advantage publishers have, in their sales force and with their editors who work with customers is that they can help faculty make those decisions by listening, filtering, and recommending. And, then, if the recommendation is chosen, supporting the choice with instructor teaching guides, and if a department uses the book or media for all sections, campus teaching workshops, often author lead, that qualify, very often, as faculty professional development.

People Ain't Perfect, But Are, in Aggregate, Effective

Not all reps are good, not all editors responsive, not all faculty even interested in talking to a rep or editor. But the key thing textbook publishers have that OER projects do not have are people whose careers center around talking to faculty about teaching. And the best reps and editors, the really good ones, the ones faculty tend to enjoy spending time with, are those who care as deeply about teaching and learning as faculty do.

Textbook companies hire a lot of people who really do see their work as a mission to better teaching and learning. It may not seem that way if you've not met those folks, but they are essential to the sales a textbook company earns.

Not Seeing the Forest of People for the Dead Trees

So my thinking is that OER won't really take off if it views how professors choose books as simply a marketing advantage, the ability to send books, or email with links to e-books, based on data gleaned from decades of reps and editors visiting professors.

Until the OER movement has more people who can do the kind of work of helping faculty match their teaching needs to OER resources, helping faculty see how the resource can work in their teaching, and helping faculty adopt and use the resources well, it will be hard to get the attention OER does in fact, where the stuff is good, deserve.

That's a lot of people. But those people, more than any other resource, do the most to make get textbook publishers the business they do have.

I am sure that there are good OER initiatives emerging that harness the enthusiasm of OER advocates, people who will promote and recommend OER. That's to be encouraged because without it, OER growth will take longer.

The best publicity is people talking to people.

Thursday, November 13, 2014

Three Peer Review Tools at Glance, and Four Other Options to Boot

Here's an e-mail that I thought Gmail wouldn't let me send to two discussion lists I am on because it was flagged as spam.  I'm posted it here so I could send to the lists the link for this post instead. Turns out, the message did go through. Ah well. Since then, I've revised this a bit so it works better as a post.

When we (Bedford/St. Martin's, now an imprint of Macmillan Education) were ramping up our project with Eli (, my colleague, Melissa Graham Meeks, did a quick review of the peer review tools landscape, including Calibrated Peer Review from UCLA (  and from U. of Pittsburgh, SWoRD (now called Perceptiv --

Here're her notes on those:

  . . . Calibrated Peer Review (CPR) from The University of California and SWORD from the University of Pittsburg.   Calibrated Peer Review works by training writers to read instructor-supplied models before completing a peer review activity and a self-reflective scoring of their own drafts; the system produces a single score that accounts for student’s performance when calibrating to the models, when scoring drafts as a reviewer, and when self-evaluating (comparison of peers’ ratings with writer’s own rating). The appeal of CPR for instructors is the single-score derived without instructor intervention once the models and rubrics are set-up. Like CPR, SWORD generates a single score. SWORD uses an algorithm to assign a grade to students’ writing based on the feedback given (reviewer accuracy is accounted for) and students’ reviews (accuracy as indicated by other peers’ scoring and helpfulness as indicated by the writer are accounted for).  But, like Eli, SWORD allows instructors to design rating and comment prompts. SWORD also has an established library of prompts, which instructors can customize.

These comparisons overlook the fundamental difference in these tools: Whereas CPR and SWORD are [primarily framed] as summative  review tools with gradebooks, Eli takes a formative approach to a writing process that includes both review and revision: Eli’s patent-pending* approach to capturing review feedback and allowing writers to build a revision plan from that feedback distinguishes Eli from every other product on the market. Eli’s espoused purpose is to teach review and revision by giving instructors analytics that match their personal teaching goals and giving students a scaffolded process that mirrors expert writers’ behaviors; CPR and SWORD simplify the review and grade it.

SWoRD is a cloud-based reciprocal student peer-to-peer student assessment system.  Students upload their assignments into SWoRD, which automatically and anonymously assigns the document to from 3 to 6 of the student’s classmates.  SWoRD Peer is equally effective for presentations, videos, art projects, business plans, legal briefs, and any other task requiring formative feedback.

CPR trains students to provide good feedback by having them rate 3 “source” texts first; reviewers must pass this calibration phase before they can rate 3 peers’ work and then they rate their own using the same criteria used to evaluate the source texts. The system compares reviewers’ scores to each other as well as the writers’ self-assessment to reviewer’ assessment; these two scores are combined with the writers’ performance during calibration to derive a single score.
Eli Review is a software service that supports and reports student engagement. As they write, give feedback, and use feedback for revision, students learn from each other. Reports give students and instructors a clear window into the peer exchanges, making it easy to assess effort, identify exemplars, and motivate revision. A revision planning tool let's students choose which comments they'll work from in their revision, sort the comments in order of priority, make notes on how they'll use the comments. The revision plan can then be reviewed by the professor for further guidance. Reviewers get the most boost to their helpfulness rating when comments they write make it to revision plans.


The folks at Perceptiv have done a good bit of publishing -- . They developed their approach with NSF grants. (The creator of the program is a cognitive psychologist; the creator of CPR was a chemistry professor, I believe; both professors took peer review seriously and sought to use writing more fully in their teaching.) The research from Perceptiv is good and useful stuff and shows, contra MOOCs which didn't do it too well, that peer review, done well, can be a reliable means for feedback, especially if done early and often.

Eli Review's approach -- see for a link to their first professional development piece -- draws directly from composition theory and seeks to turn writing classrooms into writing workshops (If you visit writing classrooms, you'd be surprised, or maybe not, at how much lecturing goes on.)

All three tools require some forethought and planning to use fully and richly. And that's the challenge in their use for most faculty, whose first instinct (understandably) begin digital teaching by taking their analog assignments and putting it on the Web. In the work Melissa did, CPR and SWoRD found purchase in WAC-focused science and social science courses, often larger course (though not exclusively).

Other tools for Peer Review - that are open to folks from other campuses to try . . .

  1. Joe Moxley's Myreviewers:
  2. Mike Palmquist's WritingStudio:
  3. MARCA from UGA's Ron Balthazor & Christy Desmet:
  4. Les Perlman and Irv Peckham might weigh in here, but I also imagine iMoat could be readily adapted to do peer review even though its original use was for writing placement:

* This is not a cliche. Eli did apply for a patent because they have a unique algorithm for deriving a reviewer's helpfulness rating based on how their review comments are rated, but more importantly, which of their comments make it to a writer's revision plan.

Monday, November 10, 2014

The Ineluctable Attraction of Grammar Quizzes and the Case for Reviewing Grammar Sites

Writing in the Chronicle of Higher Education's Lingua Franca Blog, Geoffrey Pullum has a piece on the decline of grammar education as measured by the quality of grammar advice and quizzes that proliferate on the Web. He opens with a telling observation:
Google fetches more than 300,000 hits for the term "grammar quiz"; yet if quizzes on chemistry were as uninformed as those on grammar, they would ask silly questions on peripheral topics (“Who is the Bunsen burner named after?”), and would make no reference to the periodic table, or atoms or molecules. The web’s grammar quizzes deal in minor pieces of puristic flotsam with roots in dimly understood 18th-century grammatical analysis.
People love doing grammar quizzes online. I don't know why. But the sites seem to multiple like weeds (and some weeds are very prettier than some flowers, so nothing against weeds) and draw hits. I guess they're fun if you're doing them by choice.

I work for a college textbook publisher, and I know how hard it is to get a good grammar quiz written. Even with dedicated editors, authors who write grammar handbooks and sweat the details, experienced question writers, and professional proofreaders, some questions still don't come out quite right. The volume of questions needed, the difficulty of writing a question that isolates one issue, often involving writing a sentence and trying to come with three or four variations of it, where one item is correct, is very hard to do well.

I know we have some questions that are better than others. And every once in a while, faculty alert us to a question that is just wrong or confusing or too hard or too easy, and we either revise it, replace it, or simply remove it.  Even correct and well written questions can pick up an in the conversion of manuscript to live quiz because of production errors that are missed in the quality assurance review. It's really trick stuff to get right.

So it's not surprising that a lot of the stuff on the Web that is free, open, and done by people who love the topic perhaps not wisely but too well can cause more problems than it solves. Pullum looks at one site in particular, Collins Dictionary --, and details the quiz's faults, before coming to this summary judgment:
And that’s all there is! Irrelevant, pointless, or misconceived questions on a minuscule range of topics, set by people who don’t know what they’re talking about. Not a single sensible question with a clearly correct answer on a noteworthy point of English grammar.
Now, I want to be careful. Grammar and the degree to which writing teachers address it is fraught ground.  So two things up front.

First, I make plenty of usage and occasional grammar errors in my own writing, especially one draft writing like this blog post. If you see something, say something (constructively, please), and I'll make an edit.

Second, when I teach sentence level revision, even when I teach the most novice of writers, I don't assign grammar quizzes in bulk. My primary concern as a writing teacher is not to teach grammar, but to give students enough understanding of grammar terms and guidelines to make good decisions as writers when they attend to their sentence level revisions during proofreading and copy editing stages. So I'm in the Grammar in Context, patterns of error, minimal marking camp by and large. I don't ignore this stuff, don't shy away from teaching or using grammar handbook language (independent clause, dependent clause, names of parts of speech, and other fun stuff) as needed in that context. For more on this, see the NCTE Guideline on Some Questions and Answers about Grammar at

That said, I do find grammar quizzes can be useful if used judiciously, as a part of sequence that takes students back to their writing. Briefly, the sequence is: discern an issue that remains after revising, experimenting with sentence combining, voice, and other fun stuff. Teach the writer how to look the issue up in a handbook, have the writer read what they need to in the handbook. Often, to help a writer process handbook advice, an exercise or quiz on the issue can help. Handbooks explain rules by showing incorrect and then correct sentences. Or incorrect sentences edited to become correct. Bedford/St. Martin's usability research on handbooks shows that writers look at those sentences first, and might, if they feel they need to, read about the issue. Sometimes they don't. At any rate, I find that having a student do an exercise or two in the form of a grammar quiz helps to reinforce the correct/incorrect lesson a bit. But I don't think even doing that is useful unless very soon after -- immediately is best -- the writer applies what they learned to their own writing, or in an editing workshop, classmates' writings. (See for a handout that helps with this.)

When I do the sequence above, I use the site and exercises associated with the handbook I'm using. I know more and more professors do not require a textbook publisher's handbook, but I've tried teaching without one, and found that even good resources, like Purdue's OWL, which I love for lots of reasons, is to hard to use for too many students compared to a well-designed, professionally edited -- and this is important, consistently authored -- handbook. Of course plenty of people do use a handbook and also use one of the many free exercise sites out there.

Assign Students to Review Grammar Advice and Exercise Sites

Whether as an instructor you use a handbook and its exercise site; or use a handbook and an alternative free exercise site; or don't use a handbook at all and ask students to use a free grammar advice and exercise site, given Pullum's observation about the quality of the sites out there, it's really important to only ask students to use sites that can be trusted. And no site should be taken for granted. Because free sites get a lot of traffic, which means a lot of writers, novice writers who may not know better, use them and learn from them. For example, a well regarded free site,

Capital Community College's Guide to Grammar and Writing, says it gets 30,000 hits a day.

At Bedford/St. Martin's, we have, for now, a free and open -- no code, no book purchase required to use it -- grammar exercise site called Exercise Central --   It's a site that gets traffic from all over the world, and from people who aren't using any of the books or media we sell. A majority of the accounts are unassociated with professors, that is, quizzes don't report to a professor grade book. They're simply being done by writers concerned about sentence level issues because I think a lot of writers like doing quizzes and/or think they'll help.

So it would be a good project, either as part of a proof reading class, a linguistics class, a tutor training course, a grammar course, a style course, or other course that touches on addressing sentence level issue and choices, would be to ask students in those courses to review some of these sites, especially since a lot of students seem to find them on their own and engage them on their own.  And not just the free sites that are put up by enthusiasts, or dedicated teachers such as Charles Darling, but also the stuff that textbook publishers provide, whether as open websites or behind a free code given with a new book, or, as increasingly is the case, products that are sold on their own.

Notes Toward Review Criteria

Criteria for a review, gleaned from Pullum's blog post, might include, but do not need to be limited by (so add you're own if you do this assignment) the following:

The puristic flotsam factor:  Does the quiz ground itself in the overly prescriptive bĂȘte noires of its author?
The confusion factor:  Is grammar used too broadly, with quiz questions under that name for issues of convention, spelling, punctuation, and style?

The instructional factor: Are there explanations -- either before a question is asked, in response to a quiz takers choice, or by reference to a print or online book? 

The accuracy factor: Not to be confused with bĂȘte noires of one's own, or even misnaming something as grammar that is not, but does the site get stuff fundamentally wrong in the ways Pullum illustrates?

The pedagogical factor:  Hello, are you reading, writer? I once visited a computer lab at a community college where students had to complete a battery of grammar exercises. This counted for one credit. Students could take the tests as often as they wanted. I watched students open a question -- not read the question nor the choices -- and just guess. If the answer was right, the noted the choice. If it was wrong, they noted the choice. When the question came around again, if they had gotten it right, the stayed with choice; if they had gotten it wrong, they tried a different choice. They were hacking the quizzes to get high scores.  Now, this behavior happened in part because of how the site was used, and how the lab was graded. But often teachers do in fact, where free sites support doing so, grade based on exercise activity. Does the site in use support the grading logic?

And most importantly . . .
The learning factor: A lot of sites are a lot of fun -- -- is a hoot.  And a lot of sites (see Exercise Central) aim for friendly and engaging (you be the judge) but more traditional in tone and approach. Whichever the approach, is there a way to tell whether the use of the site teaches anything?

For example, grammar sites of the kind we're discussing don't teach proof-reading. And having students do a ton of exercises does not mean you'll suddenly see error free prose. You may see more constricted writing, shorter pieces, loss of details, less ambitious sentences as writers seek to avoid all the grammar issues they've been drilled on, but even then, writing teachers who have students do lots of exercises regularly observe, and sometimes complain, that final drafts still come in with errors.

So what do exercise sites teach? Are they primarily teaching grammar terms and standard edited English rules about punctuation and spelling? Is the test of effectiveness of a grammar exercise quiz site that ability of a student to get better scores on grammar exercises tests?