There is an important connection between my thoughts on peer grading and my thinking on peer review. Peer review in the familiar sense of having two or three qualified scholars read and comment on a paper before it is published in a journal was once necessary because publication was a costly process and there had to be some fairness about who got access to the scarce resource of printed pages. The process had to be blinded (in my opinion) to encourage frank assessment of work and discourage nepotism that would have wasted valuable space. Similar practical considerations justify the traditional approach to grading, in which students submit a single copy of their essay and the teacher gives it a grade that is none of anyone else’s business. For centuries it was simply impractical to demand that students produced 25 copies (or more for larger classes) of their essays. But information technology has made the cost of publishing and making copies negligible. Or, rather, once a platform exists, the cost of publishing an additional paper or making an additional copy is essentially zero.

This makes it possible to imagine an “open” peer review system in which papers are posted online along with the reviews as they come in. I’m not against such a system but I’m not sure it’s actually necessary. To me, it seems easier just to have people post their work to their own website (which may be personal or hosted by a university, but should in any case link to an institutional repository to provide a stable URL) and see what other people do with it. “Publication” just means uploading it to a server and sending an email announcing it to your peers. All review in this world would be “post-publication”. That is, our ideas would be evaluated as we would, ideally, evaluate the results in published papers. Today, unfortunately, the “peer-reviewed” stamp has a tendency to discourage serious criticism of published work. That’s an important reason we’re in the middle of a “replication and criticism crisis”.

Now think of students and their grades. They work alone (or in groups isolated from the rest of the class) on papers they submit to the teacher. After some time they get a grade that passes some final judgment on them. They do of course hear about the grades their peers received on the same assignment, but how many of them then sit down and read those papers to learn what a “better” or “worse” paper looks like? No, once the grade is received, closure has (in the vast majority of cases) been reached. Just as too many researchers put their published work behind. it’s time to focus on the next assignment, the next grant proposal.

But we have long had the technology to make a completely different approach possible. In addition to their assigned reading, students can be asked to read each other’s work. They can be expected to “publish” weekly assignments to a class website, where their peers can comment on them. Here, they can teach others what they’ve learned from their readings and discuss the issues that have come up in class. And they can do so by way of acquiring the writing skills that scholars have. The peer-feedback can be as mandatory as the assignments themselves. And it, too, can be graded. Indeed, even the students can grade each other, trying to apply the same criteria that their teachers apply. And the can in turn be graded on their ability to do this, their eye for quality work.

I have long suspected that the culture of peer review is an extension, or continuation, of the culture of examination. There are many who argue that the peer review system is broken, or at least seriously bent. Perhaps we can only change the way we engage with the published work of our peers by changing the way we engage with the work of students, and the way we ask them to engage with each other’s. After all, today’s students are tomorrow’s scholars.

Suppose we listened to two pianists play the same sonata. Suppose one of them is a student who has been playing for a couple of years and the other is professional concert pianist. Or suppose we look at two drawings of the same hand, one by a high-school student of ordinary ability, the other by a professional illustrator. I’m going to presume we would quickly agree about which is the better performance. What I want to emphasize is that I can make this presumption without assuming anything about your abilities or claiming anything about mine. We can be less competent than all four of these “artists” and still judge them. We can say something about “how good” they are at playing the piano and drawing hands without being particularly good at it ourselves.

In the cases I’m imagining, our judgment–of relative competence–will often be quite accurate. Obviously, we’ll get into difficulties if we’re given twenty-five professional pianists to rank in order of mastery, but I don’t think we’d be as hard pressed if we’re given, say, a randomly selected group of twenty-five 18-year-old college students. Put them in front of a piano and ask them to play Bach, or give them a piece of paper and pencil and tell them to draw a hand, and we can pretty quickly sort them according to their relative skill at these things. In fact, if we’re asked to assign 3 As, 6 Bs, 8 Cs, 6 Ds, and 2 Fs, we could probably do it. That’s a rather marvelous fact to think about.

Now, if the students weren’t randomly selected but, instead, were taking a piano class or a drawing class, we’d want to have the competence of a teacher to confidently distribute grades like that. But what about the competence of the students? Could students in such a class distribute grades on the curve I’ve proposed? I believe they could, with a certain degree of accuracy, and I believe that the effort of doing so would be instructive–it would teach them something. Indeed, I believe that the ability to accurately discern the relative skill of two pianists or two artists (and therefore accurately predict the outcome of a competition between them) is itself a valuable skill that comes with mastering the relevant art. That is, as I suggested in my last post, it is in principle possible to grade students on their ability to grade each other. Developing their own eye for competence in the disciplines we teach should be part of the learning goals of any course. Even students who would give themselves failing grades in a course can make a qualified guess at what grades their fellow students will get by reading their essays. Just the effort to make that guess in 25 concrete instances will teach them something.

Let me say one last thing. I think it would be good for students to grade a class set of peer essays every once in while, even if just to develop an appreciation for the task that faces their teachers as a matter of course. Students often don’t take the problem of judging the competence of others seriously enough. By giving them this task, and by putting something at stake (i.e., grading them according to how well they predict the teacher’s grades), they will learn to value good, clear writing that is easily judged. After all, the purpose of academic writing is not impress the reader with your intelligence but to open your thinking to criticism from your peers. I think this idea needs to be tried.

I’m happy to hear from people who have.

I’ve been focusing on students lately, not the craftsmanship of accomplished scholars. But it’s important to keep in mind that there is a close connection between what students are taught and what their teachers are able to do. When you give a student an assignment, even one that involves a significant amount of “independent research”, you are telling them to do something that you yourself could do. Like a conservatory teacher or an athletics coach, you need not presume that you could do it better or more easily than them. It’s reasonable for an aging master to be weaker in practice than the young apprentice. The important thing is that the teacher already knows what is possible, and is therefore able to evaluate the result, while students are only just beginning to discover what they are capable of. “The Great Learning,” said Confucius, “is rooted in watching with affection the way people grow.” Learning is always implicit in scholarship, and the student, therefore, is always implicated in the work of the scholar.

With this in mind, I want to suggest some assignments that can train and test the skills that scholars apply to their own learning. By means of these assignments, the skills can be passed on to their students and a culture of learning can thereby be conserved. These are high-minded ideals, but the assignments themselves are altogether practical and grounded.

Assignment #1

Give the students two short texts by writers on opposite sides of a debate that is, or was once, central to your discipline. Let them be around 3000 words long, which implies that they can be read–once–in about fifteen minutes. Make sure that they either are or represent “classic” positions. That is, make sure that the students are exposed to the ideas of two major figures in your field.

Give them a week to read the texts and, on that basis alone, make up their own minds about the issue. What is the debate about? Which arguments do they find most compelling? Who is right and who is wrong? Is there a middle ground that seems more reasonable to them than either of the two positions that are being defended?

Have them write a five-paragraph essay of no more than 1000 words to present their position on the question. Have them read and grade each other’s essays. (You can give each student a number of essays to grade. But in a smaller class of, say, 20 students, you can also tell them to spend 10 minutes on each paper and have them grade them all. That’s 200 minutes of, I hope you will agree, very instructive work. I would recommend having them distribute the grades on a normal curve.)

Assignment #2

Send the students to the library to inform themselves about the history of the issue you exposed them to in assignment #1. Have them locate the texts you assigned and the most interesting (to the student) sources they cited. Have them study the “impact” of the papers you assigned (or the authors who wrote them, or the classics they represented) on the discourse of your discipline by finding papers that cite them.

Give them a week to see what they can come up with. How important is this question today? What is the consensus among scholars? What sort of research has been done to test the relevant theories? What research is being done to extend and develop them? Encourage them to seek out work that disagrees with the position they took in assignment #1.

Have them write a five-paragraph essay of no more than 1000 words to present their results. Demand that they use a particular citation style. Again, have them read and grade each other’s essays.

Assignment #3

Have them reread their solution to assignment #1. In light of their work in the library, what do they think now? Have them rewrite essay #1 to take stock of this research, either defending their position in a more sophisticated way, or taking a new position in light of the better arguments they have encountered.

This time, you give them the grade.

Assignment #4

While you are grading assignment #3, have them grade each other too. This time, however, their job isn’t to evaluate it according to their own standards, but to try to predict the grade that you will give them. This is easiest to do if you grade on a curve, since the students will now have to distribute each other’s papers on a normal distribution. Their grade on this exercise will be based on how well they match the grades you give. If you don’t distribute the whole class set of essays to everyone (perhaps the class is too big), their job will simply be to rank-order them. You will then also have to rank each essay relative to the others.

I’ll let this stand without comment and offer some reflections on why I think this is a great way to spend a few weeks of your student’s time (and yours!) in subsequent posts. Do please offer your thoughts below in the meantime.

I have suggested that the key sentence of the first paragraph of a five-paragraph essay is, either implicitly or explicitly, self-referential. It does not assert the thesis statement of the essay; it asserts that this essay will demonstrate its truth. It refers, not to the world of fact that makes it true, but to the text before us that will show us how true it is. That gives it the task of justifying the writing of the essay by situating it in a practical and theoretical context, outlining the argument, and indicating the consequences for practice or theory or both. I will now suggest that the last paragraph will actually assert the thesis and present (not merely indicate) the consequences.

If the key sentence of paragraph 1 was

In this essay, I will argue that the Great Depression was caused by the monetary policy of the Federal Reserve,

then the key sentence of paragraph 5 will be simply

The Great Depression was caused by the monetary policy of the Federal Reserve.

They do not have to be this obviously similar, but the content should be. Notice that there is no reference to the essay in the latter, no “I have here argued…” or “I have tried to show…” Also, I should say that this is actually best understood as a way of mirroring paragraph 3 of a longer essay with paragraph 39 (out of 40). In a five paragraph essay, the first and last paragraph will do a bit more work than their key sentences make explicit. (You might want to plan this work so that paragraph 1 actually has three key sentences and paragraph five has two, but do remember to maintain a focus.)

Think of paragraph five as having the task of asserting both the thesis and its importance. To begin with, after having written the key sentence, you can just write a version of the key sentence of paragraphs 2-4. You now have four sentences written. Then add three sentences that suggest the consequences of realizing that the Fed’s monetary policy was to blame, whether for scholars or bankers or both (but be clear about who you’re talking about). Presumably, this gives us some guidance about how to avoid similar crises in the future. Tell us what the right policy would have been, perhaps. And what policy we should adopt today.

You have all the material in place. Now work on the paragraph until it feels like a conclusion of your argument. Remember it has three distinct parts or moments: (1) the thesis statement of your essay, (2) a summary of the argument, (3) an assessment of the consequences. It has to make these three moves as though they are one coherent gesture that asserts your overall meaning. One last thing: I’ve given you the steps of a dance or the chords to a song. Please grant that merely following these steps or playing these chords will not necessarily produce a good essay. You have to practice until you can do it well. And you have to have something to say.

We are trying to imagine an undergraduate economics student writing an essay to argue that

 the Great Depression was caused by the monetary policy of the Federal Reserve.

Notice that an economics professor could just as easily believe this and might also write an essay on the subject. We could require both writers to confine themselves to five paragraphs. The professor might find the task easier, and might even do a better of job of it, but it’s a sensible one for both writers. It’s possible to do it well or badly. Doing it well requires the acquisition of a valuable skill.

The five-paragraph essay form demands that we come up with three arguments for our overall conclusion. In my last post, I suggested the following

• In the 1930s, the Fed followed a deliberately deflationary policy.
• As part of this policy, the Fed let the Bank of United States fail in 1931.
• The failure of the Bank of United States precipitated a general collapse of the banking sector.

I’m not entirely confident that I’m right about these three points and whatever debates economists have on this question would probably turn on them, but there can be no doubt that the line of argument is clear. If I can show that these three claims are true, I have made a case for blaming the Great Depression on the Fed’s monetary policy. There are a couple of implicit book-ends, namely, that deflation is the result of monetary policy and that the collapse of the banking sector turned a recession into a depression, and I can make these explicit in my conclusion. What I want to stress here is that each claim can be supported, elaborated or defended in at least six sentences and less than two-hundred words, and these paragraphs, in turn, can be more or less competently written. We could give a student any one of these sentences and ask them to write the corresponding paragraph. This would be a meaningful test, both of their knowledge and of their style.

Notice that the rhetorical posture of each paragraph may be different. The first paragraph may merely detail (i.e., elaborate on) the policy of the Fed, clarifying the sense in which it was deflationary. The writer may assume that it is neither hard to believe nor hard to agree with but that the reader wants to know more. The second paragraph, however, may be written with a reader who needs evidence in mind. Either it’s hard to believe that the Fed would let a bank fail at such a time, or it could be hard to believe that letting a bank fail had anything to do with a monetary policy. The third paragraph, finally, could be be written for reader who has already decided that the failure of the Bank of United States was not as pivotal as the writer thinks. Perhaps the objections to this notion are well-known. The paragraph will then engage with and counter them, perhaps only to let the disagreement stand without it affecting the overall point about the Great Depression.

In his masterful little book How to Draw Hands, Oliver Senior reminds us that “the better drawing is not the more elaborate attempt to reproduce the visual appearance of its subject, but that which is better informed.” This will also be true of these paragraphs. Competence will be revealed in the way the student selects, from the abundance of information that is available to us about the subject, those details that most efficiently establish the claim within the overall line of argument. As I’ve been saying in these posts, it puzzles me how often teachers appear not to see the point of getting students to develop and demonstrate this competence. It seems obviously worthwhile to me.