Many years ago, while I was still a PhD student, a mentor of mine said something very wise about examination. “An exam is an opportunity for the student to demonstrate their knowledge and intelligence to the teacher,” he said, “not for the teacher to expose the student’s stupidity and ignorance.” This is something that both student and teacher do well to remember and, as much as possible, ensure is true. After all, we’re all more stupid and ignorant than we’re intelligent and knowledgeable (on most things, most of the time, we’re not the leading scholar in the field) and an exam is intended to test only what we have most recently (been) taught in a particular discipline. If both teacher and student approach it with the right attitude, it can be one final learning experience in semester that was, hopefully, full of them.

In my last post I described what I believe to be an almost ideal exam at the end of an ordinary one-semester course (e.g., about Hamlet) during which students were asked to read and understand a variety of materials and write a series of essays about them. In my fantasy of the course, they have also been strongly encouraged to compose their paragraphs very deliberately. The learning objectives are therefore quite clear. By the end, they should be able produce an articulate and knowledgeable essay that answers a question that naturally arises from the course content. For the exam, they’re given three hours to write such an essay and are then examined orally on that basis for 20 minutes. In this post I want to say a few words about how to grade the performance.

Ideally, you will have divided your class into groups of eight students to be examined on separate days. (Less ideal conditions can work too.) In the morning they show up as a group and are given a prompt (a different prompt for each group must be designed). After three hours, they hand in their papers. Then, in the afternoon, they’re each given a half-hour slot for an oral exam.

In the first ten minutes (before they enter the room) you read their essay. (They are of course encouraged to do the same.) Since it’s is no longer than 1000 words, you should be able to get through it in that time and even form some thoughts about the competence of the student, both as a writer and as a scholar of the subject. Most importantly, it should give you some ideas for an interesting conversation with the student. If it doesn’t, that’s already a weakness of the student’s performance. Note down whatever reactions of this kind you have, then invite the student in.

Now, talk to the student about the ideas in the paper for twenty minutes. Do not evaluate the paper itself as a piece of writing; don’t talk to them about the paper and the rhetorical choices they made in it. Talk to them about the ideas they expressed. If you think they got something wrong, feel free to explore that with them and give them a chance to correct it. But what you mainly want to do is to poke around looking for the all the things they know and are able to talk intelligently about. If you think they can handle it (which is already promising for their grade) then challenge them directly on points you disagree with (especially if you won’t hold it against them when they stand their ground). Good students should expect to have a spirited (but never heated) debate during this examination. Average students should expect to have a conversation about a topic that both participants are familiar with. Poor students should expect to have a boring conversation about something they know and care little about.

Keep some notes to remind you of anything remarkable that happened during the exam. When the twenty minutes are up, thank them and bid them a good day. Score their perfomance on a scale from 0 to 100. Your criteria are entirely up to you to decide. The point is just that there has to be such a thing as a good, moderate, and bad performance. Now quickly reflect on your first impressions of the paper. Then give the paper a score as well, which may be higher or lower than the oral score. Did the conversation go exactly as the paper got you to expect. Did it go much better? Or did it go worse? The paper score will reflect the discrepancy. Note it down and put the paper, your notes, and the scores, on the “examined” pile. Next student!

When all the exams have been completed (which may take several days), you’ve got a pile of papers, notes from the oral exams, and numerical scores summarizing your impressions. At this point you could simply average the two scores and translate them into letter grades on whatever scale you already use. If you have time, and think it’s a good idea, you can also reread all the papers one more time and give a final grade in the light of the whole experience.

If you’re allowed to do so (in Denmark we are not) I would encourage you to “normalize” the grades — that is, distribute As, Bs, Cs, Ds and Fs on a curve. Let the students know that their challenge for those 3 hours + 20 minutes is to be as articulate and knowledgeable as they perhaps will ever be on the subject. Let them know that any training they do (whether in writing or conversation) will probably benefit them. It’s the students who are best prepared and in best shape for this exam that will get the top grades. All the effort they make, both in the moment and throughout the course, will be rewarded, because it will make them stand out relative to their peers, i.e., the community of those “other knowledgeable people” that their own knowledge is part of.

That’s it. That’s my suggestion for how to grade the ideal exam. Your comments are more than welcome. From the middle of next week, I’m going to take a break from blogging and tweeting until the new year. I will return with a post reflecting on Ian Bogost‘s and Marc Watkin‘s optimism about the, let’s call it, “creative destruction” of the college essay by artificial intelligence. (See also Stephen Marche and my conversation with Charles Knight, Anna Mills, Annette Vee and Marc Watkins.) My approach to writing instruction and examination does, indeed, seem a bit endangered by ongoing technological developments. But I’m not ready to give up yet. Let’s see what 2023 brings!

Suppose you are trying to teach your students to compose themselves in the manner I’ve been suggesting over the previous three posts. And suppose you’re not just trying to teach them to write in general, but you are trying to impart knowledge to them as well. You may be teaching them anything from organization theory to Elizabethan drama, and along the way you’ve been assigning essays that, you tell them, should consist of a series of coherent prose paragraphs that together present an argument. You’ve been encouraging them to think of a paragraph as a composition of at least six sentences and at most two-hundred words — about half a page — that says one thing they know (about the content of the course, of course) and supports, elaborates, or defends it for the purpose of discussing it with another knowledgeable person. That person, you have been telling them, is best thought of as a fellow student in your class, a peer. You and your students are, in this sense, now on “the same page”.

In this post, I will suggest a form of examination that I consider essentially ideal, even if we had no worries about plagiarism or artifical intelligence, but one that the increasingly sophisticated technologies in this area now make virtually necessary. That is, I’m hopeful that the fact that the take-home assignment no longer constitutes a serious test of the student’s knowledge of a subject or ability to write about it will force us to adopt a form of testing that was always much more serious. In fact, it’s one that I suggested a few years ago might go some way towards reducing the problem of impostor syndrome later on in an academic career. If done right, the competent student will literally feel like they have passed through an impostor filter. I will desribe the exam here and then discuss it in greater detail in a later post.

I’m assuming that the exam comes at the end of a one-semester course. You will have taught the course content as you think best and, like I say, encouraged them to write in orderly writing moments as I suggest. You will also have told them about the structure of the final exam, which will have a significant influence on their final grade. The exam that I will now describe will hopefully have impressed the importance of training on them; they will want to have their minds and their prose “in good shape” to be able to perform the task that is coming.

The exam has two parts.

The first part of the exam is a 3-hour, 1000-word (max.) invigilated essay. Under controlled exam conditions, using only materials and equipment you provide to them, and giving them no connnection to the internet or each other, the students are to respond to a prompt that you give them. You may give them three prompts and let them choose one, but you should not let them write about anything they want. They need to show you that they can quickly recognize what you are asking them to do (which should of course be something that is reasonable to ask of a student that has dutifully followed your class). Then, given those three hours they should be able to compose at least five coherent paragraphs that consititutes an intelligent, knowledgeable response to your prompt.

The second part of the exam is a 20-minute oral examination based on the submitted essay. You might be able to examine ten to fifteen students in a day in the manner I propose, depending on your stamina. Each session begins with you taking ten minutes to read the 3-page essay the student has written. They then enter the room and you discuss it with them for twenty minutes. That’s it. You make some notes, score them, and assign grades on the basis of your notes and scores when you’ve been through all the students. If you’re allowed to do so, I suggest giving the grades on a curve. More on this in the next post.

The test will work best with as little time as possible between the written and oral exams. You can expect students to spend the time between them reading up on the material they will be discussing, but you don’t want the whole oral exam to consist of them correcting themselves on matters they have since learned more about (and will forget again in due course). With a class of twenty-students you can get through this over two or three days. Out of fairness you don’t want some students to have a lot more time to prepare for the oral exam than others.

But there’s a simpler and less exhausting way through this, actually. If you’re willing to write a lot of prompts, you can examine an arbitrary amount of students in an arbitrary amount of days, spread out over an abitrarily long period. Simply split the students into groups of 8, assign them exam dates where their group writes in the morning and is then examined orally after lunch (between, say, 13:00 and 17:00). You read eight essays and have eight more or less engaging conversations with your students on some number of afternoons during the exam period.

Like I say, I will offer some reflections on this in a later post. I’m sure the idea itself provokes reactions, positive and negative, in many of my readers. I would love to hear about them in the comments.

My writing workshops take participants step-by-step through the process of preparing, composing, and reviewing a paragraph with a peer reader. In this post I want to say something about that last component.

In the workshop, it only takes 6 minutes, during which each participant receives three minutes of feedback from another participant. In “real life”, as part of your regular writing practice, I suggest you work yourself up to being able to receive feedback for 6 and eventually 9 minutes on an individual paragraph, which, remember, took you 18 or 27 minutes to write. By giving your paragraph to another person to respond to for about a third of that time, you’re giving yourself an opportunity to see whether it works. You’re testing it as a piece of of writing. On the basis of this experience, you can then plan your future writing moments, and you can begin to think about how to arrange your paragraphs into longer essays and papers and chapters and books. I’ll return to that at the end.

When doing this exercise, please remember that it is, precisely, an experience, not a judgment. You are not asking your reader to tell you whether you are a good writer or even a good thinker; you are not asking them to form an opinion about you as a writer, or about your paragraph as a piece of writing. You are literally asking them to read you, and to let you watch and listen to them do it. You are going to give them a paragraph that you have composed deliberately to support, elaborate, or defend a key sentence, and you’re going to find out whether your reader thinks you supported, elaborated, or defended that sentence. If the reader thinks you’ve done something else, that’s just a fact you have a face; next time, you’ll try to write a paragraph that is clearer about its intention and posture. Then you’ll test that one.

Choose a reader that can represent the sort of reader you had in mind while writing. Remember that this should be a “peer”: someone in your discipline that you respect as an intellectual equal, someone who should be able to understand what you’re trying to do, and someone who is qualified to tell you that you are wrong. Of course, you don’t think you’re wrong; if you are very uncertain about this, you’ve chosen the wrong paragraph to share; it’s just that your reader is someone you would take seriously if they thought you were. Indeed, if your paragraph is defending an idea you expect your reader to disagree with, then you want the reader to think you’re wrong. That was the point of writing the thing! In that case, the last thing you want is for them to think you’re right, which would mean they had misunderstood you (or you had misunderstood them). But if that’s the case, so be it; you want to know the truth about your reader. Sit back and learn.

Give your reader your paragraph, either on a screen or on a piece of paper, but in a format that is plain and easy to read. Set a timer for 6 or 9 minutes. The session begins with the reader reading your paragraph out loud. They have never seen this paragraph before so they have only your words on the page to work with, in the order you put them there. You are going to experience their struggle to pass them through their mind in a meaningful way. You will learn the extent of their “interpretive charity”; you will hear it in their voice and see it in their face. You are hearing and seeing a human being make sense of your words in real time. It can be an intense experience (if you let it) but it only lasts a minute or so. Take it in. Do not help the reader get past the typos and spelling mistakes you now suddenly realize you left in — let the reader puzzle over them and make the necessary adjustment and move on. Do not nod or smile or groan or cringe or otherwise communicate to your reader whether they’re doing it right. Put on a pokerface. Relax. Just experience the act of being read.

“I was never here,” Ben Lerner says to his reader. “Understand? You never saw me.” It’s like that.

Now, have your reader tell you what the key sentence is. If you have written the paragraph deliberately, like I told you to, then you know what the right answer is. You know what you were trying so say and which sentence in the paragraph declares this intention simply and plainly. Just have your reader point that sentence out, whichever one they think contains the “take home message” of the paragraph. If they’re not sure, let them think out loud about what it might be, let them weigh their options until they make up their mind. As before, maintain a stoical silence. This isn’t a conversation, it’s just you watching someone else puzzle out your meaning in real time. It may even be that they decide you’ve got two key sentences and that it should really have been two paragraphs. All this will be useful to you the next time you write.

You had decided not only what to say, but also how to say it. You adopted a deliberate rhetorical posture to try to get the reader to believe, understand, or accept something for the sake of argument. So you’ll be interested to hear whether your reader thinks you are trying support, elaborate, or defend the key sentence they have just identified. How does the reader feel you are addressing them? Or maybe the reader feels like you have posed the fourth difficulty: you are assuming that the reader is bored with the truth of the key sentence and you are trying to help them find the same excitement that you feel about it. Or, again, maybe the reader isn’t quite sure what you’re trying to accomplish. It’s not clear why they had to read this paragraph. These things are sometimes hard to hear, but ultimately good to know.

So far, this may have taken three, four, or five minutes. In the time that remains, let the reader say whatever comes to mind, preferably about the content of the paragraph, not its form. Of course, the most striking thing about the paragraph may be how well or badly it was written, but in both cases what you’re listening for is how your language got in the way (perhaps by being a little too clever!) of your ideas. Ideally, the reader will spend the next few minutes telling you what your paragraph made them think about, wonder about, worry about, curious about, etc. If that is the writing, your writing needs work. Much better if the reader says, “Nice try! But I still disagree. And here’s why…” Or the reader may say they have finally understood something that has puzzled them for a while and they may tell what that puzzle was. Or the reader may express doubts about the quality of your evidence and suggest methods that would be better suited. Once again, you’re getting information that will be useful to you the next time you write.

When the time runs out, stop. Sit in silence with your reader for a minute. Let it sink in. Then say, “Thank you,” be thankful, and let both of you get on with your days. You have learned something about your writing by learning something about your reader. As Virginia Woolf said, “Knowing who you’re writing for, is knowing how to write.” These six or nine minutes taught you how to write in precisely that sense.