Observing Competence

Much of my work is about providing students and scholars with occasions to experience their competence as writers. This also means I provide them with occasions to experience their incompetence. If you want to know how to improve at something, you have to give yourself opportunities to run into your limits. Give yourself conditions under which a task can reasonably be completed and then make a sincere attempt to do so. Now your success or failure tells you something. But do keep in mind that the attempt will rarely be a complete failure or a complete success. The idea is to discover more exactly where you can improve.

When it comes to writing, and especially academic writing, the composition of a paragraph offers such an occasion. Any attempt to write a paragraph about something you know will tell you something about how well you write. It is important here not to attempt to write something you merely should know, something your teacher has assigned or your editor has demanded. In such cases, your knowledge is part of what you are testing. If you find it difficult or even impossible to write something under these conditions, you may be running into the limits of your knowledge, not your writing ability. So pick something to write about that you know.

Also, make sure you give yourself a specific time and place (a moment) in which to demonstrate your competence. A good academic writer should be able to produce a paragraph in 18 or 27 minutes (followed by a 2- or 3-minute break). If you don’t give yourself a reasonable amount of time, you’re not really demonstrating a lack of writing ability. If you give yourself too much time, you’re not demonstrating a skill that is very useful in the long run. When the problem of knowing something has been settled, it shouldn’t take you two hours to write 180 meaningful words about it. If it does, that’s what you want to work at.

This is the most important thing about observing your competence as a writer. You have to set up an experience that is repeatable. You have to be able to sit down “at the machine” on a regular basis and do the same thing, just as an athlete can drill and a musician can practice. Practicing means working under conditions where the difference between two sessions is mainly your own competence, which is gradually improving. (Actually, you’ll experience setbacks too, but in the long run, if you’re doing it right, you should experience getting gradually better.) You don’t want to sit down and do something where success depends entirely on situational factors. You want your success to depend mainly on your skill as a writer.

Under such conditions, you’ll notice, your failures and disappointments become meaningful as well. In fact, you will be learning from them whether or not it seems that way in the moment. Athletes get stronger even when they’re having a bad day. Musicians become better even if they spend the entire session making mistakes. The next session will be different and the difference is simply the experience of the last session. It’s probably true that an important difference between people who are very good at something and people who are merely capable lies in how willing they are to experience their incompetence on a regular basis. So give yourself a moment or two every day in which to observe how good you really are. It takes a little courage, but it gets better.

Demystifying Competence

Imagine a class of a twenty students. Your task is to grade their ability to play the violin on a normal curve. How many notes would you need them to play in order to do this? How much information would you need in order to assign 4 As, 6 Bs, 8 Cs and 2 Ds or Fs? The answer will of course depend on the distribution of ability among the students.

For example, you might ask them simply to play a high B flat. Some of the students, having never played the violin, might fail at this task completely. They wouldn’t understand what you are asking them to do. If all 20 of them happily do as you ask, however, you know that they all have some familiarity with the instrument. Still, even one tone can tell you a lot about how well they can play. You could ask them to hold it until you tell them to stop. The quality of the tone they produce, and their ability to sustain it, is good indicator of their basic skills.

But suppose they all do this so well that you can’t really tell them apart. Maybe you’re beginning to suspect that you’re dealing with a group of conservatory students. At this point, you might tell them to play a B flat major scale or a phrase from a piece of music. Or you might sing a tune for them and ask them to play it back to you. Or you might play something for them and ask them to transcribe it. There are lots of simple exercises that will reveal what level the students are at relative to each other.

I’m bringing this up, not because I know anything about playing or teaching the violin, of course, but in order to say something about writing instruction. I think we let competence in this area remain way too mysterious. Indeed, university teachers often don’t know very much about how good their students are at writing at the beginning of a course, which, it seems to me, is a bit like not being able to make any assumptions about how well music students can play their instruments. We need to demystify the craft of writing. We have to insist that, once students have gotten to a particular level at university, they should know how to write a coherent prose paragraph. We should also be able to demand some content knowledge.

For example, most students at a business school will learn some organization theory during their first year. At the very start of a second-year management course, then, the teacher should be able to demand they write a short in-class essay answering the question, “What is an organization?” Give them forty-five minutes to write 1 to 5 paragraphs . The ability to complete this assignment will be distributed throughout the class. Some will do very well, others will fail completely. Some will demonstrate their ability to write but also their ignorance of organization theory. Others might demonstrate familiarity with the literature but exhibit weak writing skills.

Importantly, this will be as clear them as it is to you. I want to spend a few posts this week working through the implications of this fact for writing instruction.

Ezra Zuckerman and the Literature

I was recently asked for advice about how to write a literature review and, as always happens when this subject comes up, Ezra Zuckerman’s advice leaped to mind. “Never write a literature review,” he says. “They are boring.” It’s the last of his ten “Tips to Article Writers”, which he published online about ten years ago. I decided to go back and have a look at the whole thing, and it didn’t disappoint. It is very, very good advice. In fact, it’s such good advice that we should think seriously about treating them as more than mere “tips”. I think we should consider them as possible norms for research writing.

Ezra’s tips are a great way to specify what it means to say that scholarly writing is the art of writing down what you know for the purpose of discussing it with other knowledgeable people. It is not the art of impressing strangers with your intelligence or turning the world as we know it on its head. It is certainly not best understood primarily as the means to tenure. When you are writing an article you are trying to make a contribution to a few dozen people whose names you know and whose opinions you respect. You are opening your research to their criticism and challenging them to see the world as you do.

One of the themes that runs through “Tips to Article Writers” is a respect for the knowledge that your reader brings to your paper. The so-called “literature review” is really a theory section, and this, in turn, is best understood as the basis on which to construct a “compelling null”. Indeed, the null should be compelling enough to be worth saving. “The author’s job is to explain to the reader that s/he was right to believe x about the world, but that since x doesn’t hold under certain conditions, s/he should shift to belief x’.” Too many papers reject a null that is so ridiculous that the reader you actually want to reach, i.e., talk to, would see it as an affront. Don’t write as though you’re the only one who has thought carefully about the subject and everyone else should just feel lucky to finally be told what to think.

This also forces you to give the reader substantive, not merely “aesthetic”, reasons to accept your conclusions. It’s not the beauty of your theories that should compel us to adopt your point of view but the truth of the claims that your theories help you to make. Too often, Ezra tells us, “the contribution tends to be hollow because the end of research (figuring out how the world works) is sacrificed for the means (telling each other how much we like certain ideas).” All which are reasons not begin with the state of “the literature” but with the state of the world. Think of you and your readers (your peers) as people who are working together to solve a real problem. Don’t think of your writing as a way of getting into the good graces of some group of people who have the power to determine the course of your career. It’s bad for your style.

A Simple Assignment

Here’s a simple in-class assignment that would test an important competence in your students. It will work best in a class of around 20 students. If your class is bigger than that, simply divide them (randomly!) into groups of about that size.

At least one week before doing this, assign a text to the students that you think is of reasonable quality and which has substantial knowledge content (a book chapter or journal article you like will do fine). Let the students know that there will be an in-class assignment on this text and that it will count towards their grade. (I’ll let you decide whether to explain the assignment to them in advance after you’ve seen my idea.)

The assignment begins by giving them a key sentence: a simple declarative sentence that they should know is true after reading the assigned text. It should be substantial enough that it can be supported, elaborated or defended in a paragraph of at least six sentences and at most two-hundred words. Give them 27 minutes to compose such a paragraph. This must be done on some sort of digital platform for this assignment to work. Once they have submitted, distribute all the paragraphs to all the students anonymously. (If you have a class of much more than 20 students, distribute them within the groups you randomly assigned people to.)

Now give them  half an hour to rank the paragraphs from best to worst.

That’s it. Here’s how you grade the result: 50% of the grade is given for the paragraph itself. It is graded on a curve after you have ranked all the submissions (in a class of 20 students, for example, you might give 4 As, 6 Bs, 8 Cs and 2 Ds or Fs; or set the curve as you choose).* 50% of the grade is given based on how accurately they predicted your grade with their ranking of the paragraphs (here you just convert their rank ordering into grades and count how many they got right; this grade should not be curved since, in principle, the students can perform equally well or equally badly). You can choose to work out the final, combined grade on a class curve, or simply average the two grades.

I firmly believe that this assignment doesn’t just incentivize learning, but also constitutes a learning experience in its own right. I’m happy to discuss it.

_____

*Update: Though this is my preference, and does make it easier, you don’t actually need to do this on a curve. So long as you have a reasonably normal distribution of grades this assignment will make sense.

What Did You Know Last Week?

Or what did we know a century ago? It still amazes me to think that in 1918 we did not know that Andromeda is a galaxy outside our own. We thought that the universe was just a collection of stars. We didn’t know that we live in a spiral galaxy. Until 1953, we thought Andromeda was only one and a half million light years away. We now know it’s 2.5 (plus or minus .11) million light years away. Going further back, it wasn’t until 1610 that Galileo was able to show that the band of light in the night sky we call the Milky Way is actually a collection of individual stars. This raises a question: did the great astronomers of the past — Galileo, Newton, Halley — know anything about the stars? They were wrong about so many things! What did they really know?

The answer, of course, is that they knew a great deal. It’s just that today’s astronomers, building on everything that has been known before them, know much, much more about how the universe is organized. But I raise the issue for a reason. Sometimes I get the sense that scholars dread the prospect of writing about things they thought were true even just last week. They are worried that this knowledge is already out of date, already rendered irrelevant by some other study that someone else has done. In 1785, William Herschel drew the first map of the Milky Way (thinking it was the whole universe) and put our sun near the center of it. Two hundred years later, even a writer of light entertainment like Douglas Adams knew better,  describing it as “a small unregarded yellow sun” “in the uncharted backwaters of the unfashionable end of the western spiral arm of the Galaxy”. Herschel must be feeling pretty foolish!

Of course not! The fact that knowledge is constantly expanding (like our universe) should not make us embarrassed to claim to know anything today. Nor should we be particularly concerned with how long ago we last checked to be sure we were right. Yes, there are claims you can make about any number of things that everyone thought were true five years ago, but that someone, somewhere, working at the cutting edge of their field is just now becoming aware is wrong. Even profoundly wrong. But their discovery needs time to work itself into our total vision of the world in which we live. Until the news reaches you, you are perfectly in your rights to claim things are as you believe they are. Don’t feel the slightest guilt about it. Just sit down and write what you think and why you think it.

A good way to train this attitude is to always be writing your paragraphs about something you knew was true last week. Never try to write at the cutting edge of your own cognitive process. (That is, don’t try to write for publication at this edge; if you want to use writing to show yourself what’s on your own mind, that’s entirely your business.) Spend some of your research time there, of course. But don’t think every sentence you publish has to be “current”. It just has to accurately reflect what you knew at some reasonably recent moment before your paper was published. That’s the important thing: all published papers — most of what we read — gives us access, not to the present state of mind of one our peers, but to their past state of mind. Get used to writing about things you knew and your writing process is likely to feel less strained.

It is said that Edwin Hubble “settled” the debate about the shape of the galaxy in 1925. We’re not going to ridicule someone who weighed in on the “wrong” side earlier that year, right? Everyone was contributing to the debate, and even when they were getting things wrong, they were demonstrating how knowledgeable they were about many other things. So lighten up! You’re wrong about most things. Tell them, yes, sure … you’re as wrong Aristotle once was, as wrong as Ptolemy and  Newton! There’s nothing to be ashamed of.