“Charity I have had sometimes…” (Ezra Pound)

I’ve been thinking about Twitter lately. When Carlo Scannella suggested, back in 2008, that Twitter might be an “oral” (sub-)culture, not a strictly “literate” one, the project was still in its infancy. Like him, I think a strong case can be made that the way people ordinarily use the platform does not exhibit their literacy. That sounds almost like I’m trying to be funny, but there’s a serious meaning behind it. In this post, I want to say a little more about that but keep in mind that Scannella has made a similar argument for blogging. I’ll look at that another time, as a way of picking up a project I started on my now-retired blog but never developed fully.

Let’s begin with the strongest argument for considering Twitter a site of literacy practices. There are explicitly literary projects on the platform, like those of Teju Cole and Eric Jarosinski. These are clearly meant to be read, however ironically, as writing within a symbolic order that connects it to the full corpus of our literature. In other words, Twitter certainly can be a site of literacy practices–it can even be used to produce literature. Of course, you don’t have to produce literature in order to demonstrate literacy, so the question remains. Is Twitter willy-nilly a site of literacy–simply, for example, because it requires people to use written signs?

Well, first of all, it doesn’t actually require that, does it? Twitter now lets people share images, still and moving, as well as recorded sounds, as easily as strings of words. It is possible to retweet someone else’s picture, with a comment that (literally!) is nothing more than a thumbs up. Being able to use Twitter, then, requires a very limited mastery of written signs. But that’s too easy an argument, I’ll grant. The interesting question (especially back in 2008) is whether Twitter is a literate culture even when it uses written signs, letters, words, sentences.

My first hint that it is not comes from my own experience. I have found distressingly many typos in my tweets. Apparently, I don’t proofread them properly; I just fire them off — just as I sometimes deploy imperfect sentences in conversation or extemporaneous lecturing. I treat my sentences on Twitter as utterances that I don’t know how will turn out when I begin. I don’t seem to feel nearly as compelled to edit them as I do when I’m actually writing. I’m also not especially bothered by other people’s typos. While they might make me stop reading a book they had published, spelling mistakes and missing words don’t make me abandon someone’s Twitter feed. It seems perfectly understandable and, where errors make the tweet difficult to understand, I proceed as I would in conversation, asking my interlocutor to clarify what they must have meant. I don’t “correct” them. I think that’s a telling sign.

The same can be said about serious disagreements over facts and ideas. I approach any particular utterance as a tiny piece of a larger puzzle that doesn’t have to make sense on its own but will only be given a meaning as the conversation proceeds. Twitter is, ideally, located on what Rosmarie Waldrop called “the lawn of excluded middle”, “where full maturity of meaning takes time the way you eat a fish, morsel by morsel, off the bone.” Or, as Pound would perhaps have suggested, we must read each other’s tweets with charity, for maximum truth and meaning. Perhaps “it coheres all right / even if my notes do not cohere.”

But now we come to crux of the issue. Are we engaging correctly with what Twitter is when we seize on an individual tweet and denounce it as an expression of someone’s racism, sexism or other vile disposition? Should we not approach an out-of-context tweet initially as a mere fragment of a conversation we’ve overheard? “Did s/he really say that?” does not have to be a rhetorical question. Perhaps there’s a typo or a missing word that accounts for the apparent error in thought, i.e., “incorrectness” of opinion. Perhaps it “coheres all right” in the larger of whole of a detailed argument.

One of the characteristics of a literate culture is that it has a “record”, a body of “documents”. But not every utterance of a literate culture is, in fact, a document, even when, technically speaking, “written down”. Not every receipt or shopping list or scribbled note to oneself is a record of what a culture thinks and feels. We are pretty good at distinguishing passing remarks from serious propositions in speech. (We get a bit confused, however, when someone makes a recording of them.) Perhaps we need to practice greater charity in our approach to tweets. They do not document what any of us believes. They are merely part of the stream of language within which we have to find our composure. It is when we find it that we can produce a proper “text”, i.e., practice our literacy. Only then are we actually writing.

Researchers are often advised to have an impact “beyond academia”. Indeed, Andrew Gunn and Michael Mintrum (2016) see the so-called “impact agenda” as focused on “university-based research that has non-academic impact, meaning it delivers some change or benefit outside the research community.” The basic idea is that the value of our universities lies in their contribution to society, for which the economy often serves as a proxy.

The EU’s 2020 target is to spend 3% of GDP on research and development, about a third of which is likely to be invested in university-based research (the rest is government and business sector R&D.) While 1% of GDP may not seem like much, it’s certainly a significant amount of money (about €150 billion) and it’s reasonable to expect a return. Indeed, Gunn and Mintrum detect “an impatience to see rapid and significant returns on the investments being made.”  This agenda can certainly be felt by researchers, whose performance is increasingly measured through indicators of societal impact. They are now acutely aware of their stakeholders outside the university and they are earnestly engaged in catching their attention. But perhaps the impatience that Gunn and Mintrum detect in all this activity should give us pause.

Consider another target in the Europe 2020 strategy: 40% of people aged 30–34 are to have completed higher education. Already today, about  25% of 20-29-year-olds are enrolled in university in most EU countries, so this target (which is already close to being reached) is entirely realistic. In any case, a few percentage points either way won’t affect the point I’m going to try to make. With such a high percentage of the population being either the product of higher education or in the process of getting such an education, it seems odd to get the people (researchers, faculty) who are spending a mere 1% of GDP to invest a great deal of their time (a portion of that economic activity) thinking about how their research makes a contribution “beyond academia”.  If the  researchers confined their efforts to their peers and students, their impact would by no means be insignificant.

With the passage of time this impact would spread throughout the population as the students graduate and begin to apply the skills, facts and theories they have learned. While this will be slower than the impact a piece of research might have by directly shaping government polices and corporate strategies, it will be deeper and more lasting. This is especially true when we consider that the 40% of thirty-somethings who have completed higher education will presumably have a disproportionate impact on GDP (i.e., as a group they will contribute more than 40% of their cohort’s GDP-contribution). Finally, by letting students round off their understanding of the theories in “the school of hard knocks” before they are implemented at the policy level some of the worst excesses of Ivory Tower thinking can be avoided.

Imagine for a moment what would happen if academics came to measure themselves only by the external performance indicators stipulated by the “impact agenda”. (Perhaps we could add to this an interest mainly in student evaluations as regards their teaching.) Suppose it was a matter of complete indifference to them what the students actually learned (so long as they were “satisfied” while at university) and what happened to the society and economy in the long run (so long as they were feted by bureaucrats and entrepreneurs.) In the short term, it is possible that high-quality research would inform well-crafted policy initiatives. But, after a time, researchers will learn (for they are not stupid) how to game the system to maximize their immediate, apparent impact. Mediocre but shiny ideas will crowd out the ideas that might actually work in practice. Many mistakes will be made. Meanwhile, since the minds of the students have been mostly neglected, the next generation of researchers won’t be up to the task of delivering what the best minds of the last generation were capable of. The race to the bottom will be on.

To avoid this, I would recommend evaluating academic research by measuring the value of higher education according to its long term impact on the economy and society, not the immediate measurable effects of discrete pieces of research on the policy process. I think the short-term indicators (student satisfaction, policy impact) are ultimately distractions, and we might find that academia is performing well by them but driving the culture into the ground. If researchers think the most important audiences are outside their own disciplines and classrooms, they will stop giving their best thinking to their students and colleagues, waiting instead for them to occupy some position of power wherefrom they can commission a policy paper. Education becomes merely a way of conditioning students to draw on “academic” expertise when they get out into the real world. “Life -long learning” becomes a kind of conditioned dependence on epistemic authority. And peer review becomes the mutual appreciation society it has long been accused of being.

In short, I propose we curb our socio-economic enthusiasm, our pursuit of “impact”. Let’s direct our energy back into academia and wait for it to diffuse organically throughout society. As T.S. Eliot once said, you don’t make flowers grow by pulling on them.

Update: See also Tina Basi and Mona Sloane’s post at the LSE Impact Blog.

There is an important connection between my thoughts on peer grading and my thinking on peer review. Peer review in the familiar sense of having two or three qualified scholars read and comment on a paper before it is published in a journal was once necessary because publication was a costly process and there had to be some fairness about who got access to the scarce resource of printed pages. The process had to be blinded (in my opinion) to encourage frank assessment of work and discourage nepotism that would have wasted valuable space. Similar practical considerations justify the traditional approach to grading, in which students submit a single copy of their essay and the teacher gives it a grade that is none of anyone else’s business. For centuries it was simply impractical to demand that students produced 25 copies (or more for larger classes) of their essays. But information technology has made the cost of publishing and making copies negligible. Or, rather, once a platform exists, the cost of publishing an additional paper or making an additional copy is essentially zero.

This makes it possible to imagine an “open” peer review system in which papers are posted online along with the reviews as they come in. I’m not against such a system but I’m not sure it’s actually necessary. To me, it seems easier just to have people post their work to their own website (which may be personal or hosted by a university, but should in any case link to an institutional repository to provide a stable URL) and see what other people do with it. “Publication” just means uploading it to a server and sending an email announcing it to your peers. All review in this world would be “post-publication”. That is, our ideas would be evaluated as we would, ideally, evaluate the results in published papers. Today, unfortunately, the “peer-reviewed” stamp has a tendency to discourage serious criticism of published work. That’s an important reason we’re in the middle of a “replication and criticism crisis”.

Now think of students and their grades. They work alone (or in groups isolated from the rest of the class) on papers they submit to the teacher. After some time they get a grade that passes some final judgment on them. They do of course hear about the grades their peers received on the same assignment, but how many of them then sit down and read those papers to learn what a “better” or “worse” paper looks like? No, once the grade is received, closure has (in the vast majority of cases) been reached. Just as too many researchers put their published work behind. it’s time to focus on the next assignment, the next grant proposal.

But we have long had the technology to make a completely different approach possible. In addition to their assigned reading, students can be asked to read each other’s work. They can be expected to “publish” weekly assignments to a class website, where their peers can comment on them. Here, they can teach others what they’ve learned from their readings and discuss the issues that have come up in class. And they can do so by way of acquiring the writing skills that scholars have. The peer-feedback can be as mandatory as the assignments themselves. And it, too, can be graded. Indeed, even the students can grade each other, trying to apply the same criteria that their teachers apply. And the can in turn be graded on their ability to do this, their eye for quality work.

I have long suspected that the culture of peer review is an extension, or continuation, of the culture of examination. There are many who argue that the peer review system is broken, or at least seriously bent. Perhaps we can only change the way we engage with the published work of our peers by changing the way we engage with the work of students, and the way we ask them to engage with each other’s. After all, today’s students are tomorrow’s scholars.

Suppose we listened to two pianists play the same sonata. Suppose one of them is a student who has been playing for a couple of years and the other is professional concert pianist. Or suppose we look at two drawings of the same hand, one by a high-school student of ordinary ability, the other by a professional illustrator. I’m going to presume we would quickly agree about which is the better performance. What I want to emphasize is that I can make this presumption without assuming anything about your abilities or claiming anything about mine. We can be less competent than all four of these “artists” and still judge them. We can say something about “how good” they are at playing the piano and drawing hands without being particularly good at it ourselves.

In the cases I’m imagining, our judgment–of relative competence–will often be quite accurate. Obviously, we’ll get into difficulties if we’re given twenty-five professional pianists to rank in order of mastery, but I don’t think we’d be as hard pressed if we’re given, say, a randomly selected group of twenty-five 18-year-old college students. Put them in front of a piano and ask them to play Bach, or give them a piece of paper and pencil and tell them to draw a hand, and we can pretty quickly sort them according to their relative skill at these things. In fact, if we’re asked to assign 3 As, 6 Bs, 8 Cs, 6 Ds, and 2 Fs, we could probably do it. That’s a rather marvelous fact to think about.

Now, if the students weren’t randomly selected but, instead, were taking a piano class or a drawing class, we’d want to have the competence of a teacher to confidently distribute grades like that. But what about the competence of the students? Could students in such a class distribute grades on the curve I’ve proposed? I believe they could, with a certain degree of accuracy, and I believe that the effort of doing so would be instructive–it would teach them something. Indeed, I believe that the ability to accurately discern the relative skill of two pianists or two artists (and therefore accurately predict the outcome of a competition between them) is itself a valuable skill that comes with mastering the relevant art. That is, as I suggested in my last post, it is in principle possible to grade students on their ability to grade each other. Developing their own eye for competence in the disciplines we teach should be part of the learning goals of any course. Even students who would give themselves failing grades in a course can make a qualified guess at what grades their fellow students will get by reading their essays. Just the effort to make that guess in 25 concrete instances will teach them something.

Let me say one last thing. I think it would be good for students to grade a class set of peer essays every once in while, even if just to develop an appreciation for the task that faces their teachers as a matter of course. Students often don’t take the problem of judging the competence of others seriously enough. By giving them this task, and by putting something at stake (i.e., grading them according to how well they predict the teacher’s grades), they will learn to value good, clear writing that is easily judged. After all, the purpose of academic writing is not impress the reader with your intelligence but to open your thinking to criticism from your peers. I think this idea needs to be tried.

I’m happy to hear from people who have.

I’ve been focusing on students lately, not the craftsmanship of accomplished scholars. But it’s important to keep in mind that there is a close connection between what students are taught and what their teachers are able to do. When you give a student an assignment, even one that involves a significant amount of “independent research”, you are telling them to do something that you yourself could do. Like a conservatory teacher or an athletics coach, you need not presume that you could do it better or more easily than them. It’s reasonable for an aging master to be weaker in practice than the young apprentice. The important thing is that the teacher already knows what is possible, and is therefore able to evaluate the result, while students are only just beginning to discover what they are capable of. “The Great Learning,” said Confucius, “is rooted in watching with affection the way people grow.” Learning is always implicit in scholarship, and the student, therefore, is always implicated in the work of the scholar.

With this in mind, I want to suggest some assignments that can train and test the skills that scholars apply to their own learning. By means of these assignments, the skills can be passed on to their students and a culture of learning can thereby be conserved. These are high-minded ideals, but the assignments themselves are altogether practical and grounded.

Assignment #1

Give the students two short texts by writers on opposite sides of a debate that is, or was once, central to your discipline. Let them be around 3000 words long, which implies that they can be read–once–in about fifteen minutes. Make sure that they either are or represent “classic” positions. That is, make sure that the students are exposed to the ideas of two major figures in your field.

Give them a week to read the texts and, on that basis alone, make up their own minds about the issue. What is the debate about? Which arguments do they find most compelling? Who is right and who is wrong? Is there a middle ground that seems more reasonable to them than either of the two positions that are being defended?

Have them write a five-paragraph essay of no more than 1000 words to present their position on the question. Have them read and grade each other’s essays. (You can give each student a number of essays to grade. But in a smaller class of, say, 20 students, you can also tell them to spend 10 minutes on each paper and have them grade them all. That’s 200 minutes of, I hope you will agree, very instructive work. I would recommend having them distribute the grades on a normal curve.)

Assignment #2

Send the students to the library to inform themselves about the history of the issue you exposed them to in assignment #1. Have them locate the texts you assigned and the most interesting (to the student) sources they cited. Have them study the “impact” of the papers you assigned (or the authors who wrote them, or the classics they represented) on the discourse of your discipline by finding papers that cite them.

Give them a week to see what they can come up with. How important is this question today? What is the consensus among scholars? What sort of research has been done to test the relevant theories? What research is being done to extend and develop them? Encourage them to seek out work that disagrees with the position they took in assignment #1.

Have them write a five-paragraph essay of no more than 1000 words to present their results. Demand that they use a particular citation style. Again, have them read and grade each other’s essays.

Assignment #3

Have them reread their solution to assignment #1. In light of their work in the library, what do they think now? Have them rewrite essay #1 to take stock of this research, either defending their position in a more sophisticated way, or taking a new position in light of the better arguments they have encountered.

This time, you give them the grade.

Assignment #4

While you are grading assignment #3, have them grade each other too. This time, however, their job isn’t to evaluate it according to their own standards, but to try to predict the grade that you will give them. This is easiest to do if you grade on a curve, since the students will now have to distribute each other’s papers on a normal distribution. Their grade on this exercise will be based on how well they match the grades you give. If you don’t distribute the whole class set of essays to everyone (perhaps the class is too big), their job will simply be to rank-order them. You will then also have to rank each essay relative to the others.

I’ll let this stand without comment and offer some reflections on why I think this is a great way to spend a few weeks of your student’s time (and yours!) in subsequent posts. Do please offer your thoughts below in the meantime.