The Presumption of Criticism

Scholars often make claims based on research done by other scholars. It is standard practice to rely on the work of others to support or frame your own work. This practice is justified by a set of presumptions that it is our obligation, as scholars, to make true. Doing so does not guarantee that everything you read in a peer-reviewed article is true, but it does justify the (measured) confidence with which we draw on such claims when conducting our research.

In  a word, we presume that the claims made in the literature are subject to ongoing critical scrutiny by qualified peers. Suppose you read in a journal article from 2014 that “between 16% and 40% of expatriate managers return prematurely from their assignment” abroad. What impact should that fact have on your own research? Well, you could be happy to see that the subject you are interested in is, it seems, part of a big problem in the real word. Your ethnographic work on cross-cultural business appears much more relevant in that light. In your own introduction, then, you make this claim, duly citing the source that you found the figure in. You submit the paper for publication, your reviewers recommend publication, and the paper is published. Your claims, including the 16-40% expatriate failure is now opened to the aforementioned “critical scrutiny” of your peers. What happens next?

Well, the reason that you provided a source is that people want to be able to check your facts. Not all readers will do this, but some might. Suppose someone does. And suppose they find the claim embedded in a sentence like the following: “Previous research, reported on by Black and Mendenhall (1989), reveals that between 16% and 40% of expatriate managers return prematurely from their assignment.” Please understand how shocking that is. Your paper made it look like the rate was reported in 2014. We find here that this rate is almost thirty years old! But it gets worse than that. Checking Mendenhall 1989 they will see that the figure is asserted, not on the basis of empirical evidence, but still other studies, going back as far 1971. Looking at those studies, finally, does not solve the mystery either. It’s simply not possible to track down anyone who provides evidence of the 16-40% range.  This is what’s not supposed to happen in scholarship. You should not have cited the rate you did because you, too, should have tried to trace it to its source and failed. You should then have written to the authors of the 2014 paper and pointed out their mistake. The journal should have issued a correction.

It’s only when we believe that such an error-correcting mechanism exists that we can trust the literature on a particular subject. Seeing something we think we can use in an a journal article from four or five years ago, we go to the library and try to see if there’s been any published criticism of it. If not, we check the underlying sources (or evaluate the methods) of the paper in question. We decide that we trust this result and that our readers would trust it too. Then we include it in our own paper. Simply citing the first appearance of a convenient fact is not good enough.

I use the example of expatriate failure rates advisedly. Over twenty years ago, Anne-Wil Harzing discovered that her peers had not been as critical as they should have been when citing high reported rates of expatriate failure. As she put it in a follow up paper in 2002, the paper she wrote as a PhD student about this problem was “was borne out of sheer amazement and indignation that serious academics seemed to get away with something students at all levels were warned not to do.” (Indeed, my example wasn’t pulled out of thin air either, though I have left out the names to protect the guilty. Click here for a more detailed critique.)

We can’t make too much of the courage it takes to challenge your entire discipline in this way as a PhD student. Indeed, I’m not sure it’s even advisable, though Harzing’s hard work, also on other topics, has clearly paid off for her in the long run. What she did was “presumptuous” in a good way. She assumed that standards of scholarly rigor applied in her field even if many scholars seemed to be entirely innocent of them. She acted as though good research was a norm. That’s how we should all work.

Indeed, that’s how most people presume academia works. Mistakes are made but they don’t remain for long. They are caught by critically minded peers and eventually corrected. You can play your part. I highly recommend reading Harzing’s 2002 paper, which is organized around the rules you should be following and examples of how they are broken. Learn them the easy way now. The hard way is not pleasant to think about.

Knowing with Others

What impact should what someone else knows have on your life? That sounds like a pretty big question, but let’s think about this in epistemological terms, as a problem of the theory of knowledge. I want to show that this ultimately tells us something important about specifically academic knowledge and, even more specifically, about academic writing.

First, what does it mean to say that someone knows something? Philosophers often begin with the idea that to know is to have a “justified, true belief” about something. We might want to dispute that, but if we play along for a moment we can consider our original problem as one of deciding what consequences someone else’s true beliefs should have for me. The fact that these beliefs are true tells us that something specific is the case. So, at first pass, we should live our life in accordance with other people’s knowledge on pain of being “unrealistic”. True beliefs, after all, are accurate representations of reality.

Notice that this does not imply that we should believe what other people merely believe. It’s only if they have knowledge that we need to get in line with them.

But a great deal of knowledge doesn’t have immediate practical implications. Astronomers, for example, know that Andromeda will collide with our own Milky Way in a few billion years. Not only is that a long time from now, I’m told it’s not even going to be particularly inconvenient for life in either galaxy when it happens because it will happen very slowly. But astronomers do, in fact, know that this is going to happen. And the consequence for me, I’m happy to admit, is that I believe it. And I think that’s really the first and most important impact that other people’s knowledge should have on our lives. If someone else knows something, then we should believe it.

I mean this “should” in an essentially logical sense. If I say, “Astronomers know that Andromeda will collide with the Milky Way but I don’t believe it,” I am contradicting myself. I can say that astronomers “think” or “claim” or “speculate” or “argue” or, of course, “believe” this, and then declare my own skepticism about it, without contradicting myself. But I can’t claim both that they know it and that I don’t believe it. Why, after all, would I not hold a belief myself if I believe it is true?

But can I say something like, “Astronomers know that Andromeda will collide with the Milky Way but I just don’t understand it”? There’s often something mind-boggling about astronomy and physics and mathematics. We want to grant that astronomers and physicists and mathematicians “know what they’re talking about” but this doesn’t always make things clearer for us, and I’m inclined to take a hard line on this. We should not here say that they know it. We should say that they “say” it and that until we ourselves understand what they’re saying we’re not going to believe it. After all, if I believe something I don’t understand I might not, in fact, be adopting the belief of the people who know it. Maybe we mean two different things by “collide” and in their sense the belief is true but in my sense it is nonsense.

I’ll develop these ideas in subsequent posts this week. But I want to declare my intentions clearly at the outset. I believe that academia should be a place where we are able to believe things that other people know, and where this way of forming beliefs, i.e., on the basis of other people’s knowledge, allows us to claim this knowledge of others as our own. It’s not a place where we believe everything we’re told. It’s a place where people present what they know in a way that opens it for criticism from other knowledgeable people. And the specific ways in which we do this, especially the way we use our writing to foster criticism, means that when we make a claim, and cite our source, we can, at that moment, say we “know” what we’re talking about.

It’s not perfect knowledge. Sometimes we merely believe something that will turn out later to be false. But, within the critical environment of the university, it should be okay to call it knowledge. We are not fools to believe in this way. Or, rather, we expect our peers not to make fools of us. This trust is an important part of what it means to be a scholar, an “academic”. It is sometimes violated, of course; but it is, in fact, the norm.

The Difficulty Setting

If you’re following the rules, working in a rational and deliberate way to write down what you know in order to allow other knowledgeable people to discuss it with you, I’m ready to promise you that it will get progressively easier. One of the reasons for this is that you will get better and better at choosing doable writing tasks. You will become better and better at setting up your work for tomorrow such that the problem is well within your skill set, well within your ability to solve.

Think of the “difficulty setting” on a video game. Games are not fun if they are too easy or too hard. If you easily kill all your enemies just by pointing your gun in a general direction and pressing “fire” repeatedly, there is no satisfaction in your victory, and you learn nothing from it. On the other hand, if you always die in the first few seconds because the manoeuvres that are required to save you are beyond your talents or training to complete you’re not going to feel any particular respect for the problem either. You’ll quickly get tired of trying and, again, you’ll see no improvement. That’s why many games let you decide whether to play as a beginner, or novice, or expert.

The important thing to keep in mind when transferring this analogy to your writing (according to my rules) is that you choose the difficulty setting the day before you write. This happens mainly in your articulation of the key sentence. Will it be easy or hard to compose at least six sentences and at most 200 words that support, elaborate or defend it in 27 minutes? Will it be hard to come up with six sentences? Or hard to keep it under 200 words? Or will it be hard to pull it all together in 27 minutes? The answer will vary from key sentence to key sentence. And the point is that you can turn the difficulty up or down simply by making minor changes to the sentence.

“Sensemaking poses a number of problems for managers.” “Sensemaking poses a number of problems for managers in crisis situations.” “Sensemaking is hard.” “Sensemaking, argues Weick, is an imaginative retrospective process that shapes action.” For some of these key sentences, only a meticulously constructed argument, based on precise areas of scholarship, will satisfactorily solve the problem. For others, any old paragraph will do. You define the problem of writing and its difficulty by deciding what you are going to say…

…and who you’re going to say it to. But please remember  that the more knowledgeable you imagine your reader to be the easier the writing will be. It’s hard to explain something very complicated to someone who lacks the conceptual apparatus, background knowledge and general intelligence to make sense of it. Your task becomes easier and easier (in your mind) as you increase the burden you lay on the shoulders of your reader. A lot of bad student writing comes from this–they imagine their reader to be their teacher and that their teacher knows much more than they do. That’s very likely, but it’s not good for your writing. It leaves too much for the reader to do.

When writing for academic purposes always imagine your reader to be an intellectual equal, a cognitive peer. Don’t let yourself change this component of the difficulty setting. Don’t make it too hard by imagining, as it is sometimes suggested, that you are explaining it to your grandmother. But don’t make it too easy either, by imagining that your reader is 100 years in the future and understands everything much better than you. Look around you. Think of your peers. And ask yourself how hard it is to explain this to them. Then choose a difficulty setting that’s right for you by deciding exactly what you’re going to tell them. Give yourself a proper challenge–one that you can imagine will be fun to meet. Tomorrow you’ll find out if you got the setting right.

Difficulty #3

“Yeah, well, you know, that’s just, like,
your opinion, man.”

Jeffrey “the Dude” Lebowski

Sometimes your reader understands perfectly well what you are trying to say. Nor is the reader’s problem one of believing you after you present them with the evidence you have. No amount of evidence, in these cases, is going to persuade the reader that your claim is true. The reader has already made up their mind on a basis that is independent of the evidence you could provide. Once you make your claim you will not have time to support it. You will have to defend it.

This situation arises often in academic writing and its important not to be afraid of it. You don’t have to get your reader to believe this claim, only to accept it “for the sake of argument”. The paragraph merely has to survive the reader’s objections, so it will be your awareness of these objections that will structure the writing. You can take a hard-line view and simply try to defeat them; that is, you can try to show the reader why their prior beliefs are wrong on this matter. A softer touch involves merely convincing them that their objections to your position are not relevant in the context of your particular argument. A still softer touch is to try to show that, even if they don’t agree with you on this issue, it doesn’t undermine your larger point. The whole point of this kind of paragraph is to invite a reader that disagrees with you into the conversation. A writer who can’t tolerate disagreement isn’t much of a scholar.

Indeed, the difficulty here isn’t overcoming disagreement, but dealing with it. You are trying to position your work in a field of possible truths and you want to be aware of what might be the case, and who might be right, if you are wrong. You want to show that it wouldn’t be the end of the world; indeed, you know what the world would be like if you are. You do this by acknowledging those who take a different view of the facts. Even those who have an entirely different sense of what the facts are. Part of your reading (and therefore part of your theory) should be built out of these alternative perspectives.

But you don’t want to fall into an easy “perspectivism”. To tolerate disagreement is not to let everyone be entitled to their opinion. It requires us, for the moment, to form our own opinion and, therefore, an opinion about who is right and who is wrong about the matter at hand. Much as we admire the Dude, much as it gives us comfort to know he’s out there “taking her easy” for all us scholars, we have to look beyond the strikes and gutters, ups and downs, of the everyday hustle and bustle of our research. We’re not just swept along, from town to town like some tumbleweed in the wind. We have our reasons for believing the things we do, and we know of others who have their reasons for believing the opposite. “Live and let live,” sure. But we also know what we think.

Though we don’t get it right every time we’re not just haphazardly guessing at what it is. We’ve collected and analyzed our data very carefully and it should surprise us if we’re wrong about it. But — and here’s the important point — we are comfortable with that possibility precisely because we know that someone else, in another department (or perhaps our own), in another corner of the academy, knows the truth we missed, holds a different view of the same facts. They will pick up the slack if we are forced to abandon our current position. The truth, too, abides.

See also: “The Fourth Difficulty”

Difficulty #2

Before your readers can believe that what you say is true, they must understand what you mean by your words. When reflecting on your key sentence, therefore, ask yourself whether the reader’s main difficulty will one of belief or understanding. (We’ll get to the third difficulty tomorrow.) Then write with the aim of addressing this difficulty head-on. Should you be providing evidence for the claim in your key sentence, for example? Or should you be defining your terms? Is the reader surprised to learn that the practice you are talking about even exists? Or is your reader just curious to know more about it because it is somewhat novel or otherwise exotic? Notice that you cannot begin to resolve these issues without knowing who your reader is. Please begin there. 

The difficulty need not be great, just worth a paragraph of prose to overcome. After reading your key sentence (or before getting to it) the reader should want to spend about minute hearing more. But remember that your reader is already a very knowledgeable person, so when you decide that their difficulty will be to understand what you are saying make sure you don’t talk down to them. Begin with what the reader does understand. Do not try to teach your reader your theory from the ground up. This is academic writing, and you are supposed to assume a great deal of common knowledge. Indeed, you are supposed to respect the learning that your reader has already acquired.

So, if you’re going to tell the reader that “Foucault’s ‘archive’ constitutes a practice that governs the emergence of statements in a discourse,” for example, you have to address yourself to readers that already know this or who would, at least, be embarrassed to admit they don’t. You have to present a claim like this one ostensibly as a reminder that merely announces that you intend to use the concept of the archive to guide your analysis of a discourse. Your readers do not want you to teach them what an archive is, and they’re happy (let’s assume) to believe that Foucault proposed the concept as you say. What they want to know now is why you’ve chosen to characterize it using the words “constitute”, “governs”, and “emergence”. This, after all, is your way paraphrasing Foucault’s definition of “a practice the causes a multiplicity of statements to emerge as so many regular events, as so many things to be dealt with and manipulated,” right? Your focus here will prepare the reader’s mind to follow your analysis. Finally, remember that your reader is not someone who thinks that Foucault is an intellectual impostor .

Or, if you’re going to say that that you set your alpha at .001 using a Bonferroni correction, don’t talk to your reader like a first-year statistics student who needs you to tell them what an “alpha” is or what a Bonferroni correction does. Just tell them what they need to know: how many statistical tests you did and what kind of tests they were. Justify your alpha to someone who knows what they’re doing, someone who can recognize a suitable significance level when they see one. Even a reasonable person sometimes wants to know your reasons for doing things in particular ways, however obvious or reasonable those ways may seem to you. They are ready to believe you did exactly as you did; they just want to understand exactly why so they can decide whether they think it was good idea. Here, too, remember that you are not writing for someone who thinks p-values are never a good idea.

There are times when you will explain a medical procedure to a social scientist or a piece of military hardware to a historian. Here you are, indeed, engaging with the reader’s ignorance and teaching them something they don’t already know. You are not here merely specifying the meaning of your terms, or establishing their application in your analysis. You are actually providing new information to the reader. Sometimes, you’ll have to recognize that the difficulty is epistemic (i.e.,  believing things, i.e., Difficulty #1). The reader may be shocked to know we can cure a particular condition or have built a particular bomb, and you will want to provide some evidence, some reliable source, to regain your reader’s trust on matters of shocking fact. But often the reader just wants to be able to imagine it. Learning how to write well is really learning how to produce a useful image of a fact in your reader’s mind.

None of this is easy. But remember that writing is hard only to ease the reading. Making yourself understood, one paragraph at a time, is a skill worth learning. Don’t be satisfied with getting past your reviewer. Try to get through to your reader.