A recent paper by Mark Coeckelbergh and David Gunkel in AI & Society has got me thinking. Since I know that David will now immediately think his work is done — getting us thinking is his goal — let me stress from the outset that it has mainly got me thinking that they’re wrong. Since their aim is “deconstructive”, however, telling them they’re “wrong” is not so straightforward. Properly speaking, in order to be wrong, there have to be facts of the matter and you have to say something about those facts, and deconstructive writing often resists being read that way. Still, I think I’ve found a sense in which Mark and David are simply and plainly wrong and I invite them to consider it here.
Although it may not be their explicit thesis, I take the underlying message of their paper to be that large language models constitute a “fundamental challenge to long-standing metaphysical, epistemological, and axiological assumptions,” which they sort, roughly, under the rubric of “logocentrism”, and which therefore also gives them an “opportunity” to remind us of the not-quite-so-long-standing but nonetheless familiar “postmodern” critique (or deconstruction) of these assumptions as found in the work of Barthes, Foucault, and Derrida. Specifically, they put the challenge of generative AI as follows: “these algorithms write without speaking, i.e. without having access to (the) logos and without a living voice.” This is the statement that I think is wrong. But I want to make clear that, although I’m not myself ashamed of my logocentrism, I don’t just think it is wrong on those “long-standing metaphysical assumptions” they propose to deconstruct. I want to offer a critique on Mark and David’s own terms.
I should say that I’ve tried this sort of thing before in my conversations with David about robot rights, with rather limited results. I disagree with him that we can “face” a robot as an “other” in Levinas’ sense; and I don’t think they provide the correlative “incidents” of “rights” in Hohfeld’s sense. As far as I can tell, he has so far been unmoved by my arguments, which are based both on my understanding of how robots work and my reading of Levinas and Hohfeld. Past failures notwithstanding, I can’t think of a better way to do critique than that, and I’m going to offer something similar here.
Mark and David pass somewhat lightly over how language models work, encouraging us to take them more or less at face-value (or to abolish any distinction between face-value and “real” value). But we have to remember that there are many ways to imagine a machine putting words together that we would not consider writing. In a previous post, I suggested that LLMs are not, in fact, “artificial intelligences”; they are merely “electronic articulators”; and I asked us to consider the following example of a (non-)”writing” machine:
Imagine you have three bags, numbered 1, 2, 3. In bag number 1 there are slips of paper with names on them: “Thomas”, “Zahra”, “Sami”, “Linda”. In bag number 2 there are phrases like “is a”, “wants to be a”, “was once a”, and so forth. In bag number 3, there are the names of professions: “doctor”, “carpenter”, “lawyer”, “teacher”, etc. You can probably see where this is going. To operate this “machine”, you pull a slip of paper out of each bag and arrange them 1-2-3 in order. You’ll always get a string of words that “make sense”. Can this arrangement of three bags write?
My suggestion is, appearances notwithstanding, that what language models in fact do is not something Barthes, Foucault, and Derrida would countenance as writing, any more than they would call our system of three bags an “écrivain”. Since these authors are “dead” in all the relevant senses, that’s not going to bother Mark and David, of course, so let me put it in more technical terms: what large language models do cannot be construed as writing even according to the “innovations” of the “postmodern literary theory” that Mark and David propose to “capitalize on”. The operations of ChatGPT are not “grammatological”; they do not make a “différance”. Their output, as a consequence, are not “texts” that can be “subject” to “deconstruction” or, even, I dare say, “reading”. It can of course easily be turned into text by a writer who puts their name to it, authorizing it and then, in order that may it be read, politely dying.*
I wish to make this argument by quoting passages form Barthes, Foucault, and Derrida as they appear in Mark and David’s text and simply challenging them to explain how they imagine ChatGPT carries out the necessary operations required of even the postmodern conception “writing”. Let’s start with Barthes.
Text is made of multiple writings, drawn from many cultures and entering into mutual relations of dialogue, parody, contestation, but there is one place where this multiplicity is focused and that place is the reader. … A text’s unity lies not in its origin but in its destination.
Roland Barthes, “Death of the Author”
Given what we know about how ChatGPT generates its output, it’s hard to see it “drawing from cultures” or “entering into mutual relations”. That is, this “multiplicity” that produces a text is entirely foreign to ChatGPT, which merely computes the next probable token in a string of tokens. I’m certainly curious to hear the analysis (or even deconstruction) of how the output is “made” as “text.”
Next, here’s Foucault:
Although, since the eighteenth century, the author has played the role of the regulator of the fictive, a role quite characteristic of our era of industrial and bourgeois society, of individualism and private property, still given the historical modifications that are taking place, it does not seem necessary that the author function remain constant in form, complexity, and even in existence. I think that, as our society changes, at the very moment when it is in the process of changing, the author function will disappear.
Michel Foucault, “What is an Author?”
But it’s important to recall that the disappearance of the author function does not mean anyone or anything can now “write”. Rather, new questions arise: “What are the modes of existence of this discourse? Where has it been used, how can it circulate, and who can appropriate it for himself? What are the places in it where there is room for possible subjects? Who can assume these various subject functions?” How, I want to know, can ChatGPT occupy these positions, execute these new functions?
Finally, let’s consider Derrida. Mark and David seem to think that for ChatGPT in particular “there is nothing outside the text”: “For the text of an LLM to make sense, the texts (and the contexts to which they refer) are enough. For this purpose, nothing more is needed.” ChatGPT, on their view, becomes not just a possible writer but an exemplary writer of non-logocentric text (better than Beckett? Better than Gertrude Stein?). But would Derrida agree?
‘There is nothing outside the text.’ That does not mean that all referents are suspended, denied, or enclosed in a book, as people have claimed, or have been naïve enough to believe and to have accused me of believing. But it does mean that every referent, all reality has the structure of a differential trace, and that one cannot refer to this ‘real’ except in an interpretive experience”
Jacques Derrida, Limited inc
Does it not seem like Mark and David’s “nothing outside the text” is, in the case of LLMs, a matter of suspending, denying, or enclosing all referents in a book? Where, in the operations of ChatGPT do we find it actually referring, i.e., producing a “differential trace” of the real? Where is ChatGPT’s “interpretative experience”?
Like I say, I want to leave this as a challenge. Mark and David have forced me to read Barthes, Foucault, and Derrida very closely these past few days, and that is of course rewarding all on its own. But the more I read them, the less likely it seems to me that they would countenance what ChatGPT does as any kind of “writing”. Sure, Barthes suborned the murder of literary authority, but he didn’t leave only a reader in its place. A “scriptor” was to take the author’s place. We could look more closely at what he thought this writer was doing. But I doubt we could ever conclude that ChatGPT is doing it.
*”Writing,” says Barthes, “is that neutral, composite, oblique space where our subject slips away, the negative where all identity is lost, starting with the very identity of the body writing.” It seems to me that this implies a body to begin with, a subject to slip away. Perhaps, when he so famously says that “the birth of the reader must be at the cost of the death of the Author,” we misunderstand him if we think the Author must die once and for all — that all authors die to make all readers possible. Rather, the author must precisely live, to do the writing, to be the body writing in practice, but must then die, if only in principle (or on principle, if you will), in order for the text to be read.