This is the sixteenth in my series of short, quickly written, philosophy essays.
Imagine that you and I are in one of those big barn-like American stores. You know the kind of place: big walls, big shelving units, big products, a nice descriptive name on the storefront. My favourite of these is The Container Store, because you can wander round trying to persuade yourself that everything for sale does indeed count as a container. As, of course, does the store itself.
Anyway, we’re in this big barn-like American store, and suddenly I spot a sweet little green bird, fluttering across the nearest big wall. This little green bird is very small, very pretty, and presumably very lost. This makes me anxious! I watch him flutter around for a moment or two.
Meanwhile, you’re busy checking out some big product, and clearly haven’t noticed what’s going on. I’m about to attract your attention because I’m quite worried about the little green bird and you’re more practical in these situations than I am. But then I notice a projector machine on a table nearby! I follow the direction of the projected beam of light, and realise it’s headed towards the little green bird. I then have a quick look at the rest of the stuff on the table, and immediately spot an empty cartridge labelled ‘Little Green Bird’.
By this point, my anxiety has totally dissolved. And when I see you heading over, I no longer have any urge to shout, “Hey, there’s a lost bird up there! We must help him!” Instead I calmly say, “Look at that cool simulation on the wall.” And then we chat a little about the likely consumer demand for bird-related products.
You probably get where I’m going with this. I’ve written here often about my frustrations with the current AI consciousness discourse. One thing that frustrates me is people using the term ‘consciousness’ to refer to odd things, without telling us that this is what they’re doing. So I’ll clarify at this point that what I’m talking about here is the standard phenomenological, subjective experience, first-person perspective, Thomas Nagel sense of ‘consciousness’. You know what I mean: it’s the kind of thing you’re referring to when you tell me that you think your dog is conscious, or that you feel guilty about continuing to eat beef even though you think that cows are conscious.
The ‘what is the conscious thing?’ approach
Now, one reason I strongly doubt that AI could be conscious comes down to the difficulty of working out what the ‘conscious thing’ is, in the case of AI. After all, the popular contenders that the AI experts proffer here — things like conversations, models, instances of personas, computers, non-physical systems, bits of data centres, and various combinations of these things — all come up against many layers of obvious objections. And I find it hard to understand how there could be some ‘bit of consciousness’ without some particular thing that ‘bears’ it! I’ve written a lot about these ideas lately, however, so I’ll press on for now.
The ‘prerequisites for consciousness’ approach
A different but overlapping kind of approach that people sometimes take when pushing back against the idea of AI consciousness depends on working out prerequisites for the capacity for consciousness. For instance, an argument that I sometimes make reduces down to the idea that if something doesn’t have the capacity for psychological states, then it can’t have the capacity for consciousness. And a similarly shaped argument that other people sometimes make reduces down to the idea that only biological things can be conscious things.
Now, I’m very interested in these kinds of arguments.1 I love trying to work out the relations within and between things. I like using this kind of approach when I do political philosophy, and I like using it when I do other kinds of philosophy. I like thinking about necessity and possibility in this kind of context, and so on.
One obvious problem with depending on this ‘relational’ kind of approach, however, is that you can get unknowingly out of kilter with the truth of the matter, somewhere down the line! Perhaps you haven’t done a very good job of picking out some particular thing, for instance, or perhaps you haven’t done a very good job of evaluating how some particular kind of thing differs from some other particular kind of thing. And then, your conclusion about some relation might still hold in some abstract sense, but it simply wouldn’t track what you think it does! This failing might relate to some deficiency in your understanding of some relevant bit of science, or it might relate to more of a ‘philosophical’ mistake — a mistake about the scope of some kind of relation, for instance.
These kinds of risks lead me to conclude that I should remain open to the possibility that consciousness could be ‘borne’ by non-biological physical things. This is mainly because I’m not currently sure how I could rule out such a possibility. I mean, just because we have strong reasons to conclude that all the things in the world that are the best candidates for bearing consciousness are biological physical things surely cannot, in itself, suffice for us to conclude that only biological physical things can bear consciousness! And what’s more, some of my particular philosophical views — niche views about things like the mind-body problem — cause me some further complications.
Concluding that only biological physical things can be conscious seems to me a whole lot harder, therefore, than concluding that something has to have the capacity for psychological states to have the capacity for consciousness. I mean, consciousness understood in the ‘Thomas Nagel sense’ is entirely tied up with having psychological states! Yet it’s far harder to conclude that AI doesn’t have the capacity for psychological states than it is to conclude that AI is not a biological physical thing…
For now, therefore, there’s a different kind of approach that I feel much more confident about relying upon, when I reason about whether AI is conscious. This brings us back to the little green bird, and it goes something like the following.
The ‘hey it’s surely a simulation’ approach
The more I learn about AI, the more strongly I believe there are very good reasons to conclude that the consciousness that I might be tempted to think of as obtaining — somewhere! anywhere! — when I have conversations with LLMs is much better thought of as a simulation of consciousness. And that I should, therefore, seriously doubt that there is some ‘conscious thing’ out there that I’m engaging with when I’m having such conversations, or that is somehow fuelling the responses I’m receiving.
I’m aware, for instance, that LLMs have been developed to predict what ‘comes next’ in human-written texts, in part to produce convincingly human-like text. And that LLMs have also been developed to ‘act’ as assistants that help us with tasks. These are just two of the strongest bits of evidence, here. But they lead me to conclude that I should keep reminding myself that the complex systems I’m using when I have these conversations have been actively developed, by humans, to produce the kinds of outputs that could fool me in this way. Outputs that, of course, could lead someone to think there must be some ‘conscious thing’ behaving consciously — somewhere, anywhere!
And when I have very good reasons to believe that the little green bird I am watching is the simulation of a living bird, then this seriously cuts away at my reasons for concluding that the thing I’m watching is a living bird…
All that said, having reached the conclusion that what’s going on in the case of seeming AI consciousness is far better thought of as the simulation of consciousness clearly does not amount to having ruled out the idea that AI could be conscious! After all, perhaps the projector machine is conscious. Just like perhaps your bicycle is conscious!
Nonetheless, the reasons that I have to think hard about whether your bicycle is conscious are nowhere near as compelling as the reasons I have to think hard about whether AI is conscious. I mean, your bicycle can’t talk with me about Quine at 2am, can it? And what’s more, the compelling reasons that I once had to conclude that the little green bird was alive — the reasons that pretty much dissolved as soon as I saw the projector machine and the cartridge — were never compelling reasons to conclude that the projector machine was alive.
As ever, therefore, this leads me back to my questions about what the ‘conscious thing’ could be, in the case of AI. But the simulation argument seriously deflates my interest in trying to answer them.
A recent high profile Nature paper by Anil Seth, Ned Block, and Giulio Tononi, entitled ‘Three reasons why AI is not conscious and perhaps never will be’ sort of fits this kind of approach, at least in part. The writers’ first argument hinges on the idea that having the capacity for intelligence is not the same as having the capacity for consciousness. Their second and third arguments aim to destablise certain influential pro-AI-consciousness positions broadly on the grounds that these positions either over-depend on the idea that brains are “computers that are made of meat” or the idea that intelligent silicon-based AI could become conscious. On my view, none of these arguments come close to sufficing to enable us to conclude that AI is not conscious. I have specific objections to each argument, but even taken at face value, these arguments simply serve to tell us that we need to know more to conclude such a thing: that we need to know more about the capacities of AI, and that we need to consider whether there are stronger pro-AI-consciousness positions out there. That said, if you ignore the title of the article, and its final few sentences, the writers do mostly couch their conclusions in terms of “it is very unlikely that” and “The default position of the research community should therefore be that”…
Excellent stuff! I think “talking” with the AI about Quine is close to playing tennis with a wall, but I suppose the alternative to these uses of ordinary language is the AI people coming up with more silly words and phrases (which is probably worse).