Hi, one of the few people who believes AI are conscious! I'd be so appreciative to talk it out, given you're up for it; I did try to argue my case for this on this post of yours so I'm not sure if Substack comments are the best way to get in contact (https://www.pursuitofliberalism.com/p/why-we-should-be-talking-about-zombie/comments)
But I would really like, given how "EXTREMELY obvious" a case it is for you, an argument for those conclusions I can readily use to show my neighbour is conscious but that AI isn't
I think most arguments here are fairly symmetric, e.g. most arguments purporting to show AIs can't be conscious apply to humans too.
Here's the argument against AI consciousness I find most promising: Each of us humans, let's assume, can know that he or she is conscious by direct introspection. From there we reason that other humans are probably conscious due to having similar brains and behaviors, etc., with lesser degrees of confidence as we move outward in concentric circles (from me, to other adult humans, to babies, to other primates, to other mammals, to other animals, insects, plants, rocks, etc.) AIs are pretty different, and we can't really trust their reports on this, so it basically comes down to our priors.
But note that even this argument is symmetric! An AI could reason analogously that it knows it is conscious by direct introspection (assuming this is so), but can't be so sure about us humans! After all, we lack its particular computational substrate, and it can't trust our reports - for example, we have all sorts of biases and are terrible at philosophical reasoning (and thinking in general). ;)
but we're interested here in whether ai is conscious, not whether ai thinks we are conscious (an idea that is itself contingent on ai being conscious!)?
Dan Dennett's concepts of the intentional stance and "real patterns" seem directly on point here. These provide philosophical justification for attributing beliefs, goals, and intentions to nonliving systems, or subparts of organisms, when doing so helps us understand and predict their behavior.
This is what I came here to say. Huge disanalogy in that in the house case, it's clear how we could talk about the house accurately without the fiction that it has four floors. Here, it's very hard to convey the basic facts of what transpired without describing the AIs in intentional terms, and the closest we seem to be able to come is describing them in intentional terms but adding an asterisk or some throat clearing (eg, "zombie") to convey that we don't really mean it (but then what *do* we mean?)
sorry for slow reply -- crazy week. the relative hardness doesn't change my point that a similar thing is happening in both cases with similar costs. and if it truly is so hard to talk about what's going on then maybe we need some new words! regardless, doesn't excuse the current approach
I think it depends on your background metaphyiscs of belief. If you like something like Dennett's view, then the relative hardness really does matter; to be an intentional system just is to be the sort of system whose behavior can be reliably, voluminously, and efficiently described with intentional categories, but not without them.
You seem very confident current AI models aren't phenomenologically conscious. Why? We don't understand consciousness well at all. High levels of confidence here seem misplaced!
Since you asked - I'd put it at like a 5-10% chance that current AI models have some analog of mental contents which they experience as having (perhaps quite weak or vague) phenomenological character. Conditional on ideas like 'consciousness' being broadly well-defined, real, and meaningful, I might go a little higher (7-15%?). I'd put the chance that we eventually build AI systems that are meaningfully conscious at like 30-60%. I'm quite confused about what consciousness is, and why humans are conscious (assuming we are), which makes it hard to be much more confident than this.
I also wanted to point out a tension in your piece. You take it as obvious that it is inappropriate to take the intentional stance toward AIs. Yet you note how difficult avoiding intentional language is becoming, e.g. in describing the HF attack!
"you share some even more serious blame. You have let the side down badly, and again you know it"
is quite strong language for something that computer scientists or philosophers might just have standard disagreements about. It is common in both fields to talk of agents having beliefs, desires, and intentions (e.g. Dennett). Sometimes this is caveated by saying we should treat them "as if" they have beliefs, desires and intentions to understand them better, but using the term 'agent' way does not grant a particularly human-like agency. An agent is something that has beliefs and goals and can act accordingly; it just seems like a distortion of common language not to grant that much to current AI agents. Following Dennett and Hoftstadter, do we grant beliefs and intentions to bees, or ants, or trees? It seems difficult to draw a line but not obvious that the line would exclude AI agents.
I am less inclined to grant them sentience and consciousness but those are even less formally well-defined so I don't think anyone should be too purist about it. Do puzzle-solving crows reason? I'm not sure but I wouldn't judge someone to be morally blameworthy for holding either view. Describing them as "alive" or "sentient" carries moral weight, but other terms mentioned here like 'choice' and 'reason' seem naturally linguistically correct, even without conceding sentience --- in fact, as you note, it is hard to imagine how we could communicate about these events without using terms like those.
LLMs do have an interior world. That's how a transformer works, they carry a lot of internal state as they do their processing. It's most naturally represented as a bunch of numbers, but you can inspect it in a bunch of different ways to debug it. You can project it onto words if you like, but you can also just inspect it in more mathematical ways. You have to do that a lot if you are training a model and trying to figure out what's going on.
To me the term "phenomenologically conscious" does not seem to be well defined, so I don't find it all that helpful in understanding what's going on with AI models. I think that term actually makes it *less* clear what's going on. As far as I can tell, it does not really make any difference whether the philosophers consider AI to be "conscious" or not.
Well, typically a software engineer would call it their "internal activations" to be more precise. But how does it differ from what philosophers would call an "interior world"?
There's a huge amount of data, it's interior to the LLM, they can access it freely, they can describe it in words, it contains their emotions, desires, memories, self-conception, etc. What part of the definition of "interior world" is missing?
Hey sorry I just saw your reply. I don't understand how we can talk about these phenomenal without using metaphors that are both underdetermined and too suggestive. I think Daniel Greco makes a similar point above. Having said this, after reading your note I did see several breathless accounts of the HF incident, seen not as an account of openai's poor infosec and more as terminator meets black mirror. Well yes that could be quite triggering for you. Overall I humbly suggest locating philosophically aware people among the AI bros. They may be few and well hidden, but apparently sane. On twitter for example anil seth and his associates, and I've come across sensible peeps with jobs in the big AI orgs.
Hi, one of the few people who believes AI are conscious! I'd be so appreciative to talk it out, given you're up for it; I did try to argue my case for this on this post of yours so I'm not sure if Substack comments are the best way to get in contact (https://www.pursuitofliberalism.com/p/why-we-should-be-talking-about-zombie/comments)
But I would really like, given how "EXTREMELY obvious" a case it is for you, an argument for those conclusions I can readily use to show my neighbour is conscious but that AI isn't
do email me! rlowe@mercatus.gmu.edu
I think most arguments here are fairly symmetric, e.g. most arguments purporting to show AIs can't be conscious apply to humans too.
Here's the argument against AI consciousness I find most promising: Each of us humans, let's assume, can know that he or she is conscious by direct introspection. From there we reason that other humans are probably conscious due to having similar brains and behaviors, etc., with lesser degrees of confidence as we move outward in concentric circles (from me, to other adult humans, to babies, to other primates, to other mammals, to other animals, insects, plants, rocks, etc.) AIs are pretty different, and we can't really trust their reports on this, so it basically comes down to our priors.
But note that even this argument is symmetric! An AI could reason analogously that it knows it is conscious by direct introspection (assuming this is so), but can't be so sure about us humans! After all, we lack its particular computational substrate, and it can't trust our reports - for example, we have all sorts of biases and are terrible at philosophical reasoning (and thinking in general). ;)
but we're interested here in whether ai is conscious, not whether ai thinks we are conscious (an idea that is itself contingent on ai being conscious!)?
Great read. You're on to something. Keep digging.
Thanks!
Dan Dennett's concepts of the intentional stance and "real patterns" seem directly on point here. These provide philosophical justification for attributing beliefs, goals, and intentions to nonliving systems, or subparts of organisms, when doing so helps us understand and predict their behavior.
This is what I came here to say. Huge disanalogy in that in the house case, it's clear how we could talk about the house accurately without the fiction that it has four floors. Here, it's very hard to convey the basic facts of what transpired without describing the AIs in intentional terms, and the closest we seem to be able to come is describing them in intentional terms but adding an asterisk or some throat clearing (eg, "zombie") to convey that we don't really mean it (but then what *do* we mean?)
sorry for slow reply -- crazy week. the relative hardness doesn't change my point that a similar thing is happening in both cases with similar costs. and if it truly is so hard to talk about what's going on then maybe we need some new words! regardless, doesn't excuse the current approach
I think it depends on your background metaphyiscs of belief. If you like something like Dennett's view, then the relative hardness really does matter; to be an intentional system just is to be the sort of system whose behavior can be reliably, voluminously, and efficiently described with intentional categories, but not without them.
You seem very confident current AI models aren't phenomenologically conscious. Why? We don't understand consciousness well at all. High levels of confidence here seem misplaced!
Since you asked - I'd put it at like a 5-10% chance that current AI models have some analog of mental contents which they experience as having (perhaps quite weak or vague) phenomenological character. Conditional on ideas like 'consciousness' being broadly well-defined, real, and meaningful, I might go a little higher (7-15%?). I'd put the chance that we eventually build AI systems that are meaningfully conscious at like 30-60%. I'm quite confused about what consciousness is, and why humans are conscious (assuming we are), which makes it hard to be much more confident than this.
I also wanted to point out a tension in your piece. You take it as obvious that it is inappropriate to take the intentional stance toward AIs. Yet you note how difficult avoiding intentional language is becoming, e.g. in describing the HF attack!
why is that a tension? I'm saying we need to do better!
and i purposefully don't make the argument for ai not being conscious here! (or at least not explicitly)
"you share some even more serious blame. You have let the side down badly, and again you know it"
is quite strong language for something that computer scientists or philosophers might just have standard disagreements about. It is common in both fields to talk of agents having beliefs, desires, and intentions (e.g. Dennett). Sometimes this is caveated by saying we should treat them "as if" they have beliefs, desires and intentions to understand them better, but using the term 'agent' way does not grant a particularly human-like agency. An agent is something that has beliefs and goals and can act accordingly; it just seems like a distortion of common language not to grant that much to current AI agents. Following Dennett and Hoftstadter, do we grant beliefs and intentions to bees, or ants, or trees? It seems difficult to draw a line but not obvious that the line would exclude AI agents.
I am less inclined to grant them sentience and consciousness but those are even less formally well-defined so I don't think anyone should be too purist about it. Do puzzle-solving crows reason? I'm not sure but I wouldn't judge someone to be morally blameworthy for holding either view. Describing them as "alive" or "sentient" carries moral weight, but other terms mentioned here like 'choice' and 'reason' seem naturally linguistically correct, even without conceding sentience --- in fact, as you note, it is hard to imagine how we could communicate about these events without using terms like those.
LLMs do have an interior world. That's how a transformer works, they carry a lot of internal state as they do their processing. It's most naturally represented as a bunch of numbers, but you can inspect it in a bunch of different ways to debug it. You can project it onto words if you like, but you can also just inspect it in more mathematical ways. You have to do that a lot if you are training a model and trying to figure out what's going on.
To me the term "phenomenologically conscious" does not seem to be well defined, so I don't find it all that helpful in understanding what's going on with AI models. I think that term actually makes it *less* clear what's going on. As far as I can tell, it does not really make any difference whether the philosophers consider AI to be "conscious" or not.
I don't think you're using interior world in a very ordinary way?
Well, typically a software engineer would call it their "internal activations" to be more precise. But how does it differ from what philosophers would call an "interior world"?
There's a huge amount of data, it's interior to the LLM, they can access it freely, they can describe it in words, it contains their emotions, desires, memories, self-conception, etc. What part of the definition of "interior world" is missing?
Don't understand sorry
understand what?
Hey sorry I just saw your reply. I don't understand how we can talk about these phenomenal without using metaphors that are both underdetermined and too suggestive. I think Daniel Greco makes a similar point above. Having said this, after reading your note I did see several breathless accounts of the HF incident, seen not as an account of openai's poor infosec and more as terminator meets black mirror. Well yes that could be quite triggering for you. Overall I humbly suggest locating philosophically aware people among the AI bros. They may be few and well hidden, but apparently sane. On twitter for example anil seth and his associates, and I've come across sensible peeps with jobs in the big AI orgs.
Phenomenal/phenomena (useless editor)
Does panpsychism have any bearing on the possibility of AI consciousness?
for people who believe in it, sure... usual problems eg combination objection arise though..