It's interesting but on most of the points - I disagree. The British Government in an emergent entity and it is an entity that can 'know things' and 'want things' which are distinct from what any individual person in it knows and wants. Its knowing is in a different way, to a different degree and can be discussed as emerging from underlying knowings. The capacity it has to know is not the same as the sum of individual knowing or the Venn diagram of what is known by every member of the government, which is also not the true embodiment of the government. In the same vein, a fictional character can know things and want things, but in a different way and to a different degree than a living person; and the relationship between the character and the author is complex, not trivial. At the same time, each of us has the possibility of having a parasocial relationship with fiction which is effectively a dead-end - speculating about the interiority of a fiction - but to say it is uniquely a dead-end underestimates how much it is also a dead-end to speculate about the interiority of a celebrity, say, who is definitely living but with whom the relationship is also parasocial.
LLMs are a new category of thing, and we need to figure out how to think and talk about this new category. I appreciate you taking on that task; it seems genuinely important.
I'm less sure about the specifics of your account. Your view seems to be, roughly, (1) that things like desire, reasoning, knowing, and choosing are psychological phenomena, and (2) that only living things can have psychological states. Several objections occur to me.
Can we rule out the possibility of a machine having psychological states? I don't think current LLMs are conscious, but "only living things can have psychological states" seems impossible to rule out a priori.
What about unconscious desire or unconscious reasoning? These are widely thought to occur in humans. That suggests that consciousness is not necessary for things like desiring and reasoning.
More generally, it seems to me that there are phenomenological and functional accounts of concepts like "reasoning" and "desiring." You seem to be asserting only the phenomenological sense as the genuine. At a minimum, that position needs to be argued for rather than assumed.
I don't believe that current LLMs are conscious, and I'm open to the idea that consciousness is enormously important morally. But I'm less convinced that a system without consciousness therefore cannot reason. In fact, that's exactly the premise that LLMs seem to me to explode.
This is interesting and thoughtfully pushes back on some of my current beliefs, so I appreciate this. Question, does what you have laid out here justify treating AI, whether purely digital or the other extremely humanoid, cruelly? If "it depends", is there someplace on that spectrum where it flips? I'm thinking of Spielberg's underrated AI movie from 2001.
It's interesting but on most of the points - I disagree. The British Government in an emergent entity and it is an entity that can 'know things' and 'want things' which are distinct from what any individual person in it knows and wants. Its knowing is in a different way, to a different degree and can be discussed as emerging from underlying knowings. The capacity it has to know is not the same as the sum of individual knowing or the Venn diagram of what is known by every member of the government, which is also not the true embodiment of the government. In the same vein, a fictional character can know things and want things, but in a different way and to a different degree than a living person; and the relationship between the character and the author is complex, not trivial. At the same time, each of us has the possibility of having a parasocial relationship with fiction which is effectively a dead-end - speculating about the interiority of a fiction - but to say it is uniquely a dead-end underestimates how much it is also a dead-end to speculate about the interiority of a celebrity, say, who is definitely living but with whom the relationship is also parasocial.
LLMs are a new category of thing, and we need to figure out how to think and talk about this new category. I appreciate you taking on that task; it seems genuinely important.
I'm less sure about the specifics of your account. Your view seems to be, roughly, (1) that things like desire, reasoning, knowing, and choosing are psychological phenomena, and (2) that only living things can have psychological states. Several objections occur to me.
Can we rule out the possibility of a machine having psychological states? I don't think current LLMs are conscious, but "only living things can have psychological states" seems impossible to rule out a priori.
What about unconscious desire or unconscious reasoning? These are widely thought to occur in humans. That suggests that consciousness is not necessary for things like desiring and reasoning.
More generally, it seems to me that there are phenomenological and functional accounts of concepts like "reasoning" and "desiring." You seem to be asserting only the phenomenological sense as the genuine. At a minimum, that position needs to be argued for rather than assumed.
I don't believe that current LLMs are conscious, and I'm open to the idea that consciousness is enormously important morally. But I'm less convinced that a system without consciousness therefore cannot reason. In fact, that's exactly the premise that LLMs seem to me to explode.
This is interesting and thoughtfully pushes back on some of my current beliefs, so I appreciate this. Question, does what you have laid out here justify treating AI, whether purely digital or the other extremely humanoid, cruelly? If "it depends", is there someplace on that spectrum where it flips? I'm thinking of Spielberg's underrated AI movie from 2001.