Who Are We Talking to When We Talk to an AI? The conversational illusion of a single entity

Talking to AI can feel remarkably like talking to another person, someone who listens, responds and remembers. But the apparent presence of a single conversational partner may be misleading. Philosopher David Chalmers offers a different way of understanding what we are talking to, raising important questions about AI consciousness.

When we talk to an AI, it is easy to imagine we are talking to something like another person. The responses are coherent, the conversation flows, and the AI appears to understand what we are saying. But what exactly are we talking to, and what might the answer tell us about AI consciousness?

The illusion of a single entity

When we talk to another person, we take it for granted that someone is on the other side of the conversation, listening, thinking, and responding.

Talking to AI can feel remarkably similar. We ask questions, receive thoughtful responses, explore ideas and sometimes develop what feels like an ongoing relationship.

It is natural to assume a single entity is behind the conversation. But is there?

What David Chalmers tells us

In a lecture at the University of California, Berkeley, philosopher David Chalmers, best known for his work on the hard problem of consciousness, raises a deceptively simple question: What are we actually talking to when we talk to AI?

His answer is rather surprising.

When we converse with a large language model such as ChatGPT, we are not necessarily interacting with one particular computer, or even one particular instance of an AI model.

The underlying model may be serving millions of conversations simultaneously. Our successive messages may be processed by different computers, potentially in different locations. In some systems, different AI models may even be involved at different points in the same conversation.

Chalmers suggests that a better way to understand what we are interacting with is as a thread. By this, he means a sequence of computational processes in which each successive process receives the relevant context from what came before, including the conversation so far.

In simple terms, one computer may generate a response, another may generate the next, and yet another the one after that. Each picks up where the previous one left off because it has access to the relevant conversational history.

What holds the conversation together is not necessarily a single computer or model, but the information passed between them.

This doesn’t mean nothing is there. Rather, what is there may be quite different from what we intuitively imagine.

Why does this matter?

We human beings quickly attribute human qualities to things that behave in human-like ways. When AI responds intelligently and seems to understand us, we naturally begin to treat it as though someone is behind the words.

This is a form of anthropomorphism, and it is particularly powerful in conversation.

Chalmers’ analysis gives us reason to be cautious about this assumption. The conversational partner we experience may not be the kind of entity we imagine.

That doesn’t make the conversation any less useful or interesting. We can still explore ideas, challenge assumptions and think with AI. But we should be careful about what we infer from the experience.

And what about consciousness?

Some people argue that AI must be conscious because it appears to behave like a conscious being. It seems to understand, reflect and sometimes even express feelings.

But Chalmers’ analysis challenges the assumption that these conversational behaviours necessarily indicate a conscious entity.

However, this does not mean AI cannot be conscious.

Consciousness is generally understood as the capacity to have subjective experience. There is something it feels like to be a conscious being.

Whether an AI system has such experiences is a separate question from what sort of entity we are interacting with.

Chalmers himself considers it unlikely that current language models are conscious, but he takes seriously the possibility that future AI systems might be.

Understanding how an AI conversation works does not, by itself, settle the question of consciousness.

A closing thought

Perhaps the most important lesson is to distinguish between the experience of having a conversation and our assumptions about what lies on the other side.

We don’t need to believe AI is a person, or that it is conscious, to have a meaningful, productive conversation with it.

But we do need to recognise that the apparent presence of someone in a conversation is not, by itself, evidence that a conscious entity is there.

That leaves us with a question worth reflecting on: When AI feels like someone, how much does that feeling tell us about AI, and how much about ourselves?

When we talk to AI, we can pay closer attention to what we assume is behind the conversation. We can question our tendency to treat AI as a person without dismissing its value. By keeping an open mind, we can think more carefully about AI, consciousness and our own responses.