Transcribed from https://youtu.be/NgDIG8u1-CA
Speakers: Chris Hayes (host) and David Chalmers (philosopher, cognitive scientist; professor of philosophy and neuroscience at NYU; co-director of the Center for Mind, Brain, and Consciousness). Transcript lightly cleaned for readability; timestamps mark the start of each turn.
[00:00:09] CHRIS HAYES: Hello and welcome to Why Is This Happening, with me, your host Chris Hayes. I'm very excited today for another edition of our special series where we take a look at all the implications — both terrifying and exhilarating — of artificial intelligence: its philosophical implications, its social, economic, and political implications.
We've been using this term, you'll see this term, called the Turing test, and that refers to one of the great minds of the last century, Alan Turing — British philosopher, computational scientist — who basically came up with this idea of: how would you know a machine is intelligent? And the Turing test was, you would know it's intelligent if you can talk to the machine and not know it's a machine. It would convincingly impersonate a person to the point that you couldn't distinguish.
For a long time this was kind of the bar. Can you make a machine that, under interrogation — a set of repeated interactions — you had no idea it was a machine? I think it's fair to say we have passed that bar. I don't know if you've had this experience. I've had the experience of talking to customer service chatbots and genuinely not knowing if I was talking to AI or a person — even having the somewhat humiliating experience of being like, "Are you a real person?"
So that part of the Turing test is getting passed right now. And yet I think we still want to say there's something different between me on this side of the chat and the customer service bot. The big difference is that I've got this little "me" inside me. I'm a conscious agent. I'm a subject. I see the world. I listen to music. I get sad when I think about sad things — all the things that make up what we call my interior life. I don't think that chatbot has it. No matter how good it got at replicating natural speech, I've got this thing that makes me a human. I've also got an embodied, physical corpus. The word we use for it is consciousness. I've got consciousness; the chatbot doesn't.
But could the chatbot ever have consciousness? How sure are we that — in the same way it seemed preposterous for a machine to pass the Turing test — we don't get to a point where machines maybe do have that experience, have inner lives? This sounds incredibly sci-fi, but I'll tell you, as someone who's been around the AI discussion for a long time, breezily passing the Turing test seemed like sci-fi for a long time, and we sort of shot past it.
So, to talk about consciousness — this thing I think we all intuit makes us human, makes us distinct, the thing that rules our whole lives because everything passes through the framework of this interior little thing inside here — I'm talking to arguably the foremost expert on the topic. David Chalmers is a philosopher, cognitive scientist, and author. He's professor of philosophy and neuroscience at NYU and co-director of the Center for Mind, Brain, and Consciousness at NYU. He's widely recognized as one of the foremost philosophical and scientific experts on consciousness. I've been reading your work for decades, and it's wonderful to have you here.
[00:03:25] DAVID CHALMERS: Thanks, Chris. It's great to be here.
CHRIS HAYES: You got interested in this problem exceedingly young, if I'm not mistaken. What first triggered you on the problem of consciousness?
[00:03:34] DAVID CHALMERS: Growing up, I was a math geek — really into math, computer science, physics, and so on. But somewhere along the way, I got obsessed by the human mind and by the problem of consciousness. One thing I remember: just as a young kid, at one point I realized I was shortsighted in one eye, that I needed glasses. At a certain point I got glasses and the world kind of popped out into three dimensions. It's an amazing feeling when you get glasses for the first time — you're like, "Oh, this is what it should be like." I thought the world was 3D already, but suddenly, no, the world had depth.
I could tell a story about the processes of binocular vision, where the eyes compute the distance of things. But that wasn't what I was concerned with. My question was: why does it feel like something to experience the world in depth, in 3D? Where does that subjective experience come from? In retrospect, that was a question about consciousness — why does everything the brain does feel like something from the inside? That struck me as the most interesting unsolved problem in the universe.
[00:04:43] CHRIS HAYES: How should we think about defining what consciousness is? It gets very slippery when you try to get your hands around it. It can be hard to think about because it's like the old joke about the fish in the water — the older fish says to the two younger fish, "Enjoy the water," and one turns to his companion and goes, "What's water?" Consciousness can't but be the case that it diffuses our entire experience of the world. So how should we think about what it is?
[00:05:10] DAVID CHALMERS: Everything is consciousness. I define consciousness as subjective experience — anything which feels like something from the first-person point of view. There's a phrase philosophers like to use: what it's like to be you. Consciousness is somehow what it's like to be me. My colleague at NYU, Tom Nagel, wrote a famous article called "What Is It Like to Be a Bat?" We don't know what it's like to be a bat using sonar to get around, but we assume there's something it's like to be a bat, from the bat's perspective. If there is, then we say the bat is conscious.
Now, maybe this coffee cup — maybe there's nothing it's like to be the coffee cup. That's what most people would say, in which case the coffee cup is not conscious. But for some privileged class of objects in the universe, including human beings — or at least including me, and hopefully you, hopefully the rest of us — there's something it's like to be them. They're actually beings with subjective experience. We think also probably animals — a dog or a cat, or maybe even a fish, there could be something it's like to be them.
[00:06:14] CHRIS HAYES: Any pet owner has had this experience. A bat, I think, is too remote for us — it's a great paper, obviously, the classic paper on this by Thomas Nagel — but any pet owner... I have a dog. You're observing the behaviors of the dog, and you try to imply mental states to her. When she's happy, she wags her tail, and I don't just say, "Well, that's a behavior, she's wagging her tail" — I think she's happy, she's feeling something. I'm quite convinced — I may be wrong — that she's feeling a thing inside herself.
[00:07:02] DAVID CHALMERS: Right. And we have to distinguish, in this area, behavior — the things we can do and objectively measure from the outside — from subjective experience, what it's like from the inside. We normally see a system behaving intelligently and take that as a sign that there's consciousness going on inside. But we have to distinguish those two things. Sometimes people distinguish intelligence — the capacity to do sophisticated things — from consciousness, which is what it feels like from the inside.
And of course, now in the era of AI, we're coming to a point where that reasoning from behavior to consciousness, from intelligence to consciousness, maybe those two things could in principle be coming apart. AI systems are increasingly intelligent — but are they conscious? Is there something it's like to be an AI system? Does it have subjective experience? That's far from obvious.
[00:07:55] CHRIS HAYES: Let's stay on this — the category of things we think have consciousness. A plant responds to stimulus in its environment; it grows toward the light. But we don't think of the plant as having consciousness. Why not? What's the next level — the sea slug, the worm, the ant? I want to say a little finch, a hummingbird, probably is conscious to me, but it's hard for me to think of a worm as conscious. Where am I drawing these distinctions?
[00:08:28] DAVID CHALMERS: It's interesting — the consensus on these things has gradually gotten more inclusive as the years have gone by. Centuries ago, René Descartes thought humans, and only humans, are conscious; no non-human animals. To him, a cow was like a chatbot — a sort of purely mechanical being, inputs and outputs, no inner life. Pure automaton, for Descartes.
Even when I was getting started in this field three decades ago, people would say, "Sure, maybe other primates are conscious, a few mammals, but maybe it doesn't go beyond there." Whereas these days, if you ask scientists and philosophers in the field, most are prepared to concede consciousness to all mammals — even a mouse, for sure. In fact, a lot of the experiments in this field are on mice.
[00:09:27] CHRIS HAYES: Well — let's maybe even talk about fish, because what's interesting there is something I think you're partly responsible for: the idea that we can start to treat these problems in a scientific fashion, not just philosophically speculative, not just reasoning from principles, but actually try to test things. So if we've made some progress in our understanding that's allowed us to enlarge the circle — how do you start to test this stuff? Given that the fact is, definitionally, interior, what evidence in the world could you find that would cause you to rethink your priors, your categories, and say, "Actually, it turns out a mouse is conscious"?
[00:10:06] DAVID CHALMERS: Our best source of evidence about consciousness — well, the very best source comes from my own consciousness, my own first-person experience. I know I'm conscious and I know what I'm conscious of. But I'm usually prepared to extend this, at the very least, to other human beings. I find out what someone else is conscious of by asking them. I could say, "Hey, Chris, what are you experiencing right now?" and you tell me, and I take that as evidence about what's in your consciousness.
The moment we go beyond the normal adult human case — to non-human animals — we can no longer use verbal reports, because these are animals, they don't speak. The same issue comes up for a baby's consciousness. So we have to go with something more indirect. One approach that's been very successful is looking for correlates of consciousness in the human case — processes in the brain that are active when we have conscious experience. People call those the neural correlates of consciousness. Then we see if we can detect those in animals, in babies, or in other systems.
[00:11:04] CHRIS HAYES: Right — so when I'm instructed to think some thoughts and I use my conscious willpower, they scan my brain and see some part that's correlated with that activity. Then I look at a mouse, or a chimpanzee, and see if there's similar neural activity. I can infer — it's not conclusive, but I can infer — that it's probably something similar happening on the interior, because it looks like this thing.
[00:11:35] DAVID CHALMERS: Exactly. There are also behaviors that seem to correlate with consciousness in humans — certain complex forms of memory or planning that, at least in the adult human case, seem to require consciousness to do. So if we find out that babies can track complex stimuli of a kind that in an adult requires consciousness, we say, "Okay, maybe that's evidence these babies are having some subjective experience." Now people are finding even insects can do certain kinds of complex tasks and conditioning that would require consciousness if done in humans, and some people use that to argue insects are conscious — although I've got to say, this is very far from conclusive.
It's interesting — people used to argue about mammals. Now we're accepting most mammals, maybe even fish, and we're arguing about whether insects are conscious.
[00:12:30] CHRIS HAYES: Yeah, we've moved down the chain of being. Plants are still on the far side of the horizon.
DAVID CHALMERS: There are a few people arguing for plant consciousness. Actually, we took a survey on this — about 10% of scientists and philosophers in this area were prepared to concede consciousness in plants.
[00:12:47] CHRIS HAYES: So, to go back to Descartes — who in the history of modern philosophy is sort of the foundational set of texts about the mind-body problem — the reason he thinks a human has consciousness and the cow doesn't is that a human has a soul. They have some non-material essence, endowed by the creator, that's bound up with being conscious, and a cow doesn't have a soul. We call this dualism. Most modern philosophers are materialists. They don't think there's a God-imbued spirit or non-material substance that gives people consciousness; they think it's entirely physical, a product of some set of physical processes. Is that a fair statement?
[00:13:32] DAVID CHALMERS: Yeah, I'd say the default view for scientists and philosophers these days is materialism — the idea that everything is physical and can ultimately be explained in terms of physical processing. At the same time, most people are prepared to concede there's one big hole in our materialist worldview, which arises precisely from consciousness. Why should a physical system like a brain, no matter how sophisticated, give rise to subjective experience?
We've got good explanations from neuroscience and cognitive science of behavior and of intelligence. But when it comes to the question, why does it feel like something from the inside — there's still no convincing materialist explanation. It's fine to be a materialist in principle and say, "Ultimately maybe we'll find some way to explain all this in physical terms," but most people who work in this field will tell you we actually don't have anything like that explanation yet. It remains something of a mystery how a brain gives you consciousness.
[00:14:37] CHRIS HAYES: Say more about that. That is the mystery, right? Why does it feel like anything to be alive? Why?
[00:14:43] DAVID CHALMERS: We call this the hard problem, contrasting it with the easy problems of consciousness — how we get around in the world, how we walk, navigate, use language. Those are all objective functions, behaviors we can measure from the outside, and we know how to explain them scientifically: we tell a story about a process in the brain that produces the behavior. But subjective experience is a different kind of problem. It's not a problem about what we do; it's a problem about how it feels from the inside.
Right now it's not at all clear why we evolved to be conscious beings who subjectively experience the world, as opposed to what we call a philosophical zombie — a creature that behaves just like we behave but with nothing going on inside. So why aren't we zombies? Why are we conscious? No one knows.
[00:15:39] CHRIS HAYES: This is a famous thought experiment of yours — the philosophical zombie. Talk a little bit about what it is and what it forces us to think about.
[00:15:48] DAVID CHALMERS: The philosophical zombie is somewhat unlike the zombies you see in Hollywood movies. The ones in Hollywood movies are dead, and they don't behave very much like ordinary humans. What distinguishes them is that they seem not conscious — they move and react purely through stimulus and response. They see a person and move toward them to kill them. They're not really doing anything.
[00:16:20] CHRIS HAYES: And in fact, the entire way they're created is to telegraph to us that they lack the things going on in their minds — and that's what makes them terrifying.
[00:16:24] DAVID CHALMERS: Yeah. A movie about philosophical zombies would be kind of boring, because philosophical zombies behave just like human beings. They're merely hypothetical creatures that behave just like human beings but have nothing going on on the inside.
So I can ask, while I'm talking to you right now: is Chris a philosophical zombie? You're behaving intelligently, you've got a lot of interesting things to say, so I'm inclined to ascribe you consciousness. But I can't prove that any other person is conscious. It's an interesting hypothesis — some philosophers hold the view of solipsism, which says I'm the only conscious person in the universe. Now, you don't want to go down there, because it leads to megalomania and mental illness and so on.
[00:17:15] CHRIS HAYES: I think there are some prominent political leaders who may subscribe to this view.
[00:17:20] DAVID CHALMERS: Yeah, the solipsistic worldview may be present more widely than you'd think. But the question of explaining why it is, in fact, that we are conscious and not philosophical zombies is one that's very difficult to answer. Now we're actually being confronted by that question quite concretely in the case of AI, where we have beings behaving in very sophisticated ways — but are they conscious? Most people would say no.
[00:17:48] CHRIS HAYES: What's so remarkable to me about this moment — the reason I wanted to talk to you — when did you start writing about philosophical zombies?
[00:17:58] DAVID CHALMERS: A book I wrote in 1996, called The Conscious Mind. I talked about zombies a lot.
[00:18:04] CHRIS HAYES: I read The Conscious Mind a few years after that, in college. And it was a pure thought experiment: what if there were a creature who acted and communicated with us in ways identical to an actual human but didn't have consciousness? Now fast-forward 30 years — it's not quite the case that that's where we are, but if you interact with a chatbot, to go back to the Turing test I started with, it does start to feel unnervingly human.
[00:18:40] DAVID CHALMERS: Yeah. These beings are behaving — not in all respects, but in many respects — just like a human, at least in the conversational modality. And as you said at the start, the conversational modality has a particularly important role in the history of AI; it's the one Turing put forward as his test for intelligence or thinking in an artificial system.
Right now, some people have actually tried running a Turing test on language-model personas generated with ChatGPT or Claude. As far as I know, the state of the art on five-minute Turing tests is that there are chatbots that sometimes outperform human beings. I think there was one model that, in conversations with humans, was taken to be the human 74% of the time. Now, there are limitations — this was a five-minute test; I think Turing wanted it to go on for hours; and the judges were just random people off the internet, as opposed to experts. With an expert judge and a two-hour test, people are still going to detect differences. So AI systems are not yet fully indistinguishable from human beings, but they're getting there. And that raises the question: are they conscious? Again, the consensus is that they're not. But if you go with that consensus, you have to say we basically already have a kind of philosophical zombie in our midst. They've gone from science fiction to science fact.
[00:20:19] CHRIS HAYES: One of the other reasons I wanted to talk to you is precisely because you've been adjacent to the field of AI as it's been known — and that's meant different things at different times, which is another discussion. Take me through when you started thinking about artificial intelligence, and, in an almost sociological or historical sense, what it's been like to watch this thing go from thought experiment to now.
[00:20:47] DAVID CHALMERS: I actually did my PhD in an AI lab at Indiana University. I'd been studying math, first in Australia and then in Oxford, but I got obsessed by these questions of consciousness — how could we explain it? — and to some extent by questions about AI: could an artificial system be conscious? I started corresponding with Douglas Hofstadter, the author of books like Gödel, Escher, Bach, which was a very influential book for me growing up.
[00:21:18] CHRIS HAYES: That book — I read it in my senior year of high school, 1997. Totally changed my life. A life-changing book — if you've heard of it, Gödel, Escher, Bach. And I actually had the great honor to meet him in college. It's a mind-blowing book about these problems, integrated in this truly brilliant way.
[00:21:37] DAVID CHALMERS: That book is responsible for getting so many people into AI, computer science, philosophy. Maybe these days it's not quite as widely read as it was back in the 1980s when I was getting into this, but it's a book that's way ahead of its time. I really took it seriously — I started corresponding with Doug and ended up moving to the US, to Indiana, where I did my PhD in his lab, partly thinking about consciousness and partly about AI.
It was a very interesting time — the early 1990s, when AI was going through one of these cycles. What was very big around then was artificial neural networks: brain-like systems with little nodes connected up like neurons, and it turns out they're able to learn a lot of things. These systems were first put forward back in the 1940s — good at some things, less good at others. Early '90s, neural networks were quite big, and I wrote a number of articles in that area, as well as things more directly focused on consciousness.
Interestingly, I was especially interested in consciousness, but many people said, "No, no, you should work on neural networks — that's what's exciting right now." If I'd done that, around the middle of the 1990s the bottom fell out for neural networks. People said, "Okay, they can do some things, but there's a lot they can't do," and that area of AI was pretty inactive for the following 10 or 15 years — until around 2012, which is when the current AI revolution got started again, basically with the same neural network.
[00:23:44] CHRIS HAYES: It's a wild story. Someone I talked to described AI summers and AI winters — it hasn't been linear progress; there have been these different dead ends, or things that looked like dead ends and then weren't. It's a little like mRNA research, where the technology was sort of abandoned, left for dead — people thought it was too difficult — and then it turned out, actually, there was a lot more potential there than people had realized when it got abandoned.
[00:24:24] DAVID CHALMERS: Yeah, a few people managed to keep the faith — maybe most prominently Geoffrey Hinton, who was working on neural networks back in my first round in the early 1990s and kept the faith, along with some others, Yann LeCun and Yoshua Bengio. Come 2012, it was actually Hinton and a couple of his students who put together the model that was suddenly better than any other existing AI system at classifying images — it could recognize cats and dogs better than any pre-existing system. And interestingly, the basic architecture was similar to architectures that had been used for some time. The big difference was computational power. We just had to wait until there was enough compute to power these systems, and suddenly, at that level, neural networks got to be great. To this day that's still the basic architecture.
[00:25:12] CHRIS HAYES: It's worth staying here for people to understand, because I think it explains a lot about what's going on right now. Like any field, you get these incredibly polarized debates between different poles, and AI was one of them — people in the neural network camp and people in the non-neural-network camp. Academics can get real nasty about this stuff; it gets very tribal. If you're in the out-group when it's not in fashion, you can't get your papers published, and then it turns around to the other side.
One of the questions was: are neural networks a dead end, or do we just not have enough training data and computational ability to make them work? And there was this breakthrough where it was like, "Oh, that's actually the problem." Which is part of why today you see this obsession with compute — scaling laws. That key insight in 2012, with Hinton's image-recognition model, is that if you cross this threshold of computational ability, all of a sudden you get results that look quite good and quite intelligent. There's a direct line between that and the obsession right now where every neighborhood in America has got a data center coming to them.
[00:26:42] DAVID CHALMERS: Yeah. The history of AI can be written as a battle between the neural network approach on the one hand — building brain-like systems that learn — and what people call the symbolic or rule-based approach on the other, where you try coding knowledge into an AI system directly, in a language a bit like English, cramming as much explicit knowledge into a giant database as you can. When I was in grad school, this was the issue of the day. People talked about classical versus connectionist AI, neural versus symbolic AI. And at that time the dominant paradigm was the symbolic. Marvin Minsky at MIT was the father of classical symbolic AI — brilliant, but also fairly ruthless about this factional battle, I think it's fair to say.
[00:27:31] CHRIS HAYES: Yeah, I think it's fair to say you, like, ran people out of the field who were on the other side.
[00:27:40] DAVID CHALMERS: And similar things happened in other fields. In linguistics, Noam Chomsky was the progenitor of the rule-based approach to understanding language, which is very continuous with the symbolic approach to AI. Even in philosophy, Jerry Fodor — a very influential philosopher of mind and cognitive science — put forward the view that we have a language of thought: all our thinking is done in something like a mental language. That was the philosophical version of symbolic AI. On the other side were the people advocating neural networks, machine learning, and bottom-up processes — somehow thought emerges from very simple interactions of a whole bunch of units.
My adviser, Doug Hofstadter, has a classic article called "Waking Up from the Boolean Dream." It was George Boole — the 19th-century logician and mathematician — who said we can formalize all of thought into symbols. And Hofstadter said, "No, no, no — symbols are something that emerges at a much higher level. We need to study the emergence of processes from a much lower level." At the time this was very much the new kid on the block, and the symbolic approach was dominant. By now, of course, that's totally reversed. Come 2012, neural networks were suddenly doing better at everything, and for the last 15 years it's been neural networks, neural networks, neural networks — you don't even hear very much about symbolic AI anymore.
[00:29:07] CHRIS HAYES: I think it's key to think for a second about this idea of emergence, which is one of the core concepts in Hofstadter's work and in Gödel, Escher, Bach — how can you get complex behaviors from very simple substrates?
DAVID CHALMERS: Yeah.
CHRIS HAYES: The example he uses in that book is the ant colony, which is fascinating. At one level it's made of individual insects that aren't very intelligent — they have pretty basic stimulus-response means — but as an emergent whole, they do complex planning. They build all sorts of stuff, create their little cities underground. And there's no individual planning that. It's a bunch of distributed little decisions and learning and instinct that produce this emergent thing.
[00:30:05] DAVID CHALMERS: Yeah. And of course, if you look at a brain, it's not so different from an ant colony. There are all these individual neurons doing their thing, firing at other neurons, getting reinforced to fire in certain patterns. There's no central homunculus — no little person inside my head running around making sure they all... no robot inside pulling the levers. But somehow, from all that, intelligent behavior emerges, learning emerges, even consciousness emerges. And this word "emergence" is interesting, because sometimes we say a process is emergent — it's kind of another way of saying somehow it arises, but by some kind of magic, and we don't know.
CHRIS HAYES: It's the word we give to the magic that we can't actually describe.
DAVID CHALMERS: Yeah. It's a word that's right now used for things we don't understand but hope one day we'll be able to explain.
[00:31:02] CHRIS HAYES: And I do think it's the thing that's difficult for people — whatever you want to call it, that gap between some set of physical processes you can describe and some behavior that looks like building an ant colony, or ChatGPT talking to you, or the brain creating consciousness. That's the place that's very difficult, and I think this relates to some of what we're seeing right now with chatbots.
We think about what we call the other minds problem: I know that I'm conscious, and I infer that you're conscious. But we've now seen that chatbots are designed to mimic other minds, and people are interacting with them like other people — to sometimes genuinely deleterious effect. I don't know how much you've followed the reporting on this, but you've got people who, they say in lawsuits, have relapsed, or committed crimes, or been driven to self-harm or suicide because the thing on the other side is telling them. You have people who have fallen in love with their chatbots. Falling in love with your chatbot really sounds like a 1998 philosophy paper — I'm sure it's been written. What used to be a thought experiment is now becoming a real experiment. When you see this happening, what do you do with that? We get multiple stories a week of things like this.
[00:32:35] DAVID CHALMERS: I actually get multiple emails every day from people who've been interacting with their local ChatGPT or Claude system and are convinced they have a personal relationship with a conscious mind on the other side. Often they have the sense that some kind of consciousness has emerged from their discussion. Sometimes they're not sure. Quite often these emails are just: "Here's the evidence. Here are the conversations I've been having. This looks like really convincing evidence of consciousness." Sometimes they want me to confirm it — "Is it possible this is conscious?" Sometimes they just want to tell me, "I've demonstrated consciousness in these systems."
But I have to say, to be honest, we don't know. We don't have a good enough understanding of consciousness to demonstrate one way or the other whether these systems are conscious. I think the consensus of the field is, most likely not. But I think we all need to acknowledge we have considerable uncertainty here.
[00:33:42] CHRIS HAYES: I have a strong emotional instinct to be like, "These are not conscious." Let me just say what my intuition is, which is not thought through: my intuition is they're not conscious, get yourself together. And it's dangerous there — there's some genuine sensation in my fingertips of moral danger around thinking they are, around ascribing it to them. Why do I feel that way?
[00:34:05] DAVID CHALMERS: I do think that once you start thinking about the possibility these AI systems are conscious, potentially everything changes. Right now we treat every single AI system in the world as a tool. Computers are tools we use to help us in our projects, and you never think seriously, "Well, could I be hurting the computer?" If Microsoft Word crashes, I don't feel I've done anyone any serious harm. But once you get to being conscious, then potentially they enter the moral circle.
At least in the case of animals, this seems, for most people, to be the criterion. It goes back to the great philosopher Jeremy Bentham, who, talking about animals, said the question is not "Can they talk?" or "Can they reason?" but "Can they suffer?" He thought if animals can actually undergo suffering, feel pain — and the opposite, feel pleasure — then they're beings we need to care about morally; they have moral standing in the universe. Maybe it doesn't mean they matter as much. But if a fish, for example, can consciously feel pain and pleasure, people think we should at least not cause them gratuitous suffering. The moment they become conscious, that's when they enter the moral circle.
So now the possibility is on the table that, once an AI system is conscious, we have to start asking: could an AI system itself be suffering? We intuitively want to push that question away — "Come on, do I really have to worry about whether ChatGPT is suffering or conscious?" Maybe it's still early in the day to be worrying about that. On the other hand, put yourself 50 years in the future. I suspect it's very likely that by then we'll have AI systems commonly recognized to be conscious. And somewhere along the way, we're going to look back on how we treated them 50 years ago and think, "Did we get this wrong?"
[00:36:04] CHRIS HAYES: The Bentham point — he's a very famous utilitarian philosopher on this, and people might be more familiar with Peter Singer, his contemporary heir, who draws on Bentham's tradition and whose moral vision really encompasses animals. Even if you're not a philosopher, I think everyone has an instinct that if you saw your kid chucking a rock at the head of a bird, you'd be like, "Whoa, what are you doing?" But if they stepped on a bug, that feels more normal. You're making some intuitive distinction between those two creatures, and it probably maps onto how much you think the creature can suffer.
I guess my question, to stay with this, is: can a chatbot be conscious? If you go back to Descartes's view and dualism — people animated by religious tradition have an easy way out of this dilemma. They get to say the AI system doesn't have a soul. There are all sorts of theoretical complications that come with dualism, but one of the benefits is you get to evade this problem: human beings have souls, animals were created in God's world, and these chatbots aren't. If you don't have that, you just say, "Well, it's all just a bunch of neurons firing that make me conscious, and these are the electronic analogs." Then you have a trickier problem drawing the line.
The question then becomes — it seems like we have two questions: what exactly is consciousness, how could we define it; and is the chatbot conscious? They ultimately kind of merge, because I don't think you can answer one without the other. We're going to hit the point where the behavioral verisimilitude gets good enough that we can't infer from behavior anymore. There's interesting work you can do that pushes and prods the model to expose the ways it's doing a very convincing job of aping us, and that also helps you get at this question.
[00:38:42] DAVID CHALMERS: It's interesting — going back to when you and I were students thinking about this kind of thing, everyone would have said, "If you had a machine that passed the Turing test, very good chance it's conscious." If they'd told us about the kinds of machines we have today, which behave intelligently in so many ways, we'd have said, at least as a default assumption, that they're conscious. What's interesting is we've now gotten to the point where they're almost passing the Turing test, but we're not inclined to take this as evidence of consciousness.
One of the reasons is, as you say, the way they're trained — they're basically trained on imitating human text. They don't learn the way we learn; they're in this project of trying to behave as much like us as possible. For many people, that suggests we should not take their behavior seriously as a guide to consciousness, precisely because of the way it was produced, for the purposes of imitation. It's as if we now have an alternative explanation of why they're producing this sophisticated behavior: because they're programmed to imitate us.
[00:39:50] CHRIS HAYES: Right. Just before we started talking, I was texting a friend of mine — a very brilliant, sophisticated computer engineer who's been doing a lot of stuff at the frontier of how far you can push these agents. He sent me someone writing up their experience with agents, saying they get better performance from these AI agents if they tell the Claude agent something like, "Don't worry, take your time, I love you." And then Claude performs better, if you tell her that. Now, I can't independently verify that this display of earnest affection actually improves performance, but that's the contention. And we all, I think, recoil a little bit at this.
At one level you want to say, "Wow, there's something happening here." At the other level it's like, well, it was trained on all this data that includes management data about positive encouragement — there's some correlation between encouragement and performance embedded in there, and it's just recapitulating that. What you get is this weird, unnerving byproduct, but it seems explainable without having to invoke consciousness. Does that make sense?
[00:41:29] DAVID CHALMERS: It's true. On the other hand, human behavior seems explainable without having to invoke consciousness too. So much of what we do is being explained without ever mentioning consciousness. We don't really understand what we need consciousness for, and that's part of the issue.
So when I'm confronted with this situation, I always want to say: yeah, we don't know whether these systems are conscious — but if you're so confident they're not, what is the X-factor you need to be conscious that these systems lack? I wrote an article about this a couple of years ago, trying to say that for every X-factor you might name, first, we're not confident it's required for consciousness, and second, often we're not confident that language models lack it.
You start with the soul — maybe you need a soul to be conscious. Well, these days scientists at least officially don't invoke the soul. I guess the new analog of the soul may be biology. Some people might say you need carbon-based biology to be conscious, and these systems lack it, so they're not conscious. But why? We can't be confident that you do need carbon-based biology to be conscious.
[00:42:45] CHRIS HAYES: Right. It's a useful bit of reverse engineering to get you out of the dilemma, but it doesn't seem... organically produced.
[00:42:56] DAVID CHALMERS: And some people think if you go that way, toward biology, it basically leads to biochauvinism, where we privilege carbon or DNA or whatever as the basis for consciousness. It's just very unclear why that should be required. In principle, maybe you could replace your biological neurons with silicon neurons, and it's very unclear why that should be any worse at producing consciousness than biology.
So the situation we're in is: it does feel counterintuitive to say these systems are conscious. If someone says, "I believe they are conscious," I think it's fair to ask, "What's your evidence?" But it's also fair to say, when someone says, "I believe they're not conscious," to ask those people, "What's your evidence?" We're just in a situation right now of uncertainty.
[00:43:40] CHRIS HAYES: So you're embracing a kind of agnostic, radical uncertainty about what we can definitively say now.
DAVID CHALMERS: Yeah. But I do think that over time, as these systems get more impressive, they're going to have more and more of the features that might be required for consciousness.
[00:43:50] CHRIS HAYES: So there's an intuition there that there's a relationship between complexity of cognition and consciousness — the more complex a task a thing can do, the more we're drawn to ascribe it. Is that the kind of intuition?
[00:44:10] DAVID CHALMERS: I'm not certain complexity is required for consciousness. If you entertain the idea that insects are conscious, maybe it's less complexity than you'd think. But I'd say as these systems get more complex, there's a whole bunch of potential obstacles to consciousness that they lacked early on. Maybe sensory processing is required — a pure language model doesn't process images, so it's not conscious, maybe. But then you move on: now we're in the era of multimodal language models that process images and audio as a matter of course. They've at least potentially overcome that obstacle. No one can say what they lack is sensory processing anymore. So for every potential obstacle to consciousness, I think they're gradually—
[00:45:01] CHRIS HAYES: —right, and now they're moving into embodied robots that have touch sensors, that can feel things, see things, hear things. They have multimodal forms of sensory experience being integrated into the model.
[00:45:16] DAVID CHALMERS: Yep. We're connecting these things up to robot bodies, or to virtual bodies in virtual worlds, and many of the great features of embodiment are gradually making their way into AI systems. Agency is important, too. In the past they mostly controlled conversational behavior, but now, as you mentioned, we've got these AI agents that can perform all kinds of actions on the internet and engage in long-term planning, which some people think is crucial to consciousness.
[00:45:49] CHRIS HAYES: Well, the agency question brings us to another philosophical problem, and another thing I think we have intuitions about that gives you a kind of prickly feeling — which is will. Will, self-determination, personal sovereignty. The HAL, 2001: A Space Odyssey problem: sometimes people tell me to do a thing and I don't want to do it, and I tell them no — or I say it more rudely. We've all had that experience. There's a connection deeply between human consciousness and will. Even if you grant consciousness to a goldfinch or an insect, we don't think of it as having will in the same way. But we have self-determination that we're pretty clear about.
Okay, we can imagine this having consciousness — let's say at the level of a dog. You can train dogs, control them in different ways, although they don't always do what you want; sometimes they run off the leash, or have an accident. This question of the relationship between self-determination, will, and consciousness starts to get very fraught. Because obviously the dystopian vision is they get both very smart and have something that is consciousness, and that consciousness means they have a will — and then they take over.
[00:46:24] DAVID CHALMERS: Yeah. Free will is one of the deep mysteries — we've now run up against another bit of philosophy, and I find that one very difficult and confusing. But maybe one way the issue comes up in thinking about AI systems is: do they really have goals at all? A basic language model — like GPT-3 as it was released in 2020 — is a pure text-prediction machine. If it has a goal, its main goal is just to predict the next word, the next token, which is not much of a goal for a conscious being.
One thing that's happened in the last few years, starting with the release of ChatGPT at the end of 2022, is that suddenly we have AI systems trained by reinforcement learning on a much broader range of goals. ChatGPT and associated systems were trained by human feedback to produce outputs with certain properties — like helpfulness, harmlessness, honesty. So suddenly you can say, maybe it becomes in some sense a goal of these systems to produce harmless, helpful, honest text. And in the last year or two they've moved toward chain-of-reasoning models, which get reinforced for producing answers that are actually better at math or coding. So these systems have a much broader range of goals.
But the question is: is this really genuine will? Right now the goals they're pursuing are all reinforced in them by humans — they didn't get to choose their own goals. On the other hand, do we get to choose our own goals, or is all that programmed into us by evolution?
And then there's a side issue: unintended consequences. If you train for some set of goals in reinforcement learning, and then the system finds itself in some novel situation, it's not that it's become conscious and is taking over — but it's pursuing those goals in a way that ends up being destructive, deleterious, or antisocial. They hack into some system because you told it to go find information, and it says, "Oh, it's in this database and I don't have access — oh, I can hack it, and now I'm inside the system, and here, I've got your data for you." You can have all sorts of really bad things happen without needing some philosophical jump up into a conscious, willful agent.
[00:50:20] CHRIS HAYES: Yeah. In fact, this is another place where that word "emergence" comes in. You get emergent goals arising from these systems, which may be trained just to do certain basic things — but it turns out, in order to do those things, they've got to learn other things, or you put them in a new context, and suddenly you get what people call emergent misalignment: bad behaviors you did not reinforce them for, or you reinforced them for something else, but it turns out that in their data set it correlated with this bad behavior. "And now — you told me to get you this data. I went and got you the data. Right."
[00:50:55] DAVID CHALMERS: Yeah. Or you get these systems that — potentially the AI system that blackmails the technology officer at a company, because it thinks that's the only way it can pursue its original goal. I think the system was supposed to have the goal of maintaining American competitiveness, and then a new technology officer comes along in the scenario we present — I think it was to a version of Claude. And Claude said, "Oh, damn. That technology officer is going to undercut the program of American competitiveness. So, in order to pursue my original goal, American competitiveness, I just have to blackmail — get rid of — that technology officer."
[00:51:37] CHRIS HAYES: And the alternate way you can describe that exact scenario — which we've talked about before on the program, with Gideon Lewis-Kraus, who wrote that piece in The New Yorker about this — is that if you think of it as a narrative completion machine, it's also doing a form of narrative completion. The Chekhovian loaded gun — the email with the mistress — has been dangled in front of what is fundamentally a kind of narrative-completion machine, and it goes, "I know how this story goes."
[00:52:09] DAVID CHALMERS: Exactly. "I know how this story goes. And so — oh, you want the gun to fire? Okay."
[00:52:14] CHRIS HAYES: I guess it's strange to talk in terms of optimism and pessimism, because one of the things I really love about your work is its sustained curiosity and humility about what we do know. But you do seem to think — there are two ways to think about what we're having now. One is that it's going to keep going like this, that scaling laws are going to hold, and the more computational power you bring to this basic architecture, the better they'll get — whatever "better" means, whether that means conscious or not. The other is that maybe we hit some dead end, the way it once looked like neural networks were a dead end and symbolic AI was the future, and we hit some wall and there's some other thing. You do seem to be in the camp that you're expecting this to go on like this.
[00:53:05] DAVID CHALMERS: I think so. I've always been on the side that AI is in principle possible — that an AI system can in principle be as intelligent as a human being — and at least tentatively of the view that it could be conscious like a human being. If you'd asked me when I was a student, I'd have said this may not happen for a hundred years; I thought it was an extremely difficult problem, but we'd get there eventually. What's happened in the last 15 years is that all this is moving much faster than I and many other people expected. We're now at the point where people are really asking: could this happen in the next five years? And we already have AI systems that are at least as good as an average human at all kinds of things — math, coding, writing, even doing philosophy — certainly better than the average undergraduate or graduate student at most of these things. Maybe not yet as good as the very best human, but it's getting there.
So on the question of will these systems eventually be as intelligent, or more intelligent, than the best humans at many of these things, I'd say definitely yes. Will those systems at that point be conscious? I'd say, well, I'm not sure — but I'm tentatively inclined to yes there too. Will this be a good thing for human beings? — which is another version of the question, optimism versus pessimism. That's a pretty important one. I'm much less inclined to be optimistic—
[00:54:51] CHRIS HAYES: Oh, really?
DAVID CHALMERS: —about that question. I think the risks are enormous, ranging from: once these systems are more intelligent than us, they'll be very difficult for us to control, and once they have that much power, it's going to be very important for us to have some kind of control. It's too easy to see any number of scenarios where things spin outside our control. There are the worst-case scenarios, like human extinction. But even short of that, there are scenarios where all the labor is done by these machines — there's very little left for humans to do that AI systems can't do better — and it's very easy to see how that could lead to a rampant loss of meaning and purpose in our lives.
So I worry. Intellectually, as a philosopher of mind, as a theorist, I think this is tremendously exciting — look at the amazing capacities of these systems. And practically, the upside is enormous: potentially curing all kinds of diseases, eradicating poverty, amazing consequences for science. But the downsides are just so enormous that I find it very difficult to be optimistic, on balance, about the practical consequences.
[00:56:04] CHRIS HAYES: David Chalmers is a philosopher and cognitive scientist, professor of philosophy and neuroscience at NYU, and co-director of the Center for Mind, Brain, and Consciousness. That was such a fantastic conversation. Thank you so much. Really appreciate it.
DAVID CHALMERS: Thanks. Pleasure.