Comments?
I'd love to hear!
Searle
and the Chinese Room
........................................Rob
Bass
Searle is frustrating. In some
ways, in his arguments about the Chinese Room and the more general
argument about the relation between syntax and semantics, he comes so
close to getting things right, but then turns in the wrong
direction. In fact, as I’ll try to indicate in the latter case, he
gives the right argument, draws the correct conclusion, and then
misinterprets his own conclusion. In other words, I’m going
to claim that he is right that semantics can’t be reduced to
syntax, right that running computer programs can be characterized
entirely in terms of syntax, right that genuine understanding depends
(in humans) on the causal powers of the brain (surprised?) – and
wrong about Strong AI!
But before we get to that, let’s say something about the Chinese Room thought experiment itself. I think it is both over- and under-imagined (charges which may come to much the same thing), and its persuasiveness depends critically upon the extent to which this is so.
First, it is over-imagined. Searle, in effect, asks us to imagine that he’s locked in the room, that he can follow instructions (in English), that he has written transformation rules at hand and that, by following the rules and passing slips of paper through the slot in the door, he appears (to Chinese speakers) to be engaging in intelligent conversation in Chinese. He asks us to imagine it and – by virtue of the fact that he proceeds to tell us more about how the Chinese Room works, what goes on there, what impressions observers have, and so on – implicitly claims to have imagined it himself.
But has he? Have we? The important question for Searle’s argument (and for many similar thought-experiments) is not whether, in some sense, we have imagined a postulated state of affairs but whether the state of affairs that we have imagined is a coherent possibility. After all, in some sense, I can imagine inventing a perpetual motion machine. (There it sits on the lab bench, spinning away without any external power source.) I can even imagine logical impossibilities, like finding a proof that there’s a largest prime number. (I know what a mathematical proof is and what the last line would have to look like. I can be a bit vague about just what steps are supposed to lead up to the last line, can’t I? After all, if I had to be completely definite about every step, I’d already have the proof; I wouldn’t just be imagining it.) But that kind of imagining cuts no philosophical ice. It might be better to call it “imagining we imagine” rather than “imagining” (or, in a different terminology, we could distinguish between imagining something and conceiving of it.)
So far, this is only a possible ground for suspicion: Perhaps Searle is asking too much of our imagination, and is persuading us that we have imagined a coherent possibility when we have not. Whether there is positive reason to think this is actually so can be pursued by considering the ways in which the Chinese Room is under-imagined.
The most obvious way in which the Chinese Room appears to be under-imagined has to do with the extent to which Searle has under-estimated what would be required for it to work in the way he describes. If Searle is going to be able to pass slips through the door, in response to slips passed in, that convince native Chinese speakers that a Chinese speaker is in the room, then the rules with which he works will have to encode large amounts of information about the Chinese language itself, including grammatical and structural features that are not normally available explicitly to typical Chinese speakers. But the key lesson (for present purposes) of post-Chomsky linguistics is that this is by no means a trivial task. In fact, one way of representing those results is to say that what Chomsky and successors have shown is that language-learning is impossible. More precisely, it is impossible to learn language from experience alone, given the amount of experience that children typically have through exposure to speakers of their native languages. If, as they do, they manage to reach reliable conclusions about the features and structure of their languages, and are therefore able rapidly to acquire competence in language-use, they must be guided by something other than their own experience. The solution seems to involve some kind of evolutionary assist: There is, so to speak, a language module in the brain that predisposes children to interpret their experiences of language in a certain way. Language, or better, the cognitive capacities drawn upon in language acquisition and use, is largely (for each individual) innate rather than acquired through experience. (In this connection, I highly recommend Steven Pinker’s The Language Instinct.)
But this is just the beginning. Not only will the rules have to encode enough information about the structure of Chinese for the slips of paper he passes out through the door to look like grammatically competent conversation in Chinese, they will also have to make provisions for the (non-existent) Chinese speaker to have something to say. The rules will have to encode “beliefs” about his personal history, general knowledge of the world, adherence to customs that are ‘second nature’ for real Chinese speakers, and much more. It will also be necessary for the rules to encode inference patterns, so that the “beliefs” that Searle in the room expresses get updated in normal ways.
Granting that a set of rules complicated enough to allow for all of this and more is possible in principle, it is frankly incredible to think that it would also be a set of rules that Searle could apply and follow in real time – i.e., fast enough to convince genuine Chinese speakers outside that anyone inside understood what was going on. It would be even more incredible to imagine that Searle could memorize and apply the rules without the help of external rule-books. If we insist on realistic limits to what a mere syntactic rule-follower can do, we get either quick responses that native speakers can easily recognize as not showing understanding of Chinese or else we get painfully slow responses that, equally, no one would be tempted to regard as exemplifying real understanding. In neither case do we get slips passed through the door that native speakers find indistinguishable from carrying on a conversation with a real Chinese speaker.
In practice, if symbols that appear to represent intelligent conversation in Chinese are passing in and out of the door, we can be virtually certain that someone or something that understands Chinese is inside (or connected to the inside of) the room. That it is logically possible that some alternative arrangement (Block’s ‘Blockhead’, maybe) could produce the output without understanding it is nothing to hang one’s contrary intuitions on. (The Turing test is a good test without being a logically sufficient criterion.)
The Chinese Room, as Searle describes it, coaxes a certain intuitive response from us, but only because, in accepting the description, we fail to realize that what has been described is not really possible. Searle over-imagines it and convinces us that we have actually succeeded in envisioning a coherent possibility because he has under-imagined what would be required for the scenario to be realized.
Now, let me pause for a moment. I may appear to be taking back something I’ve said. I think there’s a useful point in the Chinese Room story that’s brought out by an amendment. Someone tells Searle (whom I will assume has internalized all the rules) in Chinese that his family has suffered a horrible disaster. In Chinese, by manipulating symbols according to the rules, Searle responds appropriately. But later, in English, Searle is cheerful, talks about what he’s doing with his family on the weekend, etc. So, we are invited to conclude that he doesn’t really understand after all. His verbal competence in Chinese doesn’t show that he really understands. Since Searle with the internalized rules has everything (syntactical) that an appropriately programmed computer has, the computer or its running program doesn’t understand either. (Or, at least, if it understands, its understanding is not due to the syntactic features alone.)
I think that’s exactly right. The purely syntactic manipulation of formally defined symbols, receiving such symbols as input and producing them as output does not amount to understanding. But I don’t think this leads to quite the conclusion that Searle has in mind. What it vividly highlights is that apparent verbal competence alone does not make Searle with the internalized rules (or a computer running a program) indistinguishable from a native speaker. He is distinguishable because he doesn’t behave in the right way.
The point here can be put in terms of Davidsonian charity of interpretation: We have someone, Searle, who has internalized the relevant rules, who appears to participate in conversations in Chinese. What is the right way to interpret what he is doing? Is he really speaking and understanding Chinese or not? Perhaps he’s speaking some language – or just producing a series of sounds – that phonemically matches his part of a conversation in Chinese. Whether it is reasonable to interpret him as speaking Chinese depends on what we take him to believe and to aim at. If he had sufficiently bizarre beliefs (his family having been murdered wouldn’t interfere with a pleasant weekend at the lake with them) or sufficiently bizarre objectives (his idea of a pleasant weekend at the lake is something that can be carried out with corpses), then maybe his English-language response to questions about his weekend plans wouldn’t be so puzzling.
But there are limits – almost certainly transgressed in this example – to how far we can take that. To understand speakers (or ‘speakers’) as intelligent, we have to assume that, by and large, they have reasonable aims and beliefs, that, in Dennett’s phrase (I think), they are “believers of the true and lovers of the good.” There’s a certain looseness of fit here; treating somebody as intelligent and as speaking a language that we understand doesn’t require thinking that he gets everything right (by our lights), but this occurs, and must occur, against a background of substantial agreement. To the extent that we can only interpret him as speaking and understanding the language in question by attributing bizarre beliefs and objectives to him, we have reason to reject that interpretation. To put it differently, if we insist on the assumption that Searle’s verbal behavior in the example exhibits understanding of Chinese, then what we gain on one side, in being able to interpret him as using a language we already know, we lose on the other because we are no longer able to understand him as rational and intelligent.
To begin to draw the strands of this discussion together, facility in the manipulation of a formally specifiable symbol-system is not sufficient to attribute understanding: The system to which understanding is attributed must also be causally engaged with the world via perceptual and behavioral links. In the absence of the right sort of causal engagement, we have no basis for attributing understanding.
We can elaborate this by looking at Searle’s argument which he takes to be encapsulating the results of the Chinese Room thought-experiment. Here’s a stripped-down version adapted from his presentation in Minds, Brains and Science (p. 39):
1. Brains cause minds.
2. Syntax is not sufficient for semantics.
3. Computer programs are entirely defined by their syntactical structure.
4. Minds have semantic contents.
Therefore, programs are not minds, and they are not by themselves sufficient for having minds.
That’s valid and the premises are true (or can be interpreted so they’re true) – so the conclusion must be true as well. We can add that since brains do cause minds, they must have causal properties, other than anything specifiable syntactically, in virtue of which they do so.
So far, so good. And so far, it sounds exactly like what Searle says. So where does Searle go wrong? He just misinterprets his own conclusion. When he talks about these causal properties of the brain, he seems to suppose they involve some weird “secretion” of semantic content or intentionality that somehow computer programs can’t manage. But, in fact, there’s another and much more attractive option available. The causal properties the brain needs in order to produce intentionality or semantic content or ‘aboutness’ are just (reasonably) ordinary and well-understood causal properties.
But before going on, let me head off a possible misunderstanding. What I’m about to do is offer a sketch of a theory of semantic content or, better, of intentionality, the genus of which semantic content is a species. Some people – Searle may be one of them – will object that it can’t be right because it doesn’t say anything about conscious meaning. (This may be part of what drives the Chinese Room argument. The inference may run from ‘Searle in the room does not consciously understand Chinese’ to ‘Searle in the room does not understand Chinese.’ Plainly, that’s not valid unless all understanding is conscious understanding. But that sideline is not really worth pursuing since there are other and more basic things wrong with the argument.) Now, it may be, for all I’ve said to this point, that genuine semantic content does presuppose consciousness. In fact, I think it’s almost the other way around. Intentionality is a precondition for the appearance of consciousness, not something that can be tacked on later. But be that as it may, Searle’s argument does not hold that semantic content is necessarily conscious; rather, it’s about the relation between syntax and semantics (and what that implies about minds, brains and computers). Adressing his argument does not require that the account of semantic content be an account of conscious content as well.
To return, I’ve already mentioned a couple of the more important causal properties that systems with semantic content need to have in discussing the Chinese Room, but I need to add a little, so consider the following (over-simple) example:
Suppose you have a dog-detector. It gives you a signal when there are dogs about (and maybe a different signal when dogs aren’t about). At least, that’s what it’s supposed to do. It may fail. But nobody would be tempted to call it a dog-detector if it didn’t typically signal the presence or absence of dogs reliably.
Note now that there’s more than one way it could be unreliable as a dog-detector. It could be unreliable by not indicating the presence or absence of dogs. It could also be unreliable by not being fast enough. What’s fast enough depends on why you carry around a dog-detector. If you’re a mail carrier who needs to grab the pepper spray before a hostile dog attacks, then even an infallible detector that delivered its signals 30 minutes after the dog was running toward you would be worthless. Better to have a fallible detector that delivers its signals fast. At the risk of paradox, a detector that gets things wrong sometimes can be more reliable than one that never gets things wrong – because, when the detector’s output has to modulate behavior, getting the right answer too late is a way of getting the wrong answer.
What, however, makes it a dog-detector? Part of the answer has already been suggested. It has properties such that it tends to be the case that it signals “dog” just when dogs are about. That, of course, is a kind of causal property. It still isn’t enough, though. What if there are other things that tend to be around when dogs are – such as high flea concentrations? What makes it a dog-detector rather than a high-flea-concentration-detector? Well, the fact that it’s supposed to detect dogs, that detecting dogs is what it’s for, that when it misses a dog it’s malfunctioning, but when it misses a flea circus, it’s not.
And what sense can we make of that? How do we cash out “supposed to”? I shall be very short and peremptory here – because the alternative is to be very long, and the central arguments have been well-made elsewhere. (For something much more detailed and fleshed-out, I recommend Ruth Millikan’s Language, Thought and Other Biological Categories. For my money, it’s the state-of-the-art discussion. Of especial importance for this issue are Chapters 1, 2 and 5.) The detector is supposed to detect dogs if detecting dogs is its function, and detecting dogs is its function if the properties of the detector that enable it to reliably signal the presence of dogs are there because they result in the reliable detection of dogs. In short, it must have been designed, either by a conscious designer or by an evolutionary process, for the detection of dogs. That kind of causal history is what can make sense of saying that the detector is functioning properly when it detects dogs and not when it detects flea circuses. That’s why the “dog” signal is about dogs and not about something else.
That’s an extremely rudimentary case of semantic content. But, rudimentary as it is, it’s enough to make some important points. Having semantic content is not just a matter of having the right internal syntax. It depends also on the right kind of causal engagement with the world and the right sort of causal history. There must, often at least, be reliable causal relations between what we are thinking about and the fact that we are thinking about it. Those causal relations have to be such that they can modulate real-time interactions with the environing world. And the system that has the properties which subtend those causal relations must be as it is because it has the function of real-time modulation of behavior in light of the causally relevant facts.
In the end, this is one way of making the point that Putnam expressed by saying that “meaning just ain’t in the head.” That is, it’s not in the head alone. To mean something is to be in a certain kind of relation to the world. It’s because Searle is looking for some kind of meaning that’s in the head alone – or in the computer program alone – that he’s driven to the desperate expedient of postulating mysterious causal powers of “producing semantic content” for the brain. He’s right, of course, that that kind of semantic content can’t be found in computer programs. He just fails to realize that it can’t be found in brains, either. (It can’t be found anywhere, because it doesn’t exist.) However, the ordinary causal properties of the brain – and of any computer program that can be similarly causally engaged with the world – are quite sufficient for the kind of semantic content that does exist.
