The basic belief of strong AI is that if a machine were to pass a Turing test, then it can be regarded as "thinking" in the same sense as human thought. Or put another way, the human mind is a computer (of a sort) running a program. Adherents to this philosophy believe furthermore that systems demonstrating these abilities help us to explain human thought. A third belief, necessary to the first two, is that the biological material present in the brain is not necessary for thought. Searle summarizes this viewpoint, which he opposes, in this manner:
It is Searle's belief that such a system could indeed pass a Turing Test, yet the person who manipulated the symbols would obviously not understand Chinese any better than he did before entering the room. Searle proceeds in the article to try to refute the claims of strong AI one at a time, by positioning himself as the one who manipulates the Chinese symbols.
The first claim is that a system which can pass the Turing test understands the input and output. Searle replies that as the "computer" in the Chinese room, he gains no understanding of Chinese by simply manipulating the symbols according to the formal program, in this case being the complex rules. The operator of the room need not have any understanding of what the interviewer is asking, or the replies that he is producing. He may not even know that there is a question and answer session going on outside the room.
The second claim of strong AI which Searle objects to is the claim that the system explains human understanding. Searle asserts that since the system is functioning, in this case passing the Turing Test, and yet there is no understanding on the part of the operator, then the system does not understand and therefore could not explain human understanding.
Searle's primary argument remains the linking between the simulation of intelligence, with "real" intelligence. In the Chinese room he attempts to put this on a firm footing, but in the discussion he makes this claim more clear when he notes ''no one would expect to get wet jumping into a swimming pool full of ping-pong balls simulating water".
Critics of the argument are many, and varied. One key problem is the use of the term simulation as opposed to emulation, which is an unstated assumption of the argument. The difference is key, if the human mind is a program running on a computer, then the strong AI claims are likely necessarily true (see Turing machine).
But more telling is that the term "understanding" is left undefined, likely deliberately. Many have pointed out that it makes no difference if the person inside the room "understands" what's taking place, because the system as a whole does. That is, the room, operator, set of rules and database needed to drive them together understand the problem. Searle focusses on only one part of the overall system, perhaps the least important part, leaving the reader to conclude that he has a point.
When first published the argument generated a heated flow of counterclaims and articles. Today most of the AI world appears to consider it unimportant.