We are trapped by assumptions that unravel as soon as we think about them: "we" meaning not only laymen but many philosophers and scientists. Here are three important wrong assumptions.
Many people believe that "thinking" is basically the same as "reasoning."
But when you stop work for a moment, look out the window and let your mind wander, you are still thinking. Your mind is still at work. This sort of free-association is an important part of human thought. No computer will be able to think like a man unless it can free-associate.
Many people believe that reality is one thing and your thoughts are something else. Reality is on the outside; the mental landscape created by your thoughts is inside your head, within your mind. (Assuming that you're sane.)
Yet we each hallucinate every day, when we fall asleep and dream. And when you hallucinate, your own mind redefines reality for you; "real" reality, outside reality, disappears. No computer will be able to think like a man unless it can hallucinate.
Many people believe that the thinker and the thought are separate. For many people, "thinking" means (in effect) viewing a stream of thoughts as if it were a PowerPoint presentation: the thinker watches the stream of his thoughts. This idea is important to artificial intelligence and the computationalist view of the mind. If the thinker and his thought-stream are separate, we can replace the human thinker by a computer thinker without stopping the show. The man tiptoes out of the theater. The computer slips into the empty seat. The PowerPoint presentation continues.
But when a person is dreaming, hallucinating — when he is inside a mind-made fantasy landscape — the thinker and his thought-stream are not separate. They are blended together. The thinker inhabits his thoughts. No computer will be able to think like a man unless it, too, can inhabit its thoughts; can disappear into its own mind.
I give you DAVID GELERNTER's DREAM-LOGIC, THE INTERNET AND ARTIFICIAL THOUGHT. Read it at least a couple of times.