The big question here is whether you believe it's theoretically possible for humans to create a robot as sophisticated as a human. If you're a person of faith, you've got the whole immaterial soul thing to throw a monkey wrench into the works, at least until we manage to replicate the human soul using a powerful lens, some magnets and the tears of an orphan.
Even if you're not religious, creating a robot that functions on the same level as a human being seems like a tall order, but think about it this way: the guy who built Deep Blue could probably shit a diamond more easily than beat it at a game of chess. And I'm guessing the guy who invented the hydraulic press couldn't crush a car, either. And just try to recreate two midgets having sex with a horse more realistically than your computer monitor.
— Cracked
Princess Voluptua: "Well, the Empire is surely better equipped to raise an artificial consciousness than you Earth folk. No offense. You are conscious, aren't you? Not just artificially intelligent?"
Roofus the Robot: "I... don't know."
Princess Voluptua: "You are, then. A simple A.I. would lie about it."
We're anthropomorphizing computation, sort of urging it to have consciousness. And a lot of our anxieties about it are about our own anxieties that we just don't know what consciousness is. Or a sense that we ourselves are sort of blundering through life in a daze, and wouldn't it be great if we sort of had God in a box. I think there's a lot of unexamined psychological baggage in that idea. But it's also, it's just part and parcel of the whole series of metaphors that have surrounded computation since its birth. They're all metaphors which are drawn from cognition, such as that computers have 'memory' and so forth, you know. Well they have circuitry in which ones and zeroes can be put in and removed but that's not memory, it's not like the memory that a mouse would have, or even like an angleworm. It's really something more like shuffling huge decks of cards. You know, it's a vast, but we're used to this, I means that is how we decided to describe this technology as a "thinking machine.' They don't think. They don't. But we've never come up with an alternative vocabulary which is sort of convincing which describes what computation does, and we probably won't be able to do that until we have a vocabulary which describes what cognition does. I mean what it really means to be conscious. What parts of our brain are really carrying the load there, and how that sort of grey matter really works, and we are far away from that. You know, there are chunks of brain the size of your fist inside your head, we have only the vaguest idea what these things are up to.
— Bruce Sterling, "The Singularity: Your Future As A Black Hole," a speech to the Long Now Foundation, June 11, 2004
