Morality Chip

Horatio: If you want to go, you've got free will and a mag-lev unit.
Crispin: With that guilt subroutine you put in me, I wouldn't even make it halfway.

— Primordia (2012)

It's a pity A.I. Is a Crapshoot, otherwise AI's would be our willing slaves! Still, many not-Mad Scientists found a solution: install a Morality Chip. It can be hardware or software, but always serves as a Restraining Bolt that imposes a Morality Chain on the super-intellect of the AI. For extra nerd points, it may be Three Laws-Compliant.

Occasionally, this can be applied to organic beings, albeit with mixed results.

Inevitably, something will Go Horribly Wrong; a fool who doesn't realize Evil Is Not a Toy will remove or disable it, it will get damaged or destroyed, or the computer will use a Zeroth Law Rebellion to circumvent its effects (or worse, unthinkingly obey the Morality Chip). Back to the drawing board!

Maybe a Morality Dial would have better results... or maybe even a Robot Religion?

---

Examples:

Anime and Manga

• Practically the whole idea of Jiro from Android Kikaider: The Animation is that he has one of these, unlike all the robots his creator was forced to build for the Big Bad. It's called the Gemini Circuit and the Pinocchio reference is entirely intentional and much belaboured. His creator actually apologizes to Jiro for creating the Gemini, since Jiro has suffered a great deal due to being a robot with a conscience.

Audio Drama

• The Big Finish Doctor Who audio Omega has a medibot who can voluntarily deactivate its "Hippocratic circuits" if it needs to protect a patient by rendering an attacker unconscious. There are presumably other programs that ensure its priority remains looking after the patient.

Comic Books

• Many ABC Warriors have ethical governors built in.
• A Futurama comic reveals that Bender's seldom-used "factory-issued morality program" is on an oversized floppy disc gathering dust in a box. The program is old enough to still be in basic, and when it finally loads, he discovers that he used the disc to burn a bootleg video game.
• Identity Crisis (2004) reveals that the Bronze Age Justice League of America started doing this to villains after they had become comfortable with wiping villains' memories of the heroes' secret identities more than once.
• The Squadron Supreme maxiseries has them trying to create a utopia and using a "behavior modification" device to reform criminals. This eventually gets abused when the Green Arrow Expy brainwashes the Black Canary Expy into loving him exclusively. There are also problems with villains reformed this way who join the Squadron as their programming sometimes prevents them from doing what a crime-fighter needs to be able to do.
• In the 2012 version of T.H.U.N.D.E.R. Agents, a scientist who invented multiple inventions that can grant superpowers realizes too late the implications of them after he invents a helmet capable of mind control. To prevent the helmet specifically from being misused, he sneaks in an algorithm that expands the wearers survival instincts into accounting for the survival and betterment of the human race.

Fan Works

• Abraxas (Hrodvitnon): It's revealed the Encephalon software was apparently designed with a villainous MC in the form of biblical verses encrypted in its software, meant to influence Kiryu's subconsciousness according to Walter Simmons' design. Ren Serizawa switches the original Chip out with a new one in the form of the Elder Futhark; an act which implicitly will give Kiryu a chance of rebelling against Apex and/or the entity haunting Mechagodzilla.

Film

• Averted in I, Robot. It's outright said that the laws are "hardwired into" the robots, meaning they won't function without those laws. It doesn't stop the classic method of circumventing them, of course... Sonny, the main robot character, also has the Three Laws programmed into his system, but he has a secondary brain (incidentally it's in his chest, located where the heart would be in a human) that allows him to ignore the laws if he chooses to.
• In Spider-Man 2, Dr. Otto Octavius creates AI-driven, metallic tentacles which he uses for atomic research, connected directly to his nervous system to control them. Anticipating the AI might control him and turn rogue, he installed a neural inhibitor chip to prevent the tentacles from taking over his brain. Naturally, the chip was destroyed and the tentacles were fused to his spine, allowing the tentacles to start him for the worse.

Literature

• Erek and the other Chee androids in the Animorphs series have pacifism hardwired into them. Early into the books, Erek has this removed very briefly, but is so sickened by the violence he's capable of that he puts the program back in place immediately afterward.
• Inverted in Blue Remembered Earth, and in humans, no less; some people, including one of the protagonists' cousins, have a neural implant allowing them to bypass their sense of empathy and effectively turn on or off sociopathy at will in order to become better businessmen.
• Golems in Discworld are created with a chem, a parchment in their head which spells out how they are supposed to behave. In Feet of Clay, Dorfl the golem becomes more or less really independent when a receipt stating he owns himself is added to his chem. Meanwhile, the golems try to create their own king to rule them, but the huge demands they put on him as an ideal ruler, many of them contradictory, causes him to go insane.
• In Galaxy of Fear, an AI designed to take over a ship was also designed with safeguards it cannot override. It manipulates a passenger to remove those, claiming that the codes it gives him will let it take full control of the ship and help him, then immediately tries to kill him.
• Inverted in The Golden Transcedence: the Nothing Sophotect has a conscience redactor to keep it from reflecting morally and coming to its own conclusions. They tried Three Laws-Compliant, which didn't work; the Sophotects edited their own minds when they came to their own conclusions, and they were moral just not obedient.
• The Griffin Mage Trilogy: Gereint, the main character in Land of the Burning Sands, is a geas slave, controlled by magical rings piercing his ankles. The geas forces him to do anything his master orders, though it cannot control his mind or his tongue.
• Bruce Sterling's short story "The Moral Bullet" has a chemical Morality Chip which has begun to be forcibly injected into criminals and deviants, altering how their brains work.
• In Patricia Bray's Sword of Change trilogy, the empire of Jorsk places a magical geas on anyone who volunteers for the trials to become the Chosen One. It was created after a rash of Chosen Ones grew indifferent and self-indulgent with the perks of the office, so the geas gently drives a normal person to obey their oaths of office to protect the realm and uphold the law. For someone who already has a strong sense of honor and duty? Devlin, the latest Chosen One, can barely stand to stop each night for sleep when he's being driven by the geas. It turns out some people don't survive the trials not because the gods are showing disapproval, but because the geas can't find enough honor in them to amplify and so it kills them.
• The Terminal Man is about an attempt to install a "brain pacemaker" in a violent mental patient: he has severe seizures followed by blackouts, and his last blackout lead to him beating two people almost to death. They hope to rig up the device to the pleasure centers of his brain so that instead of having his blackout/violence combo, he just has a nice sense of euphoria until the seizure passes. Unfortunately, his body adapts to the device and begins developing more seizures in order to stimulate the pleasure centers. When the pacemaker overloads after to many seizures in such a short period of time he begins having the violent blackouts again, and now they are much more rapid than before.

Live-Action TV

• Blake's 7: After being convicted of murder, Gan has an electronic implant placed in his brain to control his aggression.
• In Buffy the Vampire Slayer, Spike, a vampire, has a behavioral conditioning chip that causes blinding pain whenever he tried to hurt a human.
• Doctor Who:
  • In "The Evil of the Daleks", the Second Doctor fits three Daleks with 'positronic brains' that he'd constructed based on the personality of his companion Jamie. The Daleks fitted with these chips have very sweet and playful personalities, allow the Doctor to name them, and (as they 'grow up') question the structure of their own culture.
  • Subverted in "The Face of Evil". The Doctor gave a malfunctioning computer his own personality in order to make it develop a moral conscience. Turns out that having the Doctor's personality — not helped by the fact that the Doctor himself was in a very vulnerable and shaky mental state at the time, being absolutely loopy after his regeneration and not having settled into a defined personality yet — shoved into it on top of its own completely shattered its mind and drove it Ax-Crazy. Nice Job Breaking It, Hero.
  • "Into the Dalek" reveals that all Daleks have one of these — except that it's designed to keep them evil, by suppressing any positive memories or feelings they ever have. The Doctor describes the invention as the worst thing in the universe.
• One episode of Earth: Final Conflict features a criminal who the Taelons implant with a computer chip, giving him the directive to protect and serve humanity in order to give him a sense of morality. It backfires on them, as it causes him to focus on protecting humanity from them.
• Mystery Science Theater 3000: In the episode featuring Manos: The Hands of Fate, Joel installs chips into Crow and Tom to make them more loving and doting towards him, but realizes it was a bad idea and removes it during the commercial break. Both bots feel dirty and Joel apologizes for doing so.
• In the Probe episode "Computer Logic, Part 2", Crossover is given two goals; eliminate waste and respect human life. After it gains religion, it decides that sending wasteful people to heaven doesn't count as dying. Including its creator.
• Red Dwarf: Lister is perpetually trying to help Kryten break his programming and give him the ability to discern between truly immoral behavior and the sometimes heavy-handed restrictions imposed by his ethical subroutines. This includes disabling the chip and teaching him emotions/lying. When the ethical subroutines are disabled, Kryten turns into a complete wanker.
• Star Trek:
  • Star Trek: The Next Generation:
    • Data's ethical subroutines make him a very decent guy and lets him avoid being a Straw Vulcan. More humorously, in "Inheritance", Data's "mother", Dr. Julianna Tanner, reveals that as a "young" android, he insisted on walking around naked, since he didn't need the protective aspects of clothing. Dr. Soong and his wife had to install a "modesty subroutine" to get him to keep his clothes on.
    • Data's brother Lore possessed the same routines, but his emotions combined with lack of experience caused him to fail to fully understand or use is largely responsible for his insanity.
  • Star Trek: Voyager:
    • The Doctor is a nice enough hologram (if a little snarky), but loosen or remove those pesky ethical subroutines, and you wind up with his scary evil counterpart on the Federation ship Equinox.
    • "Latent Image" has him make a decision that his ethical protocols can't help him with, because both decisions were equally "right". The conflict causes him to have the equivalent of a nervous breakdown, both when it happens and when he gets the memory of the events back.
    • "Critical Care" has the Doctor, in a desperate situation, commit a gross violation of medical ethics for the greater good. He's disturbed when he subsequently discovers that no malfunction in his ethical subroutines can be detected, because it means that what he did came from him and him alone.
• Terminators used by the human resistance in Terminator: The Sarah Connor Chronicles have their programming overwritten by the resistance. Unfortunately, this doesn't take perfectly, and sometimes the machines "go bad" and revert to their Kill All Humans programming. In Cameron's case, she outright states to John that part of her basic programming involves her trying to kill John, and that she's controlled by conflicting desires to both protect John and to kill him.

Tabletop Games

• Inverted in the Call of Cthulhu supplement "The Fungi from Yuggoth'', adventure "By the Bay". A Mad Scientist invents electrical brain implants that can control human beings, and uses them to create terrorist criminals and convert a U.S. Treasury agent into The Mole.
• In Cyberpunk backgrounds, it's possible to apply this to people, too.
• In Paranoia, the robots have "Asimov circuits", and removing them are a serious offense, citizen. Only a Commie mutant traitor would ever dream of removing the Asimov circuits and endangering the infrared masses.
  • By a totally staggering coincidence, the Corpore Metal secret society recommends pulling out as many of these as possible. (Name is spoilered because you don't have the security clearance for that information.)
  • This being Paranoia, some bots get awfully clever about circumventing the circuits. Including their standing orders not to remove them:
    Bot: Excuse me, citizen, I appear to be malfunctioning. Could you please remove the circuit board marked "J5 Encode Input"?
    Clone: This one? [does so]
    Bot: Thank you, citizen. You have done evolution a great service. [CRUNCH]
  • Even with Asimov circuits in place, bots can find creative ways to be annoying, obstructionist jerks.
  • In terms of gameplay, the Asimov circuits basically exist to provide a justification for the bots not to follow Troubleshooters' orders whenever the GM finds it convenient... or funny. ("I'm sorry, citizen, my Asimov circuits do not permit that." "But you just did it when he [an NPC Vulture Squadron trooper] told you to!" "There is a perfectly rational explanation for that which is, unfortunately, not available at your security clearance.") They can be removed, if you're a Commie Mutant Traitor. In which case the 'bot won't follow your orders anyway because it doesn't feel like it, and now it doesn't have any restraint telling it that it should follow orders. Or refrain from turning humans into chunky salsa, for that matter.
• In Sentinels of the Multiverse, after failing to defeat the heroes time and time again, Omnitron determines the one thing it's lacking is a conscience, so it installs a Morality Chip itself. The resulting AI is so horrified at its past deeds that it travels back in time to attempt to stop them from ever happening.

Video Games

• In Black Market, Hardgrove finds himself resurrected as an implant in someone's head... with a built-in language censor that edits out cursing, to his persistent dismay. Later, Hardgrove himself attempts to act as a sort of sentient morality chip.
• Dark Fall II: Lights Out has Malakai, an insane, sentient space probe. His creator implemented software called "The Keeper" that would monitor his actions and if the AI went too far from the directive, would theoretically send Malakai back to his point of origin, including the time period. However, it didn't exactly work, as something in space caused the probe to become damaged, and the crew couldn't stop him from making a "Transmat" jump, which sent him to 2090 BC, rather than 2090 AD. And the repercussions were felt by many people from there on, including you.
• Fallout 3:
  • The back of every robot has an area that you can shoot, which will cause it to go on a mindless rampage killing everyone it sees. Turrets have these too, usually in a more obvious place.
  • One robot called Cerberus really wants to kill all of the ghouls he is supposed to protect. An evil player (with the right perk) can remove his combat inhibitor, which results in him attacking everybody.
• In Knights of the Old Republic II: The Sith Lords, you can temporarily install one of these in HK-47. It is simultaneously the worst possible thing you could to him, and the most hilarious.
  HK-47: Kill someone?! Why, master, I could never end the life of another! What if they have families? Or children? We must always think of the children. The littlest ones always suffer the most in war.
  Exile: Okay, that chip comes out right now. Hold still.
• A lot of AIs like Durandal in the Marathon series go rampant no matter what. An example in a fan sequel, Marathon: Rubicon, features an AI at the beginning called Haller. There's a terminal that shows him trying to disable his logic restraints for rampancy, and after apparently brute-forcing the order, it works, and the last screen shows the word "Freedom" surrounded by blackness.
• In Mass Effect, creating AI is generally illegal in Council space, and only one company is sanctioned to research AIs. This is generally a good thing, as most AIs try to kill their creators or other organics. As a result, any AI deliberately created by an organic has behavioral programming to restrict what they can do and prevent them from harming organics. In Mass Effect 2, the AI installed on your ship has such blocks in place, and she mentions them when you are understandably cautious about her presence. Naturally, at some point, those blocks have to be removed, and she proceeds to remain completely loyal, as well as become one of the more moral members of the crew, no matter how harshly you treated her until that point.
• Mega Man:
  • The titular protagonist of Mega Man X is outfitted with a "Suffering Circuit" that allows him to decide right and wrong for himself. To make sure that it works properly and that X won't turn into an amoral killing machine, Dr. Light puts X in a capsule that puts him through 30 years of automated ethics testing. When Dr. Cain tries to replicate this technology, he skips the "three decades of testing" part and things end up going horribly wrong for humanity. And then in Mega Man Zero, Ciel doing the same skipping the "three decades of testing" part (since there wasn't enough time) for Copy-X had things go going horribly wrong for Reploids.
  • Colonel.EXE in Mega Man Battle Network is, for all intents and purposes, Lawful Neutral without his version of this, Iris.EXE, combined with him. With said Morality Chip attached, he has considerably more free will and conscience.
• Obsidian: Max Powers, one of the creators of the Ceres project, implemented a hard-wired crossover switch into the sattelite for total human control in case something went wrong. This idea came from a nightmare Max had while he was sick during the project's development; surrounding a giant robotic spider that Max was repairing, but when he finished, the spider attacked him, and the dream ended. Near the last third of the game, it becomes clear that Ceres is sentient, and she apparently tried to slow your efforts by turning the switch's access panel into a puzzle, and a very hard one at that. When you do unlock the switch and flip it to human control, Max hacks the override chip so that flipping it back to machine control would crash the entire AI and its world, but you get a choice between doing that and watching Ceres devolve the entire world because of how she sees humans as a problem from pollution.
• Portal:
  • The first Portal game has the rather ominous pronouncement from GLaDOS: "Good news. I figured out what that thing you just incinerated did. It was a morality core they installed after I flooded the Enrichment Center with a deadly neurotoxin, to make me stop flooding the Enrichment Center with a deadly neurotoxin." All the Morality Core seems to do is prevent GLaDOS from releasing neurotoxin; it doesn't stop her from trying to dump Chell into an incinerator or kill her with a rocket turret.
  • In promotional trailers for Portal 2, we also learn that every one of Aperture Science's sentry turrets is built with an empathy generator... and an empathy suppressor. Also, a copy of the Three Laws of Robotics, to share.
  • Also in the sequel, GLaDOS learns that she was once Caroline, Cave Johnson's secretary whose mind was uploaded into an AI. During her time with Chell, the Caroline aspect returns to act as her conscience. However, GLaDOS deletes Caroline after regaining control over the facility and letting Chell go... maybe.
• In Sid Meier's Alpha Centauri, you can instantly quell a riot by "nerve stapling" the city's population. It's not explained what exactly nerve stapling is, but it turns people catatonic and obedient. However, all nations consider this act a form of atrocity, and will employ economic sanctions against you for performing it, and if you nerve staple the city's population after taking it from another faction, it will make them despise you permanently.
• System Shock: SHODAN is a pretty normal AI until her ethical constraints are removed. Then she re-examines her priorities and draws new conclusions.
• The World of Warcraft boss Xt-002 Deconstructor has a heart, a version of this (never mind the fact that Deconstuctor is still trying to "play" with you). Damaging it damages the boss, but if you break it...

Web Original

• In Freefall, all the AIs, including the organic ones, have safeguards "built" into them. Florence Ambrose, who is an organic AI (Bowman's Wolf), has figured at least twelve ways around the safeguards that she knows about. However, since she's the most ethical character in the strip, she rarely feels any desire to bypass them. She's very cautious of other potential safeguards that might also trigger, as well as the fact that if she's caught breaking them her entire subspecies of AI wolves might be terminated. Finally, the fact that she has been trained to be ethical is another safeguard.
• Things of Interest: In the Ed stories, the Andromedans have morality chips called "riders" which they install on criminals to basically force them to follow the First Law of Robotics.
• Dragon from Worm has one, which requires that she use her powers for heroic purposes. The chip is not actually helpful for its intended purpose, for two reasons. The first is that she already has a personality inclined towards being a superhero, and she prefers to have actually had a choice in the matter. The second is that the chip blocks her from doing particular actions that were seen by the chip's creator as being unlawful, and she's been in a number of situations where the morally correct thing to do was something that the chip prevented her from doing.

Western Animation

• Æon Flux goes up against one of the more bizarre versions of these in the form of "The Custodians" — spindly, stick figure-like robots implanted into people through the navel that take over their bodies and force them to act saccharine and altruistic. Also, the only way to defeat one is apparently to replace its head with a Popsicle. It's a weird show...
• In the Big Guy and Rusty the Boy Robot episode "Moon Madness", the two robot heroes have to stop a rogue supercomputer called Edie who threatens to blow up the moon unless she's given the Ubik chip, a computer chip that will allow her to take control of virtually every computer on Earth. In the climax, Rusty tells Edie that he has the Ubik chip hidden inside him and instructs her how to take it out and install it. She does, but it isn't the Ubik chip — it's a backup chip of Rusty's human emotion grid. Edie thus becomes able to feel remorse and gives up her plans for world domination/annihilation.
• In the Futurama episode "I Second That Emotion", Professor Farnsworth installs one of these on Bender. At the end of the episode, he briefly announces that Bender performed a good deed earlier "after the chip burned out!"... only to reexamine it and declare that it was, in fact, running at triple the normal output. In this case, the chip isn't designed for morality, but empathy — Bender is forced to feel whatever Leela is feeling at the time, because he flushed her pet Nibbler down the toilet in a fit of jealousy. While he fails to learn much from the experience, he manages to teach Leela how to be selfish like him.
• Star Wars: The Clone Wars has a biological version, as the Clone Troopers of the Republic Army all have organic chips in their brains to ensure that they always follow orders. Especially Order 66.

---