7th Sea: There is a red jewel known as "Legion's Spike". While not everyone has been affected by it, some unfortunate cases who have stared into its depths have suffered from catatonia, madness and homicidal rampages. And there's apparently more than one such gem.
Arduin RPG, The Complete Arduin Book 2: Resources:
The Hell Cat's bone-rattling caterwauling causes intense fear in all creatures within 60 feet.
The Sun Demon's movement causes a squealing and groaning like tortured metal that inflicts a -5 penalty of the combat abilities of all creatures within 30 feet.
Reading Cthulhu Mythos books or seeing Mythos monsters can cause a loss of sanity and eventual insanity.
In Delta Green, a certain document contains a "formula" that, to an average person, appears to be nothing more than a random sequence of numbers. However, a character whose skill in math is good enough will instantly recognize it as a "formula" that proves that there are higher dimensions of existence... and will then immediatelyAscend to a Higher Plane of Existence by just thinking about it.
Cyberpunk 2020 has an option for cyberarms that can cause disorientation and seizures.
Deadlands has the Whateley family treeshrub. Looking at it is more than enough to drive someone insane, thanks to its terribly tangled composition and some definitely non-euclidean branches being portrayed. Then there's what the Whateleys are actually doing...
An article in Dragon Magazine, the late official magazine of the game, described a sage who delved into the study of the Lords of the Nine, the nine arch-devils who rule the Nine Hells of Baator. He went missing; all that turned up of him were a few spots of blood on his floor. It's speculated that either he attracted the attention of the devils, who spirited him away; or that that the sheer evil of the tomes he was reading caused him to spontaneously implode. This is also similar to the legend of Faust, who gave rise to the term "Faustian bargain" and was ultimately found splattered all over the floor... and the walls... and the ceiling.
In D&D, it's possible to place spell traps on objects, which are triggered by looking at, or reading them.
Urban Arcana takes this trope to the 21st century, by including rules on how to send spells over the Internet. Be careful next time you open that email attachment...
Bards can charm other creatures using singing and music.
Waaaaay back in the 1E era, there was a Dragon Magazine article about a high-level bard ability called the "Last Jest". Properly delivered, this joke could make villains laugh themselves to death.
This ability was brought to 3.5 as a Gnome PrC ability. Took three rounds to finish the target off and the second round had another effect.
There are a number of monsters that can harm with sound: Wolfweres (singing = sleep), androsphinx (roar = deafness), cloaker (moaning), tyrg (howling) and so on. And banshee.
The constant babbling of a gibbering beast does weird things to the mind. People who hear the creature's gibbering might run away in terror, lash out randomly at anything within reach, or stand transfixed as the horror creeps forward to devour them.
The 9th Level Wizard/Sorcerer spell Wail of the Banshee. Like the actual banshee's power, it was an awful scream that killed anyone who heard it while too close to the caster.
Up to the 3rd Edition, nymphs were so beautiful, looking at one could blind or even kill humans. As a rule, looking at a clothed nymph could blind you, while looking at a nude nymph could kill you. In the 3rd Edition, clothing didn't matter, they could focus their beauty as a sort of overwhelming aura (though it only stunned you rather than killing you), and in the 4th Edition, they could not harm humans with their beauty at all. (Seduction and trickery, on the other hand...)
The Obyriths in 3.5 are a species of demons modeled universally on Lovecraftian concepts: to look on them is to invite madness and insane terror, even in those otherwise magically immune to such emotions. Dagon evokes terror of the sea, Ugudenk the Squirming King causes any viewer to realize he can burst from the ground at any time and thus to be terrified of the ground, etc. The most powerful of the Obyriths was supposed to be Obox-Ob, the first of the species who has a shape that could be very roughly analogued to something like a titanic scorpion, but with the head and tails (yes, plural) switched around, and horrible tentacle-tongues and worse. But the deadliest of the Obyriths, insomuch as their ability to cause madness, is Pale Night. She takes the shape of a softly curvaceous humanoid female, wrapped in a billowing shroud. Attempting to pull the shroud aside and see her true form is difficult, but if you do manage it, you must immediately make a saving throw. Success means you failed to understand what you saw beyond the veil. Failure means you understand what you see: a shape so alien, horrifically indescribable and anathematic to all existence that you are instantly slain. What's more, if a victim of this effect is brought back to life or magic is used to communicate with his spirit, he is unable to describe what he saw. As it happens, the shroud is something reality itself imposes on her to cloak her true shape as a way of protecting the rest of existence. Even the Far Realm, home to true Lovecraftian horrors in the D&D mythos, is not as innately lethal to witness (though entering it can do worse than just kill you... )
And the Devils have their own example, from second edition up to 3.5 in the form of Asmodeus and the tale of the Serpent's Coil. Supposedly, the Asmodeus all creatures understand to exist isn't the real thing, but rather a highly advanced illusion, or an avatar. The King of Hell's true form was hurled down into Hell from the Celestial Realms long ago. Asmodeus' impact into Baator is what split the plane into nine layers. Asmodeus' true body came to rest in a tunnel of rock hundreds of miles long created by his landing, called The Serpent's Coil. And it rests there still, slowly recovering its strength. Hearing this story didn't harm the listener at all, but anyone who told the tale of Asmodeus' "True Form" died within 24 hours. Anyone.
Cyric of the Forgotten Realms created a tome called the Cyrinishad that would brainwash anyone who read it into being a devoted worshipper of Cyric. Things got bad when he accidentally read it himself. As a result he became even crazier and came to believe his own hype. He eventually got better.
The Book of Vile Darkness sourcebook (3/3.5 material) introduced "Dark Speech", a language so evil that hearing a single word spoken in it can potentially drive people away screaming in terror, as it's simply that awful. (Trying to speak even a single word of it without the proper training is lethal, and only evil entities like Archdevils can manage more than a short phrase.)
Demon Lords qualify. The Fifth Edition core rulebooks include a set of optional Madness mechanics for horror themed campaigns. There are three stages of madness, that is, short term, long term, and indefinite. Then the Out of the Abyss campaign came out, and each Demon Lord listed can ignore those rules, sending you straight to Indefinite Madness if you so much as look directly at them. Demogorgon can cause a character to develop a second personality or solve all of their problems with murder, among other things. Yeenoghu, among other things, can give a character a taste for the flesh of intelligent beings. Orcus can make someone take pleasure in the suffering of the weak, want to become undead, or develop an unhealthy fascination with death. And all of this can happen just by looking at a demon lord.
On the topic of Demon Lords, Mordenkainen's Tome of Foes speaks of Demogorgon's symbol, which appears to be an unassuming Y shape. Simply gazing at a "true" copy of his symbol, crafted either by Demogorgon himself or one of his more powerful demons, will instantly bring a person under his sway, turning them into lone serial killers who spend their lives hiding in the shadows. The guidebook doesn't give any in-game stats for this effect, but the fact that Mordenkainen himself, an archmage who has traveled the multiverse and has even snuck into and out of the 8th layer of Hell, had to be rescued from the effects of the symbol, suggest that you wouldn't even get a saving throw. It's worth noting that Mordenkainen never even looked at the actual symbol — he was studying its reflection, and that still nearly destroyed him.
The Windswept Depths of Pandemonium is an Outer Plane in some D&D settings associated with Chaotic Neutral and to a lesser extent Chaotic Evil. It got its name, windswept depths, from the constant howling wind. The wind in Pandemonium never stops, and the incessant howling can cause deafness and insanity. Worse, there's a place in Pandemonium called Harmonica. There, the wind blows through stalactites and stalagmites riddled with holes, causing a sound so hellish it can cause people to simply drop dead.
Earthdawn. Simply reading about the Horrors can cause psychological problems and attract their attention upon the reader.
There is a weaponized brown note, in the form of "basilisk hacks", combinations of sensory input which essentially crash the human brain. Also, low exposures only cause seizures, but longer doses can cause Exurgent infection.
Pandora gates are noted to be odd enough that they hurt your head and cause some asyncs to wig out. Most of the solar system bases that contain gates keep them covered at all times... just to be on the safe side.
The spell called Rune of Singular Hate. It's described as a single word full of such vile and complete hatred that, when uttered at someone, curses them to debility at best, and outright death at worst. It's such a powerful word that it even affects the caster in a similar way, and can only be cast once in a lifetime.
Similarly, the Deathlord known as the Bishop of the Chalcedony Thurible is working on a mammoth collection of books about the theology of death. Some are used as holy books for ancestor cults, some are gibberish he keeps in his own personal library... and some describe Oblivion so seductively the reader goes insane.
And when it comes to damnable books in Creation, there's none better than The Broken-Winged Crane, which often instills madness in those who read it and compels them to try demon summoning and Yozi worship for fun and profit.
In other Deathlord wonkiness, there's the Monstrance of Celestial Portion, the cages used by the Deathlords to put Solar Exaltations through the spin cycle of evil so they come out as Abyssal shards. Solars can't even look at the Monstrances without feeling violently ill.
There is also the Yozi called She Who Lives In Her Name - her true name traps lesser beings into endlessly repeating it should they ever hear more than a few words of it.
Also has the unusual example of an Eldritch Abomination who experiences Brown Notes; the Yozi Adorjan, the Silent Wind, abhors all noise and will avoid it unless sufficiently motivated, and is most effectively warded off by the magical Demon Wracking Shout (said to be the death cry of the demon that was her heart), the music played by certain demons descended from her, and the sound of a child's laughter that has been caught in a shell, ground into paste, and mixed with silver or gold.
Fate of Cthulhu gives us Cthulhu himself (even when viewed remotely or through recordings), as well as the Necronomicon. Downplayed in the case of the Necronomicon, since only reading it won't set off the game's corruption mechanic. Taking the time to delve into it and understand it, however, will.
The Terror advantage (caused by whatever aspect of yourself you wish) can terrify victims beyond all reason. At its worst Terror can cause permanent insanity and actually make someone so horrified by the effect that he becomes stupider. Of course, normal human characters can't acquire this advantage.
GURPS Ultra-Tech has a more literal brown note. Sonic nauseators make people void their bowels as side effect of knocking them out. Just don't mix one up with a sonic screamer, which produces a sound that melts the target.
The Madness Dossier: The monstrous, mind-controlling "Anunnakku" actually made or remade humanity to be their slaves, and can command obedience simply by displaying special "glyphs" or in other ways. Having to obey the Anunnakku is rarely a good thing for a human being.
One Pyramid vol 2 article features an album called Iniquus Veritas which makes 30% of listeners commit suicide. Whether this is because of some kind of mind-control harmonics, a supernatural effect, or simply the music itself (which is depressing and upsetting, but compelling enough to keep people listening) is left open.
Mage: The Awakening deals with vastly alien concepts that can confuse and bewilder mortal minds, but that's just from trying to grasp extraordinarily complex ideas.
Intruders: Encounters with the Abyss suggests that Abyssal intrusions can take forms such as a poem or work of art. Banishing such intrusions is extremely difficult when merely perceiving them is deleterious.
A specific example from Intruders is Dark Angel Aphasia. At first it manifests as a strange obsession with the works of others with the condition. Then the sufferer begins to lose the ability to communicate in either speech or writing, but continues to obsess over whatever concept has caught their fancy. Eventually they become completely incapable of using language, and not long after that they either go comatose, instantly recover with no memory of what they were talking about, or disappear. And how does this condition spread? By reading something a sufferer wrote or talking to them before they lose the ability to communicate.
Magic: The Gathering's joke set Unhinged has a card called Stone-Cold Basilisk that can temporarily turn players to stone. The ability is triggered by reading the card.
Varanae generic RPG supplement Monstrum 1
The Garmen is an undead dog-spirit whose howls cause fear in all unintelligent animals, possibly causing them to flee.
The Sandmyrk is a small dog/rat hybrid. A pack of them can surround victims and begin a horrible howling that causes chilling fear in their targets, The terror is so great that animals will flee in panic and intelligent creatures have a good chance of doing the same. Even if a victim manages to resist the effect they are still severely penalized while fighting the Sandmyrks.
Mummy: The Curse: Mummies who have high levels of Sekhem (usually, those who just woke up) are leaking so much divine energy that they can't maintain the illusion of being anything other than a magically reanimated corpse. This sheer tide of divinity also triggers a state of holy terror in mortals known as Sybaris; those who witness the mummy in this state will either run in terror, collapse into catatonia, or start groveling at its feet. Sybaris also gives mortals visions that can be interpreted for prophetic insight, which makes them somewhat valuable to mummies; a revived mummy who finds a would-be grave robber has entered a state of holy ecstasy and insight has usually just recruited the first member of their cult.
In Mutants & Masterminds, any attack can be made into one of these by adding the Sense-Dependent flaw.
Second and Third edition both have flavour text describing a book on the true nature of beauty. Because the book is a sacrosant object not meant for mortals, it kills the first to read any word within. The vignette wraps up with "It is a statement on the nature of beauty, and the nature of scholars, that [...] over half of its text had been read, understood, and transcribed."
Any picture of Ananda, Lord of Murder, the Infinite, and the Fourth Age, induces physical and/or psychological damage in those who see it, due to his incredible beauty. Actually seeing him in the flesh is worse.
The Flash Pak is a device that fires light bulbs in a random stroboscopic sequence that caused disorientation in anyone who viewed it.
Piricurus are Awakened thrushes capable of producing cries pitched at 30 kilohertz. This is above the range of human hearing, but large flocks can cause significant auditory pain when all calling at once. Piricuru calls are also painful and disruptive to people with cybernetically-enhanced nervous systems, as well as to animals capable of hearing them, and disrupt both natural and artificial sonar — bats and other sonar-dependent animals become disoriented and blind when piricurus sound off.
The Alter Tongue in Unknown Armies is a language invented by a pair of twins who were locked in a basement with nothing but each other's company. It's a language that embodies a world view where darkness is safer than light and anything you don't know could be a danger. Hearing it is enough to imprint it on your brain (if you fail the Mind roll); you find it easier and easier to speak in the Alter Tongue, with new words and grammatical rules appearing out of nowhere, and start to see very weird things out of the corner of your eye. As the Tongue spreads, it draws those who know it together; it doesn't quite create a Hive Mind, but it definitely forces a clannish nature on its speakers.
Symbols of Chaos are capable of making men nauseous at best to insane at worst, and that's saying nothing of actually gazing upon daemons.
This a rather favorite tactic of Chaos. There was an old story about the forces of Chaos capturing a Janitor or somesuch, and then returning him back home... after telling him a word. Cue the inquisition purging the planet continuously for a thousand years, before finally resorting to Exterminatus.
Any mortal who looks directly at Slaanesh's true form will instantly lose their soul and willingly become his slave for all eternity. Anything involving Slaanesh would result in this. His champion, Lucius, has a set of armor that turns you INTO him if you kill him and feel even the slightest sense of accomplishment. Then there's also the ability to make yourself so irresistable that the enemy will lose the will to shoot at you.
Warhammer 40,000: The Imperium has its own Brown Note in the form of the Culexus Assassin. These rare mutants are like psychic black holes, whose bizarre non-presence seriously puts the wind up anyone (including their own allies) within a few feet. And that's on regular humans; psykers cannot get anywhere close to a Culexus-tier blank without suffering horribly and possibly dying from the strain. And daemons just straight up cannot perceive them in any way.
The worshippers of Slaanesh have a word that, when whispered into your ear, can kill you.
The Jabberslythe unit for the Beastmen is apparently so hideous it drives enemy units insane. Understandably, it's the only unit without a picture in its entry.
Hunter: The Vigil: Screamer pistols, weapons used by Task Force: VALKYRIE against mages, produce blasts of low-frequency sounds intended to interfere with the inner ear. At range, they induce dizziness and poor balance; close-up, they induce hallucinations and can resonate with the large intestine to cause sudden bowel movements. VALKYRIE agents refer to this particular effect as "the brown note".
It's quite possible for Malkavians with high Dementation and Auspex to booby-trap books, paintings or songs with their discipline's powers. It's even possible to modify your aura in that way, to punish curious Auspex users.
The Daughters of Cacophony are a vampire bloodline that specializes in madness inducing sounds. They select good singers, but the vampiric magic makes it dangerous.
Werewolf: The Apocalypse: The sight of any shifter by a normal human causes what is known as Delirium, a primal fear inherited from the times when werewolves exercised tyrannical rule over all humankind. The effects depend on what kind of shifter and the witness's willpower (usually in an adjustment in what is the witness's effective willpower score). While not lethal, it can induce fainting or severe panic attacks and usual amnesia as people just don't want to remember what they have seen, though high willpower witnesses can remember or straight up shrug it off. Naturally, other shifters, kinfolk, vampires, mages... ANY supernatural is more or less immune to it, as they are either more than or not at all human.
World of Warcraft: In the tabletop RPG, Eredun, the language of demons, is said to be inherently evil and has a will of its own; it slowly rots the brain of any nondemon who speaks it, driving them mad and corrupting them towards evil. It's one of the reasons that warlock magic is considered so taboo, as it's required for the casting of spells.
Yu-Gi-Oh!: The card Des Croaking requires 3 Des Frog on the field to destroy your opponent's entire field. The card art depicts the effect as literal brown notes.
that can temporarily turn players to stone. The ability is triggered by reading the card.