Abbott and Costello Meet the Killer is full of this. In particular, the Peek-a-Boo Corpse scenes in Costello's room and several on-screen murder attempts. But the scariest part is the climax in the local caverns, where Costello is alone with the masked killer who is consistently one step ahead of him.
Another Peek-a-Boo Corpse scene in Hold That Ghost: While in a guest bedroom in a haunted tavern, Costello discovers a hidden door behind a curtain. He assumes there's a dead body behind the door, but the body is actually hidden in the curtains next to it.
Earlier, the above-mentioned murder victim is searching the basement when a pair of hands reach out of a secret door and strangle him to death.
In Abbott and Costello Meet Dr. Jekyll and Mr. Hyde, the boys chase Mr. Hyde backstage at a theater. Hyde hides from Costello, first in a dressing room, then later in a nearby wax museum, and ambushes him when no one else is around.
Abbott and Costello Meet Frankenstein is often called the most legitimately frightening slapstick comedy ever made. From Costello finding himself alone in a morgue-like wax museum as the Monster and Dracula come to life, to his later being cornered in a stairwell by the Monster, to the final sequence on the pier, it's pure nightmare fuel. (To the extent where it's the only one of the A&C horror comedies to be officially considered part of the 1931-1948 Universal Monsters series.)
One scene has Lou innocently wandering around Larry Talbot's apartment. He goes around in his usual goofy, childlike manner, unaware that the Wolf Man is stalking him the whole time...
In Abbott and Costello Meet the Mummy, Costello stumbles into the mummy's burial chamber and is nearly strangled to death by it. When he returns with Abbott, they try to trick the mummy into attacking again so they can capture it, but it turns out that it's hiding behind a statue instead of laying in its sarcophagus.
Costello encounters plenty of scary stuff on the way to the mummy's tomb, including several hidden skeletons and a giant lizard.
The part in Abbott and Costello In The Foreign Legion where Costello is stalked by the dagger-wielding members of a hostile Arab tribe while Abbott is trying to check into a hotel room. This scene even uses the Scare Chords normally reserved for the horror-themed Abbott and Costello films.
In Go To Mars, the Venusian queen declares a curse that causes any woman who kisses a man to lose her youth. And since these women are all hundreds of years old, guess what happens when Costello kisses one of them?
