In The Darkest Shrine, Agent Brown has to do a lot of sneaking around at one point, in order to avoid being spotted by agents of S.H.R.I.N.E. Since he's invisible, one would think this wouldn't be too difficult... except the problem is that he's in the Alps, and in order to make use of his invisibility, he would have to remove his clothes. Given his location, this would kill him, so sneaking it is. Double bonus when you realize that the devs (who do their best to avoid the spoilered detail whenever possible) probably had him go to that location for this exact reason.
Phantoms of the Past has a few.
It's noted that the blue roses affect almost anybody to such an extent that they are effectively mind controlled. But Agent Brown has been in Restville for two months at the start of the game, and is still completely himself - just like his brother Mortimer. The roses were cultivated by their family, and as Mystery Trackers: Raincliff indicated, they are immune to the effects.
And just how did Agent Brown manage to stay in Restville for two whole months without being noticed? He's invisible, so all he had to do was remove his clothes! Even Mortimer, who has the same ability, can't detect him if he chooses to do this.
Speaking of the blue roses, it's observed that the reason the bad guys consider James to be dangerous is because he's mysteriously immune to the roses - meaning they can't control him. What the game doesn't explain is why. But we already know the answer! As the main page's folder for Origins notes, the Blackthorne family signet ring contains the stone called Estra, which can create its own form of mind control. While it wasn't strong enough to protect him from Morpheus in the bonus chapter of Timeless Voyage (or maybe he wasn't wearing it at the time), it's probably strong enough to shield him here.
Agent Brown's interactions with James make a lot more sense after playing Phantoms and learning that he has a brother who is his Evil Twin. Having James around must be like finally having the brother that Agent Brown has always wanted, and as a result, he is both cheerfully willing to pester the younger man and also fiercely protective of him. To a lesser extent, knowing that he also has a big sister whom he hasn't seen in a long time might explain much of his demeanor toward Anna.
Richard's absence falls under this, in a meta sense. It's shown in the bonus chapter of Timeless Voyage that one of his supernatural abilities is to defuse mind control in other characters; there, he helps to break James free of Morpheus's hold. Had he been brought along for the events of Phantoms, he would never have allowed Anna to be affected by the blue roses and could theoretically have even freed other characters from the influence, which would have drastically altered the plot of the game.
