Because a small handful of people are making it seem like every member of a fandom acts in a toxic manner, the creator stops whatever fun interaction with the fans they were having, ruining it for everyone. Even the people who were behaving themselves. This may give the rest of the fans the impression that said creator is a Nice Character, Mean Actor, even though they were very much provoked to do so. This can be especially true for those who act respectfully, were unaware of any other fans causing problems, or are part of the Silent Majority who consume media without going into fandom spaces. At the end of the day, creators are only human, and a person shouldn't reasonably expect to be berated at all hours for decisions they make at their job, their passion, or their hobby. In many cases, this may result in a no-win situation for a creator: attempt to avoid the bad fans, and the fans will hate you; alienate the good fans, and the fans will hate you.
On other occasions, it may be due to the behavior of the fandom towards others, rather than towards the creator. It's difficult to respect your fans when you constantly see them starting fights with other people for no good reason. This is especially true when this bad behavior involves offenses that are flat-out illegal, like DDOS attacks, doxxing, or death threats. All of this behavior is likely to reflect badly on you and your work, especially if these fans did it out of some misplaced desire to protect your "honor" or avenge a perceived injustice.
This can get especially dark when this kind of negative behavior is done towards things that people do as a hobby or out of personal enjoyment, rather than something the creator does as their job. Many a rant has been made by harassed creators/producers/personalities who state that they could be off doing better things or that their life is already stressful enough with their other job(s) that they really don't need to come home from a long day's work just to be hassled by the Fan Dumb. And many a rant towards pirates have been made saying that they actually need to make money or else they won't be able to produce further installments.
Most frustratingly, the fans who cause creators the most problems are also the ones least likely to understand, appreciate, or accept the effect that their actions have had. Instead of using it as a lesson, they will almost always turn on the creator, decrying them as sellouts who only care about money, egomaniacs who just want clout, performative allies who stand for nothing, or whatever else they can demonize the creator as, and will use that as a blanket justification for whatever bad acts may follow.
In some extreme cases, the creators can mistake a rowdy behaviour of a troll or other non-fan as a bad side of the fandom. Trying to discipline fans because of them for that can lead to bigger backlash than normal. Sometimes, such individuals can even try to evoke such reaction from the creators to disrupt the fanbase.
