Subtitles and actually having to seek shows out turns off a surprising amount of viewers. A majority of the entertainment that we watch is only watched because it's presented to us. It plays at a theater within driving distance or it airs on a channel that we frequent. It's what often leads us to believe that the glut of movies that we watch now are worse, on a basic level, than the movies that we watched back in the '70s. Not true. We're still getting all of the good s**t from the '70s, like The Godfather and The Texas Chain Saw Massacre and Taxi Driver. That's all still being presented to us at a fairly steady rate. But if we lived back in the '70s, when s**tty movies were presented to us at the same rate as the good ones, we'd probably be nostalgic for the '30s. Not searching for things skews our idea of how our modern entertainment compares to past entertainment.
Anime, on the other hand, is very rarely presented to us, unless someone with money has decided that American kids will love it and buy its Nintendo 3DS tie-in fighting game.
The ones that we are presented with are usually the ones with the most digestible concepts. People get in giant robots and they fight. People own monsters that they use to fight. A guy writes names in a killer notebook and thinks. They're the ones that require the least amount of hand-holding because, for the most part, the people in control of what goes on TV think that the people who watch TV are the dumbest creatures in existence. The same concepts over and over will eventually start to wear on a person. And since anime doesn't have a real "golden age" on American TV that most people can misconstrue as something that was perfect, all you're left with is a bunch of people that just really don't like anime.