Well everything's obscure until it eventually becomes mainstream. And there always are people who look outside the box and then other people follow. Don't think it's a problem of Westerners, it's like that everywhere. It depends on the individual if you're shrugged of as a weirdo or nerd or whatever, if they harass you, or if they actually check it out and then either like it or don't.
Anime and manga is far from mainstream in my eyes, but it's certainly past the point of just being something totally obscure, almost everyone in their 10s or 20s had contact with it to some extent. Still seen as something childish and weird by the majority of older people I guess. Visual Novels are something like that to your average anime fan I think, but it has grown quite a lot in popularity in the last 10 years or so. Basically a niche within a niche, but you would be surprised how many people actually are a fan of them. Fate/Stay Night obviously is the most popular among them, the whole franchise has a pretty big forum in the west. I consider myself a pretty big Fate fanboy, but even I'm surprised and not able to keep up when I see some threads with 20k-150k posts xD
I'm sure you're curious but I can't tell you

We already had them once though, long before your time here