That’s probably because people hype it up to such an extreme standard that once they read it doesn’t reach there expectations so they drop it. As of now mha is 3rd favorite currently running jump series only after CSM and JJK but personally for me the season 1-3 portion was just ok to me. The pacing was way to slow for me, there was only a handful of good fights in the manga version, and the non action focus arcs (especially in season 1) were mediocre at best.Really the only reasons why I was reading was it’s popularity, amazing art, and well written hero characters (and stain).
The season 4 portion is where it got good but it still had some of the mentioned issues. The endeavor arc definitely is where it has gotten great but then it went back to being just ok during the joint training arc. However the liberation army arc and after is when it became one of my favorite series where there isn’t really a dull moment and slow moments are done better, the fights still having some issues has become much more intense and exciting, the league of villains are developed into being actually intimidating, multi dimensional, and believable.
While I feel that the manga gets way more hate than it deserves, when people constantly hype it up as the best shonen out and some even go as far as to say it is a deconstruction of shonen tropes I think when they look at season 1 and don’t find what they are looking for people will stop wanting to watch/read it and call it overhyped.