Just because a person is in a powerful CREW and had legendary adventures doesn't mean they're strong. Look at the third level Straw Hats, Nami and Ussop (I consider Chopper middle level with Robin and Franky, because of his Devil Fruit powers). Ussop is probably about the same strength as Buggy, and he keeps up with Luffy's adventures, yet he's still weak, for a Straw Hat pirate. As long as you have some sort of skill that will help the crew, even if it is just to have an extra pair of hands to help out, you're useful, if weak, and Buggy was probably protected in fights by other, more senior crew members. So was Shanks, probably, he was just a cabinboy then. It's just that after Roger died, Shanks went back into the Grand Line and got alot stronger, strong enough to be a Yonkou known throughout the world, while Buggy went to East Blue, the weakest ocean, and did get a bit stronger, but not as much as his former crew mates. Although Impel Down is right, he certainly has gotten stronger since Volume 3 when Luffy defeated him, during his journey to get back his body parts and fighting his way through the Grand Line. But he's still not at a legendary level, and probably won't be ever. He just wasn't born for greatness, Buggy.
[/mod]