Gildards was strugling only due to Azuma's ability. When he got his magic back, he just trashed Bluenote. You also seem to forget, that Gildards survived encounter with Acnologia, and Acnologia one-shoted God Serena.
He was described as most powerful mage in Fairy Tail once, and i doubt Mashima...
