If they were to fight it'd probably be more along the lines of Luffy fighting to prove he's become a great pirate (so he can give the hat back) As per chapter 1.