Well, I'm not sure what would happen in the manga, but I think it would be interesting to discuss.
First of all, the most obvious would be that 4 of his elite 10 supporter would be gone. If Kuga, Isshiki, and Erina doesn't change their mind, there would be a 3:2 supporter, with Azami losing...