I know right, I found it odd he called him "his" men, Altho Doflamingo is a sadist fucker, He seems to value crew members a lot, even fallen ones, Reminds me of whitebeard/squardo. So this is not the plain old stupid villain, even "villains" have morals, ideas and beliefs.
Oda's story telling...