If the wiki is to be trusted then the ages are 58 for kizaru, 49 for aojiki and 55 for akainu. Its true kizaru is the oldest one there however I don't think the 3 years he has over akainu would be enough to deem akainu the youthful one there. And even then, if such a thing was a factor then aokiji would have been the only choice. Also worth noting, sengoku made it as fleet admiral into his 70s and even then he was the one who quit, he could perhaps still easily hold the position otherwise. Continuity is without a doubt important for such a position but I doubt it necessarily has to be a 20 year plan so to speak.
I also doubt a fight between the two would some down to haki mastery. To begin with at this point we should just assume all admirals are proficient at it. The difference in haki skill would have to be gargantuan for it to be defining aspect IMO. Just look at law, 2 years ago he probably didn't even know what haki was and not to long ago he defeated vergo, the single most proficient haki user we have seen go all out yet (no one has ever done a full body haki hardening so far).
I still don't think kizaru's speed is that much of a defining factor here. IMO the whole speed thing between him and akainu would end up playing pretty much the same way as it did between him and rayleigh. Kizaru is by no means lightspeedblitzing anyone, what little we have seen already outright denies that such a thing could work on someone of roughly his level.