Whoever that third guy is, I really wanna know what happens to him. I can't believe that he'd die or something if both Hongou and Sakaki are around him, unless he takes a death blow for them or something.
I doubt that, I'm guessing Saiga and Akisame are roughly the same age which would make them in their early thirties late twenties during the time of the flashback. Sakaki, Hongo, and Third guy all seem to be in their early twenties late teens. so It's unlikely that this guy is Saiga, or the possibility that the elder had a second son.
He is very talented though, I'm guessing he dies either through a dangerous situation like when they met Hongo, or he gets in between one of Hongo and Sakaki's fights and is injured. I think the last fight they had would not have this third guy as a referee, possibly due to one of the aforementioned situations.