It is clearly stated in Anime that The barbarians attack caused the destruction of The Kingdom or the village where Escanor lived .
I don't remember anywhere it is said that Escanor killed his brother or anyone innocent for that matter .
You seems to be suffering from the Sin of Envy from...