for all that's worth Oda didn't use a Swastika (crossed "SS" 卐) but a Reverse Swastika (crossed "ZZ" 卍).
The reverse swastika "卍" or "Manji", being a kanji and all, is part of the japanese language. It doesn't mean "Nazi" just because they used it a thousand years after it was commonly used in...