I agree with almost everything you said, the Hyogo fight isn't really necessary, and Gon might have been traded for Ito (a trade I'd take again any day). But, the door isn't necessarily closed on those fights either. After all, Kojiro just arrived, so there could be plenty more opportunity for other events and setup before the fight. All those possibilities aren't necessarily precluded by this episode, it's even laid the groundwork for how Inoue can touch on those subjects without going through the series' usual pacing.
Personally though, I was hoping Inoue
would flesh out all that material into dozens more volumes, and even do the largely untold story of Musashi's life post-Kojiro. Though, ultimately, this is now, more than ever, Inoue's story, rather than Yoshikawa's or even Musashi's, and what we expected, or thought was important, from those stories might not be important to him. Only he can show us that, and I prefer it that way, since, unlike those other events, we can't read about it anywhere else.
On that note, once Vagabond ends, I recommend everyone read Eiji Yoshikawa's
MUSASHI, whether you have before or not, as well as
Miyamoto Musashi: His Life and Writings, by Kenji Tokitsu. The latter is perhaps the definitive history on Musashi and his work. It contains detailed historical accounts and objective analysis of his life, as well as new translations and analysis of Musashi's own writings. I wouldn't be surprised if Inoue himself read it, or at least researched the same sources, as preparation while producing Vagabond.