Well, at least from my point of view it changes you a huge amount. No matter how much Japanese you know, you're never ever prepared for the first time you go there, so the language barrier is so horribly terrifying at the start, but when it passes everythingg starts to even out well.
Living on your own in a completely different culture definitely teaches you to fend for yourself, anyway. I brought over this fantastic cookery book full of recipes for students. Was useful about twice in my whole stay, because what you can get over there is totally different. So yeah, it makes you grow as a person, definitely.
Student-wise, you have again a whole different college system, so at the start you again have the *flail* period, but it doesn't take long to figure out that the hardest part of Japanese uni is actually getting into it, and it's smooth sailing from there. So it leaves you a lot of time to go find something Japanese-ish to do.
So yeah, go. You won't regret it. No matter how much you might wee yourself the first couple of weeks, once you get into the swing of things it's the best thing ever. Especially as a student, you can do a huge amount that a working person could never do.