It doesn't say you're not a fan, it's just the name of the button, telling you that if you press the button, you'll be saying you're not a fan. However I understand what you're saying, it can be confusing. Maybe it would be better to have an image saying I am a fan, and underneath have "Change image/Not a fan" and vice versa, maybe that might work. :_
Sorry, the line about it saying I wasn't a fan was sarcasm. I understand that it is a button, it is just unintuitive.
I've never really liked the way music players handle play/pause in this way either. Although, at least in that case it is fairly obvious if the music is playing or not, as there are other indicators (sound coming out of speakers, time elapsed incrementing, etc). Here, there really isn't any other indicator.
Don't mind me, I'm just roving the universe vacuuming up more knowledge for the archives