First of all, the show never particulary fell in my taste.
The first seasons was somewhat good, but I didn't get into the show until the second half of the 2nd season. Third seasons started of fine, but towards the break it became epic, imo. But afterwards, it became rather shitty.
I would've preferred if this show went full-on survival mode, then end up somewhere and get big and try to rebuild. The whole governor thing? I hated that.
The prison was nice, but they stayed too long, which I didn't expect, I would've expected them to have moved on somehow, but picking up all those other people stopped that.
And then there's the governor.. He's still there.. He's coming back, then probably everyone's going to die and the group moves on. That could somehow be fun, but I really, REALLY don't like the governor.
And no, it's probably not something that isn't important, if it is, I don't like it. But I bet that woman will be important in some way or another in a later scenario or something.