candy factory in chapter 629, after someone gave him some sweets. I think those sweets came from the candy factory.
I think the point of Big Mam being brought up has something to do with Shirley's prediction. Whether Luffy destroys Fishman Island or whether someone else destroys it, it will be reported to the world that the island was destroyed by Straw Hat Luffy's hands (that's what Shirley was probably seeing). Fishman Island is an island under Big Mam's protection, by destroying it, the Straw Hats are basically declaring war on Big Mam.
Alternatively Big Mam may be even more eccentric than that, she may not even care about the island, but the fact that her Candy Factory was destroyed (reportedly by Luffy) may be enough for the Straw Hats to become her enemies.
I think Big Mam is due for an appearance soon...
All the Yonoku have been shown or mentioned so far..
We have seen Shanks, Whitebeard... we have learned about Kaidou attacking Moria, fighting shanks, having a cyborglike island.... The only thing about Big Mama we know is her name is Charlotte Linlin. Since this is her island, i am sure she will turn up.
Imagine if what Hodi said, after being asked by Fukaboshi,"Just what the hell...to cause this?" was none other than,"nothing," which is what shocked everyone to death hahaha
Notice here http://www.mangareader.net/one-piece/641/6 Shirahoshi shed not a tear over her bro while after seeing Luffy getting bitten by a Hodi here http://www.mangareader.net/one-piece/641/7 she was about to cry.
I saw some people predicting that she might fell for him. Apparently, you guys hit the spot.