I have an idea:
Luffy's grand-father (not Garp, the other) is the owner of a little restaurant, somewhere in the East Blue. Then his daughter (Luffy's mother) work as waiter and stuff like that. One day, Dragon arrived at the place looking for food and met the girl. Somehow, they fall in love and start a family. But Dragon has an irresistible urge to go against the government and leaves. Somehow, the grandpa and mother die, and Garp has to take Ace and Luffy to that island where the were raised.
It's similar to Ussop story...
I've just thought about another posibility

Gol D. Roger is Luffy's grand-father. Monkey
D. Luffy, Gol
D. Roger... think about it, Luffy's family is full of great famous men, it can be posible. And the ones who met the Pirate King are always saying that Luffy acts like him and says similar things, there is probably a family tie between them.
Well, then his mother must be Gol D. Roger's daughter, who probaly grew as a normal child who barely met his father.