You can be born to a drug addicted mother or you can be born to billionaire parents

It's not a statement trying ignoring genetics or parental impact on a child and saying it's all random. It's a framing that only really works if you believe in a soul as the defining feature of a person. The "lottery" aspect is looking at it as if you're a soul about to be born onto earth. Then which parents and their circumstances are the random part.

It's just saying you didn't pick your parents and have to work with the opening hand you were dealt.

If you don't accept the framing then the sentiment doesn't make sense. The crack whore's kid could never have been born to the hardworking parents and vice-versa. In that case there's no possibilities because there's nothing comprising "you" before birth.