I don't think it's as much not being able to afford it, but a child being a burden rather than an asset. There is a reason peasants have a lot of children, while urban folks have far fewer - to the point that even in the middle ages, cities did not reproduce themselves (though this was also due to the unhealthy environment).