unless it's too hard

This is exactly what I'm getting at. It was the exact right hardness to keep them from being able to evolve much further, but not completely extinct them.

You can see the exact scenario play out on the individual level right now with the extreme poverty/homeless people. Those who can just skirt by enough to not die, but can never actually get anywhere near enough to begin improving their lot.