Lacking winter might help, but many areas just straight up had drought and dry seasons that made food just as difficult to grow. Often times more than just for a winter, which is why agriculture was far less developed if at all.

Its also not just "its hot" level problems, the fauna there is far more dangerous than in most other areas, with a lot more bite and venom to go around. Because even the animals themselves adapted to the harshness of it by becoming more lethal. Which means even the abundance of it isn't nearly as helpful because of how many you'll lose dealing with them.

There is a reason why nearly every "civilized" nation came out of somewhere that had cold/temperate climates, no matter how bad winter is, and most of the tribal slower to civilize ones came out of tropical and desert ones.