They out here sending basic grunts and zombies whereas we have XCOM elite Templars, Skirmishers and Reapers that can devastate them if they got hot.

I think you're right just due to history, A LOT of empires fell because of insular information, nepotism and believing what was built before is enough to sustain them forever. Then they always get trashed by a smaller, more disciplined group that was forced to constantly adapt and innovate that is able to hit their weak spot so decisively they buckle.