They'll fuck everything up before reaching that. Their attempts to make that happen are going to basically involve handing a LLM root access to some servers and then giving everyone the surprised Pikachu face when it wipes out their production environment.

That's not a hypothetical either.

https://archive.is/npzpa

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue