The only way FSD could even have a chance to take over is a network of vehicles that work in sync to ensure constant flow of traffic. No jackasses rubbernecking, running red lights, swerving lanes to get a few cars ahead, causing random stops and starts that are the main cause of traffic jams. Effectively you'll have just chained all cars together which would allow for more seamless travel in heavy traffic conditions.

But that's not going to work unless they start heavily pushing for all vehicles to be like this, otherwise you have too many vehicles not on the chain disrupting it, so people don't have a choice to buy what would later be called a manual vehicle, as in one you manually drive.

But also, as you point out, there are massive flaws that we cannot currently overcome with the technology we have. Now, maybe, sometimes in the future that might change, but I don't see it as feasible due to so many factors that can come up on the road that an FSD would not be capable of handling.