It understands the structure of making an argument, just like how Dall-E can seamlessly replace a bird's head with Darwin's head because it understands what a person and animal's head is, making a Darwin-headed raven (blog).

That's also how LaMDA creates its arguments. They read like actual English, but the content makes about as much sense as a Darwin-headed raven.

What it's actually really good at is projection. It can pick up on what you want it to say, and then you say wow it's really smart because it thinks just like me -- just like this clown did.