Your conclusion is objectively false, no matter what your feelings say. An LLM is a word frequency prediction calculator. That’s all. It doesn’t “understand” anything.
So when it doesn't hallucinate, is that good enough to say that sometimes it understands and sometimes it doesn't? Or do you need there to be no possibility of it ever being wrong?
When it hallucinates about things that no human being would ever hallucinate, you can’t trust it even when it isn’t, and you can never know if it isn’t unless you check against another source. At that point, you have no reason to use it whatsoever.
Your conclusion is objectively false, no matter what your feelings say. An LLM is a word frequency prediction calculator. That’s all. It doesn’t “understand” anything.
What is your criteria for true "understanding"?
Not hallucinating things that take five seconds for a human to prove.
So when it doesn't hallucinate, is that good enough to say that sometimes it understands and sometimes it doesn't? Or do you need there to be no possibility of it ever being wrong?
When it hallucinates about things that no human being would ever hallucinate, you can’t trust it even when it isn’t, and you can never know if it isn’t unless you check against another source. At that point, you have no reason to use it whatsoever.