Like, it literally creates answers out of thin air then sells it as if it's correct. It doesn't even try to get it right. What sort of redundancy is there in analyzing if the answer is correct before spewing it out? I thought LLMs were supposed to discern what the best answer is given what was said to it based on its training, yet it'll give answers that don't exist based on any training. It's not like it learned the wrong answer from a Reddit post and just posted what Reddit said. It legit is making up wrong answers then citing correct answers. It just outright gets it wrong almost on purpose.
Anyone understand why LLMs fail so much?
I understand they run correlations but how does it determine a wrong answer is the most correlated to the correct response given the prompt instead of the actual correct answer...
Older AIs are like the Plinko game from The Price Is Right.
You drop the input piece down from the top and it usually ends up in a bucket near where you dropped it, but sometimes chaos makes it goes across to the whole other side. You give it a picture of a tree and by adding the right initial condition noise to make the plinko go off track make it say it's a cat or anything else.
LLMs are the same except they have 'attention' that groups parts of the input together first, so it's harder to have input that makes the plinko go way off. It doesn't just see words like "correct" "wrong" "answer", but "correct answer" and "wrong answer" as different things.
The bucket with the grand prize answer in it is bigger in LLM, but still sometimes just the way you phrased the input or some extraneous detail sends the plinko off into some crazy answer bucket. They also don't have side walls so if the plinko goes off the side the answer won't even be from training data but extrapolated and completely made up.
amazing analogy, I'm stealing that