Like, it literally creates answers out of thin air then sells it as if it's correct. It doesn't even try to get it right. What sort of redundancy is there in analyzing if the answer is correct before spewing it out? I thought LLMs were supposed to discern what the best answer is given what was said to it based on its training, yet it'll give answers that don't exist based on any training. It's not like it learned the wrong answer from a Reddit post and just posted what Reddit said. It legit is making up wrong answers then citing correct answers. It just outright gets it wrong almost on purpose.
Anyone understand why LLMs fail so much?
I understand they run correlations but how does it determine a wrong answer is the most correlated to the correct response given the prompt instead of the actual correct answer...
I have seen it do exactly that. Ask it a niche enough game question and you'll see it happen, especially if it's a tool that provides links to its claims. Someone makes a post with bad information, gets corrected in the comments, edits their post. But since an LLM is just looking at which words happen in which order.
Still tells them LLM "can always X after Y." There's a much stronger relationship between the words in the original statement and it doesn't grasp the concept of a correction. Nor does it pick up on the strikeout markup.
That's why it excels at things that are well documented. Ask it about a function that's in a thousand tutorials and it will nicely distill the gist of the function and be fairly accurate. Ask it about something obscure only a handful of people have been guessing at, it hallucinates.