Like, it literally creates answers out of thin air then sells it as if it's correct. It doesn't even try to get it right. What sort of redundancy is there in analyzing if the answer is correct before spewing it out? I thought LLMs were supposed to discern what the best answer is given what was said to it based on its training, yet it'll give answers that don't exist based on any training. It's not like it learned the wrong answer from a Reddit post and just posted what Reddit said. It legit is making up wrong answers then citing correct answers. It just outright gets it wrong almost on purpose.
Anyone understand why LLMs fail so much?
I understand they run correlations but how does it determine a wrong answer is the most correlated to the correct response given the prompt instead of the actual correct answer...
I'll get into it. I'll try it with some metaphors but the TLDR; is that they're probabilistic, not deterministic, and that probability plays out in the growing and training phases as well as the prompting phases. I'll leave out talk about transformers and some of the nitty gritty around prompting.
The metaphor for it might be:
So your LLM studies. It has weights that help it ground and sort initially, but ultimately it's grown. It may or may not "understand" something. Like it might see numbers and go monkeys on typewriters trying to write a black box bit of code that represents a relationship. Adding or subtracting or what not. It might do it like us, but it's unlikely. Usually it comes up with thousands or millions of explanation functions and goes with the shortest and simplest, Occam's Razor.
YOU PROMPT IT
A LOT CAN GO WRONG
These things are getting more correct though as you throw more money at it, you buy more islands and more portal networks. There is less need for ambiguity and superposition on the islands (less need for tokens to occupy the same position in the vector database + dimensions). There are also more monkeys on more typewriters, so there are more lottery tickets and simpler and smaller Occam's Razors emerge as functions.
Also there are better legacy models to train the new models so if the tuning is known to be good, it can fire millions or billions of questions to post-train refine the model.
*They do relate content to other content weighted but mostly on their own. They do have algorithms that "understand" data in their way. They certainly learn. They probably think in something akin to a reflex arc that responds to stimuli but is not sustained, unprovoked and emergent, or certain other properties we consider to be thinking.
*But they are doing something and that something will augment humans to where they can probably get it to self-improve. Also assisted humans will find a more human path more quickly if a self-improved LLM isn't good enough to stop making stuff up so much.