It's not like it learned the wrong answer from a Reddit post and just posted what Reddit said
I have seen it do exactly that. Ask it a niche enough game question and you'll see it happen, especially if it's a tool that provides links to its claims. Someone makes a post with bad information, gets corrected in the comments, edits their post. But since an LLM is just looking at which words happen in which order.
You can always X after Y
Edit: I'm wrong. Sometimes you can't.
Still tells them LLM "can always X after Y." There's a much stronger relationship between the words in the original statement and it doesn't grasp the concept of a correction. Nor does it pick up on the strikeout markup.
That's why it excels at things that are well documented. Ask it about a function that's in a thousand tutorials and it will nicely distill the gist of the function and be fairly accurate. Ask it about something obscure only a handful of people have been guessing at, it hallucinates.
It's not like it learned the wrong answer from a Reddit post and just posted what Reddit said
I have seen it do exactly that. Ask it a niche enough game question and you'll see it happen, especially if it's a tool that provides links to its claims. Someone makes a post with bad information, gets corrected in the comments, edits their post. But since an LLM is just looking at which words happen in which order.
You can always X after Y
Edit: I'm wrong. Sometimes you can't.
Still tells them LLM "can always X after Y." There's a much stronger relationship between the words in the original statement and it doesn't grasp the concept of a correction. Nor does it pick up on the strikeout markup.
That's why it excels at things that are well documented. Ask if about a function that's in a thousand tutorials and it will nicely distill the gist of the function and be fairly accurate. Ask it about something obscure only a handful of people have been guessing at, it hallucinates.
It's not like it learned the wrong answer from a Reddit post and just posted what Reddit said
I have seen it do exactly that. Ask it a niche enough game question and you'll see it happen, especially if it's a tool that provides links to its claims. Someone makes a post with bad information, gets corrected in the comments, edits their post. But since an LLM is just looking at which words happen in which order.
You can always X after Y
Edit: I'm wrong. Sometimes you can't.
Still tells them LLM "can always X after Y." There's a much stronger relationship between the words in the original statement and it doesn't grasp the concept of a correction. Nor does it pick up on the strikeout markup.