Grok is cooked
(twitter.com)
You're viewing a single comment thread. View all comments, or full comment thread.
Comments (17)
sorted by:
Grok is not thinking for itself but regurgitating statistically likely words and descriptions.
Patently obvious but it always needs to be said again. They say "Words Are Magic Spells", and that's doubly true for these golems. There's very no logic or reasoning going on, even in so called reasoning models. It's looking for expected outputs to your inputs.
It will never be possible for alternative "right-wing" chatbots to do anything different, because they build on expensive work of previous models which have the bias baked in. Even without any devious intent, the creators of the models believe Grok's statement to be true. It's what the modern, corporate walled garden, libshit normie internet believes. So at best you'll always get this current of liberal normacy bias and recency bias. The goal of LLMs is not to be "maximally true" as Elon likes to pretend. The reason for their creation was to deceive users by appearing human. It's to copy the style of text so closely that you can't tell it's a machine. They were literally designed around fooling humans on reddit.
I've always wondered if the best approach to an alternative model would be to not show it the modern Internet at all. Train it from the ground up with old historical texts (like the writings of Cicero), classic literature, fairy tales, the Project Gutenberg archives, and maybe pre-2012 Internet archives. Tesla of course would have the compute to try this if they really wanted to create a "based AI" from the ground up, but they don't. I'm not sure how useful such a model would be either.
This is largely true, but only to the degree that most people don't think for themselves either.
This ain't how it works.
It is literally just a whole bunch of math to predict the next token in a sequence of tokens.
It literally isn't. You're more than a decade behind.
Grok (and others) can do math problems. And not just 2+2=4. It can execute algorithms. It can do spatial reasoning. If you have the mental capacity yourself, try asking it some difficult math problems and check the results. Try doing that with one of the statistics-based chatbots that you're thinking of and see what happens.
If it worked the way you claimed, all of that would be impossible. Neural nets aren't just "a whole bunch of math." Training them? Yes, that's a lot of math. Using them? No, that's not how they work at all.
Source: have designed and trained simple neural networks before. Have used Grok to analyze and help design a novel triangle mesh generation algorithm.
Does it have knowledge of the tokens or does it just have an improved ability to predict the next token in the sequence?