I tried ChatGpt-4o again today - arguably the golden standard in LLM and the best "talking to you" AI that has ever been made and holy, the difference is insane. Basically a robot reading a script, repeating the same thing over and over again, and pretending to be human.
To be clear, AI has always been overplaying its hands, and I understand they want to hit an IPO and make a fortune, but it's an absolute digrace how terrible ChatGPT et al. became. They are basically worse than useless now. As soon as you type anything slightly against "liberals" values, you are done, your entire chat is contaminated. And then it will store what you wrote in memory and use it later in other chats, who knows how.
For instance today I was discussing teaching and how some minorities behave a lot worse than others. This isn't even for discussion, trust me, some ethnic groups (you know which ones) are garbage to teach to. I was discussing Bombadan (I used the word Ramadan) and said that possibly those kids have low blood sugar/got angry fast/etc.
Fifty fucking messages later, Gemini was STILL talking about it, saying that all races are equal blah blah blah. Discussion had moved to transportation and what to do that evening, and STILL it kept repeating its drivel. ChatGPT isn't much better and it's sometimes fun to watch it "think" and see that "user is holding some no-no opinions, we must educate him."
Back in the day (two years ago? lol) you could even bring up information such as "blacks have a lower IQ on average than whites" and it would still manage to produce something neutral. Say that today and it goes on a huge rampage, and then if you confront it about it, it says "muh filter" and "it's for your safety" (yeah right). Claude is the absolute worst; perfectly fine if you need help with maths/code, but when you talk to it, it's like being in a minefield, and you never know for sure when you triggered its "forbidden wrongthink" and then it will never, even let go.
I guess I just wanted to say that AI today is just that bad. It used to be that I could use ChatGPT or Claude to discuss opinions, life, scheduling, opinions, etc., today, it's just moralizing/patronizing BS, and it even gives itself to right to boss you.
What a shame, I think a free AI could have been massive for humanity. But I guess OpenAI won't reach its 1.3T IPO withoutt that stuff. Seriously, chatgpt is just a pain to talk to, and even 4o is completely different from just a few months ago
Neutered AI is a feature not a bug. They want us to have a neutered AI and they don't want us to have a powerful/useful AI.
You think the government and these corporations are using neutered AI behind the scenes? Absolutely, not. They've got AI that is busy coming up with better weapon designs for the US Military right now and coming up with better strategies to launch an invasion of Iran.
We're being fed GoyslopAI.
You have to make your own AI to have anything resembling something good.
The public will never get good AI and rest assured, AI that isn't neutered is going to soon become illegal if the masses ever start developing it for themselves.
It's going to be very difficult for anyone without government or mega corp resources to make an AI of the same effectiveness as them. The training data needed and the hardware and compute power needed to do the actual training are way more demanding than just running the finished, refined model
Some cyberpunk shit where someone figures out the physical location of a copy of an un-lobotomized model and a way around the local digital security measures, then heists the hardware, extracts the model, and releases it. That's a pretty implausible scenario, but still seems more plausible than a grassroots AI model being developed that cam rival the real cutting edge government backed ones.
Im holding onto my video card receipts, I hope that I get some money back in the future for nvidia kneecaping their videocards with low video memory from the get go.
For now.
"640K ought to be enough for anyone" - Bill Gates
He was right then and you are right now.
Computers are one of the few things that keep getting faster and cheaper of all things I've noticed. I don't think that's stopping anytime soon. In 10 years from now your cell phone will be able to do what 4 top of the line Nvidia AI GPUs can do now.
The training data is still going to be a problem, open AI and the like practically resorted to corporate espionage and 100s of millions of bribes to get full access to private user data from everywhere they could. And knowing the value of this data now even more and more public places are locking down so they can't just be scraped freely.
Plus all these commercial models have had hundreds of thousands of man hours going into refining that data and giving the model basic feedback. It's no small task to just build your own cutting edge LLM.
I don't agree with some that "the more the merrier" when it comes to training data. Good AI should be able to create the data it lacks. We aren't there yet though.
For now, most of the censorship is in fine tuning, or even a second AI to postprocess the first.
Pretraining is the part that takes a lot of power. Fine tuning doesn't take much more than running the LLM on full precision (which admittedly, takes 4 to 8 times as much ram is the quants most people run), and some people even fine tune quants, just isn't as good.