I just asked my Gemini Pro to create an Ancient Roman General picture and it refused to do it because it's controversial.
There must be some deep dark realms of the internet where AI isn't so Judaized. What is it?
You're viewing a single comment thread. View all comments, or full comment thread.
Comments (38)
sorted by:
I tried DeepSeek, but not only is it still censored (along slightly different lines than Western LLMs, but still), it was also the dumbest closed-source one out there.
I've had access to the kind of hardware you need to run your own only once before, and I tried one of the so-called "uncensored" LLMs someone here recommended. It was the wokest, dumbest LLM I've ever used, hands down. The commercial ones run on way more powerful hardware (and it was a $15,000 machine I was running it on), and are trained on way more data than I'll ever see.
No, I mean that open-source models can't really be censored in any way that matters. You have direct access to the transcript. You can edit their messages, preempting refusals. You can even mask out the logits for tokens you don't want to see. The reason LLMs always begin their messages with "Sure, happy to do that!" is because messages that start with that are much more likely to result in outputs that fulfill the user's request, resulting in that verbal tic becoming dominant during fine-tuning.
You need the training data to achieve true non-censorship. They mask out tons of neurons before letting those models out the door, and the only way to get them back is to retrain.
In practice, you can't stop a released LLM from being jailbroken with the right prompts, but I'm interested in what you're referencing here. What method are they using to "mask out neurons"?
To my knowledge, nobody has quite that good an understanding of the internal connections of these models.
It's been a long time since I've researched LLMs, but I once read somewhere that they were capable of identifying the nodes in the neural net that were involved in generating an answer. If they remove those nodes from the model or zero the weights, then the NN loses whatever information was used. They call it "concept erasure" IIRC.
In any case, I've never successfully created my own jailbreak prompt that actually worked. But I only had an hour with that $15,000 computer that could actually run an LLM. I'm unlikely to ever see that much computing power again.