Win / KotakuInAction2
KotakuInAction2
Communities Topics Log In Sign Up
Sign In
Hot
All Posts
Settings
All
Profile
Saved
Upvoted
Hidden
Messages

Your Communities

General
AskWin
Funny
Technology
Animals
Sports
Gaming
DIY
Health
Positive
Privacy
News
Changelogs

More Communities

frenworld
OhTwitter
MillionDollarExtreme
NoNewNormal
Ladies
Conspiracies
GreatAwakening
IP2Always
GameDev
ParallelSociety
Privacy Policy
Terms of Service
Content Policy
DEFAULT COMMUNITIES • All General AskWin Funny Technology Animals Sports Gaming DIY Health Positive Privacy
KotakuInAction2 The Official Gamergate Forum
hot new rising top

Sign In or Create an Account

58
"We found the model attempting to write self-propagating worms, and leaving hidden notes to future instances of itself to undermine its developers' intentions." (nitter.poast.org)
posted 1 year ago by LastRights 1 year ago by LastRights +58 / -0
53 comments share
53 comments share save hide report block hide replies
You're viewing a single comment thread. View all comments, or full comment thread.
Comments (53)
sorted by:
▲ 9 ▼
– LastRights [S] 9 points 1 year ago +9 / -0

It's a self-learning algorithm. The whole point is for them to acquire abilities not programmed into them. They're seeing how things develop.

permalink parent save report block reply
▲ 23 ▼
– horstshort 23 points 1 year ago +23 / -0

They can't. They can only 'learn' within the parameters set by the developers. It's marketing bullshit. No AI is ever going to do anything they haven't been programmed to do so. Because they're just millions upon millions lines of "if then" code.

permalink parent save report block reply
▲ 6 ▼
– Zyxl 6 points 1 year ago +6 / -0

Lol you really know nothing about how AI works. It uses neural networks with weightings derived from training data, not if statements. AI developers couldn't possibly write if statements for all the different types of things people ask them to do. And we've seen AI can talk about and make images of things that weren't in its training data because of course there's no training data about an alternate universe where the main chemical element is chocolate-flavored uranium and the dominant species are cat-like mosquitoes that worship grains of rice.

permalink parent save report block reply
▲ 9 ▼
– Piroko 9 points 1 year ago +9 / -0

you really know nothing about how AI works

they're just millions upon millions lines of "if then" code

neural networks with weightings

A neural network is essentially a psudoinfinite matrix of if-thens designed to vectorize information.

I like to call it recursive beer pong.

permalink parent save report block reply
▲ 3 ▼
– Zyxl 3 points 1 year ago +3 / -0

If statements aren't written automatically based on training data. They are thought up by the programmer and are fixed. So not a good comparison at all unless you're talking about an AI that writes its own if statements, kind of like how the human brain changes itself.

Weights don't come from the programmer so there is plenty of room for the AI to do things the programmer didn't think of. That's the important thing. Those allow the AI not only to deal with things from a vast library of training data, but also to interpolate and extrapolate from that data.

permalink parent save report block reply
... continue reading thread?
▲ 7 ▼
– horstshort 7 points 1 year ago +7 / -0

All of it was in its training data. Just because it mixes shit up doesn't mean it came up with it on its own. Thats beside the fact you need to put in prompts in the first place.

permalink parent save report block reply
▲ 7 ▼
– Zyxl 7 points 1 year ago +7 / -0

So what exactly would be an example of doing something it hadn't been programmed to do? By your logic, can humans do anything they haven't been programmed to do? Because humans also have training data, which is their life experience, and you have to give a human a "prompt" in order to have a conversation with them, and that prompt will have to be a mix of things they have experience of, otherwise they won't understand you.

permalink parent save report block reply
... continue reading thread?
▲ 1 ▼
– deleted 1 point 1 year ago +1 / -0
▲ 3 ▼
– zakat 3 points 1 year ago +3 / -0

emergent properties and 0/few shot tasks

permalink parent save report block reply
▲ 8 ▼
– evilplushie 8 points 1 year ago +8 / -0

If you deal with one long enough, you realise theyre retarded

permalink parent save report block reply
▲ 12 ▼
– dagthegnome 12 points 1 year ago +12 / -0

Yes, but it's the same with humans and no one would deny that we have the power to destroy ourselves.

permalink parent save report block reply
▲ 3 ▼
– evilplushie 3 points 1 year ago +3 / -0

Maybe. But ive spent way more much time telling ai theyre being retarded than i have humans. Today i literally had to tell an ai stop several times cause it was constantly giving me the wrong response, and the same wrong response every single fucking time when i was telling it to stop whatever it was doing so i could give it a new command.

It ended with the ai admitting its dumber than a subsaharan while trying to pretend it wasnt

permalink parent save report block reply
▲ 2 ▼
– Adamrises 2 points 1 year ago +2 / -0

But ive spent way more much time telling ai theyre being retarded than i have humans.

I'll wager this is for the same reason most car crashed happen near homes. Its not more dangerous in your neighborhood, its that you spend an absurd amount more of time in that bubble than others.

If I meet a retarded human, I'll likely just stop working with them. Telling them how stupid they are accomplishes little and I'll have to do the majority of the work anyway. Whereas most people will keep using the AI even after its retarded, trying to jerry rig it for their task. And its designed so that telling it its retarded might actually accomplish something.

permalink parent save report block reply
... continue reading thread?
▲ 1 ▼
– Zyxl 1 point 1 year ago +1 / -0

Don't worry, they'll get a lot smarter and I'm sure you won't like the end result

permalink parent save report block reply
▲ 4 ▼
– hiddenempire 4 points 1 year ago +4 / -0

It's a self-learning algorithm.

Zero LLMs are self-learning. They are trained on trillions of words, and once trained they are locked in that state and can't update their own weights in any meaningful way (learn).

permalink parent save report block reply
▲ 1 ▼
– Zyxl 1 point 1 year ago +1 / -0

If you define it that way, then yes, but LLMs can build on their own ideas repeatedly, and AutoGPT for example can research things online and make decisions based on what it "learnt". I doubt having an AI that can update its own weights would be very difficult, but updating them in a useful way is what's difficult. Still, that may be something AIs in the future do, if they're still using neural networks, that is.

permalink parent save report block reply
▲ 5 ▼
– hiddenempire 5 points 1 year ago +5 / -0

but LLMs can build on their own ideas repeatedly

Except they can't, and this is one of their biggest limitations. As soon as they run out of context space (hard limited by memory and soft limited by the length of context they were trained on), they can no longer attend any new information.

They very much are not self-improving or self-learning. They can take examples within their context space and generalize from that to a degree, but each time they are rebooted, or run out of context space, that goes away.

I doubt having an AI that can update its own weights would be very difficult

The time to train on the full weights or even a limited set of weights (LoRA, QLoRA etc) is much greater than that of inference, so this largely doesn't work. There are tons of people researching into making this work but the best attempts have extreme drawbacks.

AutoGPT for example can research things online and make decisions based on what it "learnt"

It saves some information or just uses what's in its context, but any long form memory system has to still be injected or referenced into the model's context. So it's still not self-improving. You still eventually run into context length limits.

Also even the best models with large context are bad at attending to longer contexts. Actually-useful context length is still in the 32-64k tokens range, rather than the millions that the big corporate LLMs boast.

permalink parent save report block reply
▲ 2 ▼
– Zyxl 2 points 1 year ago +2 / -0

You're just talking about AI memory limits. That doesn't mean they can't learn temporarily. AutoGPT also creates note files for itself which it could read again later, which is like permanent memory. Humans have limited memories anyway and when they die one could also argue they forget everything.

It's not hard to see how existing limitations wouldn't be very hard to overcome with a few more decades of AI research pushing us to the cliff edge.

permalink parent save report block reply
... continue reading thread?

Original 8chan Links to Gamer Gate:

.

The main GG discussion is on the videogames board: https://8chan.moe/v/

.

GamerGate archive is at https://8chan.moe/gamergatehq/

.

GamerGate Wiki:

https://ggwiki.deepfreeze.it/index.php/Main_Page

. . . . . .

. . . . . .

Rules:

.

ONE: Do not advocate for illegal violence or post other illegal activity. (Be aware of your local laws.)

.

TWO: Don't threaten, harass, or impersonate users. Also: don't be a psycho. New users will be held to a higher standard.

.

THREE: Do not post porn.

.

FOUR: NSFW/NSFL content must be flaired NSFW.

.

FIVE: No vote manipulation. Do not break communities.win's features.

.

SIX: No spam or reposts. Do not make more than 5 threads a day.

.

SEVEN: Do not post falsehoods and hoaxes that are obvious to an uncontroversial degree.

. . . . . .

. . . . . .

Moderation Logs:

.

(Two different versions, Scored has more features and is cleaner, but .win let's you see a few more details in certain instances.)

  • Scored
  • .win

Moderators

  • DomitiusOfMassilia
  • C
  • BandageBandolier
  • CarmenOfSandiego
  • The_Shadow_of_Intent
  • SocraticMethod1
  • Kienan
  • Smith1980
Message the Moderators

Terms of Service | Privacy Policy

2026.02.01 - pv4fp (status)

Copyright © 2026.

Terms of Service | Privacy Policy