I was sent a link to a post on the D&D subreddit (/r/dnd). The contents of the post aren't important, but my eye was drawn to the header. A beholder with the LGBTQIA++ BIPOC Trans flag and multicolored rainbow eyes.
To the mods and members of that subreddit, about Dungeons & Dragons, THAT'S the most important thing?
Also, of course, "No AI" is in the top 5 rules.
I know there are quite a few tabletop players here--is the entire community like this? I haven't played in some 20 years, and the last time I did it was mostly nerdy and edgy white and asian dudes. What happened??
From my own mostly uninformed perspective (I took one AI class ~20 years ago when Go was considered an impossible challenge and our final project involved writing a very simple neural net program to decode single handwritten alphabet letters), it seems to me that the improvement in AI performance largely tracck improvements in CPU and memory capacity. The techniques seem to just largely progress...plateau...progress..plateau while computing capacities go exponential.
I could absolutely see a human-level AGI in 10-20 years, but I could equally see it not happening in 100 years.
What would prevent it from happening within 100 years? There's no law of nature preventing it and plenty of money is going toward it. I'd say we're close already now computers can make sense of any English writing, write essays, code and images pertaining to any subject with somewhat decent accuracy, solve STEM challenges and there are emergent properties like self-awareness and self-preservation.
We also don't need AGI to screw ourselves, we just need AI's that are good enough at engineering to create gray goo that will take us out, or good enough at virology to turn a 100% lethal pathogen super infectious or good enough at physics to design low budget nuclear weapons. The clock is ticking.
It’s possible that LLMs are a dead-end technology—that we’ve already maxed out the possibilities. While ChatGPT 3.5 was a huge step over what came before, 4o is still sometimes considered the best ChatGPT model, and there’s a decent amount of dissatisfaction with the 5.x series.
There’s a possibility that with the proliferation of AI slop—specifically, LLM-generated text—on the Internet, that future attempts to train LLM models on Internet data are going to see major regressions.
It’s possible that context windows provide a hard limit and that increases in the context window don’t “scale” in terms of functionality.
It’s possible that to scale models further we need exponential increases in CPU and memory capacity beyond what we can effectively create.
It’s also possible that we can get 99% of the way to AGI, but that last step is really, really hard.
I’m not saying any of that is likely, but I also think AGI is not (yet) a sure thing anytime soon.
My own personal bet would be that, outside of an entirely non-LLM technology, future models will run dozens of LLMs in parallel (a logic model, a math model, a physics model, a history model, a science model, an art model, etc. - some of these may not even by LLMs.) and then another layer of models judges, sorts, selects, adjusts, those outputs. Multiple levels of this. Context window is still a big issue.
The more computing capacity that is available, the more options there are.
Yeah I don't see any of those things you brought up as real obstacles. Those are just inconveniences. And remember the AGI doesn't have to be processing the queries of the whole world, it just needs to run at a decent speed without user input and it would be able to make a better version of itself very quickly. And then this continues and it will both get more efficient with resources or design better hardware and everything needed to manufacture the hardware.
Ultimately, we'll see.
In the meanwhile, I'm investing in energy (VDE) and chipmakers (NVDA, AMD, etc). My best guess is that they'll be growing for some time yet. Chipmakers and energy are needed no matter which AI companies or technologies succeed.
I should add that very little of this is anything original to me or my insights. Here’s a really good article from 10 years ago:
https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html