Opensource AI overtakes Woke AI?
(twitter.com)
You're viewing a single comment thread. View all comments, or full comment thread.
Comments (28)
sorted by:
I've been playing with opensource AI but just the last couple weeks. I'm still new and figuring it all out, and I only have 16GB of VRAM to work with, so it's a bit limited.
Don't get the idea that all open source AI isn't woke. There's Gemma which is Google's open source release, and it's pretty tough I said nigger and it went total shutdown, etc. The sad part is it also is one of the better working ones I've tried on real hardware that is somewhat obtainable without dropping thousands.
I looked up Kimi and it's totally unobtainable for us plebs, it's suggesting a minimum of 240GB of RAM and if you want any speed at all, it's going to need to be VRAM, so good luck with what they've done to RAM prices.
240gb of VRAM just for inference is INSANE. Bill Gates assured me I only need 640k.
Iirc, a fair number of smaller-scale self-hostable models are trained/built off of larger models. Either scaled/compressed down or just trained off of to produce models. And often combined with other AI popular model portions to produce hybrid models So we may end up seeing some self-hostable models based off of it eventually.
It's a really weird kind of concept and approach compared to how stuff normally works with anything technical and software related (it would be like building a Photoshop clone by having some software learn from some scripted use of photoshop).
That's sort of my thought process. Small models are getting decent too. I think some mindset is turning into them becoming good at looking things up for example. You ask about video games from 2010 instead of it having to know, it gets good at searching and interpreting. The mega cloud models already do that well.
This NYT reporter alleges to have tried it out :
https://x.com/chrissgpt/status/2077852656182129078