This is not viable in the current hardware and software landscape. If new software architecture comes out that replaces transformers, maybe. If chinese memory manufacturing comes online and drops the cost of vram by 90%, maybe. I don't think we're going to quantize our way into a 24GB gddr card replacing a 140GB hbm module.
Right now if you want to run a top end model you're gonna need $50,000 of hardware and it's still not going to be as good as claude. The only sector where local is out performing cloud is generating porn.
Local outperforms cloud for very specific tasks not for general AI. If the AI model is niche then local will work better simply because it isn't censored unlike general AI.
You're right, for now. In 15 years though, our phones will probably be able to run local AI better than the $50k tech today.
I don't see any other avenue if you want anonymity though.
Doubt they will increase phone performance much in the future. They will just keep the best tech for governments and companies. And the end user will get a little more storage and maybe a better battery.
Maybe but that hasn't been the case so far. I could see the government and companies colluding to ensure people never have sufficient hardware to run local AI.
It has been the case with graphiccards for a while already. Took Nvidia forever to upgrade their Vram for their middle class cards in 2020 they finally put 8gb vram on them and in 2015 they had like 3-4gb vram, now they are at 12gb vram some games already require that to run.... And most of the other speccs maybe improve like 10-15% in 2years and the power needed goes up and up. They take their sweet time over the years to not release too powerful cards or people might sit on them for 10+ years if nvidia has no breakthrough in their tech.
No but you can get a 128 GB unified memory mac that can send 124 GB of it to the graphics card with an admin command for less than a 19 year old Ford Crown Victoria.
This is not viable in the current hardware and software landscape. If new software architecture comes out that replaces transformers, maybe. If chinese memory manufacturing comes online and drops the cost of vram by 90%, maybe. I don't think we're going to quantize our way into a 24GB gddr card replacing a 140GB hbm module.
Right now if you want to run a top end model you're gonna need $50,000 of hardware and it's still not going to be as good as claude. The only sector where local is out performing cloud is generating porn.
Local outperforms cloud for very specific tasks not for general AI. If the AI model is niche then local will work better simply because it isn't censored unlike general AI.
You're right, for now. In 15 years though, our phones will probably be able to run local AI better than the $50k tech today.
I don't see any other avenue if you want anonymity though.
Doubt they will increase phone performance much in the future. They will just keep the best tech for governments and companies. And the end user will get a little more storage and maybe a better battery.
Maybe but that hasn't been the case so far. I could see the government and companies colluding to ensure people never have sufficient hardware to run local AI.
It has been the case with graphiccards for a while already. Took Nvidia forever to upgrade their Vram for their middle class cards in 2020 they finally put 8gb vram on them and in 2015 they had like 3-4gb vram, now they are at 12gb vram some games already require that to run.... And most of the other speccs maybe improve like 10-15% in 2years and the power needed goes up and up. They take their sweet time over the years to not release too powerful cards or people might sit on them for 10+ years if nvidia has no breakthrough in their tech.
No but you can get a 128 GB unified memory mac that can send 124 GB of it to the graphics card with an admin command for less than a 19 year old Ford Crown Victoria.