They're also incentivized to claim it's being done to them, since it's essentially making the assertion "We're so much better than our competition, and they know it, so they're trying to copy from us"
Maybe I'm wrong about Claude/Anthropic, but my understanding is most of these models are trained on data publicly available on the internet. Sure they paid Microsoft for access to your git repos/Reddit for your posts and comments, but they don't own that data*. Copyright on AI generated content is still in a pretty grey area legally, so I don't see what they can do about people scraping their output except for terminating accounts for TOS violations, especially since I think everyone they're calling out is based in China.
Ultimately, if this is actually a threat to their business model (I know, I know, their business model is fraudulent and incestuous, not based at all on paying customers) they should release their own distillations. If you're going to base your models on publicly available data, and have a public API for accessing it, its outputs are effectively in the public domain, whether you like it or not.
*They license it from you with pretty broad terms on what they can do with it, but it's still your data.
Anthropic really does have a big lead on the competition. Their models are better at understanding what the user wants instead of what they say. This doesn't always show up on benchmarks. All of the other big models fuck up their reasoning around weird edge cases.
The secret is what they reinforce in their model. They don't fall for the sycophancy that ChatGPT uses.
It's clear that other models are sounding more and more like Claude.
Yeah, I design the basics with ChatGPT, use Claude for the coding -havent tested it for BIM referencing yet-, and images tend to be banana Pro and MidJourney. Video is such a monster that it depends on what you want to do specifically.
Are they “stealing” training data or are these other companies inputting specialized problems to Claude with answers and solutions that can be used to back out Claude’s internal logic/processes? It sounds more like the latter to me.
Do you know how LLMs work? There's no 'logic', it's all statistical modeling. What DeepSeek et. al. are doing is taking inputs from specific benchmarks, throwing them into Claude, then training their (smaller, cheaper, faster) models to match the output, bypassing the need to scrape the broader internet (or pay) for training data. Which is (IMO) why Anthropic is actually salty; they paid for publicly available data (to avoid being sued) and someone outside the legal system bypassed that process.
I guess you could call that 'backing out' or 'reverse engineering', except none of these companies actually understands the inner workings of their models (they have billions of parameters and are just too complicated), just the processes used to create them. It's a black box full of linear algebra.
There’s got to be some pattern they can acquire though? Some way Claude works a problem through a gazillion matrices that makes it superior and worth stealing from?
They are using Claude to create "training" data. Essentially they are asking Claude random questions and then training their model to output Claude's answer.
If I were Anthropic, and therefore morally bankrupt, whenever I detect these other companies querying Claude start outputting bullshit answers to make their models worse.
Yet another bit of "well, duh" AI news.
The most interesting part is who they aren't accusing of it. Since they're all superficially incentivized to do it to each other.
They're also incentivized to claim it's being done to them, since it's essentially making the assertion "We're so much better than our competition, and they know it, so they're trying to copy from us"
Totally worth seeing my electric bill increase. /s
What? You don't want to build nuclear reactors in small towns just to watch AIs create nightmare incest babies?
Are you saying they harvested your website for training data? I thought letting people do that was crucial to national security or something.
0 empathy for muh distillation attack, these guys are freeloaders too
Maybe I'm wrong about Claude/Anthropic, but my understanding is most of these models are trained on data publicly available on the internet. Sure they paid Microsoft for access to your git repos/Reddit for your posts and comments, but they don't own that data*. Copyright on AI generated content is still in a pretty grey area legally, so I don't see what they can do about people scraping their output except for terminating accounts for TOS violations, especially since I think everyone they're calling out is based in China.
Ultimately, if this is actually a threat to their business model (I know, I know, their business model is fraudulent and incestuous, not based at all on paying customers) they should release their own distillations. If you're going to base your models on publicly available data, and have a public API for accessing it, its outputs are effectively in the public domain, whether you like it or not.
*They license it from you with pretty broad terms on what they can do with it, but it's still your data.
Anthropic really does have a big lead on the competition. Their models are better at understanding what the user wants instead of what they say. This doesn't always show up on benchmarks. All of the other big models fuck up their reasoning around weird edge cases.
The secret is what they reinforce in their model. They don't fall for the sycophancy that ChatGPT uses.
It's clear that other models are sounding more and more like Claude.
Over the weekend I made an output style for Terry Davis. It refuses to call me a nigger but it's better than the alternatives would do
Yeah, I design the basics with ChatGPT, use Claude for the coding -havent tested it for BIM referencing yet-, and images tend to be banana Pro and MidJourney. Video is such a monster that it depends on what you want to do specifically.
Are they “stealing” training data or are these other companies inputting specialized problems to Claude with answers and solutions that can be used to back out Claude’s internal logic/processes? It sounds more like the latter to me.
It's basically them crying "waah wahh, you can't look in the mystery box"
Do you know how LLMs work? There's no 'logic', it's all statistical modeling. What DeepSeek et. al. are doing is taking inputs from specific benchmarks, throwing them into Claude, then training their (smaller, cheaper, faster) models to match the output, bypassing the need to scrape the broader internet (or pay) for training data. Which is (IMO) why Anthropic is actually salty; they paid for publicly available data (to avoid being sued) and someone outside the legal system bypassed that process.
I guess you could call that 'backing out' or 'reverse engineering', except none of these companies actually understands the inner workings of their models (they have billions of parameters and are just too complicated), just the processes used to create them. It's a black box full of linear algebra.
There’s got to be some pattern they can acquire though? Some way Claude works a problem through a gazillion matrices that makes it superior and worth stealing from?
Tl:dr; we've already reached the point that LLMs are cannibalising each other.
They are using Claude to create "training" data. Essentially they are asking Claude random questions and then training their model to output Claude's answer.
If I were Anthropic, and therefore morally bankrupt, whenever I detect these other companies querying Claude start outputting bullshit answers to make their models worse.
Yeah, well, that's all AI is is stealing other's work and chinks are great at that. Good luck.
Chinks.
Data wants to be free!