Maybe I'm wrong about Claude/Anthropic, but my understanding is most of these models are trained on data publicly available on the internet. Sure they paid Microsoft for access to your git repos/Reddit for your posts and comments, but they don't own that data*. Copyright on AI generated content is still in a pretty grey area legally, so I don't see what they can do about people scraping their output except for terminating accounts for TOS violations, especially since I think everyone they're calling out is based in China.
Ultimately, if this is actually a threat to their business model (I know, I know, their business model is fraudulent and incestuous, not based at all on paying customers) they should release their own distillations. If you're going to base your models on publicly available data, and have a public API for accessing it, its outputs are effectively in the public domain, whether you like it or not.
*They license it from you with pretty broad terms on what they can do with it, but it's still your data.
Are they “stealing” training data or are these other companies inputting specialized problems to Claude with answers and solutions that can be used to back out Claude’s internal logic/processes? It sounds more like the latter to me.
Do you know how LLMs work? There's no 'logic', it's all statistical modeling. What DeepSeek et. al. are doing is taking inputs from specific benchmarks, throwing them into Claude, then training their (smaller, cheaper, faster) models to match the output, bypassing the need to scrape the broader internet (or pay) for training data. Which is (IMO) why Anthropic is actually salty; they paid for publicly available data (to avoid being sued) and someone outside the legal system bypassed that process.
I guess you could call that 'backing out' or 'reverse engineering', except none of these companies actually understands the inner workings of their models (they have billions of parameters and are just too complicated), just the processes used to create them. It's a black box full of linear algebra.
There’s got to be some pattern they can acquire though? Some way Claude works a problem through a gazillion matrices that makes it superior and worth stealing from?
Maybe I'm wrong about Claude/Anthropic, but my understanding is most of these models are trained on data publicly available on the internet. Sure they paid Microsoft for access to your git repos/Reddit for your posts and comments, but they don't own that data*. Copyright on AI generated content is still in a pretty grey area legally, so I don't see what they can do about people scraping their output except for terminating accounts for TOS violations, especially since I think everyone they're calling out is based in China.
Ultimately, if this is actually a threat to their business model (I know, I know, their business model is fraudulent and incestuous, not based at all on paying customers) they should release their own distillations. If you're going to base your models on publicly available data, and have a public API for accessing it, its outputs are effectively in the public domain, whether you like it or not.
*They license it from you with pretty broad terms on what they can do with it, but it's still your data.
Are they “stealing” training data or are these other companies inputting specialized problems to Claude with answers and solutions that can be used to back out Claude’s internal logic/processes? It sounds more like the latter to me.
Do you know how LLMs work? There's no 'logic', it's all statistical modeling. What DeepSeek et. al. are doing is taking inputs from specific benchmarks, throwing them into Claude, then training their (smaller, cheaper, faster) models to match the output, bypassing the need to scrape the broader internet (or pay) for training data. Which is (IMO) why Anthropic is actually salty; they paid for publicly available data (to avoid being sued) and someone outside the legal system bypassed that process.
I guess you could call that 'backing out' or 'reverse engineering', except none of these companies actually understands the inner workings of their models (they have billions of parameters and are just too complicated), just the processes used to create them. It's a black box full of linear algebra.
There’s got to be some pattern they can acquire though? Some way Claude works a problem through a gazillion matrices that makes it superior and worth stealing from?
Tl:dr; we've already reached the point that LLMs are cannibalising each other.