- OpenAI said this week that DeepSeek may have used OpenAI model outputs “inappropriately.”
- OpenAI has been accused of doing the same thing with copyrighted content.
- What’s that old saying about karma? Go ask ChatGPT. Or DeepSeek.
In the brave new world of generative AI, there’s a moment that everyone will experience. It’s the realization that your original work is being used to train AI models that could be competing against you.
This moment has arrived for OpenAI and CEO Sam Altman.
The US startup said this week that Chinese lab DeepSeek may have “inappropriately” used OpenAI outputs to train new AI models in a process called distillation.
Translation: We think you used our content without permission, and that’s not allowed.
For some AI experts, these complaints are hypocritical. OpenAI’s success is built on a similar process. The startup has for years collected outputs and data from the internet and used that to train its own models. This includes scooping up copyrighted content and other original work from thousands of companies that have not authorized this use.
In fact, this is what most model developers do, according to Nick Vincent, an assistant professor for computer science at Simon Fraser University, who studies how data is used in AI.
“These firms are simultaneously arguing for the right to train on anything they can get their hands on while denying their competitors the right to train on model outputs,” he wrote in a blog this week. “Rules for thee, but not for me?”
DeepSeek = just deserts
Vincent sees the rise of DeepSeek as the inevitable outcome of a training data free-for-all where AI companies take whatever content they want and ask for forgiveness later.
This has now backfired on OpenAI, which may be having its own outputs plundered in the name of AI progress. The startup “will struggle to defend itself in the court of public opinion on this,” Vincent told Business Insider on Wednesday. “There’s a reckoning coming.”
He hopes this reckoning will encourage tech companies to create a new system that gives appropriate credit and compensation to content creators.
“So far, none of the AI labs have seriously thought about this, so DeepSeek is their just deserts,” Vincent added.
Fair use just for OpenAI, or everyone?
High-quality training data is a crucial ingredient of powerful AI models. Many of the companies that created this information want to be paid for providing intelligence to these new products. Tech companies don’t want to be forced to pay. This dispute is being fought in court.
OpenAI is being sued by authors who claim the startup is breaking copyright law by using their books to train AI models. The New York Times is pursuing a similar complaint.
OpenAI has also been accused of using YouTube content to train its Sora video-generation model. YouTube CEO Neal Mohan said last year that if this is true it would violate the video platform’s rules.
OpenAI has denied breaking copyright laws, citing the “fair use” doctrine, which allows unlicensed use of copyrighted works in certain situations, including teaching, research, and news reporting.
So, would DeepSeek’s use of OpenAI’s outputs also constitute fair use?
“Very potentially, yes,” Vincent said.
Fair use isn’t just for yourself when it’s convenient. That would be, well, unfair.
I asked OpenAI about all this on Wednesday and it didn’t respond. The startup has partnerships with some companies that authorize the use of their content for AI model training. Axel Springer, the owner of Business Insider, struck one of these deals in 2023.
Distillation and karma
How do AI model outputs get scooped up for competitive means anyway?
Distillation is the technical term for extracting intelligence buried in one model and weaving it into a new one, according to Vincent. AI godfathers, including Geoffrey Hinton, wrote a research paper about this in 2015 called “Distilling the Knowledge in a Neural Network.”
Back then, the researchers described a tamer version of this, where a lab or company would take its own old models and use outputs from them to cleverly infuse a new offering with more intelligence.
Distilling intelligence from someone else’s AI model without permission is frowned upon in some research circles but happens a lot, according to Vincent.
DeepSeek’s research paper about its new R1 model described using distillation with open-source models, but it didn’t mention OpenAI.
“We demonstrate that the reasoning patterns of larger models can be distilled into smaller models, resulting in better performance,” the Chinese lab’s researchers wrote.
Since these new offerings began rolling out late last year, some AI researchers have theorized that DeepSeek used outputs from OpenAI’s new “reasoning” model, called o1, as synthetic data to improve its own models, such as R1.
In December, when DeepSeek was beginning to wow the AI field, Altman seemed to take a dig at his new rival.
“It is (relatively) easy to copy something that you know works. It is extremely hard to do something new, risky, and difficult when you don’t know if it will work,” he wrote on X.
What’s that phrase about karma? I can’t write it here. If you don’t know, go ask ChatGPT. Or DeepSeek.
