AI giants learn what everyone else on the modern internet already knows

By Alistair Barr

You're currently following this author! Want to unfollow? Unsubscribe via the link in your email.

Author of the Tech Memo newsletter

Here's some delicious AI irony for you.

For years, tech giants have argued that if information is available on the internet, it can be used for AI model development and outputs. They call it fair use. Content owners have tried to prevent this, with no success.

Now Anthropic, OpenAI, and Google are discovering what the rest of the internet has already learned through painful experience: once you put something online, people will find ways to use it in ways you don't like and can't stop.

The latest flashpoint is something called "distillation," using the outputs of one AI model to improve another. Anthropic says competitors are harvesting its outputs at scale, turning billions of dollars of research into a shortcut for rivals. OpenAI and Google have made similar warnings recently.

The fear is obvious. Why spend billions building the best AI models if someone else can recreate much of this intelligence for a fraction of the cost?

That's a legitimate business concern. But here's the awkward part.

Symmetry

From 30,000 feet, distillation looks an awful lot like what AI companies have been doing to the rest of the internet. Scrape web content for free and without permission. Turn it into a product you sell. Argue it's fair use. Hope the lawyers sort out the details later.

Anthropic says rivals are extracting intelligence from its top models. Website owners have spent the past three years saying Anthropic extracted intelligence from them. Both sides argue this is against their terms of service. The symmetry is hard to ignore.

And despite pitching itself as the most ethical AI company, Anthropic is by far the worst actor here. Its data-sucking bots crawl webpages thousands of times for every one referral the company sends back to the web.

Bots on both sides

Anthropic, OpenAI, and especially Google, frame this as a cybersecurity issue, pointing to swarms of bots "attacking" their models to extract intelligence. But, they've been doing the same to many websites, bombarding them with so much bot crawling activity that site owners have seen their operating costs skyrocket. Not only are some websites having their content used without permission, they are paying more for the privilege.

AI researchers say distillation is different from web scraping. But the AI industry can't even decide whether distillation is OK or not, or where to draw the line.

There's the original, benign form of distillation, where labs use outputs from their own models to create different, often smaller, models. Then, there's what Anthropic calls "distillation attacks," where rivals use other people's AI outputs to develop or improve their own offerings.

Even here, though, the lines blur, with some AI researchers now worrying that Anthropic's aggressive stance will hurt all types of distillation. Open-source AI expert Nathan Lambert calls this "distillation panic."

So, let me wrap this up for you, from the AI giants' perspective: They can extract intelligence from the web for free and without permission. That's different from distillation, which is OK. Oh, but not when distillation involves using their content in ways they don't like.

"A cat-and-mouse game"

This contorted argument is being demolished by the brutal realities of the modern internet. Anthropic has spent months tightening access to its top models to stop competitors from learning too much. Those efforts have either backfired, or they're just spurring more elaborate workarounds.

Once information goes online, clever people will figure out how to collect it, remix it, and profit from it. That's true for blogs, photos, software code, videos, and yes, AI giants' precious model outputs.

"It's always a kind of a cat-and-mouse game," Zilan Qian, a researcher at the Oxford China Policy Lab, told Business Insider. As long as AI model outputs are out in the world, "people will probably find a way to get access to it."

Indeed, distilling another company's AI model may even be fair use. These legal arguments can cut both ways.

Welcome to the new internet, Anthropic, OpenAI, and Google. Get used to it.

Sign up for BI's Tech Memo newsletter here. Reach out to me via email at [email protected].

Read next

Alistair Barr

You're currently following this author! Want to unfollow? Unsubscribe via the link in your email.

Alistair Barr is the author of Business Insider's Tech Memo newsletter. Sign up here. Before that, he was BI's Global Tech Editor and the Big Tech team leader at Bloomberg, following a reporting career at The Wall Street Journal, USA Today, Reuters, and MarketWatch. Alistair won a Gerald Loeb Award in 2007 for coverage of short selling and was a finalist in 2013 for scoops on the Facebook IPO. More recently, he won a 2024 San Francisco Press Club award for commentary. Got a tip? Reach out using the secure messaging app Signal (+1 415-341-4927) or via email on [email protected] oversees all things Big Tech, along with startups and venture capital. He writes analysis and columns about topics including generative AI, large language models, cloud computing, semiconductors, online search, e-commerce, EVs, robotics, and autonomous vehicles.Popular StoriesArtificial Intelligence:It's getting harder to make big leaps at the frontier of AIOpenAI's AI-adjusted earnings numbers have echoes of Groupon and WeWorkDeath by LLM: Stack Overflow's decline, and its plan to survive, shows the future of free online data in an AI worldCloud computing:Amazon dominated the first cloud era. The AI boom has kicked off Cloud 2.0, and the company doesn't have a head start this time.In cloud, there's AI (which is hot) and everything else (which is not)Chips:Why Intel is still so important: Real countries have fabsApple's made-in-the-USA chips signal a turnaround for the US's big semiconductor betEVs and Tesla:Tesla's AI supercomputer has a Silicon Valley town rushing to meet surging electricity demandTesla's Cybertruck is outselling almost every other EV in the USOnline Search:Google is losing its status as a verbA simple way to fix search: Bright pink ads