Model collapse is what happens when AI systems train on content produced by other AI systems, and the results degrade with each generation. Researchers gave a language model a prompt about medieval English architecture. By the ninth generation of training on its own output, the model was writing about jackrabbits with different-colored tails.[s] That is not a metaphor. That is a peer-reviewed finding.
How Model Collapse Works
Every generative AI model, whether it produces text, images, or audio, learns by finding patterns in training data. When that training data comes from the real world, the model captures the full range of human expression: common patterns, rare edge cases, unusual perspectives. But when the next generation of models trains on the output of the previous generation instead, something breaks. The rare patterns start disappearing first.[s]
Think of it like photocopying a photocopy. The first copy looks almost identical to the original. The second copy of the copy is slightly fuzzier. By the tenth copy, fine details are gone, contrast is flattened, and what remains is a blurry approximation of what you started with. AI model collapse follows the same trajectory, but with data distributions instead of ink.
Researchers at the University of Oxford and other British and Canadian institutions published the landmark study on model collapse in Nature in 2024.[s] Led by Ilia Shumailov, the team demonstrated that indiscriminate use of AI-generated content in training “causes irreversible defects in the resulting models, in which tails of the original content distribution disappear.” They identified two distinct phases: early model collapse, where rare events and minority viewpoints vanish from the data, and late model collapse, where the output converges to something that bears little resemblance to reality.
Why Model Collapse Matters Now
The internet is rapidly filling with AI-generated content. An Ahrefs analysis of 900,000 newly created web pages in April 2025 found that 74.2% contained AI-generated content.[s] A now-removed statement in an earlier Europol deepfakes report was widely reported as estimating that as much as 90% of online content could be synthetic by 2026, but Europol later said the statement came from an inaccurate source and removed it from the current version.[s][s]
This matters because most large language models are trained on data scraped from the web. If the web is increasingly composed of AI output, future models will inevitably train on the output of their predecessors. The feedback loop tightens with every generation. Nature described model collapse as a “cannibalistic phenomenon” that “could halt the improvement of large language models.”[s]
The problem extends beyond text. Bohacek and Farid showed that image-generating models retrained on even small amounts of their own output produce “highly distorted images,” and that the damage persists even after retraining exclusively on real images.[s] Once model collapse takes hold, it is difficult to reverse.
What Gets Lost
The most insidious aspect of model collapse is what disappears first: the tails of the distribution. In practical terms, that means rare diseases in medical data, minority dialects in language data, unusual artistic styles in image data, and niche perspectives in text data. The models do not become obviously broken overnight. They become subtly averaged, producing outputs that look plausible but have quietly shed the diversity of human experience.
This has consequences beyond AI performance. If widely used systems gradually lose their ability to represent uncommon patterns, the knowledge those patterns encode risks fading from public access entirely.[s]
Is There a Way Out?
Researchers at Stanford and elsewhere have challenged the most catastrophic predictions. A 2024 study led by Matthias Gerstgrasser showed that model collapse can be avoided if synthetic data accumulates alongside the original real data, rather than replacing it.[s] In other words, as long as each new generation of training data includes the original human-generated material plus all previous synthetic generations, the error stays bounded.
The catch: this requires knowing which data is human-generated and which is synthetic. That is increasingly difficult. The Data Provenance Initiative, a collective of researchers from MIT and other institutions, audited over 1,800 text datasets and found license omission rates above 70% and error rates above 50% on popular hosting sites.[s] If we cannot reliably track where training data comes from, the accumulation strategy falls apart in practice.
The emerging consensus is that the value of verified human-generated data will only increase. Pre-AI datasets, content produced before large-scale generative models existed, may become some of the most valuable assets in machine learning. Model collapse is a solvable problem in theory; whether the AI industry solves it in practice depends on treating data provenance as infrastructure, not an afterthought.
Model collapse is a degenerative process in which successive generations of learned generative models, each trained on the output of their predecessors, progressively lose information about the true data distribution until the output converges to a low-variance approximation bearing little resemblance to the original.[s] First formally described by Shumailov et al. in a 2023 preprint[s] and published in Nature in 2024, the phenomenon has implications for every organization scraping the web for training data.
Model Collapse: The Mathematical Foundation
The Shumailov et al. framework identifies three compounding error sources that drive model collapse. First, statistical approximation error: finite sampling means low-probability events can be lost at each resampling step, with nonzero probability per generation. Second, functional expressivity error: neural networks are universal approximators only as their size approaches infinity; in practice, they introduce nonzero likelihood outside the support of the original distribution or zero likelihood inside it. Third, functional approximation error: limitations of learning procedures (structural bias of SGD, choice of objective function) introduce further deviation even with infinite data and perfect expressivity.[s]
The authors prove this rigorously for the Gaussian case. Given original data with nonzero sample variance, if successive generations fit unbiased sample mean and variance estimators recursively, the Wasserstein-2 distance between the nth-generation approximation and the true distribution diverges to infinity, while the variance collapses to zero almost surely as n approaches infinity.[s] For discrete distributions, the process reduces to a Markov chain whose only absorbing states are delta functions, meaning the distribution must converge to a single point with probability 1.
In the OPT-125M experiment, the team fine-tuned Meta’s open-source LLM on its own output across nine generations. Starting from a prompt about medieval English architecture, the ninth-generation model produced text about jackrabbits with colored tails.[s] The trajectory is not random drift; it follows the mathematical prediction: tails vanish first (early model collapse), then the entire distribution collapses to a narrow mode (late model collapse).
The Scale of the Contamination Problem
An Ahrefs analysis of 900,000 newly indexed web pages in April 2025 found 74.2% contained AI-generated content, with only 25.8% classified as “pure human.”[s] A widely reported 90% synthetic-content forecast came from a statement that Europol has since removed from its deepfakes report because the source was inaccurate.[s][s] For any training pipeline that ingests Common Crawl or similar web-scale corpora without filtering, model collapse is no longer a theoretical risk; it is a data quality crisis already in progress.
The contamination extends to image models. Bohacek and Farid demonstrated that generative image models retrained on even small amounts of their own output produce “highly distorted images,” and critically, the distortion “extends beyond the text prompts used in retraining.”[s] The damage is not prompt-specific; it corrupts the model’s internal representations globally. Their finding that “once affected, the models struggle to fully heal even after retraining on only real images” suggests model collapse may be partially irreversible in practice.
The Accumulation Counterargument
Gerstgrasser et al. (2024) challenged the inevitability of model collapse by distinguishing between data replacement and data accumulation.[s] The original Shumailov framework assumed each generation’s training data replaced the previous generation entirely (mixing parameter βi = γi = 0, αi = 1). The accumulation model instead retains original real data alongside all synthetic generations. In the linear regression case, Gerstgrasser et al. proved that accumulated data yields a test error with a finite upper bound independent of the number of iterations, meaning model collapse no longer occurs.
Empirically, their results held across multiple model sizes, architectures, and hyperparameters, extending to diffusion models for molecule conformation and VAEs for image generation. This is a meaningful result, but it comes with a critical dependency: the original human-generated data must be identifiable and preserved.
The Provenance Bottleneck
Reliable data provenance is the prerequisite for the accumulation strategy, and the current ecosystem fails at it. The Data Provenance Initiative audited over 1,800 text datasets and found license omission rates above 70% and error rates above 50% on popular dataset hosting platforms.[s] If practitioners cannot determine whether a dataset is human-generated, AI-generated, or a mixture, they cannot implement the accumulation approach that prevents model collapse.
The practical implication: pre-AI training corpora, datasets collected before the proliferation of large-scale generative models, are becoming uniquely valuable. The Shumailov et al. paper itself noted that “the value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of LLM-generated content in data crawled from the Internet.”[s]
Model collapse is not an unsolvable problem. The mathematics show that preserving access to original distributions bounds the error. But the engineering challenge of maintaining data provenance at web scale, while the web itself becomes increasingly synthetic, defines the central tension in AI training for the foreseeable future.
