How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text
The paper finds that AI-generated web tokens can help only briefly, then start hurting models trained to predict human text.
The authors label 27.5% of June 2026 FineWeb-filtered web tokens as AI-generated, rising to 31.1% in August. They pretrain 800 models with different AI-to-human token ratios and find the gains from added AI text saturate, then reverse, especially when human data budgets are already high. Existing scaling laws miss that sign change, so the paper proposes one with separate benefit and harm terms. The authors recommend filtering AI text when the goal is human-text performance, and they release WildAI, an 83B-token labeled corpus, along with the models and code. HF Daily Papers' note
The authors label 27.5% of June 2026 FineWeb-filtered web tokens as AI-generated, rising to 31.1% in August. They pretrain 800 models with different AI-to-human token ratios and find the gains from added AI text saturate, then reverse, especially when human data budgets are already high. Existing scaling laws miss that sign change, so the paper proposes one with separate benefit and harm terms. The authors recommend filtering AI text when the goal is human-text performance, and they release WildAI, an 83B-token labeled corpus, along with the models and code. HF Daily Papers' note
score 6