Preventing Model Collapse: A Fisher-Rao Perspective on the Dynamics of Training with Synthetic Data
The paper argues that prior minimum human-data bounds can become unusable in high-dimensional settings.
The authors recast synthetic-data collapse through the Fisher-Rao geometry of categorical probability distributions. They derive contraction and invariance bounds meant to stay meaningful as dimension grows. The result is a different estimate of how much fresh human data is needed to stabilize recursive training. ArXiv · AI/CL/LG's note
The authors recast synthetic-data collapse through the Fisher-Rao geometry of categorical probability distributions. They derive contraction and invariance bounds meant to stay meaningful as dimension grows. The result is a different estimate of how much fresh human data is needed to stabilize recursive training. ArXiv · AI/CL/LG's note
score 5