Mitigating Model Collapse through Dreaming Learning
Training on higher-temperature samples helped models retain diversity across repeated generations of synthetic data.
In a controlled Markov setting, the researchers found that sampling below temperature 1 pushed models toward deterministic behavior; accumulating data did not appear to stop the drift. Dreaming Learning adds a regularization loss using the model’s own samples drawn above temperature 1, helping preserve probabilities that would otherwise shrink toward zero. Spectral measurements over 5,000 generations supported that mechanism. The mitigation also persisted in tests with LSTM language models and a 124-million-parameter GPT-2.
ArXiv · AI/CL/LG's note
In a controlled Markov setting, the researchers found that sampling below temperature 1 pushed models toward deterministic behavior; accumulating data did not appear to stop the drift. Dreaming Learning adds a regularization loss using the model’s own samples drawn above temperature 1, helping preserve probabilities that would otherwise shrink toward zero. Spectral measurements over 5,000 generations supported that mechanism. The mitigation also persisted in tests with LSTM language models and a 124-million-parameter GPT-2.
ArXiv · AI/CL/LG's note
score 4