Data World: Scaling Data Environments from Scratch
Training on synthetic data worlds lifted a 35B model’s scores across all four reported benchmarks.
Data World defines the mechanisms behind a dataset before generating its files, tasks and scoring rubrics. Independent audits check whether the target findings can be recovered from those files. The authors built 2,135 training tasks and a separate 100-task benchmark, where ten frontier models struggled particularly with discovering insights. Reinforcement learning without trajectory distillation improved the base model by 7.06 points on DataWorld-Bench and 12.13–20.90 points on three other benchmarks.
HF Daily Papers' note
Data World defines the mechanisms behind a dataset before generating its files, tasks and scoring rubrics. Independent audits check whether the target findings can be recovered from those files. The authors built 2,135 training tasks and a separate 100-task benchmark, where ten frontier models struggled particularly with discovering insights. Reinforcement learning without trajectory distillation improved the base model by 7.06 points on DataWorld-Bench and 12.13–20.90 points on three other benchmarks.
HF Daily Papers' note
score 5