Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation
Explorative Models train by trying multiple candidate matches and learning from the best one.
The paper argues this lets predictions commit to modes instead of averaging across them. The authors report that scaling exploration improves results across images, video, and language, adding a pretraining axis beyond data and parameters. They claim exploration improves FLOP efficiency by 4.1x and sample efficiency by 6.2x, and brings an image-generation setup to 1.43 FID on ImageNet without guidance. They also say the approach enables end-to-end reconstructive generation that matches diffusion on control tasks with 16-256x fewer inference steps. HF Daily Papers' note
The paper argues this lets predictions commit to modes instead of averaging across them. The authors report that scaling exploration improves results across images, video, and language, adding a pretraining axis beyond data and parameters. They claim exploration improves FLOP efficiency by 4.1x and sample efficiency by 6.2x, and brings an image-generation setup to 1.43 FID on ImageNet without guidance. They also say the approach enables end-to-end reconstructive generation that matches diffusion on control tasks with 16-256x fewer inference steps. HF Daily Papers' note
score 6