CURIO: Curiosity-Driven Test-Time Learning for Open-Ended Discovery
CURIO adds a curiosity reward at test time so the model keeps exploring low-probability directions instead of only reinforcing early winners.
The framework pairs task feedback with an Intrinsic Curiosity World Model that rewards prediction errors on sampled tokens outside the policy’s top choices. Tested with Qwen3 models from 8B to 235B, it beat a task-only RL control on five of six math discovery objectives. It also matched the best reported Circle Packing result and improved single-cell denoising metrics at every tested scale. The authors report more structural variety in generated programs, arguing that curiosity helped preserve useful exploration. HF Daily Papers' note
The framework pairs task feedback with an Intrinsic Curiosity World Model that rewards prediction errors on sampled tokens outside the policy’s top choices. Tested with Qwen3 models from 8B to 235B, it beat a task-only RL control on five of six math discovery objectives. It also matched the best reported Circle Packing result and improved single-cell denoising metrics at every tested scale. The authors report more structural variety in generated programs, arguing that curiosity helped preserve useful exploration. HF Daily Papers' note
score 5