Continuation-Guided LLM Evolution of Program Policies and Goal Compilers in Craftax
The optimized Craftax policy reached the final stage in 50% of runs and defeated the final boss in 28%, with no inference-time LLM calls.
Refinement using episode-level feedback alone stalled below 1% final-stage arrival. The framework added denser simulator feedback on hierarchical goals and belief updates to guide policy improvements. It then separated the controller’s environment knowledge, memory and skills from its rules, allowing a reusable compiler to generate rules for new goals. Across five independent searches trained only on short tasks, the compilers transferred to longer tasks without task-specific tuning, according to the preprint.
HF Daily Papers' note
Refinement using episode-level feedback alone stalled below 1% final-stage arrival. The framework added denser simulator feedback on hierarchical goals and belief updates to guide policy improvements. It then separated the controller’s environment knowledge, memory and skills from its rules, allowing a reusable compiler to generate rules for new goals. Across five independent searches trained only on short tasks, the compilers transferred to longer tasks without task-specific tuning, according to the preprint.
HF Daily Papers' note
score 5