CAST: Game Solvers as Turn-Level Teachers for LLM Agents
CAST uses solver value changes to give LLM agents turn-by-turn credit during game training.
The paper says sparse final rewards in RLVR make it hard to know which moves mattered in long-horizon games. CAST turns changes in a game solver’s state value into scalar “solver advantages” and feeds them into training as process-level signals. The authors report gains over trained baselines on Sokoban, Minesweeper, and Rush Hour, including unseen-difficulty tests. They also claim the best average zero-shot performance on ALFWorld and WebShop. HF Daily Papers' note
The paper says sparse final rewards in RLVR make it hard to know which moves mattered in long-horizon games. CAST turns changes in a game solver’s state value into scalar “solver advantages” and feeds them into training as process-level signals. The authors report gains over trained baselines on Sokoban, Minesweeper, and Rush Hour, including unseen-difficulty tests. They also claim the best average zero-shot performance on ALFWorld and WebShop. HF Daily Papers' note
score 5