Learning to Act with World Models by Valuing What Comes Next
WM-PPO raised success to 87.5% on ALFWorld and 52.9% on ScienceWorld, using comparable real environment steps to PPO.
The method pairs a world model’s predictions of what actions would change with a critic’s estimates of success. It uses those estimates to train both the policy and critic, weighting the extra guidance by how reliably earlier comparisons predicted real outcomes. PPO’s baseline success rates were 63.5% and 39.3%, respectively. All three components learn from the agent’s own interactions without a teacher or privileged information; only the policy runs at inference.
HF Daily Papers' note
The method pairs a world model’s predictions of what actions would change with a critic’s estimates of success. It uses those estimates to train both the policy and critic, weighting the extra guidance by how reliably earlier comparisons predicted real outcomes. PPO’s baseline success rates were 63.5% and 39.3%, respectively. All three components learn from the agent’s own interactions without a teacher or privileged information; only the policy runs at inference.
HF Daily Papers' note
score 4