Rollback: Experience Backtracking in World Models for Policy Improvement
Rollback beat the strongest baseline by 10.67–11.85 percentage points across three real-world robot tasks.
The system rewinds a learned world model to critical decisions, tries alternative actions and uses their outcomes to supervise the robot’s policy without a separately learned value model. Real-robot experience also updates the world model through a lightweight LoRA pipeline. The authors report gains on block extraction, ball tossing and tabletop curling, with more stable improvement during training. On tossing and curling, Rollback exceeded human players’ average success rates within eight hours of training.
HF Daily Papers' note
The system rewinds a learned world model to critical decisions, tries alternative actions and uses their outcomes to supervise the robot’s policy without a separately learned value model. Real-robot experience also updates the world model through a lightweight LoRA pipeline. The authors report gains on block extraction, ball tossing and tabletop curling, with more stable improvement during training. On tossing and curling, Rollback exceeded human players’ average success rates within eight hours of training.
HF Daily Papers' note
score 5