RoboCasa-Shift: Stage-Aware Robustness Evaluation of Vision-Language-Action Policies in Long-Horizon Household Manipulation
An unchanged success rate hid outcome reversals in 8 of 80 matched robot-task episodes.
RoboCasa-Shift compares clean and perturbed runs under otherwise identical settings, tracking where execution fails. Researchers tested GR00T N1.5 across four household tasks, four perturbation factors and three severity levels in 1,040 formal rollouts. Stronger perturbations did not always cause greater degradation. A camera shift and Gaussian blur produced the same overall success-rate drop, but first-stage failure rates differed: 19.0% versus 36.4%.
HF Daily Papers' note
RoboCasa-Shift compares clean and perturbed runs under otherwise identical settings, tracking where execution fails. Researchers tested GR00T N1.5 across four household tasks, four perturbation factors and three severity levels in 1,040 formal rollouts. Stronger perturbations did not always cause greater degradation. A camera shift and Gaussian blur produced the same overall success-rate drop, but first-stage failure rates differed: 19.0% versus 36.4%.
HF Daily Papers' note
score 4