LIBERO-RIG: A Hardware-Setup Robustness Benchmark for Vision-Language-Action Models
Nine VLA models lost substantial success under controlled hardware/setup perturbations, even though the tasks stayed fixed.
LIBERO-RIG varies only robot hardware and setup across 11 axes, with five severity levels and 210 conditions over the four LIBERO suites. The paper reports pooled success dropping from 0.932 to 0.767 at middle severity and 0.509 at the highest severity. Degradation differed by perturbation axis and action interface. Visual perturbation training did not help with faults that do not change the camera image. Source: ArXiv · AI/CL/LG's note.
LIBERO-RIG varies only robot hardware and setup across 11 axes, with five severity levels and 210 conditions over the four LIBERO suites. The paper reports pooled success dropping from 0.932 to 0.767 at middle severity and 0.509 at the highest severity. Degradation differed by perturbation axis and action interface. Visual perturbation training did not help with faults that do not change the camera image. Source: ArXiv · AI/CL/LG's note.
score 5