DepthWorld: 3D World Model for Robot Manipulation
Depth supervision made the robot world model better at predicting RGB video, not just depth.
The paper introduces a calibration pipeline that turns DROID into DROID-3D, with dense metric depth and recalibrated multi-view camera extrinsics. DepthWorld then trains on that data to jointly predict RGB and depth from multiple views while keeping the pretrained VAE unchanged. The authors report a +1.48 dB PSNR gain over an RGB-only baseline at the same training budget, alongside metric depth useful for geometric reasoning. Accepted at CoRL 2026. ArXiv · AI/CL/LG's note
The paper introduces a calibration pipeline that turns DROID into DROID-3D, with dense metric depth and recalibrated multi-view camera extrinsics. DepthWorld then trains on that data to jointly predict RGB and depth from multiple views while keeping the pretrained VAE unchanged. The authors report a +1.48 dB PSNR gain over an RGB-only baseline at the same training budget, alongside metric depth useful for geometric reasoning. Accepted at CoRL 2026. ArXiv · AI/CL/LG's note
score 6