SiPhy: Single-Image Physical Property Reasoning
SiPhy estimates mass, density, stiffness, and elasticity from a single RGB image.
The paper frames the system as a bridge between 3D-aware visual cues, depth, and language-based material knowledge. It samples pseudo-voxel points, uses CLIP features, and grounds regions to material candidates proposed by a vision-language model. On ABO-500, MVImgNet-100, and PhysXNet-100, the authors report state-of-the-art single-image results, including large gains over several multi-view baselines. They also test it on real hand-object interaction datasets as a possible annotation engine for physical understanding. ArXiv · AI/CL/LG's note
The paper frames the system as a bridge between 3D-aware visual cues, depth, and language-based material knowledge. It samples pseudo-voxel points, uses CLIP features, and grounds regions to material candidates proposed by a vision-language model. On ABO-500, MVImgNet-100, and PhysXNet-100, the authors report state-of-the-art single-image results, including large gains over several multi-view baselines. They also test it on real hand-object interaction datasets as a possible annotation engine for physical understanding. ArXiv · AI/CL/LG's note
score 5