HarmoHOI: Harmonizing Appearance and 3D Motion for Multi-view Hand-Object Interaction Synthesis
The paper’s bet is that believable multi-view hand-object video needs 3D motion generated alongside appearance.
HarmoHOI is a diffusion framework that produces synchronized multi-view interaction videos and globally aligned 3D point tracks together. The authors represent point tracks as pseudo-videos so 3D geometry can be modeled in the same latent space as video foundation models. A separate motion-aligning diffusion step refines coarse tracks into metric-scale 3D trajectories. They report state-of-the-art results on visual quality, motion plausibility, and multi-view geometric consistency. HF Daily Papers' note
HarmoHOI is a diffusion framework that produces synchronized multi-view interaction videos and globally aligned 3D point tracks together. The authors represent point tracks as pseudo-videos so 3D geometry can be modeled in the same latent space as video foundation models. A separate motion-aligning diffusion step refines coarse tracks into metric-scale 3D trajectories. They report state-of-the-art results on visual quality, motion plausibility, and multi-view geometric consistency. HF Daily Papers' note
score 5