Luce: Relightable Gaussians for 3D Asset Generation
Luce turns one image into a relightable 3D asset with PBR materials, not just fixed appearance.
The paper represents geometry and material channels inside a voxelized multimodal Gaussian cloud, with separate Gaussian primitives for albedo, metallic-roughness, and normals. A VAE compresses that into a material-aware latent space, then a rectified-flow transformer generates it from a single input image. The decoded output can be relightable PBR Gaussians or an optional textured mesh with a tangent-space normal map. On Toys4K, the authors report a 28% FID gain over the strongest baseline, plus better CLIP image alignment on their AI-generated-image benchmark. HF Daily Papers' note
The paper represents geometry and material channels inside a voxelized multimodal Gaussian cloud, with separate Gaussian primitives for albedo, metallic-roughness, and normals. A VAE compresses that into a material-aware latent space, then a rectified-flow transformer generates it from a single input image. The decoded output can be relightable PBR Gaussians or an optional textured mesh with a tangent-space normal map. On Toys4K, the authors report a 28% FID gain over the strongest baseline, plus better CLIP image alignment on their AI-generated-image benchmark. HF Daily Papers' note
score 5