TaoMate-FVSR:Audio-Guided Facial Video Restoration
The model uses speech audio to help reconstruct lips and teeth that match what a person is saying.
The framework targets poorly resolved faces in AI-generated videos, where sharpening alone can leave mouth details inconsistent with speech. It combines partially preserved source images with references for appearance and motion. Replacing the audio also lets it support visual dubbing. The authors report the best paired perceptual similarity and synchronization scores among the restoration methods compared on talking-head benchmarks.
HF Daily Papers' note
The framework targets poorly resolved faces in AI-generated videos, where sharpening alone can leave mouth details inconsistent with speech. It combines partially preserved source images with references for appearance and motion. Replacing the audio also lets it support visual dubbing. The authors report the best paired perceptual similarity and synchronization scores among the restoration methods compared on talking-head benchmarks.
HF Daily Papers' note
score 4