CIViC: Evidence Gates for Audio-Conditioned Visual Prediction on Proxy Tasks
CIViC tests whether audio is actually doing the claimed work in visual prediction models.
The paper proposes a four-gate protocol tying audio-based interpretations to interventions, held-out comparisons, or endpoint controls. Across its proxy tasks, the checks do not give one clean story: audio replacement has only a small detected effect in the face-raster test, while constant audio beats audio-trained fits on a temporal endpoint. In SpeakerVid, zeroing five low-level audio summaries improves whole-frame grayscale RMSE, but aligned audio does better on segment-drift error. HF Daily Papers' note
The paper proposes a four-gate protocol tying audio-based interpretations to interventions, held-out comparisons, or endpoint controls. Across its proxy tasks, the checks do not give one clean story: audio replacement has only a small detected effect in the face-raster test, while constant audio beats audio-trained fits on a temporal endpoint. In SpeakerVid, zeroing five low-level audio summaries improves whole-frame grayscale RMSE, but aligned audio does better on segment-drift error. HF Daily Papers' note
score 4