KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generation
The benchmark tests whether video models can obey supplied keyframes without breaking the video.
KeyFrame-Compass includes 386 curated samples across domains, video structures, prompt detail levels, conditioning formats, and keyframe densities. Its evaluation separates keyframe execution into presence, fidelity, ordering, localization, persistence, and uniqueness, then pairs that with video-quality judgments. In tests on nine video generation systems, the paper says models trade off faithful keyframe following against natural synthesis. Performance drops as keyframe constraints get denser, and most open-source models fail to read storyboard grids as ordered keyframe sequences. HF Daily Papers' note
KeyFrame-Compass includes 386 curated samples across domains, video structures, prompt detail levels, conditioning formats, and keyframe densities. Its evaluation separates keyframe execution into presence, fidelity, ordering, localization, persistence, and uniqueness, then pairs that with video-quality judgments. In tests on nine video generation systems, the paper says models trade off faithful keyframe following against natural synthesis. Performance drops as keyframe constraints get denser, and most open-source models fail to read storyboard grids as ordered keyframe sequences. HF Daily Papers' note
score 4