REFRAMED: Towards Realistic Audio Description Generation for Movies
The paper turns movie audio description into a timing-and-content problem, not a captioning shortcut.
The authors introduce REFRAMED, a dataset covering 2,023 videos, 3,302 scenes, and 206 movies with professional AD transcripts, subtitles, and aligned screenplays. The benchmark asks models to decide both what visual information matters and where it fits into dialogue gaps. Current AD systems and multimodal LLMs beat trivial baselines, but remain far below expert human performance. ArXiv · AI/CL/LG's note
The authors introduce REFRAMED, a dataset covering 2,023 videos, 3,302 scenes, and 206 movies with professional AD transcripts, subtitles, and aligned screenplays. The benchmark asks models to decide both what visual information matters and where it fits into dialogue gaps. Current AD systems and multimodal LLMs beat trivial baselines, but remain far below expert human performance. ArXiv · AI/CL/LG's note
score 4