Ambient @ EgoLongQA 2026: Distilling Long-Video perception into a Sub-2B Model
A pruned 2B vision-language model won the sub-2B EgoLongQA division while retaining most of a much larger agent pipeline’s accuracy.
The entry answers multiple-choice questions about ten-minute egocentric videos in one greedy forward pass. It was trained by distilling only the junior perception module of a tool-using agent, using traces from cases the teacher answered correctly. The report says it reached 0.8279 on the held-out test set and 89% of the large agentic pipeline’s accuracy with 1.1% of its parameters. To meet the parameter cap, the author pruned the multilingual embedding table, bringing the model from 2.2132B to 1.9985B parameters. HF Daily Papers' note
The entry answers multiple-choice questions about ten-minute egocentric videos in one greedy forward pass. It was trained by distilling only the junior perception module of a tool-using agent, using traces from cases the teacher answered correctly. The report says it reached 0.8279 on the held-out test set and 89% of the large agentic pipeline’s accuracy with 1.1% of its parameters. To meet the parameter cap, the author pruned the multilingual embedding table, bringing the model from 2.2132B to 1.9985B parameters. HF Daily Papers' note
score 4