OmniDelta: Skill-Driven Budget Allocation for Token Compression in OmniLLMs
OmniDelta shifts a fixed token budget toward the audio or video evidence a query is likely to need.
The paper says fixed or uniform compression budgets can waste retained tokens and miss key audio-video evidence. OmniDelta is training-free and reallocates budgets first across modalities, then across audio segments and video frames. It can sit on top of existing pruning methods while keeping the same total retained-token ratio. On four audio-video benchmarks with two Qwen2.5-Omni models, the authors report a better accuracy-efficiency tradeoff; at 25% token retention on Qwen2.5-Omni-7B, they report 22.0% lower GPU memory use and 1.64x end-to-end speedup versus full-token inference. HF Daily Papers' note
The paper says fixed or uniform compression budgets can waste retained tokens and miss key audio-video evidence. OmniDelta is training-free and reallocates budgets first across modalities, then across audio segments and video frames. It can sit on top of existing pruning methods while keeping the same total retained-token ratio. On four audio-video benchmarks with two Qwen2.5-Omni models, the authors report a better accuracy-efficiency tradeoff; at 25% token retention on Qwen2.5-Omni-7B, they report 22.0% lower GPU memory use and 1.64x end-to-end speedup versus full-token inference. HF Daily Papers' note
score 4