SP$^2$ec: Adaptive Self-Speculative Decoding for Vision-Language Models
SP²ec adjusts the self-speculative draft setup during generation to cut VLM inference latency.
The paper presents a training-free bandit algorithm for vision-language models where fixed draft choices can miss the best throughput. It uses a “single-peak” throughput pattern seen across several self-speculative methods to search cheaply for the best configuration per query. The authors report up to 11% improvement over the original methods on Qwen2.5-VL-7B-Instruct and LLaVA-NeXT-Mistral-7B across visual and text tasks. Source: ArXiv · AI/CL/LG's note.
The paper presents a training-free bandit algorithm for vision-language models where fixed draft choices can miss the best throughput. It uses a “single-peak” throughput pattern seen across several self-speculative methods to search cheaply for the best configuration per query. The authors report up to 11% improvement over the original methods on Qwen2.5-VL-7B-Instruct and LLaVA-NeXT-Mistral-7B across visual and text tasks. Source: ArXiv · AI/CL/LG's note.
score 5