Language Models Can Control Their Own Attention
Declarative Attention lets a model mark which context region it needs before the inference engine reads the cache.
The paper proposes three generation modes: full-context, focused-region, and local-only attention. The engine treats those declarations like tool calls and skips much of the KV cache read during decoding. In zero-shot tests across 15 long-context tasks, the method cut attended tokens by 52.0% on Gemma-4-31B and 31.1% on Qwen-3.6-27B, with accuracy drops of 1.27 and 2.75 percentage points. The authors frame it as an intrinsic sparse-attention route that could improve with training. HF Daily Papers' note
The paper proposes three generation modes: full-context, focused-region, and local-only attention. The engine treats those declarations like tool calls and skips much of the KV cache read during decoding. In zero-shot tests across 15 long-context tasks, the method cut attended tokens by 52.0% on Gemma-4-31B and 31.1% on Qwen-3.6-27B, with accuracy drops of 1.27 and 2.75 percentage points. The authors frame it as an intrinsic sparse-attention route that could improve with training. HF Daily Papers' note
score 6