Language Models Can Control Their Own Attention
The paper proposes letting models declare which part of context they need, so inference can skip much of the KV-cache scan.
Declarative Attention has the model emit `<global>`, `<focus>`, or `<local>` modes during chain-of-thought, which the inference engine parses like tool calls. On 15 long-context tasks, off-the-shelf Gemma-4-31B and Qwen-3.6-27B cut attended tokens during decoding by 52.0% and 31.1%. Accuracy fell modestly, by 1.27 and 2.75 percentage points, with smaller drops at larger scale. The authors frame it as an intrinsic sparse-attention route that could improve further with training. ArXiv · AI/CL/LG's note
Declarative Attention has the model emit `<global>`, `<focus>`, or `<local>` modes during chain-of-thought, which the inference engine parses like tool calls. On 15 long-context tasks, off-the-shelf Gemma-4-31B and Qwen-3.6-27B cut attended tokens during decoding by 52.0% and 31.1%. Accuracy fell modestly, by 1.27 and 2.75 percentage points, with smaller drops at larger scale. The authors frame it as an intrinsic sparse-attention route that could improve further with training. ArXiv · AI/CL/LG's note
score 6