The Sirens' Song: When Proximal Background Context Overshadows Distant Evidence
The paper argues that nearby irrelevant context, not just long distance, can drown out the evidence a model needs.
Yang, Lu, Duan, and Yu call this failure mode the “Proximity Trap.” They propose LYRA, a t-distributed directional matching mechanism meant to shift more attention toward task-relevant evidence while preserving position information. The authors report gains on LongBench-v2, RULER, and LongBench, and introduce ProxBench to test distant-evidence use under growing nearby-background interference. ArXiv · AI/CL/LG's note
Yang, Lu, Duan, and Yu call this failure mode the “Proximity Trap.” They propose LYRA, a t-distributed directional matching mechanism meant to shift more attention toward task-relevant evidence while preserving position information. The authors report gains on LongBench-v2, RULER, and LongBench, and introduce ProxBench to test distant-evidence use under growing nearby-background interference. ArXiv · AI/CL/LG's note
score 5