CRISP: Cliff-awaRe Input-adaptive Sparse Prefilling with Structural-Mass-Motivated Routing
CRISP targets long-context prefilling by routing sparse attention from the proxy map itself, then cutting noise from token selection.
The paper says its `C_struct` proxy reproduces JSD-style routing decisions without the pooled matmul and KL-divergence overhead. It also introduces a sink-aware threshold to avoid accumulating background noise at long context lengths. Across InfiniteBench, RULER, and LongBench on two model families, the authors report CRISP as the strongest sparse method overall. They claim up to a 5.30x attention speedup at 512k tokens and gains of up to +28.0 points on retrieval tasks over baselines. HF Daily Papers' note
The paper says its `C_struct` proxy reproduces JSD-style routing decisions without the pooled matmul and KL-divergence overhead. It also introduces a sink-aware threshold to avoid accumulating background noise at long context lengths. Across InfiniteBench, RULER, and LongBench on two model families, the authors report CRISP as the strongest sparse method overall. They claim up to a 5.30x attention speedup at 512k tokens and gains of up to +28.0 points on retrieval tasks over baselines. HF Daily Papers' note
score 5