Sketching Language Models for Context Attribution
Under a stability assumption, the method estimates every token’s leave-one-out effect with a model-call bound independent of prompt length.
Computing those effects directly takes one model evaluation per token. For error proportional to ε, the sketching algorithm uses O(√log(1/ε)/ε) evaluations, succeeding with constant probability. The authors prove a matching lower bound in an oracle model. Empirical tests check the stability assumption and assess the method on a small language-model retrieval task.
ArXiv · AI/CL/LG's note
Computing those effects directly takes one model evaluation per token. For error proportional to ε, the sketching algorithm uses O(√log(1/ε)/ε) evaluations, succeeding with constant probability. The authors prove a matching lower bound in an oracle model. Empirical tests check the stability assumption and assess the method on a small language-model retrieval task.
ArXiv · AI/CL/LG's note
score 4