Cited but Unused: Parametric Provenance in Retrieval-Augmented Generation
Standard citation checks performed no better than chance at identifying whether models actually used the sources they cited.
The researchers replaced facts in retrieved passages with false variants to test whether answers followed the context or the model’s memory. Across the models tested, 1.2% to 15.7% of citable claims came from parametric memory, with rates rising with model size within a family. Models answering from memory selected citations that agreed with their output, leaving ordinary entailment checks unable to distinguish provenance. Of ten attribution methods, comparing output distributions with and without context worked best, but was still not accurate enough to improve provenance honesty.
ArXiv · AI/CL/LG's note
The researchers replaced facts in retrieved passages with false variants to test whether answers followed the context or the model’s memory. Across the models tested, 1.2% to 15.7% of citable claims came from parametric memory, with rates rising with model size within a family. Models answering from memory selected citations that agreed with their output, leaving ordinary entailment checks unable to distinguish provenance. Of ten attribution methods, comparing output distributions with and without context worked best, but was still not accurate enough to improve provenance honesty.
ArXiv · AI/CL/LG's note
score 5