Reading Is Not Using: Retrieval, Judgment, and the Design of AI Financial Research Workflows
The paper says LLMs can retrieve a risk disclosure accurately and still fail to let it affect an investment judgment.
Liu and Liu describe a “retrieval-integration gap” in long-context financial analysis. In their tests, unrelated context grew from 2,000 to 128,000 tokens while the focal-firm information stayed fixed, and the disclosure’s effect on judgments fell to the noise floor. The pattern held across model families and judgment tasks, including experiments that removed real disclosures from actual 10-K filings. They find that workflow design matters: chunk-and-summarize pipelines can drop relevant information, while a targeted structured restatement near the decision can restore its influence. ArXiv · AI/CL/LG's note
Liu and Liu describe a “retrieval-integration gap” in long-context financial analysis. In their tests, unrelated context grew from 2,000 to 128,000 tokens while the focal-firm information stayed fixed, and the disclosure’s effect on judgments fell to the noise floor. The pattern held across model families and judgment tasks, including experiments that removed real disclosures from actual 10-K filings. They find that workflow design matters: chunk-and-summarize pipelines can drop relevant information, while a targeted structured restatement near the decision can restore its influence. ArXiv · AI/CL/LG's note
score 5