LLMs Get Lost in Evolving User Intent
Strong benchmark scores fell when user goals changed mid-conversation.
The paper tests LLMs in multi-turn tasks where intent is revealed, revised, or redirected over time. Its framework converts static benchmarks into evolving conversations while keeping the original evaluation protocol. Across multiple tasks and model families, performance dropped substantially compared with fully specified single-turn settings. HF Daily Papers' note
The paper tests LLMs in multi-turn tasks where intent is revealed, revised, or redirected over time. Its framework converts static benchmarks into evolving conversations while keeping the original evaluation protocol. Across multiple tasks and model families, performance dropped substantially compared with fully specified single-turn settings. HF Daily Papers' note
score 5