Fewer Clarifications, Better Code: Benchmarking Cross-Session Personalized Ambiguity Adaptation in Coding Assistants
The paper tests whether a coding assistant can use a user’s past resolved sessions to answer new ambiguous requests with fewer follow-ups.
The authors define a new task, personalized ambiguity adaptation, built around recurring user-specific ambiguity across coding sessions. They introduce CAPA, a benchmark with 600 coding sessions across 60 user-ambiguity cells, including 300 held-out evaluation sessions. Twelve recent LLMs are evaluated with and without same-user history, using executable success, first-turn success, and turns-to-completion. The paper also proposes same-user history gating as a lightweight inference-time method. HF Daily Papers' note
The authors define a new task, personalized ambiguity adaptation, built around recurring user-specific ambiguity across coding sessions. They introduce CAPA, a benchmark with 600 coding sessions across 60 user-ambiguity cells, including 300 held-out evaluation sessions. Twelve recent LLMs are evaluated with and without same-user history, using executable success, first-turn success, and turns-to-completion. The paper also proposes same-user history gating as a lightweight inference-time method. HF Daily Papers' note
score 4