What Else Needs Fixing? Exploring Cost-Effective Test-Time Compute for Revision Propagation in Artifacts Generated Through Conversation
The paper tests how well LLMs carry a user’s small requested edit through the other parts of a conversation-built artifact that also need to change.
It introduces a benchmark for revision propagation when dependencies may live in the chat history, not just the current artifact. The study evaluates nine revision methods across gpt-oss, GPT-5.4-mini, and Qwen models. Baseline accuracy ranges from 68.3% to 93%. The best cost-effective approach selects from three parallel samples, improving accuracy by 2.2 to 9.7%. HF Daily Papers' note
It introduces a benchmark for revision propagation when dependencies may live in the chat history, not just the current artifact. The study evaluates nine revision methods across gpt-oss, GPT-5.4-mini, and Qwen models. Baseline accuracy ranges from 68.3% to 93%. The best cost-effective approach selects from three parallel samples, improving accuracy by 2.2 to 9.7%. HF Daily Papers' note
score 4