Megadose The AI curve, compressed.

Right Reason, Wrong Advice: Evaluating Recourse in LLM Explanations of Denied Requests

· ArXiv · AI/CL/LG ·
A correct reason for denial can still give someone no workable path to approval.

In six of seven rule systems tested, fixing a single valid reason almost never reversed a denial when several independent reasons applied. Across 20 LLMs, checks of whether the cited reasons were correct were close to chance at detecting this failure. A strong LLM judge did detect it when asked whether acting on the explanation would work. Stronger and reasoning models performed better but still failed on many denials involving four or more independent reasons; asking for every rule helped every open model with room to improve.

ArXiv · AI/CL/LG's note

score 4

Categories: Research