Untangling the Mechanisms of Misleading Context in Medical Question Answering
Bare assertions swayed the medical QA models more than fabricated evidence did.
The paper tests misleading context on MedMisBench’s medical reasoning questions, using fabricated evidence and a bare assertion as injected cues. Across three reasoning models, the assertion was adopted 10 to 27 points more often than fabricated evidence. The cues often appeared in reasoning traces, but were much less consistently disclosed in final responses, especially for assertions. An LLM monitor caught corrupted decisions far better from an open reasoning trace than from responses alone. ArXiv · AI/CL/LG's note
The paper tests misleading context on MedMisBench’s medical reasoning questions, using fabricated evidence and a bare assertion as injected cues. Across three reasoning models, the assertion was adopted 10 to 27 points more often than fabricated evidence. The cues often appeared in reasoning traces, but were much less consistently disclosed in final responses, especially for assertions. An LLM monitor caught corrupted decisions far better from an open reasoning trace than from responses alone. ArXiv · AI/CL/LG's note
score 4