Reporting Practice Matters: The Impact of Reference Choice on Chest X-ray Report Evaluation
Changing the reference report style was enough to change model rankings in chest X-ray report evaluation.
The paper tests how established metrics respond when clinically equivalent radiology references differ in wording, detail, or formatting. Its ReRef method rewrites reference reports along radiologist-defined practice variations while preserving the clinical interpretation. On MIMIC-CXR with RadCliQ-v1, condensing normal findings moved Libra from first to second and CheXOne from third to first among nine report-generation models. The authors argue that current metrics often reward conformity to reporting habits, not just clinical correctness, and release a 120-pair radiologist-validated reference set to study the issue. ArXiv · AI/CL/LG's note
The paper tests how established metrics respond when clinically equivalent radiology references differ in wording, detail, or formatting. Its ReRef method rewrites reference reports along radiologist-defined practice variations while preserving the clinical interpretation. On MIMIC-CXR with RadCliQ-v1, condensing normal findings moved Libra from first to second and CheXOne from third to first among nine report-generation models. The authors argue that current metrics often reward conformity to reporting habits, not just clinical correctness, and release a 120-pair radiologist-validated reference set to study the issue. ArXiv · AI/CL/LG's note
score 4