Accuracy and Order Sensitivity Diverge Under Label-Free Strategies
Removing answer labels cuts order bias, but it does not reliably make MCQ scores more accurate.
Hanna and Feng test two label-free strategies: generating an answer before matching it to choices, and scoring each option in isolation. Neither strategy consistently beats the standard multiple-choice baseline. Their decomposition points to withheld options as the main bottleneck, not the matching step. The setup that best holds up is showing all options and using an LLM matcher, while cyclic permutation often helps accuracy more than full positional debiasing. HF Daily Papers' note
Hanna and Feng test two label-free strategies: generating an answer before matching it to choices, and scoring each option in isolation. Neither strategy consistently beats the standard multiple-choice baseline. Their decomposition points to withheld options as the main bottleneck, not the matching step. The setup that best holds up is showing all options and using an LLM matcher, while cyclic permutation often helps accuracy more than full positional debiasing. HF Daily Papers' note
score 4