From Atomic Evidence to Logical Composition: Structured Compositional Reasoning over Compound Answer Options
The paper reports large F1 gains by hiding compound answer choices from the model and recombining atomic judgments with logic constraints.
The authors test answer options joined by AND, OR, and NEITHER/NOR, where models can get the parts right but fail the composition. Their method scores each atomic claim separately, then uses an operator-constrained integer linear program to choose the final answer. It raises Macro-F1 from 48.3 to 77.0 on LOGICAL-COMMONSENSEQA and from 47.0 to 75.6 on LOGICAL-SATA, with the biggest improvement on NEITHER/NOR cases. Source: HF Daily Papers' note.
The authors test answer options joined by AND, OR, and NEITHER/NOR, where models can get the parts right but fail the composition. Their method scores each atomic claim separately, then uses an operator-constrained integer linear program to choose the final answer. It raises Macro-F1 from 48.3 to 77.0 on LOGICAL-COMMONSENSEQA and from 47.0 to 75.6 on LOGICAL-SATA, with the biggest improvement on NEITHER/NOR cases. Source: HF Daily Papers' note.
score 4