GraphDecisionBench: Benchmarking System One Models on Graph Tasks
GraphDecide tests whether choice-based “System One” models can actually reason over graph structure.
The paper evaluates Jev and related selector-style models against language-model baselines across fourteen interface setups. Its main finding is that recognizing adjacency does not reliably translate into broader graph correctness. Adding both graph and text inputs also does not consistently improve predictions, and producing feasible answers does not mean the solutions are high quality. HF Daily Papers' note
The paper evaluates Jev and related selector-style models against language-model baselines across fourteen interface setups. Its main finding is that recognizing adjacency does not reliably translate into broader graph correctness. Adding both graph and text inputs also does not consistently improve predictions, and producing feasible answers does not mean the solutions are high quality. HF Daily Papers' note
score 4