The Value of Thinking: Learning When to Reason in Hybrid Decision Models
HDM-4B slightly beats Jev on JevBench while cutting median latency by 72.5%.
The hybrid decision model uses one frozen backbone to either answer in a single forward pass without generating tokens or run full autoregressive reasoning. A lightweight routing block estimates whether reasoning’s expected benefit exceeds a price set for each request. On JevBench, HDM-4B scores 0.857 against Jev’s 0.851, with median latency of 0.179 seconds versus 0.652 seconds. On MiniWoB, the authors report that HDM-27B cuts token cost by 80.1%.
HF Daily Papers' note
The hybrid decision model uses one frozen backbone to either answer in a single forward pass without generating tokens or run full autoregressive reasoning. A lightweight routing block estimates whether reasoning’s expected benefit exceeds a price set for each request. On JevBench, HDM-4B scores 0.857 against Jev’s 0.851, with median latency of 0.179 seconds versus 0.652 seconds. On MiniWoB, the authors report that HDM-27B cuts token cost by 80.1%.
HF Daily Papers' note
score 5