Towards a Deterministic Math Solver for Clinical Language Models
A restricted Python executor improved calculator accuracy for the 32B model, but not reliably for the 7B model.
The paper tests a setup where the language model writes case-specific Python and leaves the arithmetic to a local deterministic solver. On 1,100 verified benchmark cases across 55 clinical calculators, the 32B Qwen model rose from 83.47% with direct arithmetic to 90.53% with the solver route. The 7B model showed only a small, statistically uncertain gain. The authors also found formula issues in 16 of the 55 calculators, underscoring that an executor does not replace validated formulas or dependable variable extraction. HF Daily Papers' note
The paper tests a setup where the language model writes case-specific Python and leaves the arithmetic to a local deterministic solver. On 1,100 verified benchmark cases across 55 clinical calculators, the 32B Qwen model rose from 83.47% with direct arithmetic to 90.53% with the solver route. The 7B model showed only a small, statistically uncertain gain. The authors also found formula issues in 16 of the 55 calculators, underscoring that an executor does not replace validated formulas or dependable variable extraction. HF Daily Papers' note
score 4