Parameter-Efficient Conciseness Adaptation: Differentiable Length Control for Concise Clinical Reasoning in Smaller Language Models
A 1.7-billion-parameter model produced 72.8% shorter responses while diagnostic accuracy rose from 40.8% to 44.0%.
The authors report these results after fine-tuning on the medical-o1-reasoning-SFT corpus. Their PECA method adds a conciseness term to QLoRA training, increasing the probability of ending a response once it exceeds a token budget. It requires no preference data or auxiliary model and runs on a single commodity GPU. Adjusting the term’s strength lets users balance response length against diagnostic accuracy.
ArXiv · AI/CL/LG's note
The authors report these results after fine-tuning on the medical-o1-reasoning-SFT corpus. Their PECA method adds a conciseness term to QLoRA training, increasing the probability of ending a response once it exceeds a token budget. It requires no preference data or auxiliary model and runs on a single commodity GPU. Adjusting the term’s strength lets users balance response length against diagnostic accuracy.
ArXiv · AI/CL/LG's note
score 4