From Retrieval to Typed Decisions: Calibrated System One Models from Biomedical Sentence Encoders
Biomedical retrieval encoders can be turned into typed decision models, but the best gains depend on keeping the retrieval prior.
The paper introduces SBERT2S1, BIODECIDE, and MEDLINE-S1, a 243k-decision training set derived from NLM indexing. Retrieval training improved zero-shot matching, but after fine-tuning it helped the prior-fused residual head far more reliably than the cross-head model. In matched tests, the cross-head model still beat the prior-fused version under every objective. The paper also says the released RLCD recipe trailed cross-entropy mainly because reward normalisation amplified noisy score-function terms, while leave-one-out estimation recovered much of the gap. HF Daily Papers' note
The paper introduces SBERT2S1, BIODECIDE, and MEDLINE-S1, a 243k-decision training set derived from NLM indexing. Retrieval training improved zero-shot matching, but after fine-tuning it helped the prior-fused residual head far more reliably than the cross-head model. In matched tests, the cross-head model still beat the prior-fused version under every objective. The paper also says the released RLCD recipe trailed cross-entropy mainly because reward normalisation amplified noisy score-function terms, while leave-one-out estimation recovered much of the gap. HF Daily Papers' note
score 4