When Do Biological Reasoning Models Use Their Biological Inputs?
Some biology reasoning models scored well without relying much on the biology representations they were given.
The paper tests six models across DNA, protein, and single-cell tasks and finds sharp differences in whether biological inputs actually affect performance. BioReason and BioReason-Pro largely followed text over foundation-model representations in conflict tests, including 97.9% and 99.7% text-following rates. Shuffling DNA barely changed BioReason disease prediction accuracy, even though probes showed Evo2 and ESM3 representations contained task-relevant information. Other systems, including ChatNT, Prot2Text-V2, CellWhisperer, and Cell2Sentence-Scale, did show input contributions in the evaluated settings. HF Daily Papers' note
The paper tests six models across DNA, protein, and single-cell tasks and finds sharp differences in whether biological inputs actually affect performance. BioReason and BioReason-Pro largely followed text over foundation-model representations in conflict tests, including 97.9% and 99.7% text-following rates. Shuffling DNA barely changed BioReason disease prediction accuracy, even though probes showed Evo2 and ESM3 representations contained task-relevant information. Other systems, including ChatNT, Prot2Text-V2, CellWhisperer, and Cell2Sentence-Scale, did show input contributions in the evaluated settings. HF Daily Papers' note
score 5