InclusionMed: Benchmarking AI for Inclusive Healthcare
The ten highest-ranked models all scored better on patient communication than on clinical decision-making.
InclusionMed’s first release combines four public benchmarks selected from 388 candidates with a private diagnostic reasoning benchmark. It covers three of nine cells in a framework spanning healthcare roles and AI capabilities. Overall model scores ranged from 17.5 to 49.1 out of 100, with rankings shifting by role or capability. Models with similar overall scores also differed substantially in estimated inference cost.
HF Daily Papers' note
InclusionMed’s first release combines four public benchmarks selected from 388 candidates with a private diagnostic reasoning benchmark. It covers three of nine cells in a framework spanning healthcare roles and AI capabilities. Overall model scores ranged from 17.5 to 49.1 out of 100, with rankings shifting by role or capability. Models with similar overall scores also differed substantially in estimated inference cost.
HF Daily Papers' note
score 4