SciMIF: Understanding Multimodal Instruction Following in Scientific Domains
The benchmark tests whether multimodal models can obey complex scientific constraints, and chemistry is where they stumble hardest.
SciMIF is built from 22 tasks across five scientific disciplines, with a taxonomy of 10 constraint groups for both general and field-specific instructions. The authors use that taxonomy to inject higher-fidelity instructions into existing scientific datasets. In experiments on closed- and open-source MLLMs, larger models did not reliably improve constraint adherence. The paper says current systems still struggle with fine-grained requirements and instructions that need deeper disciplinary knowledge. ArXiv · AI/CL/LG's note
SciMIF is built from 22 tasks across five scientific disciplines, with a taxonomy of 10 constraint groups for both general and field-specific instructions. The authors use that taxonomy to inject higher-fidelity instructions into existing scientific datasets. In experiments on closed- and open-source MLLMs, larger models did not reliably improve constraint adherence. The paper says current systems still struggle with fine-grained requirements and instructions that need deeper disciplinary knowledge. ArXiv · AI/CL/LG's note
score 5