MedPixel: A Unified Pixel-Language Model for Medical Reasoning and Segmentation
MedPixel ties clinical text responses to pixel-level masks through a shared language-mask interface.
The paper introduces MedPLG-440K, about 440,000 pixel-language task samples built through clinical synthesis without external LLM annotation. Training combines multi-task supervised fine-tuning with Pixel-Level Preference Optimization, using ground-truth masks to rank responses by mask quality. The model is reported to handle grounding, spatial interaction, grounded explanation, medical VQA, and zero-shot transfer to external grounding benchmarks. Code and checkpoints are promised for release. ArXiv · AI/CL/LG's note
The paper introduces MedPLG-440K, about 440,000 pixel-language task samples built through clinical synthesis without external LLM annotation. Training combines multi-task supervised fine-tuning with Pixel-Level Preference Optimization, using ground-truth masks to rank responses by mask quality. The model is reported to handle grounding, spatial interaction, grounded explanation, medical VQA, and zero-shot transfer to external grounding benchmarks. Code and checkpoints are promised for release. ArXiv · AI/CL/LG's note
score 5