Preference-Guided Adaptation for Open-Vocabulary Semantic Segmentation via Prompt Disagreement
The paper turns prompt disagreement into a substitute for pixel-level labels.
The authors target open-vocabulary semantic segmentation in specialized domains where dense masks are expensive to get. Their method finds regions where different prompt templates produce conflicting segmentations, then asks for binary preferences on those localized regions. It adapts the model with Region-Localized Preference Optimization and a consistency term meant to avoid destabilizing the rest of the image. Experiments on MESS show gains across multiple OVSS backbones without pixel-level annotation, including under noisy preferences. HF Daily Papers' note
The authors target open-vocabulary semantic segmentation in specialized domains where dense masks are expensive to get. Their method finds regions where different prompt templates produce conflicting segmentations, then asks for binary preferences on those localized regions. It adapts the model with Region-Localized Preference Optimization and a consistency term meant to avoid destabilizing the rest of the image. Experiments on MESS show gains across multiple OVSS backbones without pixel-level annotation, including under noisy preferences. HF Daily Papers' note
score 4