PerceptRL: Making Visual Perception Rewardable in MLLMs from Answers to Interactions
PerceptRL reports 71.6% average accuracy across seven benchmarks, beating the strongest evaluated frontier vision-language model by 5.2 percentage points.
The method turns verified errors in image captions into questions with fixed answer labels and rule-based rewards. A second training stage uses multi-turn visual inspection to tackle fine-grained cases that single-turn training leaves unresolved. To address coordinate drift, which can produce incorrect crops or annotations, it corrects spatial operations in training trajectories and rewards intermediate grounding accuracy. Under the shared tool-use protocol, it also outperformed the base model using the same tools by 15.7 percentage points.
ArXiv · AI/CL/LG's note
The method turns verified errors in image captions into questions with fixed answer labels and rule-based rewards. A second training stage uses multi-turn visual inspection to tackle fine-grained cases that single-turn training leaves unresolved. To address coordinate drift, which can produce incorrect crops or annotations, it corrects spatial operations in training trajectories and rewards intermediate grounding accuracy. Under the shared tool-use protocol, it also outperformed the base model using the same tools by 15.7 percentage points.
ArXiv · AI/CL/LG's note
score 5