Enhancing VLM Reward Models Through Structure-Aware Fine-Tuning
SAFT fine-tunes VLM reward models online using task structure, without ground-truth reward labels.
The paper presents Structure-Aware Fine-Tuning, a self-supervised method that uses LoRA adapters to regularize a VLM’s latent space. The authors say it denoises reward landscapes across different base model strengths, helping RL policies converge faster. They report improved alignment by EPIC distance and argue some reward failures reflect structural brittleness rather than failed semantic understanding. ArXiv · AI/CL/LG's note
The paper presents Structure-Aware Fine-Tuning, a self-supervised method that uses LoRA adapters to regularize a VLM’s latent space. The authors say it denoises reward landscapes across different base model strengths, helping RL policies converge faster. They report improved alignment by EPIC distance and argue some reward failures reflect structural brittleness rather than failed semantic understanding. ArXiv · AI/CL/LG's note
score 4