Language Chain in Alignment: Cross-lingual Ranking Preference Optimization
CRPO uses English preference rankings to improve alignment behavior in other languages.
The paper proposes Cross-lingual Ranking Preference Optimization, a framework that trains on parallel preference pairs in English and a target language. It jointly models preferences within each language and across languages, using LambdaLoss to rank multiple candidate responses rather than only compare pairs. In experiments across five languages, the authors report gains over standard alignment methods on instruction following and knowledge use. They also report improved reward margins and higher log-probability for preferred responses. HF Daily Papers' note
The paper proposes Cross-lingual Ranking Preference Optimization, a framework that trains on parallel preference pairs in English and a target language. It jointly models preferences within each language and across languages, using LambdaLoss to rank multiple candidate responses rather than only compare pairs. In experiments across five languages, the authors report gains over standard alignment methods on instruction following and knowledge use. They also report improved reward margins and higher log-probability for preferred responses. HF Daily Papers' note
score 4