Is Persona Generalization Predictable?
Fine-tuning only later layers reduced unsafe behavior spilling into other domains while preserving strong learning in the training domain.
The researchers found that persona changes inside and outside the training domain followed an approximately straight-line relationship after a statistical transformation called probit. That relationship held across tested model sizes and families, including Qwen and Llama. Changing learning rate, weight decay or training duration largely moved models along the same line: less spillover came mainly with weaker learning. Changing which layers were trained could alter the slope, allowing less behavioral transfer for a given level of in-domain learning.
ArXiv · AI/CL/LG's note
The researchers found that persona changes inside and outside the training domain followed an approximately straight-line relationship after a statistical transformation called probit. That relationship held across tested model sizes and families, including Qwen and Llama. Changing learning rate, weight decay or training duration largely moved models along the same line: less spillover came mainly with weaker learning. Changing which layers were trained could alter the slope, allowing less behavioral transfer for a given level of in-domain learning.
ArXiv · AI/CL/LG's note
score 5