MemSFT: Mitigating Alignment Tax with an External Parametric Memory
MemSFT adds domain skill through a reusable external memory instead of pushing that knowledge into the model backbone.
The paper says the method trains a plug-and-play parametric memory to imitate retrieval over domain data. A learned router then blends the memory and base model distributions during generation, step by step. In tests across biology, geoscience, and law, the authors report stronger domain performance with negligible general-task degradation, while full supervised fine-tuning caused severe forgetting. HF Daily Papers' note
The paper says the method trains a plug-and-play parametric memory to imitate retrieval over domain data. A learned router then blends the memory and base model distributions during generation, step by step. In tests across biology, geoscience, and law, the authors report stronger domain performance with negligible general-task degradation, while full supervised fine-tuning caused severe forgetting. HF Daily Papers' note
score 5