AMRD: Adaptive Multi-Teacher Relational Distillation for Lightweight Speech Emotion Recognition
AMRD uses per-batch teacher weighting and relational matching to make speech emotion models smaller without dropping as much performance.
The method targets on-device speech emotion recognition, where larger self-supervised models are too expensive to run at the edge. It weights multiple teachers with a one-class SVM based on each teacher’s logit similarity matrix, favoring the more coherent teacher for that batch. It also trains the student to match inter-sample similarity structure, not just logits. The paper reports gains over single-teacher distillation baselines on IEMOCAP and CREMA-D across four student architectures, with ablations supporting both pieces. HF Daily Papers' note
The method targets on-device speech emotion recognition, where larger self-supervised models are too expensive to run at the edge. It weights multiple teachers with a one-class SVM based on each teacher’s logit similarity matrix, favoring the more coherent teacher for that batch. It also trains the student to match inter-sample similarity structure, not just logits. The paper reports gains over single-teacher distillation baselines on IEMOCAP and CREMA-D across four student architectures, with ablations supporting both pieces. HF Daily Papers' note
score 4