UniMoMo: Expert Merging-Based MoE Acceleration for Large Recommendation Models
UniMoMo compresses trained recommendation MoE models by merging functionally similar experts, keeping performance near the source checkpoints while cutting serving cost.
The paper frames deployment as converting a full expert bank into a smaller standard MoE under a fixed expert budget. UniMoMo uses an unlabeled calibration set to group experts by response similarity, not parameter distance. It also protects high-traffic experts from risky merges based on routing exposure. On Amazon Beauty, KuaiRec, and TenRec, four-expert exports report 99.92%–102.30% source-relative NDCG@10 with 1.28×–1.63× A100 speedups; a two-expert top-1 setting reports 98.36%–104.24% with 1.47×–2.21× speedups. HF Daily Papers' note
The paper frames deployment as converting a full expert bank into a smaller standard MoE under a fixed expert budget. UniMoMo uses an unlabeled calibration set to group experts by response similarity, not parameter distance. It also protects high-traffic experts from risky merges based on routing exposure. On Amazon Beauty, KuaiRec, and TenRec, four-expert exports report 99.92%–102.30% source-relative NDCG@10 with 1.28×–1.63× A100 speedups; a two-expert top-1 setting reports 98.36%–104.24% with 1.47×–2.21× speedups. HF Daily Papers' note
score 4