One Loop, Two Gains: Can Active Learning win the Lottery for Free?
Active learning’s retraining loop can double as a pruning loop, producing sparse models without another full training cycle.
The paper proposes Improve & Prune, which folds magnitude pruning into each active learning retraining round. Across acquisition functions, architectures, image datasets, and an active fine-tuning setup, the sparse models matched dense-model accuracy at up to 95% sparsity. The authors frame the gain as practical: smaller deployable models at every iteration, with potential savings in retraining and unlabeled-pool scoring. ArXiv · AI/CL/LG's note
The paper proposes Improve & Prune, which folds magnitude pruning into each active learning retraining round. Across acquisition functions, architectures, image datasets, and an active fine-tuning setup, the sparse models matched dense-model accuracy at up to 95% sparsity. The authors frame the gain as practical: smaller deployable models at every iteration, with potential savings in retraining and unlabeled-pool scoring. ArXiv · AI/CL/LG's note
score 4