Scheduling Recursive Reasoning in Looped Transformers
TAPS changes the step size during recurrent inference, aiming to get more from each loop without retraining.
The paper analyzes when repeated transformer updates are making steady progress versus when they are fluctuating. Its scheduler tracks that balance online and scales the recurrent update accordingly. The authors report higher terminal accuracy on structured reasoning tasks, and further gains when the same principle is used in training. They also claim up to 1.56x wall-clock speedup at matched baseline accuracy. HF Daily Papers' note
The paper analyzes when repeated transformer updates are making steady progress versus when they are fluctuating. Its scheduler tracks that balance online and scales the recurrent update accordingly. The authors report higher terminal accuracy on structured reasoning tasks, and further gains when the same principle is used in training. They also claim up to 1.56x wall-clock speedup at matched baseline accuracy. HF Daily Papers' note
score 4