A Closed-Form Upper Bound for Admissible Learning-Rate Steps in Belief-Space Dynamics
Derives a closed-form upper bound for admissible learning-rate steps under KL/Bregman geometry, replacing hyperparameter tuning with a precise formula.
Excerpt
Zixi Li, Youzhen Li — Learning-rate steps are usually treated as hyperparameters. This paper isolates a local beliefspace calculation: when an update is modeled as a projected forward step on the probability simplex, admissibility means contractivity in the natural KL/Bregman geometry. Under this model, the upper bound of an admissible step is not a tuning slogan but a formula.
Read at source: https://arxiv.org/abs/2605.06741