Prediction Is Not a Sufficient Memory State
Matching the current predictor does not preserve the information needed for the next update.
The paper formalizes that gap for exact cumulative learning and attention. It shows fixed-dimensional continuous recurrent states exist for smooth convex residual losses exactly when the loss is polynomial, with precise coordinate counts for ordinary observations. For ridge regression, the missing state is the design matrix, making exact continuation more expensive than storing the predictor alone. The author says the results identify what future updates reveal, not a new regression algorithm or evidence of trained-model gains. ArXiv · AI/CL/LG's note
The paper formalizes that gap for exact cumulative learning and attention. It shows fixed-dimensional continuous recurrent states exist for smooth convex residual losses exactly when the loss is polynomial, with precise coordinate counts for ordinary observations. For ridge regression, the missing state is the design matrix, making exact continuation more expensive than storing the predictor alone. The author says the results identify what future updates reveal, not a new regression algorithm or evidence of trained-model gains. ArXiv · AI/CL/LG's note
score 4