Backprop Alternative: Augmented Lagrangian Predictive Coding
PC-ALM trains 1000-layer residual MLPs using only neighboring-layer dynamics, landing within about two points of backprop on MNIST.
Sakana’s post frames the method as predictive coding with augmented Lagrangian multipliers added at each layer. Those dual variables accumulate local prediction errors and, in linear networks, converge to the same credit signals backprop would send. In the reported tests, PC-ALM beats standard predictive coding across the listed image benchmarks and reduces its signal-decay problem in deep, narrow networks. The authors position the work mainly as a credit-assignment model for distributed systems such as brains, with possible relevance to neuromorphic hardware. HN · ArXiv's note
Sakana’s post frames the method as predictive coding with augmented Lagrangian multipliers added at each layer. Those dual variables accumulate local prediction errors and, in linear networks, converge to the same credit signals backprop would send. In the reported tests, PC-ALM beats standard predictive coding across the listed image benchmarks and reduces its signal-decay problem in deep, narrow networks. The authors position the work mainly as a credit-assignment model for distributed systems such as brains, with possible relevance to neuromorphic hardware. HN · ArXiv's note
score 5
Discussions
- hn · 75 points · 22 comments
- hn · 79 points · 22 comments
- hn · 83 points · 22 comments
- hn · 84 points · 27 comments
- hn · 88 points · 27 comments
- hn · 93 points · 30 comments
- hn · 94 points · 32 comments
- hn · 101 points · 34 comments
- hn · 103 points · 37 comments
- hn · 103 points · 37 comments
- hn · 103 points · 37 comments
- hn · 106 points · 38 comments
- hn · 108 points · 38 comments
- hn · 110 points · 38 comments
- hn · 110 points · 39 comments
- hn · 111 points · 39 comments
- hn · 113 points · 39 comments
- hn · 113 points · 39 comments
- hn · 113 points · 41 comments
- hn · 114 points · 41 comments
- hn · 114 points · 41 comments
- hn · 115 points · 42 comments
- hn · 116 points · 42 comments
- hn · 116 points · 42 comments
- hn · 117 points · 45 comments
- hn · 117 points · 45 comments
- hn · 117 points · 45 comments
- hn · 117 points · 45 comments
- hn · 117 points · 45 comments
- hn · 117 points · 45 comments
- hn · 117 points · 45 comments
- hn · 118 points · 45 comments
- hn · 118 points · 45 comments
- hn · 118 points · 45 comments
- hn · 118 points · 45 comments
- hn · 118 points · 45 comments
- hn · 118 points · 45 comments
- hn · 119 points · 45 comments
- hn · 119 points · 45 comments
- hn · 119 points · 45 comments
- hn · 119 points · 45 comments
- hn · 119 points · 45 comments
- hn · 119 points · 45 comments
- hn · 120 points · 45 comments
- hn · 120 points · 45 comments
- hn · 120 points · 45 comments
- hn · 120 points · 45 comments
- hn · 120 points · 45 comments
- hn · 120 points · 45 comments
- hn · 120 points · 45 comments
- hn · 120 points · 45 comments
- hn · 120 points · 45 comments
- hn · 120 points · 45 comments
- hn · 122 points · 45 comments
- hn · 122 points · 45 comments
- hn · 122 points · 45 comments
- hn · 122 points · 45 comments
- hn · 122 points · 45 comments
- hn · 122 points · 45 comments
- hn · 122 points · 45 comments
- hn · 122 points · 45 comments
- hn · 122 points · 45 comments
- hn · 122 points · 45 comments
- hn · 122 points · 45 comments
- hn · 122 points · 45 comments
- hn · 123 points · 45 comments
- hn · 123 points · 45 comments
- hn · 123 points · 45 comments
- hn · 123 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 124 points · 45 comments
- hn · 125 points · 45 comments
- hn · 125 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments
- hn · 126 points · 45 comments