GoKawiilSakana AI unveils PC-ALM, a backprop-free method for training 1000-layer networks
Sakana AI researchers Jeffrey Seely and Julian Gould introduced PC-ALM (Augmented Lagrangian Predictive Coding), a training method that replaces backpropagation's forward and backward passes with local feedback control dynamics running between neighboring layers. The team reports it successfully trains residual MLPs as deep as 1000 layers, reaching performance close to standard backpropagation while relying only on layer-local computation.