GoKawiilSakana AI unveils PC-ALM, a backprop-free method for training 1000-layer networks
Sakana AI researchers Jeffrey Seely and Julian Gould introduced PC-ALM (Augmented Lagrangian Predictive Coding), a training method that replaces backpropagation's forward and backward passes with local feedback control dynamics running between neighboring layers. The team reports it successfully trains residual MLPs as deep as 1000 layers, reaching performance close to standard backpropagation while relying only on layer-local computation.
GoKawiil's interpretation of the reporting above, not reported fact.
Backpropagation's strict three-phase timing requirement has no known biological analog, making it a poor model for how brains actually learn, so a working local alternative could bridge neuroscience and machine learning. If local, biologically plausible training methods can match backprop at scale, they could also enable more parallel, hardware-efficient, or brain-inspired computing architectures.
- PC-ALM trains extremely deep residual networks (up to 1000 layers) using only local layer-to-layer dynamics instead of global backpropagation.
- It builds on predictive coding but adds augmented Lagrangian-style feedback control to better distribute credit across layers.
- The approach nearly matches backpropagation's performance while avoiding the phase-locking constraint that makes backprop biologically implausible.
Source: pub.sakana.ai, 2026-09-14
Published there as: “Backprop Alternative: Augmented Lagrangian Predictive Coding”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.