GoKawiilResearcher argues 'polysemantic' neurons are mostly misread, not superposition
A mechanistic interpretability researcher reports that after months studying neurons and circuits in image and language models, the classic examples of 'polysemantic' neurons—thought to prove networks pack unrelated concepts into single units via superposition—were largely misinterpreted. The researcher attributes most of these cases to a pattern they call 'recursive generalization,' including the famous neuron that responded to both cat faces and car fronts, and says this pattern also shows up in the final layer of InceptionV1, a widely used model organism in interpretability research.
GoKawiil's interpretation of the reporting above, not reported fact.
If the finding holds up, it suggests mechanistic interpretability researchers could study neural networks' own neurons and circuits directly rather than relying on complex reconstruction techniques designed to untangle presumed superposition, potentially simplifying a major line of safety-relevant AI research. The researcher frames this as reason for optimism that the field's outlook is better than previously feared, though this reinterpretation of long-standing canonical evidence would need independent scrutiny before reshaping research priorities.
- A researcher argues that canonical examples of 'polysemantic' neurons were largely misread, not evidence of superposition.
- The alternative explanation, called 'recursive generalization,' is described as a far less concerning phenomenon for interpretability.
- The claim suggests networks up to trillions of parameters could be studied directly without first disentangling their representations.
Source: already-clean.goodfire.com — Nick Cammarata, 2026-10-15
Published there as: “Neural Networks Are Already Clean”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.