GoKawiilResearcher argues 'polysemantic' neurons are mostly misread, not superposition
A mechanistic interpretability researcher reports that after months studying neurons and circuits in image and language models, the classic examples of 'polysemantic' neurons—thought to prove networks pack unrelated concepts into single units via superposition—were largely misinterpreted. The researcher attributes most of these cases to a pattern they call 'recursive generalization,' including the famous neuron that responded to both cat faces and car fronts, and says this pattern also shows up in the final layer of InceptionV1, a widely used model organism in interpretability research.