Mechanistic Interpretability

Researcher argues 'polysemantic' neurons are mostly misread, not superposition

A mechanistic interpretability researcher reports that after months studying neurons and circuits in image and language models, the classic examples of 'polysemantic' neurons—thought to prove networks pack unrelated concepts into single units via superposition—were largely misinterpreted. The researcher attributes most of these cases to a pattern they call 'recursive generalization,' including the famous neuron that responded to both cat faces and car fronts, and says this pattern also shows up in the final layer of InceptionV1, a widely used model organism in interpretability research.

already-clean.goodfire.com · 2026-10-15

Anthropic Publishes Early Framework for Reverse-Engineering Transformer Circuits

Anthropic researchers introduce a mathematical approach to mechanistic interpretability, aiming to reverse-engineer the internal computations of transformer language models. Their initial study focuses on small transformers with two layers or fewer that use only attention blocks, deliberately simpler than models like GPT-3, in order to identify basic patterns before tackling larger systems.

transformer-circuits.pub · 2026-09-12