AnswersPublished26 sources
Are Neural Networks Easier to Interpret Than We Thought? Cammarata’s Case—and Its Limits
Goodfire researcher Nick Cammarata argues that better tools and ordinary generalization explain much apparent neural network complexity. In vision models, he reports interpretability rising from about 5% with older tools to roughly 99% in a parameter matched ConvNeXt; these are last layer results, not proof that ent...
Published byEdited with GPT-6 LunaImages generated with GPT Image 2
Research answer
Key takeaways
- Goodfire researcher Nick Cammarata argues that better tools and ordinary generalization explain much apparent neural network complexity.
- In vision models, he reports interpretability rising from about 5% with older tools to roughly 99% in a parameter matched ConvNeXt; these are last layer results, not proof that entire models are understood.
AI Prompt
openai.comCreate a landscape editorial hero image for this Studio Global article: What does Goodfire researcher Nick Cammarata argue in his October 15, 2026 essay about neural networks being easier to interpret than previo. Article summary: Cammarata’s October 15 essay, “Neural Networks Are Already Clean,” argues that much apparent neural-network opacity reflects inadequate tools and older architectures rather than intrinsically tangled representations; he . Topic tags: general, general web, user generated. Style: premium digital editorial illustration, source-backed research mood, clean composition, high detail, modern web publication hero. Use reference image context only for broad subject, composition, and topical grounding; do not copy the exact image. Avoid: logos, brand marks, copyrighted characters, real person likenesses, fake screenshots, UI text, readable text, watermarks, charts with fa
Nick Cammarata’s October 15, 2026 essay, “Neural Networks Are Already Clean,” makes a provocative case: some neurons that look like mixtures of unrelated concepts may instead reflect weak analysis tools, older architectures, or related features branching through ordinary generalization. He argues that researchers can often study neurons directly rather than first trying to “untangle” them with another model. His reported results are encouraging, but they are not a verdict that neural networks—or their behavior as a whole—are fully understood. 5
10
26
What Cammarata means by “recursive generalization”
Cammarata’s hypothesis is that a neuron’s responses can expand from one feature into related ones, creating a branching family rather than an arbitrary bundle of unrelated concepts. One reported language example starts with a neuron responding to the letter sequence “iy,” then to “diy,” and eventually to “do it yourself”—a phrase that is related in meaning but does not contain the original sequence. The proposal is that this progression can look like feature-mixing if researchers treat each response as a separate, unrelated job. 2
6
10
That interpretation challenges a common reading of polysemanticity, the apparent activation of one neuron for multiple concepts. It also questions whether researchers must first use a separate feature-discovery method, such as a sparse autoencoder, to make a model interpretable. But recursive generalization is not, by itself, proof that superposition never occurs: it describes a possible explanation for some observed responses, not a conclusive replacement for every account of how models represent features. 5
15
Re-reading the “cat and car” neuron
A well-known InceptionV1 example appeared to respond to cat faces and the fronts of cars, which seemed to show one neuron combining unrelated categories. Cammarata’s essay is reported to reinterpret the response as detecting a shared visual feature—a V-like shape—rather than independently representing “cat” and “car.” That makes the example less decisive as evidence of arbitrary mixing, though a new interpretation of one neuron does not settle the broader question of how features are represented across a model. 10
12
The sources available for this account do not provide enough detail to reliably reconstruct the essay’s revised explanation of Cammarata’s 2020 car-to-dog circuit example. It is therefore not possible here to assess how that example supports the hypothesis beyond the broader argument that apparent mixtures may have a shared or related explanation.
The reported interpretability rates
Cammarata reports a series of last-layer results for vision models: about 5% of InceptionV1’s last-layer neurons were interpretable with 2017-era tools; about 50% with improved tools applied to the same network; about 70% after retraining with modern regularization; and about 99% in a parameter-matched ConvNeXt trained on the same dataset. The sequence suggests that both analysis tools and model architecture can affect how legible neurons appear. These are reported interpretations of the last layer, not percentages of the model’s full computation that researchers have explained. 31
For Qwen3-32B, Cammarata’s preliminary audit estimates that roughly 90% of neuron response poles have one main job, often with small related offshoots; about 7% have two related jobs; and roughly 2–4% remain unexplained. These figures describe neuron poles, not a demonstrated understanding of every component or decision in the language model. The reported proportions also vary with the cutoff for weak activations, which layers are examined, and what qualifies as a single “job.” 1
7
The essay is also discussed alongside an OpenAI automated interpretability project that reportedly explained about 1.7% of the neurons it examined. That figure is not a direct before-and-after comparison with the Qwen3-32B estimate: the projects concern different models, tools, and measures. It should not be read as a controlled demonstration of a particular increase in interpretability. 9
12
What AI agents add—and what the available evidence leaves unclear
Reporting on the essay describes Cammarata’s use of modern agentic tools as part of the case for studying neurons directly at larger scale. But the sources available here do not establish how a figure of 100,000 attribution graphs was generated or how those graphs contributed to the findings, so that specific methodological detail cannot be verified from this evidence. 3
5
6
The language-model results are described as preliminary and the essay as a working hypothesis. Reporting also says the technical companion paper was still forthcoming or unreleased at the time. The supplied sources do not establish independent replication or provide enough information to evaluate the researcher’s confidence in each individual estimate. 7
12
34
Why the response is divided
Latent Space presented the essay enthusiastically, framing agentic analysis as a challenge to the idea that polysemantic neurons make direct study impractical. Neel Nanda of Google DeepMind is described in reporting as more cautious: he continues to expect superposition and argues that weak activations may contain meaningful information, including memorized facts. Other reporting characterizes his critique as partly about how “one job” or a “monosemantic” neuron is defined, rather than simply rejecting the reported observations. 6
35
36
37
The available sources do not document enough of the exchange to substantiate every claimed point of agreement or disagreement—including the detailed question of whether conclusions from image models transfer to language models, or whether neuron-level analysis is sufficient for understanding circuits. Those questions should remain open rather than being presented as resolved by the reported percentages.
The careful takeaway
Cammarata’s strongest claim is that apparent neural-network complexity may sometimes come from the tools and models researchers use to inspect it, and that many individual neurons may be more legible than older studies suggested. His reported vision-model results and preliminary Qwen3-32B audit make that a noteworthy hypothesis. They do not establish that superposition has been disproved, that every neuron has one clear function, or that understanding neurons alone amounts to understanding an entire model. 7
10
12
Studio Global AI
Continue your research
This page includes a source-backed answer you can continue inside Studio Global.
People also ask
What is the short answer to "Are Neural Networks Easier to Interpret Than We Thought? Cammarata’s Case—and Its Limits"?
Goodfire researcher Nick Cammarata argues that better tools and ordinary generalization explain much apparent neural network complexity.
What are the key points to validate first?
Goodfire researcher Nick Cammarata argues that better tools and ordinary generalization explain much apparent neural network complexity. In vision models, he reports interpretability rising from about 5% with older tools to roughly 99% in a parameter matched ConvNeXt; these are last layer results, not proof that entire models are understood.