acceptodds
Under review as a conference paper at ICLR 2027

Manifestation Units: Representing Mechanistic Findings for Access and Intervention

Abstract

Mechanistic interpretability produces heterogeneous findings about model components, relationships, and causal effects, but these findings are often left in analysis-specific forms that complicate downstream reuse. We introduce **Manifestation Units**, a persistent structured interface that organizes findings through shared semantic roles while retaining architecture-specific evidence. We evaluate MUs through Structural Accessibility, whether recorded findings can be recovered under bounded access, and Causal Utility, whether represented numerical information preserves downstream component selection and intervention effectiveness. Across VAE, CNN, and GPT-2 settings, the MU pipeline achieves the highest answer-slot accuracy among nine matched BM25 conditions (54.58%, 79.44%, and 79.72%). In the CNN, MU-mediated access reaches 92.37% targeted success, while complete-access MU and native findings produce identical selections and activation-patching effects. In GPT-2, MU preserves the complete native head ranking and seven-head selection, which achieves 61.87% normalized recovery on fresh IOI examples versus 6.30% for layer-matched random controls. These results support treating persistent representation between mechanistic analysis and downstream use as an explicit and independently evaluable layer.

Primary area:
interpretability and explainable AI
Keywords:
Mechanistic Interpretability, Manifestation Units, Structural Accessibility, Causal Utility, Representation Interface

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_M6rdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Manifestation Units (MUs), a structured interface for storing and reusing mechanistic interpretability findings. It organizes findings through five shared roles, with architecture-specific implementations. Experiments on VAE, CNN and GPT-2 test whether this organization improves retrieval and question answering, and whether it preserves the information needed for component selection and intervention.

Critical strengths
  • The paper addresses a useful problem beyond mechanism discovery. The research perspective on mechanistic interpretability is interesting and meaningful, making interpretability results easier to reuse.
  • The evaluation goes beyond question answering. Checking component-selection fidelity and subsequent interventions gives concrete evidence that the stored findings remain actionable. Separating this from the reliability of the original findings is also valuable.
Critical weaknesses
  • The added value of the proposed representation remains unclear. The main comparisons change organization, serialization and chunking together. The entity-chunked control helps, but does not isolate the five-role design. I am therefore unsure what MU offers beyond a carefully designed entity-based schema and retrieval pipeline, and I am unsure why the five factors selected matter for MechInterp reuse, either.
  • The causal experiments provide limited additional evidence for MU’s advantage.
Questions for authors
  • What concrete capability does the E/S/R/D/G organization provide beyond an entity-based schema that preserves the same measurements, provenance and intervention conditions?
1 comment
AuthorsOpenReview ↗

Added Value Beyond Entity Schemas, Scope of Causal Utility, and Future Evaluation, We address three points.

We thank the reviewer and clarify what MUs contribute beyond entity-centered organization and what our causal experiments establish.

1. Added value beyond an entity-based schema.

MUs are not merely a chunking strategy. They provide a shared intermediate representation (E/S/R/D/G) separating mechanistic evidence organization from architecture-specific realization and retrieval.

In Algorithm 1, Ba\mathcal{B}_a organizes findings into common informational roles; Ha\mathcal{H}_a constructs architecture-specific fields for downstream information requirements. The renderer creates retrievable passages (Appendix A). Fields can combine multiple roles depending on the downstream task.

Selected mappings are:

ArchitectureRealized fieldRoles
VAEsummaryE+S+R+D
VAEredundancy_notesR
VAEmanipulation_guideG
CNNsummaryE+S+D
CNNclass_responsesS+D
CNNtechnical_propertiesD
CNNrelationshipsR
GPT-2attention_behaviorD
GPT-2mechanistic_profileS+D
GPT-2intervention_findingsG

For example, CNN class_responses combines semantic associations (S) and measurements (D). This allows shared roles to be realized into different fields without identical schemas across architectures, while retaining native evidence, conditions, and provenance (Table 1; Appendix A).

These fields support natural-language queries about recorded mechanistic findings, as illustrated by our accessibility demo [1], provided with the original submission.

We do not claim E/S/R/D/G is uniquely necessary or optimal. Under BM25, MU outperforms Generic-EntityChunked (entity-grouped native records) on CNN (79.44% vs. 33.61%), GPT-2 (79.72% vs. 62.22%), and VAE (54.58% vs. 49.86%). One shared consumer recovered 23,008 measurements and reproduced 52 complete rankings across three architectures. Under matched traversal, MU also achieved higher answer accuracy than KG across all three. These results support interface usefulness, not isolated role superiority. Appendix F's fixed-evidence ablations assess role-content dependence; Native-EntityJoined and KG controls examine organization and retrieval.

2. Causal Utility and additional matched experiments.

MUs are bounded by upstream mechanistic findings. Causal Utility tests preservation of information needed for component selection and intervention through MU construction and realization, not improved intervention methods.

Reproducing native selections and intervention effects under equivalent complete evidence is the intended fidelity result, consistent with the Selection Fidelity and Intervention Effectiveness criteria defined in Section 3 of the submitted paper.

We compared MUs against Native-EntityJoined, which retains native measurements, conditions, provenance, and intervention operators without MU roles. The matched causal studies cover CNN and GPT-2 using separate evidence corpora.

EvaluationMUNative-EntityJoinedComplete reference
CNN complete selectionSame 8-filter sets for 10 classesSameSame
CNN bounded selection78/80 choices retained78/8080/80
CNN natural patching+14.17 pp+14.17 pp+14.46 pp
GPT-2 complete selectionSame 144-head ranking/top 7SameSame
GPT-2 bounded selectionSame 7 heads; 2/7 global top headsSame7/7
GPT-2 IOI recovery23.12%23.12%61.87%
GPT-2 evidence tokens1,5762,804—

CNN uses 50 of 128 filter passages; GPT-2 uses ten head passages. Complete reference is unrestricted Direct-Numeric. CNN effects are target-probability gains.

CNN's bounded representations give the same selections and effects, just below the complete reference. In GPT-2, both recover the same bounded policy, while MU uses 43.8% fewer evidence tokens, without improving coverage or causal effectiveness.

Complete-access comparisons reproduce native selections exactly. Thus, lower bounded IOI recovery reflects incomplete retrieval of preferred heads, not failure to preserve the tested selection information.

These results establish task-specific preservation of intervention-relevant information. Equal native/MU outcomes demonstrate fidelity, not improved causal effectiveness. An entity-centered alternative with equivalent evidence can achieve the same results.

3. Future downstream evaluation.

Future work could compare MUs, KGs, and native representations using identical evidence, evaluating human judgments of clarity and usefulness alongside LLM/SLM accuracy, grounding, and token efficiency. This would test downstream comprehensibility beyond the properties evaluated here.

We will include these clarifications and matched experiments in the revision.

Reference

[1] Manifestation Units — Accessibility Demo

Reviewer_XiF3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This manuscript aims at a persistent interface between mechanistic analysis and downstream use. Specifically, it asks how to commonly represent the findings from mechanistic analysis, despite that the findings can be of various forms due to heterogeneity of architectures, so that they can be accessed and reused for downstream tasks.

To address this question, Manifestation Units (MUs) are proposed as the persistent interface to represent a finding using five shared roles: Entity, Semantics, Relationships, Dynamics, and Guidance (E/S/R/D/G). This shared representation serves as the intermediate to map to architecture-specific fields for downstream uses.

The MU interface was evaluated through two properties (Structural Accessibility and Causal Utility) and over three forms (VAE latent channels on CelebA, CNN filters on CIFAR-10, and GPT-2 attention heads, with held-out functional evaluation on WikiText), showing improved performances against eight alternative presentation conditions.

Critical strengths
  1. The manuscript presents a combination of a new problem (representing findings between mechanistic analysis and downstream use) and a new solution (manifestation units for intermediate representation that is common and not architecture-specific), although the extent of these "new" contributions relative to related works can use much more convincing presentations.

  2. Numerically, MU showed robust performance gains over various finding forms specific to various architectures and over two types of tasks. The results showed that organization of the findings matter for both Selection Fidelity (preservation of the information required for selection) and Intervention Effectiveness (the causal effect of the retained selection) and provided common organization as a feasible approach to pursue such organization.

Critical weaknesses
  1. The extent of the "new" problem and the "new" solution is not convincingly presented yet. Especially for the approach.
  • MU builds on five shared roles (E/S/R/D/G), which provide a feasible solution for common organization beyond architecture-specific representations. But there is no optimality criteria or analysis to establish a potentially desired solution. In such a case it is hard to argue for the significance of the contribution with proposed roles for common organization. What is numerically learned from the current proposition does not clearly tell how to pursue improvement, due to the lack of theoretical / analytical motivations and the lack of numerical separation of E/S/R/D/G roles from other components of the pipeline.
  • Similar critique can be said about the fixed mapping to architecture-specific fields, which can also be deemed ad hoc for specific structures.
  1. More details are needed to establish a fair comparison between MU and the eight alternative presentations. The alternatives seem to be focused on unstructured and token-limited cases, which does not necessarily present a matched comparison to MU.
Questions for authors

Central questions are as in the two weaknesses above. Answers would help clarify the significance of the contributions and the results.

4 comments
AuthorsOpenReview ↗

Novelty of the Problem and Fairness of Comparisons, We clarify two points

We thank the reviewer and address methodological novelty and comparison fairness.

1. Novelty of the problem and proposed approach.

Mechanistic interpretability investigates model components, functions, relationships, and causal effects. Yet producing findings does not ensure downstream accessibility or reuse. Sharkey et al. (2025) [1] identify human interaction with model internals as an open research direction, while Kowalska and Kwaśnicka (2026) [2] discuss specialist effort and barriers to adoption beyond research.

We address a complementary post-analysis problem: how can heterogeneous mechanistic findings be persistently represented for downstream access and reuse?

MUs support natural-language questions about recorded component functions, relationships, measurements, and intervention guidance without requiring users to navigate the original analysis files. Intended consumers include medical researchers, biologists, AI safety researchers, and model developers. Our accessibility demo [3], provided with the original submission, illustrates this interaction.

Algorithm 1 separates common information semantics from architecture-specific realization. Ba\mathcal{B}_a organizes findings into E/S/R/D/G, while Ha\mathcal{H}_a constructs architecture-specific fields for downstream information requirements. Examples include generation_role in VAE, class_responses in CNN, and attention_behavior in GPT-2 (Table 1; Appendix A).

As stated in the submitted Introduction (lines 63–65), we do not claim these roles are the only possible or minimal decomposition, nor do we claim optimality. We evaluate Structural Accessibility and Causal Utility.

Fixed-evidence ablations (Appendix F, with additional E/G tests) assess dependence on role-specific content independently of retrieval. Entity-centered and KG controls examine organization and access policy. These distinguish pipeline aspects without fully isolating E/S/R/D/G or establishing an optimal partition.

2. Fairness of comparisons.

We agree that the original eight alternatives evaluate presentation-and-access pipelines rather than isolating the five-role structure. We therefore evaluated stronger structured alternatives.

Native entity organization. Generic-EntityChunked groups native records by entity within source files, without MU roles. Under common BM25, answer-slot accuracy was:

ModelGenericGeneric-EntityChunkedMU
VAE44.72%49.86%54.58%
CNN21.11%33.61%79.44%
GPT-254.17%62.22%79.72%

We additionally evaluated Native-EntityJoined, consolidating GPT-2 findings across files without MU roles. Under matched BM25, MU achieved 74.44% accuracy versus 70.00% for Native (+4.44 percentage points; 95% CI [−7.22, 16.67]), an uncertain difference. This evaluation uses a separately prepared retrieval representation; the 79.72% result belongs to the multi-presentation comparison.

On the 20-question cross-file subset, Native performed better under whole-question retrieval (65.00% versus 45.00%). This small subset highlights a limitation of the evaluated MU retrieval pipeline. Entity consolidation is not exclusive to MUs. In bounded GPT-2 causal selection, MU used 43.8% fewer evidence tokens than Native with identical selections.

Knowledge Graph (KG). We evaluated KGs constructed from native findings across VAE, CNN, and GPT-2 using common BM25 and matched entity-aware traversal with one-hop expansion.

ModelViewBM25 acc.BM25 recallTraversal acc.Traversal recall
VAEKG44.64%53.61%60.64%88.06%
VAEMU54.58%57.22%75.83%80.14%
CNNKG50.28%50.56%71.11%83.33%
CNNMU79.44%91.67%80.00%93.89%
GPT-2KG52.78%50.00%85.83%90.00%
GPT-2MU79.72%80.56%94.17%100.00%

Answer-slot accuracy and requested-evidence recall (%), at 8,000 tokens and ten passages.

Traversal improves both representations. Under matched traversal, MU achieves higher accuracy across all three architectures and higher recall for CNN and GPT-2, while KG achieves higher VAE recall.

These controls test entity consolidation and structured graph access rather than only serialization or chunking. They support effective MU accessibility while revealing conditions where alternatives perform better. They do not isolate the five roles from the entire pipeline.

We will include these comparisons and protocols in the revision.

References

[1] Sharkey et al. (2025), Open Problems in Mechanistic Interpretability, TMLR

[2] Kowalska & Kwaśnicka (2026), Unboxing the Black Box

[3] Manifestation Units — Accessibility Demo

I thank the authors for the well-structured responses with additional experiments. Table 3 is useful in ablating the five roles individually. The responses did not increase confidence in the contribution due to the still lacking theoretical or analytical perspective on what constitutes a feasible or ideal solution and due to a narrowed numerical gain compared to a stronger baseline Native-EntityJoined.

AuthorsOpenReview ↗

Clarification of Contribution and Analytical Perspective (1/2)

We thank the reviewer for the thoughtful engagement and for finding the role ablation (Table 3) useful.

Open problem we align with. Sharkey et al. (2025) [1] identify human interaction with model internals as an open problem, including making mechanisms understandable to non-experts (Introduction, lines 45-47; Section 6, lines 506-509). Findings are typically recorded in forms conditioned by the analyses that produced them, so evidence about one component is spread across heterogeneous outputs. Our contribution aligns with this direction at the level of the findings themselves: a persistent representation between analysis and downstream use, so findings can be queried in natural language and used for intervention without navigating the original outputs (Introduction, lines 47-52). Our accessibility demo [2], provided with the original submission, illustrates this interaction.

Contribution. MUs differ from component-level inspection and storage resources such as Neuronpedia [4] (Introduction, lines 53-54; Section 6): they organize findings from heterogeneous analyses under shared roles across architectures, retaining measurements, conditions, and provenance, and to our knowledge no published resource does so. Retrieval gains are not our contribution (Introduction, lines 85-97). Our contributions are the representation and its evaluation along two separate properties. In particular, a finding that is structurally accessible is not necessarily causally informative: recovering a finding does not ensure it carries what an intervention needs, and preserved evidence can still fail to reach the selector (Introduction, lines 73-78, following Makelov et al., 2025 [3]; Section 5, lines 486-488). For example, in bounded GPT-2 selection both MU and Native-EntityJoined expose the same ten eligible heads, yet only 2 of the complete-access top seven are retrieved, giving 23.12% normalized recovery versus 61.87% under complete access (Table 36).

Analytical perspective. We do not claim a theory of an ideal representation, nor that E/S/R/D/G is unique or minimal (Introduction, lines 63-65; Discussion, lines 524-528; Appendix F, lines 1906-1908). The paper's perspective is three separable questions: can a finding be found again (Structural Accessibility), does it still carry what a downstream procedure needs (Selection Fidelity), and does acting on it work (Intervention Effectiveness) (Section 7, lines 520-523 and 534-537). One consumer operated over three architectures with 0 edits (Appendix A.2, Tables 8-9; importers remain architecture-specific), and removing a role's content removes recovery on the tasks that depend on it (Table 3; S and R only modestly above random, lines 374-377).

References

[1] Sharkey et al. (2025), Open Problems in Mechanistic Interpretability, TMLR.

[2] Manifestation Units, Accessibility Demo

[3] Makelov, Lange & Nanda (2025), Towards principled evaluations of sparse autoencoders for interpretability and control, ICLR.

[4] Lin (2023), Neuronpedia

AuthorsOpenReview ↗

Clarification: Retrieval Controls and Our Claim (2/2)

Retrieval comparisons. We compared MUs against 11 alternatives accross 3 differents architectures, VAE, CNN and GPT-2: eight matched presentations (nine matched conditions with MU) and three supplementary controls (Generic-EntityChunked, a knowledge graph under BM25 and traversal, and Native-EntityJoined). Each uses 60 questions per architecture and four evidence budgets (shown at 8,000 tokens and ten passages, common BM25 unless stated; definitions in Table 10, Appendix A.6; full grids in Tables 11-13). Ans. = answer-slot accuracy (%), ER = requested-evidence recall (%). Standard windows: up to 1,000 tokens, 100-token overlap, within source files.

ConditionVAE Ans. / ERCNN Ans. / ERGPT-2 Ans. / ERDefinition (Table 10)
MU54.58 / 57.2279.44 / 91.6779.72 / 80.56Findings organized by entity and MU field; sections packed within one MU parent
Direct-Bounded44.81 / 59.9425.83 / 29.4449.17 / 49.44Pretty-printed native records; standard windows
Generic44.72 / 54.3921.11 / 32.2254.17 / 55.00Compact native records; standard windows
Natural-Prose32.22 / 48.6136.67 / 50.5653.06 / 58.96Native paths and values rendered as sentences
Unstructured39.89 / 54.6123.33 / 31.1155.83 / 49.44Single-line native text keeping keys, values, delimiters
ID-Normalized41.44 / 49.6116.67 / 18.8956.94 / 55.00Generic text, digit-only dictionary keys padded to at least three digits
Random-046.72 / 60.7218.06 / 31.1152.78 / 50.83Generic text cut into random 800-1,000-token windows (100-token overlap), seed 0
Random-144.47 / 58.1721.11 / 30.2752.22 / 49.72Same, seed 1
Random-245.06 / 59.6116.94 / 38.5749.44 / 51.11Same, seed 2
Generic-EntityChunked (suppl.)49.86 / 63.0633.61 / 56.1162.22 / 63.06Generic native text grouped by entity; keeps parent and file boundaries
KG, BM25 (suppl.)44.64 / 53.6150.28 / 50.5652.78 / 50.00Native findings linked by entity identifiers (cf. graph retrieval [1]), ranked by BM25
KG, traversal (suppl.)60.64 / 88.0671.11 / 83.3385.83 / 90.00Same graph; direct entity matches, then one-hop neighbors
MU, traversal (suppl.)75.83 / 80.1480.00 / 93.8994.17 / 100.00Same MU passages, same traversal rule; post hoc (Appendix B.4)

Supplementary readings. MU minus Generic-EntityChunked accuracy is +4.72 (VAE), +45.83 (CNN), +17.50 (GPT-2) pp; after Holm correction only CNN is clearly separated (Table 14), and on VAE EntityChunked has higher recall (63.06 vs 57.22; Table 15). MU-traversal minus KG-traversal accuracy is +15.19 / +8.89 / +8.34 pp, with paired tests only on GPT-2 (+8.33, CI [4.17, 13.33]; Table 18); KG-traversal has higher VAE recall.

Native-EntityJoined (GPT-2 only; Table 16, Appendix B.3) consolidates the same native findings across analysis files by qualified entity identifier, keeping original keys, measurements, conditions, and sources, but without MU roles or realized MU fields (Appendix A.5). It thus offers entity-level organization similar to MUs; the comparison covers the full pipeline, not the five roles alone (Section 4.1, lines 317-319). Its evidence is prepared separately, so scores are not pooled with the grid above:

SubsetNative Ans. / ERMU Ans. / ERMU - Native (95% CI)
All 60 questions70.00 / 72.1374.44 / 76.01+4.44 [-7.22, 16.67] / +3.88 [-7.78, 16.11]
20 cross-file questions65.00 / 68.3345.00 / 46.67-20.00 [-34.17, -5.83] / -21.67 [-35.00, -8.33]

Our claim. These are controls, not our contribution. MUs have the highest observed accuracy among the nine matched conditions (MU and eight alternatives) on all three architectures, although recall does not always follow accuracy (VAE; Appendix B.1).. The gain over entity-preserving chunking is large on CNN, moderate on GPT-2, and small on VAE, and Native-EntityJoined shows parity overall and a loss on cross-file questions. We do not claim that roles improve retrieval: with identical evidence, role grouping adds +0.28 pp (95% CI [-1.67, 2.50]) (Appendix B.5), and entity-preserving alternatives may offer the same organization and causal results (Appendix A.1.1; Sections 5.1-5.2). In bounded GPT-2 selection, MUs use 43.8% fewer evidence tokens with identical selections (Table 36).

We would be very grateful, and happy, to run any additional experiment the reviewer finds useful. Given the core contribution, a persistent representation of heterogeneous findings accross three different architectures evaluated separately for access and causal utility, we would also be grateful if the reviewer would consider reconsidering their score.

References

[1] Edge et al. (2024), From Local to Global: A Graph RAG Approach to Query-Focused Summarization

Reviewer_ck8kOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces and implements “Manifestation Units” (MU) storing mechanistic findings from different models. For instance, there is a previous finding about a specific latent channel in VAE related to facial construction, and there is another finding relating a specific filter in CNN to airplane recognition. Then MU proposed by this paper can store these findings about two different models in a common structure and preserve model-specific evidence. Through experiments, this paper demonstrates that MU improves finding recovery. This paper also shows that the stored findings in MU can be used for model interventions.

Critical strengths
  1. The presentation of this paper is clear.

  2. The experiment design and metrics are clearly defined. For instance, the “requested-evidence recall” and “answer-slot accuracy” answers different questions: the ability of retrieving relevant context and the ability to retrieve the stored value in an accurate way.

Critical weaknesses
  1. I’m concerned about the methodological novelty of this paper. The main contribution of this paper is storing information in a structural way and a retrieval framework built on existing techniques. I wonder how this paper demonstrates methodological advances beyond traditional structured data storage and retrieval pipelines.

  2. There seems to be insufficient motivation for ”storing findings of different models in a unified way”. This paper does not demonstrate scenarios where a unified representation enables meaningful interaction or reuse of mechanistic findings across different models. For instance, if there is a pipeline with VAE and CNN, where VAE first generates airplane images and then CNN later predicts whether a generated image can be used for business commercials. In this scenario, I would imagine storing the importance mechanism of a VAE channel and CNN filter on “windows of airplane” in a unified way can help, because intervene on one of the VAE channels may further enhance/affect a downstream filter in CNN. However, the experiments on VAE, CNN and GPT-2 from this paper are conducted independently, making the benefits of cross-architecture unification unclear. It would be great if author(s) of this paper can elaborate on this point

  3. The decomposition of five MU roles are not justified. The choice of the E, S, R, D, G roles seem to be heuristic. This paper does not provide theoretical or empirical justification for why this particular combination of components is helpful. For instance, why is there not an additional component called Liminations (L) specifying whether this mechanism (say, a CNN filter) is also sensitive to training data imbalance? Additional ablations would help establish the necessity and value of each of the five components.

Questions for authors

See Weakness 1 to 3.

1 comment
AuthorsOpenReview ↗

Novelty, Cross-Architecture Motivation, and Choice of Roles We clarify three points.

We thank the reviewer for the comments and clarify three aspects of our contribution.

1. Methodological novelty. Our contribution is a persistent, entity-centered intermediate representation of heterogeneous mechanistic findings. Different mechanistic interpretability analyses characterize the same component through distinct measurements, relationships, and experimental conditions.

As formalized in Algorithm 1, upstream analysis (Aa\mathcal{A}_a) produces findings, MU construction (Ba\mathcal{B}_a) organizes them into shared E/S/R/D/G informational roles, and architecture-specific realization (Ha\mathcal{H}_a) constructs fields for downstream information requirements. The representation retains numerical evidence, conditions, and provenance. Rendering these fields for retrieval is a subsequent operation (Appendix A).

We evaluate evidence recovery (Structural Accessibility) and preservation of intervention-relevant information (Causal Utility).

Component-centered resources such as OpenAI Microscope and Neuronpedia already exist. Our focus is the common representation and reuse of heterogeneous findings. In CNN, MU-guided amplification achieves 92.37% targeted success versus 3.46% to 8.87% for matched random filters (App. D.5), showing that retained evidence supports targeted intervention, not that MUs improve the method. Retrieval uses conventional BM25: MU exceeds Generic-EntityChunked (entity-grouped native records) by 4.7 to 45.8 pp in accuracy, without isolating the five roles.

2. Cross-architecture motivation. We acknowledge that our wording may have suggested cross-model causal interaction. Our objective is a common representation applicable independently across architectures.

Entities correspond to VAE latent dimensions, CNN layer–filter pairs, and GPT-2 attention heads (Table 1). Shared roles are realized into architecture-specific fields, including generation_role and redundancy_notes in VAE, class_responses and technical_properties in CNN, and attention_behavior and mechanistic_profile in GPT-2 (Appendix A).

We additionally evaluated one shared consumer:

ModelEntitiesMeasurementsRankingsConsumer edits
VAE51220,480400
CNN2242,240100
GPT-214428820
Total88023,008520

The consumer performed entity lookup, conditioned measurement retrieval, relationship traversal, and provenance recovery. It reproduced complete within-model rankings against native analysis outputs (40 VAE attributes, 10 CNN classes, two GPT-2 attention measures). GPT-2 provides a smaller-scale validation. Architecture-specific importers were required, but consumer logic remained unchanged.

This demonstrates common downstream evidence operations, not cross-model causal integration. The suggested VAE–CNN pipeline would require additional cross-model analysis.

3. Choice of E/S/R/D/G. The roles reflect recurring mechanistic evidence: Network Dissection examines components, semantic associations, and measured responses (E/S/D) [1]; circuit analysis examines features and relationships (S/R) [2]; and GAN Dissection studies interventions (G) [3]. Mueller et al. (2026) [4] discuss causal units at different abstraction levels, supporting analysis-dependent component granularity.

Table 1 and Appendix A describe their shared meanings and architecture-specific realizations. We do not claim a unique or optimal decomposition; entity consolidation alone is not MU-exclusive.

The submitted Appendix F contains fixed-evidence ablations for D/S/R. We additionally evaluated E and G. E tasks select heads belonging to a specified layer; G tasks use recorded intervention-severity labels.

RemovedTaskNFull MUAblated
EEntity495.00%0.00%
SSemantics860.00%0.00%
RRelationships465.00%0.00%
DAttention1286.67%18.33%
GGuidance4100.00%10.00%

Values are mean Recall@5 over N selection tasks. Additional E/G protocols will appear in the revised Appendix F. These tests establish dependence on retained role content, not decomposition optimality.

Regarding Limitations (L), a CNN filter response measured under imbalanced training data could retain that condition in D, while intervention constraints could be recorded in G, without requiring a separate role.

We will clarify the mappings and include the additional experiments in the revision.

References

[1] Bau et al. (2017), Network Dissection

[2] Olah et al. (2020), Zoom In

[3] Bau et al. (2019), GAN Dissection

[4] Mueller et al. (2026), The Quest for the Right Mediator

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of revisions (1/2)

We thank all reviewers for their constructive feedback. The revised PDF is uploaded; Part A summarizes the new experiments and clarifications prompted by the reviews; Part B gives the location and content of each change.

A. Experiments and clarifications prompted by the reviews

New experiments

  1. Test the five roles directly (ck8k, XiF3, M6rd). We ablated all five roles, adding E and G to the submission's D/S/R, and ran a control that gives role-grouped and source-grouped views identical evidence.
  2. Compare against stronger baselines (XiF3, M6rd). Beyond the serialization and chunking baselines of the submission, we added structured alternatives that organize evidence by entity without the MU roles: knowledge-graph baselines under common BM25 and matched one-hop traversal, and a Native-EntityJoined condition that consolidates native findings across files.
  3. Check what the causal results add (M6rd, ck8k). We ran matched Native-EntityJoined controls for the CNN and GPT-2 causal studies, a seven-file CNN study, and a bounded-access GPT-2 comparison.
  4. Test a common consumer across architectures (ck8k). One consumer performed the same evidence operations over all three architectures with no consumer edits.

Clarifications

  1. Problem and motivation, from the literature (ck8k, XiF3). Sharkey et al. (2025) identify human interaction with model internals as an open problem, and Kowalska & Kwasnicka (2026) note that interpretation still demands specialist effort and sees limited use beyond research. Work on defining, finding and evaluating mechanisms (Geiger et al., 2025; Mueller et al., 2025; 2026) addresses how mechanisms are produced and compared, not how findings are carried forward for later access and reuse. This is the question we focus on.
  2. Basis for the five roles (ck8k, XiF3, M6rd). The roles synthesize evidence forms that recur in prior work: Network Dissection (Bau et al., 2017; E/S/D), circuit analysis (Olah et al., 2020; S/R) and GAN Dissection (Bau et al., 2019; G). We state that this is a synthesis, not a unique or minimal decomposition.
  3. Scope of the claims (ck8k, M6rd). The causal results show preservation of intervention-relevant information, not an improved intervention method, and the representation is applied independently within each architecture.
AuthorsOpenReview ↗

Summary of revisions (2/2)

B. Changes to the paper (location and content)

ChangeWhereHow
Motivation and problemSec. 1, 6Introduction and Related Work cite Sharkey et al. and Kowalska & Kwasnicka; Neuronpedia and similar tools (Lin, 2023) are positioned as storage and inspection, MU as the representation of findings for reuse
Basis for the rolesSec. 2.1, Table 1New paragraphs "Relation to prior work and evidence for the roles" and "Choosing the entity level" (Mueller et al., 2026; Locatello et al., 2019; Leask et al., 2025)
Role experimentsTable 3, App. F, B.5E 95.00 to 0.00, S 60.00 to 0.00, R 65.00 to 0.00, D 86.67 to 18.33, G 100.00 to 10.00 when the role is removed (random-selection level 50.00, 50.00, 58.33, 50.00, 50.00; the zeros are valid abstentions). With identical evidence, role grouping gives no measurable benefit (+0.28 pp, 95% CI [-1.67, 2.50])
Stronger baselinesTable 2, App. BGeneric-EntityChunked (already in App. B.2) moved to the main table. With matched traversal MU has higher accuracy on all three models (75.83 vs 60.64, 80.00 vs 71.11, 94.17 vs 85.83), the graph higher VAE recall (88.06 vs 80.14). GPT-2 Native-EntityJoined: 74.44 vs 70.00 (CI [-7.22, 16.67]); Native higher on the 20 cross-file questions (65.00 vs 45.00)
Causal UtilitySec. 5, App. D, E, GCNN: MU 92.37% targeted success vs 92.54% Direct-Complete and 3.46% to 8.87% matched random; entity-joined native gives the same selections and effects (78/80; +14.17 vs +14.46 pp). GPT-2: 61.87% vs 6.30% (+55.57 pp); at a ten-passage bound identical policies, MU 43.8% fewer tokens
Cross-architecture scopeSec. 1, App. A.2Independent per architecture; no cross-model causal integration. One consumer: 880 components, 23,008 measurements, 52 rankings

References (for both comments)

  • Bau, D., et al. (2017). Network dissection: Quantifying interpretability of deep visual representations. CVPR.
  • Bau, D., et al. (2019). GAN dissection: Visualizing and understanding generative adversarial networks. ICLR.
  • Geiger, A., et al. (2025). Causal abstraction: A theoretical foundation for mechanistic interpretability. JMLR 26(83):1-64.
  • Kowalska, B., Kwasnicka, H. (2026). Unboxing the black box: A survey on mechanistic interpretability for algorithmic understanding of neural networks. Machine Learning 115:190. doi:10.1007/s10994-026-07129-4.
  • Leask, P., et al. (2025). Sparse autoencoders do not find canonical units of analysis. ICLR.
  • Lin, J. (2023). Neuronpedia: Interactive reference and tooling for analyzing neural networks. https://www.neuronpedia.org
  • Locatello, F., et al. (2019). Challenging common assumptions in the unsupervised learning of disentangled representations. ICML, PMLR 97:4114-4124.
  • Mueller, A., et al. (2025). MIB: A mechanistic interpretability benchmark. ICML, PMLR 267:45069-45108.
  • Mueller, A., et al. (2026). The quest for the right mediator: Surveying mechanistic interpretability for NLP through the lens of causal mediation analysis. Computational Linguistics 52(1):331-378. doi:10.1162/coli.a.572.
  • Olah, C., et al. (2020). Zoom in: An introduction to circuits. Distill. doi:10.23915/distill.00024.001.
  • Sharkey, L., et al. (2025). Open problems in mechanistic interpretability. TMLR. https://openreview.net/forum?id=91H76m9Z94

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.