- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces Manifestation Units (MUs), a structured interface for storing and reusing mechanistic interpretability findings. It organizes findings through five shared roles, with architecture-specific implementations. Experiments on VAE, CNN and GPT-2 test whether this organization improves retrieval and question answering, and whether it preserves the information needed for component selection and intervention.
Critical strengths
- The paper addresses a useful problem beyond mechanism discovery. The research perspective on mechanistic interpretability is interesting and meaningful, making interpretability results easier to reuse.
- The evaluation goes beyond question answering. Checking component-selection fidelity and subsequent interventions gives concrete evidence that the stored findings remain actionable. Separating this from the reliability of the original findings is also valuable.
Critical weaknesses
- The added value of the proposed representation remains unclear. The main comparisons change organization, serialization and chunking together. The entity-chunked control helps, but does not isolate the five-role design. I am therefore unsure what MU offers beyond a carefully designed entity-based schema and retrieval pipeline, and I am unsure why the five factors selected matter for MechInterp reuse, either.
- The causal experiments provide limited additional evidence for MU’s advantage.
Questions for authors
- What concrete capability does the E/S/R/D/G organization provide beyond an entity-based schema that preserves the same measurements, provenance and intervention conditions?
Added Value Beyond Entity Schemas, Scope of Causal Utility, and Future Evaluation, We address three points.
We thank the reviewer and clarify what MUs contribute beyond entity-centered organization and what our causal experiments establish.
1. Added value beyond an entity-based schema.
MUs are not merely a chunking strategy. They provide a shared intermediate representation (E/S/R/D/G) separating mechanistic evidence organization from architecture-specific realization and retrieval.
In Algorithm 1, Ba organizes findings into common informational roles; Ha constructs architecture-specific fields for downstream information requirements. The renderer creates retrievable passages (Appendix A). Fields can combine multiple roles depending on the downstream task.
Selected mappings are:
summaryredundancy_notesmanipulation_guidesummaryclass_responsestechnical_propertiesrelationshipsattention_behaviormechanistic_profileintervention_findingsFor example, CNN
class_responsescombines semantic associations (S) and measurements (D). This allows shared roles to be realized into different fields without identical schemas across architectures, while retaining native evidence, conditions, and provenance (Table 1; Appendix A).These fields support natural-language queries about recorded mechanistic findings, as illustrated by our accessibility demo [1], provided with the original submission.
We do not claim E/S/R/D/G is uniquely necessary or optimal. Under BM25, MU outperforms Generic-EntityChunked (entity-grouped native records) on CNN (79.44% vs. 33.61%), GPT-2 (79.72% vs. 62.22%), and VAE (54.58% vs. 49.86%). One shared consumer recovered 23,008 measurements and reproduced 52 complete rankings across three architectures. Under matched traversal, MU also achieved higher answer accuracy than KG across all three. These results support interface usefulness, not isolated role superiority. Appendix F's fixed-evidence ablations assess role-content dependence; Native-EntityJoined and KG controls examine organization and retrieval.
2. Causal Utility and additional matched experiments.
MUs are bounded by upstream mechanistic findings. Causal Utility tests preservation of information needed for component selection and intervention through MU construction and realization, not improved intervention methods.
Reproducing native selections and intervention effects under equivalent complete evidence is the intended fidelity result, consistent with the Selection Fidelity and Intervention Effectiveness criteria defined in Section 3 of the submitted paper.
We compared MUs against Native-EntityJoined, which retains native measurements, conditions, provenance, and intervention operators without MU roles. The matched causal studies cover CNN and GPT-2 using separate evidence corpora.
CNN uses 50 of 128 filter passages; GPT-2 uses ten head passages. Complete reference is unrestricted Direct-Numeric. CNN effects are target-probability gains.
CNN's bounded representations give the same selections and effects, just below the complete reference. In GPT-2, both recover the same bounded policy, while MU uses 43.8% fewer evidence tokens, without improving coverage or causal effectiveness.
Complete-access comparisons reproduce native selections exactly. Thus, lower bounded IOI recovery reflects incomplete retrieval of preferred heads, not failure to preserve the tested selection information.
These results establish task-specific preservation of intervention-relevant information. Equal native/MU outcomes demonstrate fidelity, not improved causal effectiveness. An entity-centered alternative with equivalent evidence can achieve the same results.
3. Future downstream evaluation.
Future work could compare MUs, KGs, and native representations using identical evidence, evaluating human judgments of clarity and usefulness alongside LLM/SLM accuracy, grounding, and token efficiency. This would test downstream comprehensibility beyond the properties evaluated here.
We will include these clarifications and matched experiments in the revision.
Reference
[1] Manifestation Units — Accessibility Demo