acceptodds
Under review as a conference paper at ICLR 2027

Typed Mechanism Libraries: Organized Routing and Weak Edit Response

Abstract

Does organizing a model's internal state make that state behaviorally editable? We study this question in transformers augmented with a typed mechanism library: an addressable collection of slots with structural payloads and a reversible edit interface. Across 22.6M and 125M configurations, type supervision increases the association between evidence types and routing destinations. At 125M, a replication on fresh training seeds and worlds yields permutation-centered mutual information of 0.1025–0.1275 nats, compared with 0.0471–0.0573 for two unsupervised controls, and every paired contrast against these controls exceeds five permutation standard deviations. Yet structural edits produce little numerical response: the readout change is at most in the 125M evaluation, with zero observed collateral change. Language-modeling loss stays within 0.01 nats of a dense baseline, and rollback recovers the saved state exactly. These results separate routing organization, state-level editability, and behavioral control, and an analysis of the architecture shows why they can come apart: when routing ignores payloads, organization is invariant to payload edits, and a null edit response arises either because the edited fields are not read or because the readout does not use them. Evaluations of explicit-state models should therefore report output sensitivity alongside internal organization and calibrate organization claims against unsupervised controls in each configuration.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, causal reasoning, typed supervision, mechanism library, routing/readout boundary, preregistration

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_4ceEOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper adds an explicit editable memory to small transformer model, that is trained on synthetic "causal-reasoning" tasks. The memory is a set of slots that store small pieces of a structure, eg graph edges, and a router decides which slot each input token goes to. The memory can be edited directly. One of findings is that editing the stored structure, eg flipping signs, doesn't change the model's answers (a negative result).

Critical strengths

none

Critical weaknesses

The paper is difficult to follow, there’s a lot of invented jargon. For example: 'evidence type' (Sec. 3.1, where the six categories are listed but not defined), 'library-mediated features', 'block-structured routing prior', 'library invariants', and so on. None of these is defined precisely. The paper never says how the payload fields enter the forward pass, or what ŷ is (probability, logit?). Basic setup is also missing -- what a synthetic world and a query look like, what the 'behavioral battery' is, what the held-out probe measures.

The main finding is not explained. In Sec. 5, the authors give two possible reasons of why edits do nothing, but neither is tested.

Questions for authors

none

Reviewer_cyiAOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

A transformer with an extra "mechanism library": 200 slots (22.6M model) or 600 slots (125M model), each holding a payload (edge list, sign bits, scalars), a router that sends tokens to slots, and an edit interface (sign flip, edge add/remove/swap, small parameter update) that snapshots and restores the library. Trained on synthetic causal worlds. Claims: (1) adding a type-classification loss (weight 0.1) raises mutual information between slot and evidence type relative to two unsupervised controls (125M replication: 0.1025–0.1275 nats vs 0.047 and 0.057, paired permutation z∆ of 5–7.9); (2) sign-flip edits change the answer readout by at most 3.4e-6 with zero collateral, while restoration is bit exact and LM loss is within 0.01 nats of a dense model; (3) an analysis (Eq. 4) saying organization is invariant to payload edits and a null edit response means either the fields aren't read or the readout ignores them.

Critical strengths
  • Shows that even the unsupervised controls get nonzero MI that changes between collections, so raw MI without a matched control is meaningless. Fair point, though specific to this MI estimator.
  • Restoration is tested thoroughly (250 single restores, 1000 depth-20 rollbacks, 1250 backbone checks per seed, all exact). But that is a software property of snapshot/restore, not a scientific result.
Critical weaknesses
  • The main negative result is undiagnosed and trivial. The abstract says organization, editability and control "come apart", but control fails because the answer doesn't use the payload, which is how this model was built and trained. No variant where the answer must depend on payloads (edit-consistency loss, payload-dependent routing) is tried, so we don't know if the gap can close or is intrinsic. "Explicit, organized, reversible state is not by itself a lever on behavior" is shown for one architecture and nothing more.
  • Tiny scope with big framing. Synthetic worlds, one family, two sizes where parameters, slots and types all change together, 3 seeds, no real LM, no CounterFact/zsRE, even though the intro and related work are all about ROME/MEMIT/MEND. The closing recommendations for "evaluations of explicit-state models" are not supported by one toy setup. Also the AI statement says AI helped derive the factorization, which fits how thin it is.
Questions for authors
  1. Does the forward pass read the edited fields at all? Report answer change when payloads are zeroed or randomized, and ∂ψ/∂L. This is the main thing missing.
  2. If you train so that the answer depends on payloads, does organization still increase and does edit response become nonzero?
  3. How was the "designated control" chosen? Blocks s2 would have failed your criterion for s4.
  4. (Suggestion) Same measures on one real memory-layer LM.
Reviewer_WR7WOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper explores whether well-organized, addressable, editable explicit states (slots in the mechanism library) that are plugged into LLMs can be adjusted to control the answer. Synthetic causal data are used to train a transformer with typed routers and mechanism libraries, where the slots store mechanism-related structural information and numeric parameters. The authors conduct two kinds of experiments: directly editing the structural records; updating some parameters via gradient descent. Empirical results indicate that type-supervised routing can enhance the slot-type association, while editing the structural states barely changes the answer, and parameter update shows limited generalizability.

Critical strengths
  1. The routing result is checked against trained unsupervised baselines, and then repeated on new seeds and new worlds, which provides evidence that type supervision drives the organization.
  2. The paper reports state organization, state writability and recovery, and answer change as separate measurements, and specifies the evidential scope of each.
Critical weaknesses
  1. The paper lacks many implementation details, which weakens both the validity of its central claims and the reproducibility of its results. It does not state: (1) how a slot's structural fields are represented and read; (2) how the hidden state produces a routing distribution, including the Gumbel routing, the probability floor and the Blocks prior; (3) how the library is combined with the backbone transformer to produce the answer; (4) how the synthetic worlds are generated.
  2. The results of this paper do not establish the effectiveness of the proposed method: structural edits leave the answer essentially unchanged, with a reported change near zero (upper bound of 3.4×10−63.4\times10^{-6}); parameter updates are effective on the training points (0.884/0.935/0.917) but transfer only weakly to new samples (0.217). Also, the paper neither analyzes why the method fails nor attempts a repair.
Questions for authors
  1. Could you specify the computations that the four points above leave open: how a slot's structural fields are represented and read, how the hidden state produces the routing distribution (including the Gumbel routing, the probability floor and the Blocks prior), how the library is combined with the backbone transformer to produce the answer, and how the synthetic worlds are generated?
  2. Could you identify why structural edits have near zero effect? For the same input and matched routing randomness, what happens to the stored payload, the library features and the answer when a field is flipped, and is there a positive control showing that these fields can influence the answer when changed in a way the model must use?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.