acceptodds
Withdrawn from ICLR 2027

Mask Faithfulness Does Not Guarantee Selective Control

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

A parameter decomposition writes a weight matrix as a sum of rank-one components with learned gates. Its standard test is mask faithfulness. The network is expected to behave the same under every mask allowed by the gates. We explore a different question. Can a faithful decomposition erase one behavior while preserving others? We answer this exactly on a controlled task with independent, always-active behaviors. For every and any capacity , we show that every faithful decomposition with the fewest active gates pairs coordinates. This is because a paired read cancels on half the inputs, so it appears sparser than a single coordinate. Erasing one behavior then raises another behavior's error by at least (MSE), for any real coefficients, whereas the coordinate basis erases with no damage. The sparsity advantage that pairing creates depends on the input data. An input that activates only one behavior activates both the sum and difference reads in a pair, but only one coordinate read. If single-behavior inputs become frequent enough, the coordinate basis is the optimal choice. A second theorem extends this bound to smooth targets, small input noise, continuous gates, and a finite penalty. It holds for every -approximate optimum. In three small trained networks, the same change of training data cuts the provable cost of selective edits by more than . Therefore, selective control should be judged separately from masked reconstruction.

Primary area:
interpretability and explainable AI
Keywords:
parameter decomposition, mask faithfulness, selective control, behavior erasure, sparsity, mechanistic interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_yJtaOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The manuscript is exceptionally difficult to read. This goes well beyond minor language or proofreading issues: the exposition is unclear and poorly structured, so I struggled to understand what the paper was about, the methodology, assumptions, and claims being made. For instance, both the abstract and introduction put apparent no effort to contextualize the contribution of the paper. Also, core concepts of the paper are introduced abruptly, like Theorem 1 and most of the terminology. Given these premises, it was not possible for me to formulate any constructive feedback about strengths and weaknesses within a reasonable time and effort.

Critical strengths

see summary

Critical weaknesses

see summary

Questions for authors

see summary

Reviewer_GEmnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors shows that faithful parameter decompositions do not need to allow erasing one behavior without damaging others. On a sign-input task with target IdI_d, every optimum of all-mask faithfulness plus gate minimality pairs coordinates, so any erasing edit costs at least 1/81/8 MSE on another behavior; changing only the fitting data removes the obstruction.

Critical strengths
  1. What I find strongest is that the result covers every optimum, for all d≥2d \geq 2 and C≥dC \geq d. This makes it hard to dismiss the failure as an optimization issue or a lack of capacity. The proof is short, and the reason for the failure is easy to follow: a paired read ej+seke_j + s e_k cancels on half the co-active inputs, so the objective counts it as sparser than a coordinate read. I also like the distribution intervention in Proposition 1. It gives an exact transition at p∗=d/(d+2)p^* = d/(d+2) while keeping the target, mean, and covariance fixed, which makes the role of the fitting distribution quite clear.

 

  1. I also find the way the authors establish edit failure convincing. The collateral bounds apply to every coefficient edit in the specified box, so the result does not depend on whether an edit search happened to find a good solution. Computing these bounds from the saved dictionaries, checking them with exact or interval arithmetic, and retaining all training starts gives me more confidence in the experiments. The criteria set in advance and the preregistered confirmation runs help as well. I see the certification procedure as a useful contribution in its own right, potentially beyond this particular example.
Critical weaknesses
  1. My main reservation is that I do not yet see how often this obstruction should arise in the settings where PD is actually used. The exact result relies on cancellation between sign-valued, always-active inputs. The noisy extension only gives a nontrivial bound below roughly σ=3×10−3\sigma = 3 \times 10^{-3} in the regime examined. And the authors' own sparse-activity calculation in Remark 1 shows that, under independent activation, pairing only wins when π>2/3\pi > 2/3. That is far from the sparse regime motivating PD. The experiments do not really close this gap for me: they stop at d≤4d \leq 4 in the controlled tasks and C=2C=2 in the learned MLPs, with no evaluation on the standard TMS or residual-MLP benchmarks, or on a real model layer. At (d,C)=(4,8)(d,C)=(4,8), the cross-entropy bound is already inconclusive, although the MSE bound remains positive. I also find the “ordinary” data condition in Section 6 a limited test: the factors are always active or inactive together, so their being merged is not especially surprising.

 

  1. I am less convinced by the novelty of the broader message than by the exact theorem. Sparsity objectives merging co-active features seems closely related to the absorption and non-identifiability results the paper discusses. Once faithfulness has been reduced to a linear constraint in the first proof step, the characterization of optimal reads is essentially an L0L_0 statement about rank-one factorizations of IdI_d on sign data. I would like the paper to be clearer about what this adds conceptually, and about who has actually claimed that mask faithfulness should imply selective control. To me, the basic distinction deserves a more direct explanation: faithfulness concerns removing components that are inactive on a particular input, whereas a coefficient edit removes a component globally, including on inputs where it is active. The dense writing and repeated qualifications made it harder for me to extract that point.
Questions for authors
  1. I wonder whether co-activation, rather than marginal density, is the more relevant condition here. For two behaviors, suppose b=P(both active)b=P(\text{both active}) and s=P(only j active)=P(only k active)s=P(\text{only }j\text{ active})=P(\text{only }k\text{ active}), retaining the sign-valued inputs. I get an expected gate count of 2s+2b2s+2b for coordinate reads and 4s+b4s+b for paired reads. Pairing then wins exactly when P(k active∣j active)>2/3P(k\text{ active}\mid j\text{ active})>2/3, even if both behaviors are marginally sparse; independence recovers your calculation in Remark 1. Is this reasoning correct? A sparse example with hierarchical correlations would help me judge whether the obstruction extends to the regime of interest. I would also like to understand what happens with continuous magnitudes, where exact cancellation is lost: does the objective still favor mixing, leave a rotational tie within the co-active subspace, or recover the native basis?

 

  1. Could you explain what drives the pairing in the trained experiments at σ∈{0.04,0.1}\sigma\in\{0.04,0.1\}? These noise levels are outside the range where the reported Theorem 2 bound is informative. Is near-cancellation, together with the finite penalty and leaky gates, enough to explain it? My main uncertainty is how much the result tells us about practical use. Can the certification procedure be applied to SPD dictionaries fitted on the standard TMS or compressed-computation benchmarks, or to a real model layer? Evidence in one of those settings would materially affect my assessment.
Reviewer_g4RvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether mask faithfulness in parameter decomposition — the property that a network behaves identically under every mask permitted by learned gates — implies the ability to selectively erase one behavior while preserving others. On a controlled task with independent, always-active behaviors, the authors prove that every globally optimal sparse faithful decomposition pairs coordinates together, so erasing one behavior necessarily damages its partner. They further show that changing the input data distribution can restore the natural coordinate basis and recover selective control, and support the theory with certified bounds on small trained networks. The overall message is that masked reconstruction quality and selective control are separate criteria that must be evaluated independently.

Critical strengths
  • Theorem 1 is not just a counterexample: it characterizes every global optimum of the faithfulness-plus-sparsity objective at arbitrary capacity C ≥ d, which is stronger than most existence-style impossibility claims in this literature. The message that masked reconstruction quality and selective control are distinct evaluation axes is useful for the parameter-decomposition community.
  • The bounds for trained dictionaries (Eq. 8, Eq. 10) are proved from saved matrices alone rather than estimated by search; the 72-run audit with exact rational arithmetic, interval-arithmetic verification of witnesses, and a replayable anonymous supplement is commendably careful and raises the reproducibility bar.
Critical weaknesses
  • The identity matrix target with d independent coordinate behaviors, uniform sign inputs where every behavior is always active, and the odd-symmetry condition ϕ(t) − ϕ(−t) = t are all engineered to make pairing optimal. Real networks have correlated, sparsely active behaviors, non-identity targets, and shared computation; the paper offers no evidence that the obstruction arises outside the constructed setting. The authors acknowledge this partially (Scope, Limitations) but the paper's framing ("mask faithfulness does not guarantee selective control") reads as a general claim while the evidence is a toy.
  • All experiments use d ∈ {2, 4}, C ≤ 8, and 2→32→32→2 MLPs with two factors. The learned-network study covers a single weight matrix at one site with C = 2. The paper itself admits the bounds "weaken with spare capacity" and are inconclusive at (d,C) = (4,8) — precisely the regime (many components, large capacity) where practitioners would care. Whether anything like pairing survives in LLM-scale decompositions (the motivation given for the work) is entirely open.
  • Objective (4) is the γ→0 limit of the minimality loss; Theorem 2 is informative only for noise σ below ~3×10⁻³, averaged over targets, with exact-sum atoms and a population all-mask loss that omits the frequency term used in actual SPD/VPD training. The paper concedes it does not show the full VPD objective fails or that PGD/SGD attains optimality — so the theorems do not directly indict any method as actually trained.
  • Feature absorption, composition, orthogonality blindness, and non-identifiability of learned dictionaries are well documented (Bussmann et al., Chanin et al., Paulo & Belrose, Korznikov et al.); the closest technical note (reuellee, 2026) already contains an exact crossover analysis. The contribution over this prior art — "all optima are inseparable on this task" plus certified bounds — is incremental in conceptual terms, even if the proof craft is solid.
Questions for authors
  • Does any analogue of the pairing obstruction arise when behaviors are sparsely or correlatedly active, i.e., the regime superposition toy models identify as realistic? Remark 1 suggests density is essential — does the phenomenon vanish entirely under sparse activation, or merely attenuate?
  • What happens under the actual SPD/VPD training objective (finite γ, frequency penalty, minibatch SGD)? Even a small demonstration that trained dictionaries land in the pairing regime would substantially strengthen the practical relevance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.