acceptodds
Under review as a conference paper at ICLR 2027

A Principled Decomposition of Explanation Failure in Routed Models

Abstract

Feature scores in routed models can disagree with the model's response even when every prediction term is exposed. We decompose this failure for frozen-route gradient-times-input scores under a declared replacement into reference-origin mismatch, finite-step expert approximation, and omitted routing response. The exact decomposition connects each term to a correction or control. Centering removes origin mismatch and affine experts remove expert approximation. Routing can still reverse the response without a support switch. Across ten tabular datasets, five seeds, and four routed predictors, centering increases joint donor-replacement probability drops in 199/200 runs. Complete differentiation improves numerical-field sign agreement in 377/400 checkpoint–reference pairs, but finite-step errors remain. This motivates routing-derivative priorities for budgeted endpoint checks. An original-author SENN reproduction on three datasets extends the coefficient–response distinction to native explanations. Complete derivatives improve response agreement, while a logit-aligned training control incurs a predictive cost. The framework identifies which failures each correction addresses and why local agreement does not guarantee finite-response fidelity.

Primary area:
interpretability and explainable AI
Keywords:
Routed models, Feature attribution, Explanation fidelity, Routing response, Interpretable machine learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zxo1OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the feature attribution problem in the context of expert mixtures, with the particular focus on demonstrating the failure by a simple yet widely adopted approach, GradientxInput. The authors claim three sources of mismatch that result in the attribution failure through their analysis, which are the coordinate origin, finite-step approximation, and routing change. Following the analysis, the authors introduce MERN, which aims to measures the claimed failure sources.

Critical strengths
  • This paper presents detailed analysis of the failure by the defined explainer. The derivation is clean and algebraically correct
Critical weaknesses
  • The overall writing is extremely difficult to follow. This paper lacks logical coherence and necessary transitions between sentences, paragraphs, and sections.
    • For example, Section 3 that presents related work reads as a collection of disconnected sentences. While the authors did draw the some connection to the presented research with one-sentence comments, the paragraphs in this section lacks logical flows and fall apart.
    • The organization of and transition between sections should also be largely improved. Related work comes after the exposure of the problem/challenge, which is badly connected. The relationship between Sec.2 and Sec.4 is also unclear; the authors starts directly introducing a network design without connecting it to the analysis.
    • In fact, the design choices of MERN remains mostly unexplained; there is few intuition, justification, or connection to the previously exposed failure.
  • Another critical point is the three sources, which is determined through a questionable definition of GradientxInput. Only the finite step approximation appears sufficiently justified.
    • The authors define AjA_j with a frozen route despite the fact that zez_e also depends on the input. The second part of Eq. (4), which is from the authors themselves, more precisely describes GradientxInput in the context of expert mixture. This definition undermines the validity of the term RR.
    • Besides, GradientxInput, in my opinion, implicitly assumes a zero baseline; considering alternative baselines without necessary adaptations to its default definition is inappropriate and should of course cause misalignment. This renders the reference-origin mismatch more like an implementation side problem rather than a conceptual defect.

These two concerns largely drives my recommendation of reject in the current states of the manuscript.

In addition to the major points, I strongly recommend the authors to refine their symbol systems, pairing each symbol with proper and timely clarification.

Questions for authors
  • Could the authors comment on the two error sources discussed in weaknesses?
  • What is the connection between Sec 2 and Sec 4?
  • What is the practical utility of MERN? Does it intend to deliver a better model design or to promote explainability research? If it is the second case, in what way?
  • How should this work be positioned in the existing explainability research? What is its connection to other gradient-based approaches, e.g. SmoothGrad, IG?
5 comments

We thank the reviewer for the comments on presentation and attribution definitions. We now distinguish the frozen-route expert-side score from standard complete-model Gradient×Input, interpret the origin term as reference alignment, and analyze the residual after complete differentiation. We also strengthen the section transitions and explain MERN’s design choices and diagnostic role. Below, we address the weaknesses and questions point by point and identify the corresponding manuscript changes.

For the reviewers' convenience, we have uploaded both a clean version of the revised manuscript (revised_manuscript_clean.pdf) and a version highlighting key changes (revised_manuscript_mark.pdf).

W1: The overall writing is extremely difficult to follow. This paper lacks logical coherence and necessary transitions between sentences, paragraphs, and sections.

Answer: We agree that the original presentation did not make the progression from the diagnostic question to the model and experiments sufficiently clear. We have reorganized Related Work, added a notation guide in Section 3, and strengthened the transition to Section 4 by explaining how MERN’s representations and controls support the preceding analysis. Section 6.2 now states the experimental questions explicitly and connects the results to those questions. These changes aim to clarify both the overall argument and the purpose of the model design. We describe the revisions in detail in our responses to Questions 2 and 3 below.

W2: Another critical point is the three sources, which are determined through a questionable definition of Gradient×Input. Only the finite step approximation appears sufficiently justified.

Answer: We agree that standard Gradient×Input of the complete predictor differentiates both expert outputs and routing weights. Section 3 now explicitly identifies AjA_j as a frozen-route expert-side score, interprets the origin term as reference alignment, and defines RjR_j as a finite routing contribution under the stated endpoint convention. We have also added an analysis of the residual after complete differentiation, so the diagnostic argument does not depend on excluding routing derivatives. Our response to Question 1 below provides the definitions, derivation, and same-function construction, together with the corresponding manuscript changes.

Q1: Could the authors comment on the two error sources discussed in the weaknesses?

Answer: Thank you for raising these concerns. We agree that the original wording did not sufficiently distinguish the frozen-route expert-side score from standard Gradient×Input of the complete predictor. The revision makes that distinction explicit and treats the origin term as reference alignment. It also analyzes the residual of complete-model gradients, so the diagnostic argument does not depend on excluding routing derivatives.

We agree that standard Gradient×Input for the complete predictor must differentiate both expert outputs and routing weights. Section 3 now explicitly identifies Aj=uj∑eze(x)∂ujre(u)A_{j}=u_{j}\sum_{e}z_{e}(x)\partial_{u_{j}}r_{e}(u) as a frozen-route expert-side score, while Eq. (4) gives the complete derivative. Accordingly, the three-term decomposition of Aj−DjA_{j}-D_{j} is an exact accounting of the discrepancy for this specified score, rather than a characterization of standard complete-model Gradient×Input.

The zero-reference convention also needs to be stated explicitly. Under the field-aligned expert-coordinate score, the term

Oj=Aj−Cj=uj′gjO_{j}=A_{j}-C_{j}=u_{j}^{\prime}g_{j}

measures the mismatch between the score's zero origin in expert coordinates and the declared replacement reference. It vanishes when uj′=0u_{j}^{\prime}=0, and centering removes it by construction. A zero in expert coordinates need not be a zero in raw input coordinates. We therefore interpret OjO_{j} as reference alignment, not a newly identified conceptual defect of reference-aware attribution methods.

The routing term has a separate operational definition. We first evaluate the finite expert response FjF_{j} with the original routing weights fixed. The residual Rj=Dj−FjR_{j}=D_{j}-F_{j} then captures the finite contribution of routing-weight changes under this endpoint convention. Its interpretation depends on the chosen decomposition and does not represent a routing derivative omitted by standard complete-model Gradient×Input.

To examine the issue beyond the frozen-route formulation, we added a complete-gradient analysis:

Sj−Dj=Nj+Qj,Qj=(Sj−Cj)−Rj.S_{j}-D_{j}=N_{j}+Q_{j}, Q_{j}=(S_{j}-C_{j})-R_{j}.

For the affine fieldwise numerical encoding, QjQ_{j} compares the included local routing contribution with its finite endpoint counterpart; it may include expert-router interaction. Thus complete differentiation includes local routing sensitivity but can still disagree with the finite response. Appendix B gives the corresponding Njx+QjN_{j}^{x}+Q_{j} identity for nonlinear or shared encoders.

Appendix B provides a zero-reference construction using standard complete-model Gradient×Input. Two dense routed implementations represent the same function ℓ(x)=1+32x−x2\ell (x)=1+\frac{3}{2}x-x^{2}. At x=1x=1, with x′=0x^{\prime}=0, both yield S=−12S=-\frac{1}{2} and D=12D=\frac{1}{2}, but (Nx,Q)E=(−1,0)(N^{x},Q)_{E}=(-1,0) and (Nx,Q)X=(0,−1)(N^{x},Q)_{X}=(0,-1). Finite expert evaluation corrects the first score, while substitution of the finite routing contribution corrects the second. Neither uses a support switch.

This establishes that the same overall response error can arise from different internal approximations. The decomposition diagnoses the supplied factorization and convention; it does not uniquely attribute an implementation-independent cause or establish how common either account is in trained models. Computing the component substitutions requires endpoint information.

Manuscript changes. Section 3 (pp. 3–4) now distinguishes the frozen-route and complete-gradient scores, clarifies the reference-origin and finite routing terms, and introduces the complete-gradient residual in Eqs. (5)–(7). Appendix B (pp. 17–18) adds the corresponding derivations, affine-stream interaction analysis, and same-function construction.

Q2: What is the connection between Sec. 2 and Sec. 4?

Answer: We have moved Related Work from Section 3 to Section 2 and made the transition to the model explicit. In the revised order, Section 2 positions the problem, Section 3 defines the response target and diagnostic evaluations, Section 4 explains how the model exposes those quantities, and Sections 5-6 specify and report the experiments.

The central connection is that Section 3 defines what must be measured, while Section 4 explains how the model exposes those quantities. The analysis compares reference alignment, finite expert-output changes under the original route, and the complete response after routing changes. MERN exposes expert outputs and routing weights separately so that these evaluations can be performed on the same fitted predictor and replacement.

We have also rewritten the criticized Related Work paragraph to follow a connected argument. It now moves from internal coefficients and local derivatives, through neighborhood averaging and reference-based attribution, to methods that examine internal computation. It then explains how our intermediate finite evaluations build on these approaches. This makes the relationship between the cited methods and our research question explicit within the paragraph.

Manuscript changes. Section 2's "Internal scores and complete derivatives" paragraph (p. 2) now follows this progression. Section 3 adds the notation guide (p. 3); the opening of Section 4 and Section 4.1 (p. 4) explain MERN's diagnostic purpose and representation choices; Section 4.3 (p. 6) connects its exported scores to the analysis.

Q3: What is the practical utility of MERN? Does it intend to deliver a better model design or to promote explainability research? If it is the second case, in what way?

Answer: MERN serves as an experimental model for explanation research. We use it to distinguish intermediate expert and routing responses on the same fitted predictor, not to recommend a new deployment model. Hierarchical MERN has lower mean AUROC than CatBoost on nine of ten datasets (Table 1, p. 7). Its research value comes from the controlled comparisons enabled by the following design choices.

Our designRole in the analysis
Field-aligned expert representation and an input-dependent routing encoderA field replacement changes an identifiable expert coordinate while routing can respond to the full input.
Explicit routing weights and expert logits, including biasesTheir weighted sum reconstructs the prediction, allowing intermediate evaluations to be checked against the complete response.
Affine-expert and dense-routing controlsAffine experts remove expert approximation error; dense routing removes support switching, allowing continuous routing-weight effects to be examined.

The hierarchy supplies one setting with changing expert outputs and routing weights. The simpler controls test which observations remain after expert nonlinearity or support switching is removed. We do not claim that MERN's particular hierarchy is necessary or optimal for the decomposition.

Figure 4 (p. 9) illustrates a concrete diagnostic use. In the affine top-two Shoppers model, CC and the finite frozen-route response are both +0.1576, but the routing contribution is -0.5301, giving D=−0.3725D=-0.3725. Expert support is unchanged. This identifies a reversal due to changing routing weights under the declared convention, rather than expert approximation. The complete local score S=−0.1117S=-0.1117 repairs direction but leaves magnitude error. The example was selected by a disclosed median-event rule and establishes an illustration, not prevalence.

The broader comparisons quantify related effects: dense affine mixtures retain response reversals without support switching, and complete differentiation improves sign agreement in 377/400 checkpoint-reference conditions while giving less consistent magnitude gains. Appendix B's same-function construction shows why total fidelity alone cannot select the appropriate component substitution. The SENN study in Section 6.4 examines the coefficient-response distinction in an external implementation.

This provides a reproducible way to distinguish reference centering, local routing differentiation, and finite component evaluation when investigating explanation errors. The diagnosed substitutions modify explanatory scores, not the predictor or its accuracy; the study does not demonstrate human benefit, a cheaper alternative to occlusion, or general transfer to learned shared encoders.

Manuscript changes. Sections 4-5 (pp. 4-6) explain the diagnostic role of the representations, prediction ledger, and controls. Sections 6.1-6.3 (pp. 7-9) report predictive context and explanation-response results, including Figure 4. Appendix B (pp. 17-18) supplies the same-function construction, and Section 6.4 (p. 10) reports SENN.

Reviewer_aeftOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper addresses disagreement between feature scores and the actual responses of routed models, even when the prediction terms are transparent, by decomposing the discrepancy into reference-origin mismatch, finite-step expert approximation, and omitted routing response. The authors show that centering can remove origin mismatch and affine experts can remove expert approximation. However, routing can still reverse the response without changing the active experts. Experiments on multiple tabular datasets and routed model variants are used to support the claims empirically.

Critical strengths
  1. The paper discusses an important problem of directionally misleading feature scores in interpretable routed models.

  2. The three-component decomposition is intuitive and useful to diagnose score-response disagreement. This is especially useful in sign-reversal scenarios.

  3. The experimental analysis on multiple datasets and routed model variants provides satisfactory empirical evidence for the authors' claims. Additionally, the paper reports both magnitude error and directional disagreement, making it more complete than analyzing just one quantity.

Critical weaknesses
  1. The authors should clearly state the novelty of their work and distinguish it from existing routing-interpretability literature.

  2. Prior works like Li et al., “Decoding Knowledge Attribution in Mixture-of-Experts: A Framework of Basic-Refinement Collaboration and Efficiency Analysis,” ACL 2025, and Lasy, Cai & Ayonrinde, “RouterInterp: Understanding Superposed Specialisation in Mixture of Experts Routing,” ICML 2026, should be included in the related works section to better position the contribution.

  3. The paper is difficult to read in several places. The abstract and parts of the methodology and experimental discussion are very dense, with several concepts introduced before they are intuitively explained.

  4. The empirical benefit is not uniform across replacement strategies. For example, in Table 3, moving from the centered score (C) to the derivative-based score (S) substantially improves MAE for mean/mode replacement. However, it gives only a very small MAE improvement for fit-donor replacement, even though reversal rates improve more clearly. The authors should discuss this in more details.

  5. It would be easier for the readers to interpret the results of the tables in the paper if the authors state explicitly whether higher or lower is better for each metric and put markers like ( ↑ / ↓ ) to denote the same.

Questions for authors

Please address the weaknesses.

3 comments

We thank the reviewer for the helpful comments. We have clarified the contribution and related-work positioning, strengthened the connections between the analysis, methodology design, and the experiments, and improved the discussion of reference-dependent results. We also distinguish directional agreement from magnitude accuracy and clarify metric directions in the tables. Below, we respond to each comment and specify the corresponding revisions.

For the reviewers' convenience, we have uploaded both a clean version of the revised manuscript (revised_manuscript_clean.pdf) and a version highlighting key changes (revised_manuscript_mark.pdf).

Presentation: Difficult to follow despite careful reading.

Answer: Thank you for raising this concern. We have revised the presentation to make the progression from the research question to the analysis and experiments more explicit. The introduction distinguishes understanding how a prediction is assembled from understanding how the complete predictor responds to a specified feature replacement. Section 3 adds a notation guide identifying the definitions and roles of the intermediate scores and responses. Section 4 then explains MERN’s role as a controlled diagnostic predictor and why its representations support the comparisons introduced in Section 3.

We have also strengthened the organization of the experimental discussion. Section 6.2 now begins with three explicit questions: whether reference alignment improves the scores, whether routing effects persist with affine experts, and whether complete differentiation resolves the remaining finite-response error. The subsequent discussion connects the results to these questions and distinguishes directional agreement from magnitude accuracy. Appendix A separately presents baseline procedures, query counts, and measured latency, making the computational comparisons easier to interpret. These revisions appear in Sections 1, 3–4, and 6.2–6.3 (pp. 1–9), with computational details in Appendix A (pp. 14–17).

W1: The authors should clearly state the novelty of their work and distinguish it from existing routing-interpretability literature.

Answer: We have revised the abstract, introduction, and related work to state the contribution more precisely. Existing routing-interpretability research examines expert specialization, knowledge attribution, collaboration among experts, and features associated with routing decisions. Our study addresses a specific complementary question: how do expert-output and routing-weight changes account for a local feature score’s disagreement with the complete predictor’s finite response to a declared replacement?

The abstract now identifies the response target and diagnostic contribution. Section 3 distinguishes reference alignment, finite expert evaluation, and complete differentiation. Appendix B then shows why aggregate response fidelity cannot determine the internal source of an error: two factorizations of the same prediction function have the same complete gradients and endpoint responses, yet different effective component substitutions. This is a statement about diagnosing a supplied computational structure, not a new function-invariant attribution rule. The trained affine controls separately show that routing-related reversals occur without expert nonlinearity or support switching.

Section 2 connects this analysis to expert/routing interpretation, internal attribution, and perturbation-based response evaluation. We acknowledge their established treatment of references and gradients and specify the additional error-accounting question. The positioning changes appear in the abstract and Sections 1-2 (pp. 1-2); Section 3 (pp. 3-4) and Appendix B (pp. 17-18) give the derivations and scope.

W2: Prior works like Li et al., “Decoding Knowledge Attribution in Mixture-of-Experts: A Framework of Basic-Refinement Collaboration and Efficiency Analysis,” ACL 2025, and Lasy, Cai & Ayonrinde, “RouterInterp: Understanding Superposed Specialisation in Mixture of Experts Routing,” ICML 2026, should be included in the related works section to better position the contribution.

Answer: Thank you for recommending these works. Both are now cited in the introduction (p. 1), discussed in the "Expert structure and routing" paragraph of Section 2 (p. 2), and included in the bibliography (p. 12).

We discuss Li et al. (2025) as analyzing cross-level knowledge attribution and expert collaboration, and Lasy et al. (2026) as identifying sparse-autoencoder features predictive of routing decisions to interpret expert specialization. We then clarify how these studies motivate examining internal routing mechanisms, while our analysis focuses on expert-output and routing-weight contributions to a specified local-score–response discrepancy.

W3: The paper is difficult to read in several places. The abstract and parts of the methodology and experimental discussion are very dense, with several concepts introduced before they are intuitively explained.

Answer: We agree that readers need clearer explanations of the concepts alongside their formal definitions. We have revised the abstract and introduction to make the replacement-response question and the purpose of the decomposition explicit. In Section 3 (p. 3), the added notation guide describes both the definitions and roles of the local scores, finite responses, and residual terms.

The discussion accompanying Eqs. (3)–(7) (pp. 3–4) now explains what each discrepancy measures, why the routing contribution depends on the stated endpoint convention, and why including routing derivatives does not necessarily eliminate finite-response error. Section 4 (p. 4) explains the purpose of the field-aligned expert representation and its separation from routing.

The experimental discussion also provides more interpretation around the numerical results. Section 6.2 introduces the questions guiding the comparisons before discussing their outcomes, then explains why substantial donor replacements can exceed the accuracy of a local approximation and why signed errors may cancel. Figure 4 (p. 9) provides a concrete illustration: complete differentiation repairs the response direction in that case, while endpoint evaluation resolves the remaining magnitude error.

W4: The empirical benefit is not uniform across replacement strategies. For example...

Answer: Thank you for highlighting this distinction. We agree that the benefit of complete differentiation depends on both the replacement reference and the evaluation metric. Table 3 shows that C→SC\to S reduces MAE from .233 to .171 under mean/mode replacement, but only from .318 to .313 under fit-donor replacement. Meanwhile, the donor displayed-field reversal rate decreases more substantially, from 3.808% to 2.560%.

The two references define different endpoints and displacements, while SS remains a first-order approximation at the original input. Finite replacements can therefore involve expert and routing changes beyond what that derivative captures. The residual S−D=N+QS-D=N+Q also contains signed terms that may reinforce or cancel. These mechanisms explain why directional improvement need not yield comparable magnitude improvement; the aggregate table alone does not determine how much each mechanism contributes to the donor result.

The donor average also masks heterogeneity. Appendix O, Table 37 (p. 37), shows lower donor MAE in eight of ten dataset means, but higher MAE on Shoppers (.462 to .598) and Covertype (.855 to 1.095). Across both references, sign agreement improves in 377/400 checkpoint-reference conditions, whereas MAE improves in 325/400. We therefore describe complete differentiation as improving direction more consistently than finite-response magnitude, rather than as a uniform correction.

Manuscript changes. The "Complete-predictor gradient control" paragraph in Section 6.2 (p. 8) now contrasts the two references, reports the adverse dataset results, and explains the first-order and cancellation qualifications. Section 3, Eqs. (6)-(7) (p. 4), provides the corresponding residual and squared-error analysis.

W5: It would be easier for the readers to interpret the results of the tables in the paper if the authors state explicitly whether higher or lower is better for each metric and put markers like (↑/↓) to denote the same.

Answer: Thank you for this helpful suggestion. We have checked all tables in the main text and appendices and added the missing ↑/↓ markers to the relevant column headers or captions. We also explicitly state at the end of Section 5 (Experimental Protocol) that ↑ indicates higher is better and ↓ indicates lower is better. The annotations follow the definition of each reported quantity; for example, MAE is marked ↓, whereas MAE reduction is marked ↑. Descriptive quantities without a universal better/worse direction, such as dataset sizes and signed mechanism components, are identified as such and remain unmarked.

Manuscript changes. Section 5 (Experimental Protocol) now explicitly defines ↑ and ↓ as indicating that higher and lower values are better, respectively. Missing direction markers have been added to table headers or captions throughout the appendices. Table 27 distinguishes these markers from the within-cell C→SC\to S notation, while Table 43 clarifies that MAE reduction is a gain (↑\uparrow). Tables 13, 26, and 30 also identify descriptive quantities that have no universal better/worse direction.

Dear Reviewer,

Thank you very much for your positive evaluation and constructive suggestion. We have carefully revised the manuscript to address the concerns you raised, and we would like to briefly highlight the main changes.

First, regarding the novelty and positioning of our work, we have clarified how our mechanism-level diagnosis differs from existing studies on expert specialization and routing interpretability. We have also incorporated the suggested works by Li et al. (ACL 2025) and Lasy et al. (ICML 2026) into the Related Work section to better establish this distinction.

Second, we have improved the paper's presentation and logical flow. The abstract and methodology have been reorganized to introduce the key concepts more progressively, with additional explanations and a notation guide to make the analysis easier to follow.

Third, we have expanded the discussion of the non-uniform effects across replacement strategies. In particular, we now explain why complete differentiation improves directional agreement more consistently than magnitude accuracy, report the dataset-level variations, and discuss the limitations of local first-order approximations.

Finally, we have added explicit upward and downward arrows to the evaluation metrics throughout the tables to make the results easier to interpret.

We have made a sincere effort to address each of your concerns and to improve both the clarity and technical presentation of the paper. We hope these revisions adequately address your comments. If possible, we would greatly appreciate it if you could provide a higher rating.

There is still some time remaining in the discussion period, and we would be very happy to clarify or further revise any aspect that you feel remains insufficiently addressed.

Thank you again for your time and consideration.

Best wishes!

Reviewer_13ZoOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks why feature scores from routed mixtures can disagree with the model's finite logit response DjD_j when one field is replaced by a reference. It writes the gap between the frozen-route gradient-times-input score AjA_j and DjD_j as an exact sum of three terms, origin mismatch, finite-step expert error and omitted routing response, and pairs each with a fix (centering, affine experts, complete differentiation SjS_j, endpoint checks). Experiments use a hierarchical sparse mixture (MERN), a single-level control and two affine mixtures on ten tabular datasets with five seeds, plus a reproduction of SENN on three datasets.

Critical strengths

The decomposition is exact, needs no differentiable router, and the authors are careful about what it does not claim (Section 2, Appendix B). The dense affine controls isolate routing well, with ∣N∣<2.1×10−7|N| < 2.1 \times 10^{-7} yet F and D signs disagree in 18.1% to 19.4% of changed fields (Section 6.2). The SENN transfer is a nice external check, and the paper reports its costs openly (AUROC .910 to .899 on MAGIC for the logit-aligned arm, Section 6.4).

Critical weaknesses

The technical novelty is small, and the paper says so itself (Section 1). Equation 3 is an add-and-subtract identity. That gradient-times-input depends on the origin, and that a local gradient differs from a finite change, are the standard motivations for reference-based methods such as Integrated Gradients and DeepLIFT (Sundararajan et al., 2017; Ancona et al., 2018), and neither is cited. So centering beating AA in 199/200 runs is expected, since AA ignores the declared reference. Holding gates fixed is also a known design choice in LRP for gated and attention models (Arras et al., 2017; Ali et al., 2022), which the related work should discuss.

The missing baselines matter more. Path methods on the full model, e.g. Integrated Gradients along the single-field path from xx to x(j)x^{(j)}, give DjD_j exactly by completeness, routing included. Plain occlusion gives DjD_j with one forward pass per field, which is cheap here (at most 54 fields). This makes the budgeted-check setting hard to motivate, and the gain from the ∣S−C∣|S-C| priority is small in absolute terms (0.84% vs 1.33% residual reversals for random, Section 6.3). Also SS does not fix magnitude under donors (MAE .318 to .313) and worsens it on Shoppers and Covertype. Finally, MERN trails CatBoost on 9 of 10 datasets (Table 1), so the practical case for using it as an interpretable model is unclear.

Questions for authors
  1. Can you add Integrated Gradients (complete model, same references) and single-field occlusion to Tables 2 and 3, with their query cost?
  2. In what realistic setting is a budget of two endpoint checks binding, given that exact DjD_j for all fields costs about dd forward passes?
  3. Several analyses are post hoc follow-ups after earlier cohorts (Sections 5 and 6, Appendices J and O). Which results were fixed before seeing any test outcomes?
  4. Do the main findings hold for a model where routing and experts share one encoder, which is the common case in practice?
5 comments

We thank the reviewer for the careful assessment. We have clarified the diagnostic contribution, added full-model IG-16 and occlusion comparisons with computational costs, and introduced complete-gradient residual analysis and a same-function construction. We also clarify the practical limits of budgeted checks and the empirical scope of our findings. Below, we address each weakness and question and identify the corresponding manuscript changes.

For the reviewers' convenience, we have uploaded both a clean version of the revised manuscript (revised_manuscript_clean.pdf) and a version highlighting key changes (revised_manuscript_mark.pdf).

W1: The technical novelty is small, and the paper says so itself (Section 1). Equation 3 is an add-and-subtract identity. That gradient-times-input depends on the origin, and that a local gradient differs from a finite change, are the standard motivations for reference-based methods such as Integrated Gradients and DeepLIFT (Sundararajan et al., 2017; Ancona et al., 2018), and neither is cited. So centering beating A in 199/200 runs is expected, since A ignores the declared reference. Holding gates fixed is also a known design choice in LRP for gated and attention models (Arras et al., 2017; Ali et al., 2022), which the related work should discuss.

Answer: Thank you for highlighting these important connections. We agree that Eq. (3) is an add-and-subtract identity, and that reference alignment, the distinction between local gradients and finite changes, and fixed-gate attribution build on established ideas. We have revised the introduction and related work to clarify these foundations and make the specific diagnostic contribution of our study explicit.

We now discuss IG (Sundararajan et al., 2017), DeepLIFT (Shrikumar et al., 2017), and the relationships among gradient-based attribution methods analyzed by Ancona et al. (2018). We also discuss relevance propagation through recurrent gates (Arras et al., 2017) and Transformer attention (Ali et al., 2022). These connections clarify that reference dependence and fixed-gate attribution are established foundations of our analysis.

Our contribution is to organize these established ingredients into computable discrepancy terms and controlled interventions for a specified predictor and replacement. Compensated translations isolate reference-origin effects, while affine experts eliminate finite-step expert approximation; routing can nevertheless reverse the response without a support switch. We further extend the accounting to complete-model gradients that already include routing derivatives. This extension distinguishes the finite expert remainder from the discrepancy between local and finite routing contributions, and shows why improving one component need not improve aggregate fidelity when signed terms cancel.

The same-function construction in Appendix B makes the additional diagnostic information precise. Two implementations have identical prediction functions, complete gradients, IG scores, and endpoint responses, yet different internal sources of complete-gradient error. Finite expert evaluation corrects the score in one implementation, whereas substituting the finite routing contribution corrects it in the other. Aggregate response fidelity therefore cannot identify which of these component substitutions addresses the discrepancy. This diagnosis is conditional on the computational factorization and endpoint convention; the construction establishes possibility, not prevalence in trained models or an improvement over IG's implementation invariance.

Regarding the 199/200 result, we agree that centering addresses an established reference-alignment issue: Aj−Cj=uj′gjA_{j}-C_{j}=u_{j}^{\prime}g_{j}. We therefore interpret the observed improvement in donor-reference joint probability drop at k=3k=3 as empirical support for reference alignment, not an independent novelty claim. The identity guarantees removal of the origin-mismatch term; it does not guarantee improvement in every aggregate metric, because the signed error components may reinforce or cancel one another.

Manuscript changes. Sections 1-2 (pp. 1-2) clarify the contribution and its attribution foundations. Section 3, Eqs. (5)-(7) (p. 4), and Appendix B (pp. 17-18) add the complete-gradient residual analysis and same-function construction. Appendix A (pp. 14-17) reports the full-model IG-16 and occlusion baselines, query counts, and measured costs.

W2: The missing baselines matter more. Path methods on the full model, e.g...

Answer: Thank you for raising these concerns. The added baselines show that IG-16 and occlusion estimate the declared finite response substantially more accurately than the local scores. Our contribution concerns diagnosing the remaining discrepancy. We address the baseline, magnitude, budget, and model-selection issues separately below.

(1) Full-model baselines. We agree that single-field IG recovers DjD_{j} under the completeness conditions, and that occlusion computes it with one additional forward evaluation per changed field when the original prediction is cached. The new comparisons use the same 200 checkpoints, audit observations, and references. For hierarchical MERN with donor references, all-field MAE is .391/.335/.055 for A/C/IG16A/C/\mathrm{IG}_{16}; occlusion matches the archived target up to floating-point recomputation error. Appendix A explains both finite-resolution quadrature error and possible output discontinuities under hard routing.

(2) Magnitude limitations. Complete differentiation improves numerical-field sign agreement in 377/400 checkpoint-reference conditions, but donor MAE changes only from .318 to .313 and worsens on Shoppers (.462 to .598) and Covertype (.855 to 1.095). The residual S−D=N+QS-D=N+Q explains why including local routing sensitivity does not guarantee a better finite-response approximation: signed expert and routing remainders can reinforce or cancel. We therefore distinguish directional improvement from magnitude accuracy.

(3) Budgeted checks. These tabular experiments do not demonstrate that two checks are a binding practical constraint. On the fixed numerical ∣S∣\lvert S\rvert top-five display, two routing-priority checks leave .844% donor reversals, versus 1.328% for random and 1.108% for magnitude priority: a modest improvement of about 0.48 percentage points over random. The complete checking pipeline is faster than all-field occlusion in 0/80 singleton and 25/80 batch conditions; median latency ratios are 1.410 and 1.196 (Appendix A, p. 17). The result supports conditional budget allocation, not a general efficiency advantage. Full occlusion is appropriate when all endpoint responses are affordable.

(4) MERN's role. Hierarchical MERN has lower mean AUROC than CatBoost on nine of ten datasets. We do not infer predictive parity or a deployment advantage from Table 1. MERN exposes field-aligned expert responses and input-dependent routing for controlled diagnosis on a fixed predictor. Independent affine and dense-routing controls remove expert nonlinearity and support switching, respectively. The external SENN study examines the coefficient-response distinction beyond MERN, without establishing general transfer to other architectures.

Manuscript changes. IG-16 appears in Table 2 (p. 7) and Table 3 (p. 9); explicitly labelled occlusion results and costs appear in Appendix A, Tables 5-10 (pp. 15-17). Section 3 (pp. 3-4) and Appendix B (pp. 17-18) analyze the complete-gradient residual. Section 6.3 (p. 9), Appendix K (p. 31), and Appendix O, Table 41 (p. 39), report the checking protocol and results. Sections 4 and 6.1 explain MERN's role; Section 6.4 (p. 10) reports SENN.

Q1: Can you add Integrated Gradients (complete model, same references) and single-field occlusion to Tables 2 and 3, with their query cost?

Answer: Thank you for this suggestion. We have added complete-model IG-16 to Tables 2 and 3, using the same 200 checkpoints, 256 audit observations per checkpoint, and mean/mode and fitting-row donor references. It uses 16-point Gauss-Legendre quadrature on each single-field path and differentiates both expert outputs and routing. Numerical paths vary the selected input field; categorical paths interpolate its continuous representations in both prediction streams simultaneously (Appendix A, p. 14). The original hard routing is retained, so IG-16 need not exactly recover the endpoint difference.

Single-field occlusion computes Dj=ℓ(x)−ℓ(x(j))D_{j}=\ell (x)-\ell (x^{(j)}) directly. In the current main tables, Table 2 includes its joint-selection comparator under the label DD, while Table 3 uses DD as the response target. Appendix A, Tables 5-7 (p. 15), explicitly labels Occlusion as a baseline and reports its MAE and reversal rate alongside IG-16 and the local scores. Tables 7-8 give the corresponding numerical-field and all-field query costs.

The following results summarize the comparison for hierarchical MERN across ten datasets and five seeds; the complete results for all four predictors appear in Tables 2 and 5–6.

MethodMean/mode MAE ↓Fit-donor MAE ↓Mean/mode joint drop ↑Fit-donor joint drop ↑
Uncentered score AA0.2740.3910.2160.192
Centered score CC0.2650.3350.2310.244
Full-model IG-160.0450.0550.3020.319
Occlusion (DD)3.32×10−73.32\times 10^{-7}3.39×10−73.39\times 10^{-7}0.3030.321

MAE measures error against the single-field logit response. Joint drop reevaluates the simultaneous replacement of up to three selected fields; comparisons are within the same predictor and reference.

Under donor references, hierarchical-MERN MAE falls from .391 to .335 after centering and to .055 with full-model IG-16. The numerical-field comparison in Table 3 also favors IG-16 over both CC and the complete local-gradient score SS in MAE and reversal rate. Occlusion directly recovers the declared target; its tiny MAE reflects floating-point recomputation against archived responses, and its numerical-field reversal rate is zero. IG-16's residual can reflect quadrature error and, for hard routing, output jumps that ordinary derivative integration does not capture.

For qq actually changed eligible fields, IG-16 uses 16q16q forward and 16q16q backward evaluated rows. Occlusion uses qq additional forward rows with a cached original prediction, or 1+q1+q otherwise, and no backward rows. These are evaluated-row counts, not counts of batched software calls. The following table combines them with the all-field hierarchical-MERN latency measurements in Table 9.

MethodForward evaluations ↓Backward evaluations ↓Singleton latency (ms) ↓Batch-16 latency (ms) ↓
Centered score CC113.7300.300
Full-model IG-1616q16q16q16q18.88510.998
Occlusionqq cached; 1+q1+q otherwise04.4830.623

Latency is milliseconds per observation, averaged over both references; batch-16 values are normalized per observation. Timing uses 40 seed-1 checkpoints across the four predictors on a shared, single-thread CPU setting, whereas accuracy uses all 200 checkpoints. Counts refer to the common automatic-differentiation implementation; the numerical-field score SS also uses one forward and one backward row.

Appendix A separates scoring counts (Tables 7-8, pp. 15-16), standalone scoring latency (Table 9, p. 16), and complete checking-pipeline latency (Table 10, p. 17). The latter includes score construction, selection, and endpoint evaluation. Tables 9 and 10 come from separate timing runs and are not combined to calculate speed ratios.

For the declared finite-response target, these measurements favor occlusion when endpoint evaluations are affordable. The decomposition adds a different type of information: intermediate expert and routing evaluations identify how the local score's discrepancy arises and what each component substitution changes. Computing that diagnosis itself uses endpoint information; we do not claim that it is a more accurate or cheaper substitute for occlusion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.