- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies the feature attribution problem in the context of expert mixtures, with the particular focus on demonstrating the failure by a simple yet widely adopted approach, GradientxInput. The authors claim three sources of mismatch that result in the attribution failure through their analysis, which are the coordinate origin, finite-step approximation, and routing change. Following the analysis, the authors introduce MERN, which aims to measures the claimed failure sources.
Critical strengths
- This paper presents detailed analysis of the failure by the defined explainer. The derivation is clean and algebraically correct
Critical weaknesses
- The overall writing is extremely difficult to follow. This paper lacks logical coherence and necessary transitions between sentences, paragraphs, and sections.
- For example, Section 3 that presents related work reads as a collection of disconnected sentences. While the authors did draw the some connection to the presented research with one-sentence comments, the paragraphs in this section lacks logical flows and fall apart.
- The organization of and transition between sections should also be largely improved. Related work comes after the exposure of the problem/challenge, which is badly connected. The relationship between Sec.2 and Sec.4 is also unclear; the authors starts directly introducing a network design without connecting it to the analysis.
- In fact, the design choices of MERN remains mostly unexplained; there is few intuition, justification, or connection to the previously exposed failure.
- Another critical point is the three sources, which is determined through a questionable definition of GradientxInput. Only the finite step approximation appears sufficiently justified.
- The authors define with a frozen route despite the fact that also depends on the input. The second part of Eq. (4), which is from the authors themselves, more precisely describes GradientxInput in the context of expert mixture. This definition undermines the validity of the term .
- Besides, GradientxInput, in my opinion, implicitly assumes a zero baseline; considering alternative baselines without necessary adaptations to its default definition is inappropriate and should of course cause misalignment. This renders the reference-origin mismatch more like an implementation side problem rather than a conceptual defect.
These two concerns largely drives my recommendation of reject in the current states of the manuscript.
In addition to the major points, I strongly recommend the authors to refine their symbol systems, pairing each symbol with proper and timely clarification.
Questions for authors
- Could the authors comment on the two error sources discussed in weaknesses?
- What is the connection between Sec 2 and Sec 4?
- What is the practical utility of MERN? Does it intend to deliver a better model design or to promote explainability research? If it is the second case, in what way?
- How should this work be positioned in the existing explainability research? What is its connection to other gradient-based approaches, e.g. SmoothGrad, IG?
5 comments
Q1: Could the authors comment on the two error sources discussed in the weaknesses?
Answer: Thank you for raising these concerns. We agree that the original wording did not sufficiently distinguish the frozen-route expert-side score from standard Gradient×Input of the complete predictor. The revision makes that distinction explicit and treats the origin term as reference alignment. It also analyzes the residual of complete-model gradients, so the diagnostic argument does not depend on excluding routing derivatives.
We agree that standard Gradient×Input for the complete predictor must differentiate both expert outputs and routing weights. Section 3 now explicitly identifies as a frozen-route expert-side score, while Eq. (4) gives the complete derivative. Accordingly, the three-term decomposition of is an exact accounting of the discrepancy for this specified score, rather than a characterization of standard complete-model Gradient×Input.
The zero-reference convention also needs to be stated explicitly. Under the field-aligned expert-coordinate score, the term
measures the mismatch between the score's zero origin in expert coordinates and the declared replacement reference. It vanishes when , and centering removes it by construction. A zero in expert coordinates need not be a zero in raw input coordinates. We therefore interpret as reference alignment, not a newly identified conceptual defect of reference-aware attribution methods.
The routing term has a separate operational definition. We first evaluate the finite expert response with the original routing weights fixed. The residual then captures the finite contribution of routing-weight changes under this endpoint convention. Its interpretation depends on the chosen decomposition and does not represent a routing derivative omitted by standard complete-model Gradient×Input.
To examine the issue beyond the frozen-route formulation, we added a complete-gradient analysis:
For the affine fieldwise numerical encoding, compares the included local routing contribution with its finite endpoint counterpart; it may include expert-router interaction. Thus complete differentiation includes local routing sensitivity but can still disagree with the finite response. Appendix B gives the corresponding identity for nonlinear or shared encoders.
Appendix B provides a zero-reference construction using standard complete-model Gradient×Input. Two dense routed implementations represent the same function . At , with , both yield and , but and . Finite expert evaluation corrects the first score, while substitution of the finite routing contribution corrects the second. Neither uses a support switch.
This establishes that the same overall response error can arise from different internal approximations. The decomposition diagnoses the supplied factorization and convention; it does not uniquely attribute an implementation-independent cause or establish how common either account is in trained models. Computing the component substitutions requires endpoint information.
Manuscript changes. Section 3 (pp. 3–4) now distinguishes the frozen-route and complete-gradient scores, clarifies the reference-origin and finite routing terms, and introduces the complete-gradient residual in Eqs. (5)–(7). Appendix B (pp. 17–18) adds the corresponding derivations, affine-stream interaction analysis, and same-function construction.
Q2: What is the connection between Sec. 2 and Sec. 4?
Answer: We have moved Related Work from Section 3 to Section 2 and made the transition to the model explicit. In the revised order, Section 2 positions the problem, Section 3 defines the response target and diagnostic evaluations, Section 4 explains how the model exposes those quantities, and Sections 5-6 specify and report the experiments.
The central connection is that Section 3 defines what must be measured, while Section 4 explains how the model exposes those quantities. The analysis compares reference alignment, finite expert-output changes under the original route, and the complete response after routing changes. MERN exposes expert outputs and routing weights separately so that these evaluations can be performed on the same fitted predictor and replacement.
We have also rewritten the criticized Related Work paragraph to follow a connected argument. It now moves from internal coefficients and local derivatives, through neighborhood averaging and reference-based attribution, to methods that examine internal computation. It then explains how our intermediate finite evaluations build on these approaches. This makes the relationship between the cited methods and our research question explicit within the paragraph.
Manuscript changes. Section 2's "Internal scores and complete derivatives" paragraph (p. 2) now follows this progression. Section 3 adds the notation guide (p. 3); the opening of Section 4 and Section 4.1 (p. 4) explain MERN's diagnostic purpose and representation choices; Section 4.3 (p. 6) connects its exported scores to the analysis.
Q3: What is the practical utility of MERN? Does it intend to deliver a better model design or to promote explainability research? If it is the second case, in what way?
Answer: MERN serves as an experimental model for explanation research. We use it to distinguish intermediate expert and routing responses on the same fitted predictor, not to recommend a new deployment model. Hierarchical MERN has lower mean AUROC than CatBoost on nine of ten datasets (Table 1, p. 7). Its research value comes from the controlled comparisons enabled by the following design choices.
| Our design | Role in the analysis |
|---|---|
| Field-aligned expert representation and an input-dependent routing encoder | A field replacement changes an identifiable expert coordinate while routing can respond to the full input. |
| Explicit routing weights and expert logits, including biases | Their weighted sum reconstructs the prediction, allowing intermediate evaluations to be checked against the complete response. |
| Affine-expert and dense-routing controls | Affine experts remove expert approximation error; dense routing removes support switching, allowing continuous routing-weight effects to be examined. |
The hierarchy supplies one setting with changing expert outputs and routing weights. The simpler controls test which observations remain after expert nonlinearity or support switching is removed. We do not claim that MERN's particular hierarchy is necessary or optimal for the decomposition.
Figure 4 (p. 9) illustrates a concrete diagnostic use. In the affine top-two Shoppers model, and the finite frozen-route response are both +0.1576, but the routing contribution is -0.5301, giving . Expert support is unchanged. This identifies a reversal due to changing routing weights under the declared convention, rather than expert approximation. The complete local score repairs direction but leaves magnitude error. The example was selected by a disclosed median-event rule and establishes an illustration, not prevalence.
The broader comparisons quantify related effects: dense affine mixtures retain response reversals without support switching, and complete differentiation improves sign agreement in 377/400 checkpoint-reference conditions while giving less consistent magnitude gains. Appendix B's same-function construction shows why total fidelity alone cannot select the appropriate component substitution. The SENN study in Section 6.4 examines the coefficient-response distinction in an external implementation.
This provides a reproducible way to distinguish reference centering, local routing differentiation, and finite component evaluation when investigating explanation errors. The diagnosed substitutions modify explanatory scores, not the predictor or its accuracy; the study does not demonstrate human benefit, a cheaper alternative to occlusion, or general transfer to learned shared encoders.
Manuscript changes. Sections 4-5 (pp. 4-6) explain the diagnostic role of the representations, prediction ledger, and controls. Sections 6.1-6.3 (pp. 7-9) report predictive context and explanation-response results, including Figure 4. Appendix B (pp. 17-18) supplies the same-function construction, and Section 6.4 (p. 10) reports SENN.
We thank the reviewer for the comments on presentation and attribution definitions. We now distinguish the frozen-route expert-side score from standard complete-model Gradient×Input, interpret the origin term as reference alignment, and analyze the residual after complete differentiation. We also strengthen the section transitions and explain MERN’s design choices and diagnostic role. Below, we address the weaknesses and questions point by point and identify the corresponding manuscript changes.
For the reviewers' convenience, we have uploaded both a clean version of the revised manuscript (revised_manuscript_clean.pdf) and a version highlighting key changes (revised_manuscript_mark.pdf).
W1: The overall writing is extremely difficult to follow. This paper lacks logical coherence and necessary transitions between sentences, paragraphs, and sections.
Answer: We agree that the original presentation did not make the progression from the diagnostic question to the model and experiments sufficiently clear. We have reorganized Related Work, added a notation guide in Section 3, and strengthened the transition to Section 4 by explaining how MERN’s representations and controls support the preceding analysis. Section 6.2 now states the experimental questions explicitly and connects the results to those questions. These changes aim to clarify both the overall argument and the purpose of the model design. We describe the revisions in detail in our responses to Questions 2 and 3 below.
W2: Another critical point is the three sources, which are determined through a questionable definition of Gradient×Input. Only the finite step approximation appears sufficiently justified.
Answer: We agree that standard Gradient×Input of the complete predictor differentiates both expert outputs and routing weights. Section 3 now explicitly identifies Aj as a frozen-route expert-side score, interprets the origin term as reference alignment, and defines Rj as a finite routing contribution under the stated endpoint convention. We have also added an analysis of the residual after complete differentiation, so the diagnostic argument does not depend on excluding routing derivatives. Our response to Question 1 below provides the definitions, derivation, and same-function construction, together with the corresponding manuscript changes.