- Rating
- 1(−1)
- Confidence
- 2
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
The paper presents MUG, a method to attribute uncertainty to specific molecular substructures. The paper provides a concrete motivation and cover six benchmarks as well as a molecular design task.
Critical strengths
The paper presents a specific and reasonable motivation. The problem of identifying which molecular substructures are related with predictive uncertainty is important in many domains.
Critical weaknesses
The novelty is limited. The method mostly takes existing gradient-based attribution methods and applies them to predictive variance.
Questions for authors
What is the main novelty of MUG compared with existing gradient-based methods?
How well does the exact decomposition work when the model assumptions are not satisfied?
Does exact decomposition necessarily mean accurate uncertainty attribution?
Can the authors provide more experiments and comparisons to support their claims?
2 comments
Response to Reviewer d3xZ (2/2)
Q2: How well does the exact decomposition work when the model assumptions are not satisfied?
The decomposition is no longer guaranteed to be exact when the assumptions are violated. As reported in our original submission (Appendix F.7; now Appendix H.8, Table 28 in the revised manuscript), completeness residuals are approximately – for configurations satisfying the assumptions, compared with 0.049 for independently biased ensemble heads and 0.196 for input-level attribution. However, loss of completeness does not necessarily imply poor attribution rankings: a biased ReLU head yields a residual of 0.066 while retaining positive controlled fidelity (). Attention pooling, by contrast, loses both: a residual of 0.296 and .
Q3: Does exact decomposition necessarily mean accurate uncertainty attribution?
No, that is a separate but useful quality of attribution. Exact decomposition guarantees that contributions sum to predictive variance under the stated assumptions, but does not guarantee attribution accuracy (Section 3.2, Theorem 1). As shown in our paper, independent evaluations provide evidence of attribution quality and practical usefulness:
- Controlled fidelity: MUG achieves the highest mean controlled fidelity among evaluated methods across six benchmarks, supporting attribution faithfulness (Section 4.2, Table 1).
- Chemical withholding: The attribution share assigned directly to fluorine atoms increases from approximately 8% to 42%, with similar localization for withheld sulfone groups, supporting sensitivity to missing chemical training coverage (Section 4.1, Figure 3b; Appendix H.4, Figure 8).
- Molecular design: MUG-guided fragment selection increases the docking hit rate from 47.2% to 56.2% compared with random fragment selection within the same framework (Section 5, Table 6).
Q4. Can the authors provide more experiments and comparisons to support their claims?
We would be grateful for more specific feedback on which claims or aspects of the experimental evaluation remain concerning, and which additional experiments or comparisons would help address these concerns. In response to Reviewer wtzS’s comments, we have added an input-space comparison with Integrated Gradients targeting the same predictive variance in the rebuttal version (Appendix H.2). Below, we summarize the experimental evidence already included in our original submission, with references to the revised manuscript:
- Baseline comparisons: MUG achieves the highest mean controlled fidelity among evaluated methods across six benchmarks, supporting attribution faithfulness (Section 4.2, Table 1).
- Chemical withholding: The attribution share assigned directly to fluorine atoms increases from approximately 8% to 42%, with similar localization for withheld sulfone groups, supporting sensitivity to missing chemical training coverage (Section 4.1, Figure 3b; Appendix H.4, Figure 8).
- Prediction versus uncertainty attribution: MUG achieves positive uncertainty fidelity, whereas prediction-based Grad×Input yields negative uncertainty fidelity (Section 4.3, Table 3).
- Architectures and ablations: configurations satisfying the assumptions achieve completeness residuals of approximately –; ablations illustrate where the guarantee no longer holds (Section 4.5; Appendix H.8, Table 28).
- Molecular design: MUG-guided fragment selection increases the docking hit rate from 47.2% to 56.2% compared with random fragment selection within the same framework (Section 5, Table 6).
We hope our responses and revisions address the Reviewer’s concerns. We would be happy to clarify any remaining questions during the discussion period.
Response to Reviewer d3xZ (1/2)
We thank the reviewer for the questions on novelty and on the exact decomposition. We address each of them below, and all changes are highlighted in blue in the revised paper.
W1/Q1: What is the main novelty of MUG?
We thank the Reviewer for raising this question about MUG’s novelty. As Reviewer 4uic helpfully pointed out, Bley et al. (2025) provide an elegant and general framework for explaining predictive uncertainty through second-order effects. Their work clearly shows the value of investigating gradient methods for uncertainty attribution and offers an important reference point for discussing MUG. We have clarified this connection in the revised manuscript (Section 2; Appendix E). Please refer to our response to Reviewer 4uic for a detailed comparison between MUG and Bley et al. (2025).
In this context, MUG contributes an efficient formulation for molecular GNNs that obtains atom-level uncertainty attributions by directly differentiating predictive variance, without separately explaining each ensemble member’s prediction. We focus on final-layer atom representations, where the stated readout and head assumptions allow a signed, exact decomposition of predictive variance through a single backward pass, without retraining (Section 3.2, Theorem 1). Atom contributions can then be summed directly over any molecular fragment (Section 3.2). Our experiments demonstrate both the chemical relevance and practical usefulness of these attributions. Controlled fluorine/sulfone withholding experiments show that MUG localizes uncertainty to chemical substructures absent from training (Section 4.1, Figure 3b; Appendix H.4, Figure 8). In molecular design, these attributions guide fragment-level acquisition, increasing the docking hit rate from 47.2% to 56.2% compared with random fragment selection within the same acquisition framework (Section 5, Table 6). In the rebuttal version, we have also clarified the relationship to Integrated Gradients (Section 3.2) and added an input-space comparison targeting the same predictive variance (Appendix H.2, Table 16).
Additionally, we note that novelty of MUG lies also in applicability of this method in chemistry and drug design, where many major architectures (such as GNNs) can be easily adapted to use MUG without architectural changes. It also provides more interpretability for chemists operating on functional groups rather than single atoms (the completeness property, please also see our response to W1 of Reviewer wtzS).
Bley, F., Lapuschkin, S., Samek, W., & Montavon, G. (2025). Explaining Predictive Uncertainty by Exposing Second-Order Effects. Pattern Recognition, 160, 111171.