acceptodds
Under review as a conference paper at ICLR 2027

When Does Attribution Stability Improve Explanation-Guided Learning?

Abstract

Teacher attribution maps provide spatial guidance without region annotations, but the same teacher can highlight different regions across label-preserving views. Uniform alignment gives these inconsistent targets the same weight as repeatable ones. We propose Perturbation-Consistent Gated Attribution Alignment (PCGA), which averages class-matched teacher maps into a spatial target and uses their cross-view agreement to weight student alignment. Matched comparisons against uniform and prediction-derived weights test whether agreement improves learning. On Chest CT, gated cosine alignment improves AUC over conventionally tuned uniform alignment, although matching the auxiliary-gradient strength removes that advantage. Prediction-confidence weighting leads on RADGEN. Stability weighting has the highest mean AUC on CIFAR-10, with gains that vary across seeds. Gradient measurements and spatial interventions clarify the distinction: map agreement chiefly controls alignment strength in the Chest CT comparison and does not consistently identify regions that affect predictions. Cross-view agreement can therefore set the strength of attribution supervision, but its value as an image-ranking signal depends on the task.

Primary area:
interpretability and explainable AI
Keywords:
explanation-guided learning, attribution stability, teacher-student attribution alignment, gradient diagnostics, interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_fth3OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes PCGA, which averages teacher attribution maps across label-preserving perturbations and uses their agreement to weight student attribution alignment. Experiments on Chest CT, RADGEN, and CIFAR-10 compare stability weighting with uniform and prediction-derived weights, supplemented by gradient controls and spatial interventions. The results attribute the main Chest CT gain chiefly to auxiliary-supervision strength, while the incremental benefit of image-specific weighting remains inconsistent.

Critical strengths

The gradient analysis and scale-only control distinguish changes in auxiliary-supervision strength from the effects of image-specific weighting. These controls, together with transparent reporting of negative results, provide useful evidence about the limitations of attribution stability as a supervision criterion (§§3.3, 5.2).

Critical weaknesses
  • Insufficient support for the weighting criterion and its distinct methodological benefit. The experiments do not adequately establish that more stable explanations provide more useful student supervision. In the strongest Chest CT cosine comparison, PCGA’s advantage disappears after matching auxiliary-gradient strength: PCGA achieves 99.19% AUC versus 99.25% for scale-only uniform alignment (Table 1; §5.2). The CIFAR-10 advantage occurs in only one of three paired seeds, while the favorable norm-matched L1 follow-up is limited to validation results (Appendices F, I). These findings do not establish a reproducible incremental benefit from assigning stability-based weights to individual images.
  • Insufficient validation of when stability is useful. The reported variation across tasks, losses, and target configurations does not establish conditions that predict when stability weighting improves learning. The gradient decomposition characterizes changes in update scale and direction, but does not establish when those changes are beneficial. Although the authors acknowledge the negative findings, the analysis does not sufficiently justify why this stability score should determine supervision strength or when assessing it separately for each image adds value.
Questions for authors
  • What evidence establishes that the proposed stability score predicts an explanation’s downstream training utility and therefore justifies its use in determining supervision strength?
  • Which result demonstrates a reproducible benefit from image-specific stability weighting after controlling auxiliary-gradient strength?
  • What measurable condition predicts when stability weighting will improve learning, and where is that prediction independently validated?
Reviewer_PWfQOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors consider the effects of stability (across views) of attributions when learning from explanations. Where attributions from teachers are inconsistent across views, the authors suggest to down-weight the teachers attributions. This down-weighting is done systematically in a method they call Perturbation-Consistent Gated Attribution Alignment (PCGA). The alignment process is presented and then empirically tested on two medical imaging datasets. The experimental results, as far as I can ascertain, are weak and relatively inconclusive. 

My overall opinion is that while the paper targets and interesting problem in an interesting domain, the methodological novelty and interest is limited, the experimental results are too sparse to be conclusive, and the presentation and technical writing are not quite where they need to be for this paper to be full appreciated.

Critical strengths

The area the paper researches -- learning from explanations -- is a very interesting one. And the inconsistency the authors mention is potentially problematic for learning from explanations. In particular it is well documented that explanations are non-robust and it is also documented that learning from robust explanations yields much better results empirically.

Critical weaknesses

Technical Weaknesses: 
The problem is not well motivated. Attributions may be inconsistent across views, but why does this harm my ability to learn from explanations? In machine learning we encounter label noise all the time. I am not saying that inconsistency does not affect things, but I am saying that without this motivation in the introduction, methods, and demonstrated in the experiments, the papers impact is hindered. 

Methodologically, the approach is presented with a very particular instantiation. While this is fine, it gives little insight into why or how this approach is better than others. Rather than engaging with this on a theoretical level, the approach is explored through experimental ablations. 
 Experimentally, the results do not seem to demonstrate a clear benefit of the proposed method. Other baseline methods appear to be much stronger and less noisy. The fact that the results in Table 1, ostensibly the most important results of the paper, are not great makes it much less interesting to explore and dive into the ablations that are subsequently presented. Perhaps I have missed some critical points here and welcome the authors to correct my perspective that the experimental results are weak compared to baselines.

Rhetorical, Non-Technical Weaknesses:

Here I provide my thoughts on the presentation of the work which I think could be improved. These are, of course, all just my opinions on where the paper loses steam, clarity, or impact. The corresponding technical questions are below in the technical evaluation

I think your abstract and introduction should do more to set the scene. You dive directly into the problem without underscoring its importance. Without this, your argument does not feel anchored. By giving the clear case for impact and lineage of works on this problem you can better justify the methods you use and the argument you make. As written the abstract and introduction come off as flat to me. There have been many discussed motivations for learning from explanations, each of them promising in their own way — which of these promises is in jeopardy if we do not consider the stability of explanations? This will help the impact case of the paper. 

In your methodology, Section 3, avoid giving concrete values. These are not relevant to the method and only serve to make your methodological contribution feel pigeon-holed to the one setting you study. You should try to present your method in the most general light possible so that readers can evaluate your contribution absent the experimental details.

In your experimental tables, make sure to bold best performing methods so that I do not have to do all of the math myself as a reader when looking at the results. In experimental text, several of your short sentence structures read as AI-Generated and I struggle to get through the paragraphs without my eyes glazing over a bit. I think you can do better to tell me what you would like for me to take away from each experiment more clearly.

Questions for authors

Questions are presented in the weaknesses section of my review.

Reviewer_ZrVWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a method to use teacher attribution maps to inform student training. Perturbations of samples result in altered attribution maps, and the paper suggests incorporating these varied maps by weighing them accordingly. The authors perform experiments demonstrating cases where such methodology seems to help and where it does not

Critical strengths
  1. Good interesting questions and several arms to test such as experiments attempting to identify whether benefit arises from Teacher map specific information or simply reduced magnitude of gradient

  2. Honest reporting of results

Critical weaknesses

W1. Paper is very hard to parse and read. Lots of dense complex language and the organization makes it hard to understand the story. A rewrite focusing on clarity is probably required W2. Maybe more ablations. Why only GradCAM W3. Many confusing results eg. for RADGEN, deleting the highlighted region hurts the prediction less than deleting a random region W4. Results in tables do not seem strongly statistically significant W5. Some values seem to be underjustified (3 views, Top 25%)

Questions for authors

Q1: In Table 1 three seeds are used and most results are within 1 SD of each other. Are the authors confident that their interpreted results are not just due to randomness between runs? Q2. Could ablations be done with more and preferably 1-2 other methods of generating attribution maps eg. Integrated Gradients, in order to see if results hold Q3: How were certain parameters such as top 25% pixels for Ri in the salient-region control and V=3 obtained?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.