acceptodds
Under review as a conference paper at ICLR 2027

When Does Noise Sensitivity Reveal Head-Accessible Shortcuts?

Abstract

Can mild input noise reveal which coordinates a frozen vision model uses as shortcuts? We measure the class-conditional change of each pooled representation coordinate and use this score in Noise-based Class-Conditional Sensitivity Pruning (NCSP). The method masks sensitive connections in each row of a linear classifier and refits the unmasked head parameters on clean labeled data. The score directly measures brittleness; interpreting it as a shortcut signal additionally requires nuisance-linked coordinates to be head-accessible, coordinate-aligned, and more probe-sensitive than stable semantic alternatives. Under sub-exponential tails, empirical top- masks nearly minimize retained sensitivity mass; the constrained refit is convex and norm-controlled; and cross-row conditions yield prediction and KL-stability bounds. The full pipeline raises BAR accuracy from to , Waterbirds WGA from to , MetaShift WGA from to , and FB-CMNIST accuracy at from to . Matched controls qualify these gains: clean-data head refitting accounts for a large share of the improvement, sensitivity masks do not uniformly beat random or magnitude-based masks, and a 20-class stress test shows no sensitivity-specific advantage. NCSP is therefore a conditional audit and intervention for a particular representation and probe, not a general guarantee of worst-group performance.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
spurious correlations, spurious features, representation learning, noise sensitivity, feature accessibility, linear probing, classification heads, robustness, shortcut learning, distribution shift, deep neural networks

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_h3jmOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether adding small amounts of noise to images can help identify shortcut-related features in a trained vision model. The authors propose Noise-based Class-Conditional Sensitivity Pruning (NCSP), which measures how much each feature in the backbone’s representation changes when noise is added to the input. These sensitivity scores are calculated separately for each class, and the connections corresponding to the most sensitive features are set to zero in each row of the classification head. The backbone remains frozen, while the remaining classification-head weights are retrained using clean labeled data without requiring nuisance or group annotations. However, noise-sensitive features do not necessarily represent shortcuts, and the method is effective only when shortcut-related features are sufficiently distinguishable from stable semantic features. The method is evaluated on BAR, Waterbirds, MetaShift, and FB-CMNIST, showing substantial improvements over the original ERM baseline.

Critical strengths
  1. NCSP is computationally lightweight. It freezes the backbone and modifies the classification head after identifying the target feature dimensions.
  2. The paper does not claim that every noise-representative feature explains a shortcut.
  3. The evaluation includes multiple spurious-correlation benchmarks and reports both mean and worst-group performance. The experimental section also transparently indicates that head refitting and random pruning are competitive with each other. And the paper does not claim universal success.
Critical weaknesses
  1. NCSP has considerable overlap with EvA, DFR, and AFR.
  2. I am not sure about the exact definition of a shortcut used here. A shortcut can be a predictive feature based on how the dataset actually is, i.e., how samples of a particular class follow a certain distribution. A spurious feature, on the other hand, is not expected to remain reliable under the distribution shifts of interest. The authors explicitly define these “nuisance” attributes as noncausal to the target label. That gives them a task-dependent definition of what counts as a shortcut. However, NCSP itself does not establish which features are causally related to the label.
  3. The method measures noise sensitivity, but it remains unclear whether that measurement reliably identifies shortcut-related features rather than simply unstable semantic features.
Questions for authors
  1. What is the key methodological contribution of NCSP beyond EvA-E’s class-specific feature erasure and constrained head refitting?
  2. What evidence demonstrates that Gaussian-noise sensitivity is a more effective criterion for identifying shortcut-related features?
  3. The method assumes that shortcut-related coordinates are more sensitive to Gaussian noise than semantic features. Please clarify.
  4. How can this assumption be independently verified on a new dataset without prior knowledge of the shortcut or access to shortcut labels?
  5. How does NCSP distinguish genuinely useful semantic features from harmful shortcuts when both may be sensitive to Gaussian noise?
  6. Which exact ImageNet-pretrained ResNet-18 checkpoint and training recipe were used? Were the same pretrained weights and initialization protocols used by the competing methods? Were the baseline results in Tables 1 and 2 independently reproduced or taken from their respective publications? In particular, could the authors clarify the apparent discrepancies between the reported EvA-C results and those in the original EvA paper (ICLR 2025)?
  7. Can the authors provide a controlled comparison of NCSP, EvA-E, and head-only refitting using identical initial ERM checkpoints, data splits, optimization budgets, and validation protocols to isolate the contribution of noise-sensitivity-based pruning?
3 comments

We thank the reviewer for the questions and for highlighting NCSP's lightweight head-only intervention, its evaluation of both average and worst-group performance, and its careful distinction between noise sensitivity and shortcut identity. We also appreciate the acknowledgement that the paper reports competitive refitting and random-pruning controls. The reviewer’s questions ask us to sharpen the methodological comparison and explain the evidence, assumptions, and settings. We respond point by point below and will add an explicit paragraph describing the differences from EvA.

Q1: Contribution beyond EvA, DFR, and AFR

MethodMain signal or choiceHow the classifier is adapted
DFRRefitting data in which the spurious correlation is brokenRetrains the final layer on that data.
AFRWeights based on the base model's predictionsRefits the final layer while emphasizing poorly predicted examples.
EvA-CActivation differences relative to an unbiased reference setErases class-specific connections and refits the head.
EvA-EConfidence-related contribution of a feature to predictionErases class-specific connections and refits the head.
NCSPFeature change between paired clean and perturbed inputs, measured within each classMasks sensitive connections and refits the head under those constraints.

NCSP shares head adaptation with these methods and class-specific erasure with EvA. Its contribution has three parts:

  • A different observable: the score measures feature response to a probe, rather than clean prediction contribution or example difficulty.
  • A selection objective: removing the largest class-conditional sensitivities minimizes retained sensitivity at the population level for a fixed budget; the analysis bounds the estimation error in this choice.
  • A connection to the refitted model: norm control and retained sensitivity yield conditional logit-stability bounds, with additional assumptions for nuisance recovery and multiclass conclusions.

For fixed features and class labels, the score Sc,j=E[∣ϕj(X+ε)−ϕj(X)∣∣Y=c]S_{c,j}=\mathbb{E}[|\phi_j(X+\varepsilon)-\phi_j(X)|\mid Y=c] does not depend on the current head weights or confidence. Two coordinates can contribute similarly to clean predictions while responding differently to small input changes. NCSP measures that difference. This provides a distinct, measurable basis for selecting connections, supported by the estimation and stability analysis.

Q2: What evidence supports Gaussian sensitivity?

Table G.1 supports a useful, metric-dependent contribution from the sensitivity constraint. We reproduce its results below; entries are percentages.

SystemBAR accuracyWaterbirds Avg./WGAMetaShift Avg./WGA
Original ERM60.5190.10 / 63.7483.30 / 69.60
Clean-data head refit only91.9398.30 / 74.6091.45 / 86.98
Sensitivity mask only, no refit68.9991.83 / 40.4988.21 / 68.16
Random equal-sparsity mask + refit90.7197.58 / 81.7291.89 / 86.78
Weight × sensitivity mask + refit92.0897.39 / 78.4291.91 / 86.22
NCSP89.6497.11 / 79.4392.49 / 86.05

Relative to refit-only, NCSP adds 4.83 points in Waterbirds WGA and 1.04 points in MetaShift average accuracy. It also exceeds random masking by 0.60 points in MetaShift average accuracy. Random masking achieves higher Waterbirds WGA, and refit-only is stronger on BAR and MetaShift WGA. These are descriptive comparisons; Table G.1 supplies no uncertainty estimates for these differences.

The reverse-pruning and predictive-drift experiments test whether the score identifies removable features and whether the intervention reduces response to the probe. Together, they support using sensitivity to study feature removability and predictor stability, complementing the downstream comparisons in Table G.1.

Q3: The sensitivity-gap assumption

Section 3.1 defines the target shift through changes in P(Z∣Y)P(Z\mid Y) that preserve label semantics. An attribute can predict Y in training and still be unreliable under that shift. The definition depends on the target task and shift; identifying causal attributes requires additional evidence.

Assumption 4.2 requires a head-accessible nuisance support BcB_c satisfying min⁡j∈BcSc,j>max⁡j∉BcSc,j+γc.\min_{j\in B_c} S_{c,j}>\max_{j\notin B_c}S_{c,j}+\gamma_c. If the uniform estimation error is at most ϵN\epsilon_N, the gap exceeds 2ϵN2\epsilon_N, and the budget covers BcB_c, Proposition 4.3 establishes nuisance-support coverage. Without nuisance alignment, the same proposition still bounds retained sensitivity relative to its population optimum.

Lemma 4.1 provides a local-gradient interpretation of the finite-noise score. It does not prove that Gaussian noise selectively changes nuisance information, or that a given architecture or training recipe inevitably creates the gap. The recovery claim therefore depends on the stated gap assumption.

Q4: Verification on a new dataset without shortcut labels

Sensitivity alone cannot independently certify nuisance alignment. Appendix A.7 considers two possible representations with the same observed sensitivity vector but different nuisance coordinates. A sensitivity-only rule cannot recover the nuisance correctly in both cases.

Label-only measurements can check ranking stability, removability, prediction drift, and whether random or bottom-sensitive masks behave similarly. They can identify reasons to reject an intervention. They cannot confirm that the removed coordinates are causal shortcuts, certify unseen-group performance, or validate the gap's semantic interpretation. We do not present the recommended abstention extension as an already validated screening rule for new datasets.

Q5: Sensitive semantic features

When useful and nuisance features have overlapping sensitivity, NCSP cannot generally distinguish them. A high score marks a pruning candidate without establishing its semantic role. Row-specific masks avoid deleting a coordinate from every class indiscriminately, but they do not guarantee that the selected connections are dispensable. Refit cannot restore a useful connection that is permanently masked. If stable alternatives are missing, the method can fail.

Useful features can also be sensitive. The estimation and optimization results still apply under their stated assumptions, while shortcut recovery requires the additional separation condition.

Q6: Initialization, training details, and EvA provenance

The paper specifies an ImageNet-initialized ResNet-18 for the main experiments, followed by ERM training and a frozen backbone during NCSP. Its hyperparameter table gives the following head-refit settings:

SettingWaterbirdsBARMetaShiftFB-CMNIST
OptimizerAdamSGDAdamSGD
Learning rate0.010.010.0010.01
Epochs30305030
Batch size64128128128
Sensitivity examples per class100100500100
Noise level0.20.20.20.1
Pruning ratio0.250.250.250.25

These settings are already reported in the manuscript. We will expand the initialization details to identify the ImageNet weights and starting checkpoints for the comparisons, and distinguish initial ERM training from head refitting.

Our EvA entries are in-house implementations. We requested code from the authors but did not receive it. We will label the entries “our implementation” and explain the settings used. This will distinguish our experimental results from the original publication and make the baseline sources and settings clear.

Q7: Isolating the contribution of the score

Table G.1 already addresses the comparison with head-only refitting and alternative masks. It shows the gains and trade-offs discussed in Q2 and helps separate the complete pipeline's improvement from the effect of selecting coordinates by sensitivity. The main-table EvA entries provide an additional method comparison.

The two comparisons serve complementary purposes: Table G.1 examines the contribution of masking and refitting, while the main table compares the complete methods, including EvA-E. As explained in Q1, NCSP contributes a paired perturbation score, a guarantee for estimating its ranking, and an analysis linking the constrained refit to output stability. We will present these comparisons together with their experimental settings so that the role of the sensitivity score is clear.

NCSP contributes a distinct perturbation-based selection signal and a mathematical account of how that signal relates to the refitted classifier. We believe this is a strong foundation for work on frozen-model adaptation and for studying alternatives to confidence- or activation-based feature selection. The comparison table and question-by-question responses make that contribution explicit while crediting the shared ideas from prior work. We would be happy to provide further explanation of the EvA distinction, recovery conditions, or experimental setup if the reviewer would find it helpful.

Dear Reviewer h3jm,

Thank you for the seven focused questions. Our response answers each, and we summarise the main points here:

  • Q1 and Q7, contribution beyond EvA, DFR and AFR. A comparison table separates the signal each method uses from how it adapts the head. NCSP's score is a paired clean/noisy response that does not depend on head weights or confidence, and the paper adds an estimation bound for the ranking and stability bounds for the constrained refit. Table G.1 isolates the score's contribution against refit-only and alternative masks.
  • Q2, Q3 and Q5, evidence and the sensitivity assumption. We reported Table G.1 as it stands, including where refit-only and random masks are stronger, and stated that the sensitivity gap is an assumption (Assumption 4.2) rather than a consequence of Lemma 4.1. When useful and nuisance features are both sensitive, NCSP cannot separate them, and the paper reports such failures.
  • Q4, verification without shortcut labels. Label-only diagnostics can reject an intervention but cannot certify it, and we do not claim the abstention rule is validated.
  • Q6, checkpoints and EvA provenance. The full head-refit settings are in the paper; we will add the exact ImageNet weights and starting checkpoints. The EvA entries are our own implementation; we requested the authors' code and did not receive it, and we will label the entries as ours.

These responses address all seven questions. With two days left in the discussion period, we kindly ask you to reconsider your score in light of them, and we remain available for any further question.

Best regards, The Authors

Reviewer_GeA6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether adding mild noise to images can reveal features that a trained classifier uses as shortcuts, such as relying on backgrounds instead of the objects being classified. It proposes NCSP, which measures how much each feature changes under noise, blocks the most sensitive connections separately for each class, and retrains the final classifier while keeping the feature extractor fixed.

The contributions are a sensitivity-based intervention, a theoretical analysis of the conditions under which it can identify shortcut features, and experiments examining when it helps or fails. The method requires ordinary class labels but does not require shortcut labels during the intervention.

Experiments on BAR, Waterbirds, MetaShift, and a colored-MNIST variant show substantial improvements over the original models. However, retraining the final classifier alone explains much of the improvement, and randomly blocking connections sometimes performs equally well or better. Additional experiments, including CelebA and changes to the feature representation, demonstrate limitations. The paper therefore presents noise sensitivity as a potentially useful diagnostic under specific conditions.

Critical strengths

The paper addresses a useful practical problem. Improving an already trained model without retraining its feature extractor is relevant when full retraining is expensive. Avoiding the need for shortcut labels also makes the setting practically interesting. Additionally the proposed solution is relatively simple. It measures feature changes under noise, removes selected classifier connections, and retrains the remaining classifier weights. The core procedure is straightforward to describe and inspect.

The theoretical analysis provides conditional support for the method and the experimental study includes informative controls and failure cases. The paper explains when sensitivity rankings can be estimated reliably and establishes properties of the retraining problem. The distinction between individual class scores and the final multiclass prediction is also useful. Comparisons with classifier retraining alone and random masking help identify where the improvements come from. The authors also examine different noise types, pruning amounts, estimation budgets, and architectures. The unsuccessful settings provide useful evidence about the method's limitations.

Critical weaknesses

The experimental evidence does not establish the added value or generalizability of NCSP. The main comparisons use only ResNet-18, while the additional architectures lack the controls and repeated runs needed to establish whether sensitivity-based selection generalizes. Moreover, much of the improvement comes from classifier retraining alone, and NCSP does not consistently outperform random masking. For example, random masking achieves higher worst-group accuracy on Waterbirds in Table G.1. EvA [1] also provides a relevant baseline, as it removes class-specific classifier connections and retrains the remaining weights while keeping the feature extractor fixed. Comparisons under identical conditions, with uncertainty estimates, are needed to establish what NCSP adds beyond these alternatives. The strongest colored-MNIST result needs the same controls. The paper should also clarify the sources, distributions, and overlap of the sensitivity-estimation, retraining, validation, and test data, as well as the weighting and aggregation conventions used for average and group accuracies.

The validity of the shortcut diagnostic and its practical use remain insufficiently established. The diagnostic experiments do not clearly distinguish shortcut information from class information. When class labels and shortcut attributes are correlated, a feature that identifies the class may also predict the shortcut attribute, without necessarily encoding or relying on the shortcut. Within-class tests or controlled changes to the shortcut, such as changing only the background, would provide stronger evidence. Similarly, the theoretical analysis assumes a strong separation between shortcut features and other features in their sensitivity to noise, but reliable estimation of sensitivity does not establish that this separation holds in practice. Nor does reducing sensitivity guarantee better worst-group accuracy. Finally, although the paper suggests retaining the original model when the diagnostic is unsupported, it does not evaluate a fixed screening rule on new settings. Preserving average validation accuracy alone is insufficient because worst-group accuracy may still decrease.

[1] He, Q., Xu, K., & Yao, A. (2025). EvA: Erasing Spurious Correlations with Activations. The Thirteenth International Conference on Learning Representations.

Questions for authors
  1. What does sensitivity-based selection add beyond classifier retraining and random masking? Can you provide comparisons on identical checkpoints, with uncertainty across runs, and include these controls for the main colored-MNIST results?

  2. How do the diagnostic experiments distinguish shortcut information from correlated class information? Can the selected features predict the shortcut within a fixed class, and does changing only the shortcut-such as the background-confirm that those features affect the model's predictions?

  3. Which data are used at each stage of the method? Please clarify the sources, distributions, and overlap of the sensitivity, retraining, validation, and test sets, together with the weighting conventions needed to reconcile the reported average and group accuracies.

5 comments

We thank the reviewer for the thorough and constructive assessment and for recognizing the practical value of adapting a frozen model without shortcut labels, the simplicity of the procedure, and the informative theoretical analysis, controls, and failure cases. The reviewer’s questions focus the discussion on what sensitivity selection adds, how to interpret the shortcut diagnostics, and how the data and metrics are defined. We answer these as Q1–Q3, using Table G.1 and the analysis in the paper, then discuss architecture coverage and screening.

Q1 response: What does sensitivity selection add?

For clarity, we reproduce Table G.1 below. All entries are percentages, and the Waterbirds and MetaShift cells report average accuracy/WGA.

SystemBAR accuracyWaterbirds Avg./WGAMetaShift Avg./WGA
Original ERM60.5190.10 / 63.7483.30 / 69.60
Clean-data head refit only91.9398.30 / 74.6091.45 / 86.98
Sensitivity mask only, no refit68.9991.83 / 40.4988.21 / 68.16
Random equal-sparsity mask + refit90.7197.58 / 81.7291.89 / 86.78
Weight × sensitivity mask + refit92.0897.39 / 78.4291.91 / 86.22
NCSP89.6497.11 / 79.4392.49 / 86.05

Relative to refit-only, NCSP improves Waterbirds WGA by 4.83 percentage points and MetaShift average accuracy by 1.04 points. Relative to random masking, it improves MetaShift average accuracy by 0.60 points. It trails refit-only on BAR and MetaShift WGA, and random masking has higher Waterbirds WGA.

The contribution combines a sensitivity measurement, a head intervention, and an analysis of when it is useful. NCSP supplies a class-conditional measurement of representation brittleness, a rule for using that measurement to constrain the head, and an analysis of the conditions under which the intervention suppresses nuisance dependence. Head refitting alone supplies no sensitivity ranking. Random masks supply a capacity control but no estimate of which coordinates respond to the probe. Proposition 4.3 characterizes how accurately the ranking can be estimated; the post-refit analysis connects retained sensitivity mass to logit stability under its stated conditions. These results explain the stability objective, while Table G.1 measures the resulting accuracy trade-offs.

Mask-only also performs poorly on Waterbirds WGA. That result supports evaluating the full mask–refit procedure rather than attributing the ERM-to-NCSP gain to erasure alone. We will make the comparison against refit-only and random masks explicit wherever we discuss the main-table gains.

Q2 response: Do the diagnostics distinguish shortcuts from class information?

Conditioning sensitivity estimation on the target class makes the ranking specific to a classifier row. It does not remove class–nuisance confounding from a separate nuisance-decoding experiment. A coordinate can predict Z through its association with Y, and predictability does not establish that the head relies on that information.

The existing diagnostics answer narrower questions. Reverse pruning tests whether the most sensitive coordinates are more removable than stable or random coordinates at a given budget. The nuisance probes test predictiveness. The KL experiment measures predictive drift under the chosen perturbation. Together, these examine different consequences of the proposed mechanism, but none alone constitutes a controlled background intervention.

Estimating sensitivity reliably and identifying shortcuts are separate claims. Assumption 4.2 specifies sensitivity–nuisance alignment; Proposition 4.3 controls sensitivity estimation conditional on the statistical assumptions. Reliable estimation cannot establish the alignment assumption. Appendix A.7 also gives an indistinguishability example showing why sensitivity alone cannot identify nuisance roles without additional structure.

A within-class nuisance probe would address the class-confounding objection more directly, and a background-only intervention would test prediction dependence. We will describe the existing diagnostics at their measured scope and avoid treating nuisance predictiveness or reduced noise drift as causal identification.

Q3 response: Which data and accuracy measures are used?

The manuscript reports dataset sizes of 4,795/1,199/5,794 for Waterbirds, 1,941/327/327 for BAR, 1,608/288/288 for MetaShift, and 60,000/12,000/10,000 for FB-CMNIST, in train/validation/test order. It specifies sensitivity budgets of 100 examples per class for Waterbirds, BAR, and FB-CMNIST, and 500 for MetaShift. The backbone is frozen during sensitivity estimation and head refitting; the perturbation is used to compute the score, while refitting uses clean labeled examples.

We will make the data flow easier to follow by specifying the source distribution, example reuse, and checkpoint-selection role at each stage. Here, “clean” refers to examples without probe noise. This clarification will distinguish sensitivity estimation, head refitting, validation, and final evaluation.

For metric reporting, sample accuracy is the number of correct predictions divided by the number of evaluated examples; an equal-group mean averages group accuracies; WGA takes their minimum. A distribution-adjusted average requires explicit reference weights. We will state the convention for each reported average and identify the checkpoint and evaluation population shared by any overall/group breakdown. An unweighted group mean can differ from sample-weighted accuracy because groups can have different sizes.

Other concerns: New architectures and screening

The additional backbones demonstrate applicability beyond ResNet-18. The CelebA, coordinate-basis, and many-class experiments help characterize how the outcome depends on the representation and task.

Preserving average validation accuracy cannot certify WGA: a minority group's loss may be outweighed by gains elsewhere. Appendix G reports this limitation through the label-only selection audit. We therefore recommend using the diagnostics to inform screening and retaining ERM when the evidence does not support intervention. The selection audit explains why average validation accuracy should be considered alongside the other diagnostics.

We appreciate the reviewer’s emphasis on component attribution and diagnostic validity, and have clarified how the controls and analysis address these points. Our confidence in NCSP rests on the combination of an inspectable intervention, a formal sensitivity objective, and experiments that distinguish pipeline gains from mask-selection effects. Researchers can build on this framework to compare adaptation rules and investigate how feature sensitivity relates to performance under distribution shift. We would be glad to clarify any remaining question about the controls, data flow, or interpretation of the diagnostics.

We thank the reviewer for the thorough and constructive review. Our response answers each of your three questions. On what sensitivity adds, we reproduced Table G.1 as it stands: NCSP gains 4.83 WGA on Waterbirds and 1.04 average on MetaShift over refit-only, trails on BAR and MetaShift WGA, and sits below random masking on Waterbirds WGA. The contribution is the measurement, the constrained refit and the analysis of when it helps, and we will add spread across repeated runs so the differences can be judged. On shortcut versus class information, we agree the probes cannot separate the two, and we now describe each diagnostic at its measured scope rather than as causal evidence. On data flow and metrics, we listed the splits, budgets and head-refit settings and will add a per-stage table with the convention behind every reported average. On screening, we agree average validation accuracy cannot certify WGA; the Appendix G audit shows that.

These responses address each of your three questions and the screening concern. As the discussion period ends in a few days, we kindly request you to reconsider your assessment, and we remain available for any further question.

Best regards, The Authors

Reviewer_TR6sOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

In this work, authors suggest that the predictions of visual classifiers may rely on environmental cues such as background, texture, and color. The authors regard this as shortcut reliance, and changes in these shortcut features may cause the classifiers to fail. The authors argue that if shortcut-linked features are more sensitive to mild input noise than stable semantic features, noise sensitivity can help locate and suppress these shortcut features. Based on this assumption, they propose Noise-based Class-Conditional Sensitivity Pruning (NCSP), which prunes connections to highly sensitive feature coordinates in the linear head.

Critical strengths

The method is clearly presented and comprises a well-defined workflow, encompassing class-conditional sensitivity estimation, connection masking, and constrained head refitting. And, the paper provides theoretical analysis regarding sensitivity estimation, the convexity and norm control involved in refitting, and output stability, while also explicitly stating the relevant assumptions and theoretical guarantees.

Critical weaknesses

The writing falls well below the standard expected for an academic paper. The motivation is not clearly articulated in the introduction section.

The novelty of NCSP is very limited. It seems like retaining the framework of the prior work EvA [1] and merely replaces the original scoring metrics with noise sensitivity. What is the motivation for doing so? The introduction currently provides no corresponding explanation. More explanation should be added, such as, Do EvA’s two metrics, Consistency and Evidence Energy, have any specific shortcomings? How does noise sensitivity address these shortcomings? Is it necessary to introduce a new metric?

The authors claim that mild perturbations can expose brittle features, and if shortcut-linked features are more brittle than semantic features, connections to prune can be selected on this basis. Currently, the paper only stipulates this correspondence through assumptions. However, why should shortcut-linked features be more sensitive? In which model architectures does this relationship arise? Does some training process cause shortcut-linked features to be more brittle than semantic features against noise? These questions remain unanswered. Therefore, the rationale for using noise sensitivity as a pruning metric has not been sufficiently justified.

[1] Qiyuan He, Kai Xu, and Angela Yao. Eva: Erasing spurious correlations with activations. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=zKvrOOBouT.

Questions for authors

As shown in weakness section.

5 comments

We thank the reviewer for examining the motivation and theoretical basis of NCSP, and for recognizing the clearly defined workflow, the analysis of sensitivity estimation and head refitting, and the explicit assumptions behind the stability guarantees. The reviewer’s questions give us the opportunity to clarify why perturbation response is worth measuring and when it can reveal shortcut-linked features. We address the distinction from EvA, the theoretical contribution, and the mechanism behind the sensitivity gap below, with a direct comparison and explanations tied to the paper's analysis and experiments.

1. Why measure sensitivity, and how does NCSP differ from EvA?

A feature can contribute strongly to a clean prediction while reacting either strongly or weakly to a small input change. Clean feature contribution and response to perturbation answer different questions. NCSP measures the latter to identify connections through which input changes can affect the classifier.

EvANCSP
What is measured?EvA-C compares activation distributions; EvA-E measures confidence-related feature contribution.NCSP measures the change in each feature between a clean input and its perturbed copy.
Information used by the scoreEvA-C uses an unbiased reference set; EvA-E uses the classifier's confidence and feature contribution.NCSP uses paired clean/noisy features and target-class labels. The score itself does not use head weights or confidence.
Question answeredWhich activations differ across the reference distributions or provide strong prediction evidence?Which coordinates change most under the chosen perturbation, within each class?
InterventionClass-specific erasure followed by head refitting.Class-specific sensitivity masks followed by constrained head refitting. Only this intervention structure is shared.
Analysis developed in our paperAcknowledged as prior work on activation erasure.Estimation error for the sensitivity ranking, retained sensitivity after masking, refit norm control, and conditional output-stability and nuisance-recovery results.

The score is

Sc,j=E[∣ϕj(X+ε)−ϕj(X)∣∣Y=c].S_{c,j}=\mathbb{E}[|\phi_j(X+\varepsilon)-\phi_j(X)|\mid Y=c].

Two features can have the same value on a clean example and different slopes around it. Paired perturbations expose that difference. This motivates measuring sensitivity even when an activation-based score is already available. The method needs no semantic name for a coordinate and no nuisance annotation to compute its score. The reference-data requirement above concerns score estimation; head refitting still requires clean labeled data.

We will add an explicit comparison paragraph to the introduction and credit EvA's shared erasure/refit structure. Our EvA baselines are our implementations of the published method: we requested the authors' code but did not receive it.

2. What the theory proves

The analysis proves useful properties of the measured score before making any claim about shortcut identity. The nuisance-gap assumption is needed for nuisance recovery; the other results have their own stated sampling, regularity, and optimization conditions.

ResultMeaning in simple terms
Lemma 4.1Relates finite-noise feature change to input-gradient response, while accounting for changes along the perturbation path.
Proposition 4.3Bounds the error in estimated sensitivities and how far the selected mask is from minimizing retained sensitivity.
Proposition 4.4Establishes convexity of the fixed-mask head-refit objective and norm control with positive regularization.
Proposition 4.5Bounds the change in a class logit on examples from that class after refitting.
Appendix A.5States the additional cross-row conditions needed for multiclass prediction and KL bounds.
Assumption 4.2 and the recovery analysisIdentify when a sufficiently accurate sensitivity ranking covers nuisance-linked coordinates.

For example, Proposition 4.3 gives

\qquad R_c(P)=\sum_{j\notin P}S_{c,j}.$$ Here, $R_c$ is the total sensitivity left after pruning and $\epsilon_N$ bounds estimation error. **The selected mask is close to the best mask for this measured objective, without needing to label the coordinates as nuisance features.** Interpreting it as nuisance removal requires the additional alignment assumption. Appendix A.7 explains why such additional information is necessary: identical sensitivity scores can correspond to different semantic and nuisance roles.

3. Why can shortcuts be sensitive, and when should we expect a gap?

The rationale combines a local response mechanism, diagnostic experiments, and an explicit account of failure cases. A feature that changes rapidly around an input has a larger noise response. Lemma 4.1 makes this connection precise; when the path-variation remainder is small,

Sj(x;σ)≈2/π σ∥∇ϕj(x)∥2.S_j(x;\sigma)\approx\sqrt{2/\pi}\,\sigma\|\nabla\phi_j(x)\|_2.

Why might this affect shortcut features? Background textures or fine spatial patterns can change rapidly under small perturbations, while a representation may also contain more stable object information. Appendix A.8 gives frequency as one mechanism: higher-frequency patterns can have larger input gradients. The spectral analysis reports greater high-frequency energy in the sensitive BAR features. This supports the mechanism in that setting; a feature's frequency alone does not establish whether it is a shortcut.

The paper also examines the mechanism through reverse pruning, prediction drift under noise, and downstream shift performance. Reverse pruning asks whether sensitive coordinates can be removed with less damage than stable coordinates. The KL analysis tests the response of the resulting predictor. These tests examine the proposed mechanism through feature removability, predictive stability, and shift performance.

Which architectures? The main experiments use ResNet-18, and Appendix F reports applications to ResNet-50, ConvNeXt, and ViT-B/16. The relevant condition is the structure learned by a particular backbone: nuisance information must be accessible through the head, concentrated in coordinates that the mask can remove, and more sensitive than useful alternatives. Architecture names alone do not establish that condition. The additional backbones show applicability.

What role does training play? ERM can reward cues that predict labels in the training distribution, including background or texture cues. Whether those cues become more sensitive depends on their encoding and on the chosen perturbation. NCSP measures this property directly in the trained representation, allowing the diagnostic to reflect the features the model has learned.

Stable shortcuts and sensitive semantic features can defeat the ordering. Assumption 4.2 states the favorable case explicitly, and the CelebA failure and coordinate-basis stress test examine boundaries of the method. The scientific contribution is to make this relationship measurable and analyzable, with recovery guarantees when the conditions hold and documented limits when they do not.

We are confident in the technical contribution of connecting a measurable perturbation response to feature selection and constrained-refit stability. This gives the research community a way to study which connections carry brittle responses in frozen representations and to develop sensitivity-based pruning objectives with explicit estimation guarantees. We hope the comparison and mechanism discussion address the reviewer’s concerns about motivation and novelty. If a step in the sensitivity-gap argument remains unclear, we would welcome the opportunity to explain it further.

Dear Reviewer TR6s,

Thank you for pressing us on the motivation and on why noise sensitivity deserves to be a pruning criterion at all. Those were the right questions, and they shaped our response. On novelty\color{blue}{\textbf{On novelty}}, the comparison table in our reply shows that NCSP borrows only the mask-and-refit skeleton from EvA; the quantity it measures (a paired clean/noisy feature response, with no head weights, confidence or reference set) and the guarantees built on it (estimation, norm control, stability) are not in EvA. On the mechanism\color{blue}{\textbf{On the mechanism}}, Lemma 4.1 connects the score to the input-gradient norm, Appendix A.8 explains why high-frequency background and texture cues acquire larger gradients and shows this spectrally on BAR, and reverse pruning and KL drift test it. We do not claim the gap always holds; Assumption 4.2 states when it does and the paper reports where it does not. On writing\color{blue}{\textbf{On writing}}, we will restructure the introduction so that motivation, the EvA comparison and the mechanism come before the method.

These responses address each of the concerns in the review. As the discussion period ends in a few days, we kindly ask you to reconsider your score in light of them. We remain available for any further question.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Response

Dear AC and Reviewers,

Thank you for the careful and constructive feedback. To support the final decision, we summarise below how the main strengths and concerns have been addressed, with reviewer references for clarity (TR6s\color{purple}{\text{TR6s}}, GeA6\color{teal}{\text{GeA6}}, h3jm\color{orange}{\text{h3jm}}). NCSP asks a simple question: can mild input noise reveal which coordinates a frozen model's head relies on as shortcuts? It gives a measurable answer (a class-conditional sensitivity score), a lightweight intervention (mask and refit the head, no backbone retraining, no shortcut labels), guarantees for the mask estimate and the refit, and controls that show where the gains come from and where the method fails. Our responses cover every question raised, and we hope the reviewers will consider them in their final scores.

  1. Overall assessment.\color{blue}{\textbf{Overall assessment.}} Reviewers found the workflow clearly defined with explicit assumptions and guarantees [TR6s\color{purple}{\text{TR6s}}], the frozen-backbone setting practical and the controls and failure cases informative [GeA6\color{teal}{\text{GeA6}}], and the intervention lightweight with transparent reporting of refit and random-pruning baselines [h3jm\color{orange}{\text{h3jm}}].

  2. Relation to EvA, DFR and AFR.\color{blue}{\textbf{Relation to EvA, DFR and AFR.}} The shared concern was novelty [TR6s\color{purple}{\text{TR6s}}, GeA6\color{teal}{\text{GeA6}}, h3jm\color{orange}{\text{h3jm}}]. Our side-by-side tables show that only the erasure/refit structure is shared: the NCSP score is a paired clean/noisy feature response that uses no head weights, confidence or reference set, and the paper adds the estimation bound (Prop. 4.3), refit norm control (Prop. 4.4) and stability bounds (Prop. 4.5) that EvA does not provide. We will add an explicit comparison crediting EvA to the introduction.

  3. Why shortcuts are more noise-sensitive.\color{blue}{\textbf{Why shortcuts are more noise-sensitive.}} Lemma 4.1 ties the score to the input-gradient norm, App. A.8 gives frequency as the mechanism with spectral evidence on BAR, and reverse pruning and KL drift test it [TR6s\color{purple}{\text{TR6s}}, h3jm\color{orange}{\text{h3jm}}]. The gap is stated as an assumption (4.2), and failures (CelebA, coordinate-basis, 20-class) are reported rather than hidden.

  4. Gains beyond refit-only and random masks.\color{blue}{\textbf{Gains beyond refit-only and random masks.}} Table G.1 is reported as is: +4.83 WGA on Waterbirds and +1.04 average on MetaShift over refit-only, behind on BAR and MetaShift WGA, below random masking on Waterbirds WGA [GeA6\color{teal}{\text{GeA6}}, h3jm\color{orange}{\text{h3jm}}]. Spread across repeated runs will be added.

  5. Diagnostics and screening.\color{blue}{\textbf{Diagnostics and screening.}} We agree that the nuisance probes do not separate shortcut from class information; each diagnostic is described at its measured scope, and the abstention rule is not claimed as validated [GeA6\color{teal}{\text{GeA6}}, h3jm\color{orange}{\text{h3jm}}].

  6. Data flow, metrics and provenance.\color{blue}{\textbf{Data flow, metrics and provenance.}} Split sizes, budgets and head-refit settings are listed; a per-stage data-flow table and the accuracy convention for each number will be added. EvA entries are our implementation (code requested, not received) and will be labelled so [GeA6\color{teal}{\text{GeA6}}, h3jm\color{orange}{\text{h3jm}}].

  7. Writing.\color{blue}{\textbf{Writing.}} The introduction will be rewritten to put motivation, the EvA comparison and the mechanism first [TR6s\color{purple}{\text{TR6s}}].

The remaining items are presentation changes for the revision; the method, theory and controls stand as submitted. With the discussion closing in a few days, we kindly request the reviewers to read the responses and, if their concerns are resolved, to revise their scores accordingly.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.