acceptodds
Under review as a conference paper at ICLR 2027

Counterexample-Guided Shortcut Suppression without Shortcut Labels

Abstract

Spurious correlations (SC) are correlated features in datasets that allow for better benchmark predictions, but are not robust to domain change and so might harm generalisation when relied upon. SC can be unintended, such as background bias in object classification and diagnosis markings in medical images, but they have also been used intentionally, such as in dataset protection. Traditional methods for addressing SC often require expert knowledge and task-specific operations, leading to recent research to detect and correct SC without putting shortcut labels in the training objective. In this paper, we follow this line of research, both when SC is unintended and when it is deliberately injected. We propose a method that does not use shortcut labels during training; instead, we create label-conflicting counterexample mixtures so shortcut-following predictions become costly. We study this as a clean-counterexample supervision regime: biased training data and a small clean counterexample pool are available, but shortcut labels are not used in the training objective. In a controlled hidden-watermark setting, the method suppresses watermark-only prediction to chance, validating the intended mechanism. On augmented Waterbirds, where the shortcut is a replaceable background cue, we compare against previous methods, attaining higher worst-group accuracy without domain labels in the objective. Additional CelebA results test a visually entangled attribute shortcut and show weaker, more variable gains. These results suggest that clean counterexamples can provide useful weak supervision for shortcut attenuation, with effectiveness governed by shortcut separability and counterexample quality.

Primary area:
causal reasoning
Keywords:
dataset watermarking, shortcut learning, causal reasoning, distributionally robust optimization, adversarial attacks, robustness

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YYASOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a counterexample-guided method that does not require shortcut annotations in the training objective. The authors apply MixUp or CutMix to combine biased samples with clean counterexamples from different classes, and use the labels of the clean counterexamples to supervise the mixed samples. This creates a training signal in which the shortcut evidence conflicts with the assigned label. Experiments on Hidden Watermarks, Waterbirds, and CelebA report reduced reliance on the original shortcuts and improved worst-group accuracy.

Critical strengths

The paper addresses a practically important problem: improving model robustness when per-sample shortcut annotations are unavailable. The proposed method is simple and can be readily integrated into existing training pipelines.

The Hidden Watermarks experiment provides a relatively clear demonstration that training with label conflicts can suppress reliance on a specified bias-related signal.

The paper also discusses the method’s dependence on shortcut separability and the quality of the counterexamples.

Critical weaknesses

“No shortcut annotations” does not mean “no shortcut knowledge.” The method still requires researchers to identify the factor to be suppressed in advance and to collect or construct clean counterexamples accordingly. It therefore appears better suited to suppressing prespecified attributes than to discovering previously unknown shortcuts.

The background in Waterbirds is not necessarily a shortcut. Habitat may contain genuine and useful ecological information. Whether a feature should be considered a shortcut depends on the task definition and the deployment distribution, rather than on correlation alone. Label-conflicting mixtures may introduce new shortcuts. The model may exploit mixing boundaries, region sizes, interpolation ratios, or differences between data sources instead of learning stable task-relevant features. The current experiments mainly demonstrate that reliance on the original shortcut is reduced, but they do not rule out the possibility of shortcut substitution.

Consequently, the paper’s claims regarding the learning of causal or invariant features appear somewhat overstated. The current evidence more directly supports the narrower claim that the method reduces measurable reliance on a specified attribute.

Questions for authors

Without prior knowledge of the type of potential shortcut, how can the proposed “clean counterexamples” be obtained or validated? What criteria do the authors use to determine that the background is a shortcut that should be removed, rather than legitimate contextual information?

Have the authors evaluated the model using previously unseen mixing strategies, naturally occurring unedited counterexamples, or data generated through different pipelines, to rule out the possibility that the model exploits mixing artifacts?

Can the authors provide further evidence that the model genuinely shifts toward task-relevant features, rather than merely replacing the original shortcut with a newly constructed one?

Reviewer_ptdeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper focuses on the detection and correction of spurious correlations. Authors study this issue as a clean-counterexample supervision regime, which uses biased training data and only a small clean counterexample pool while not using shortcut labels in the training objective. Experiments are conducted on augmented Waterbirds and CelebA datasets to demonstrate the effectiveness.

Critical strengths
  • The topic of this paper is important. Mitigating spurious correlations is indeed necessary in practical use.
  • The motivation is clear. Suppressing shortcut learning without directly utilizing shortcut labels in the training objective is unique.
Critical weaknesses
  • The proposed method mainly relies on existing techniques such as CutMix and MixUp, with different-class sample pairing and hard-label supervision. While the idea is intuitive, the technical contribution appears to be relatively limited.
  • The experimental improvements are not consistently convincing. On Waterbirds, the proposed Counterexample MixUp achieves better worst-group accuracy than standard MixUp, but its average accuracy is clearly lower.
Questions for authors
  • The presentation of Figure 3 is confusing and could be further improved. What core observation can be obtained from this figure?
Reviewer_8oe4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies shortcut suppression using a small, class-labeled clean counterexample pool, without shortcut labels in the training objective. It mixes biased images with clean images from other classes and assigns the clean image’s hard label.

Critical strengths
  1. The comparison with biased+clean training provides meaningful evidence beyond simply adding more clean data. Using the same counterexample pool, Counterexample MixUp improves Waterbirds WGA from 17.41% to 34.88% (Table 1).

  2. The experiments offer useful insight into when the method works. The watermark experiment shows reduced watermark predictiveness, while the counterexample-quality ablation shows substantially larger robustness gains with realistic background replacements than with blank backgrounds (Fig. 3; Table 2).

Critical weaknesses
  1. The main comparison does not isolate the proposed hard-label rule. Counterexample MixUp receives 240 off-axis clean donors, whereas standard MixUp and LISA use vanilla Waterbirds containing 240 minority examples (Table 1). Since counterexample quality substantially affects performance (Table 2), this difference matters. The biased+clean control establishes that the proposed mixing procedure helps, but does not show whether conflicting hard labels outperform ordinary soft-label mixing with exactly the same donors. This distinction is central to the methodological contribution.

  2. The headline robustness result is difficult to interpret. Table 1 reports 34.88% WGA, but Table 3 identifies this as pooled WGA and reports only 16.51% minority WGA. Counterexample CutMix also performs better on minority WGA, reaching 19.63%, despite having lower pooled WGA. The main text does not clearly explain how the evaluation views are pooled into groups, and standard MixUp and LISA are not included in the minority comparison. Consequently, the headline ranking does not establish the claimed advantage on examples that directly reverse the shortcut.

Questions for authors
  1. Matched comparison: How does ordinary soft-label MixUp perform with exactly the same clean donors, image pairs, mixing coefficients, and training budget? This would clarify what the proposed hard-label supervision adds beyond the counterexample construction.

  2. Evaluation clarification: Which views contribute to pooled WGA, and how are groups assigned to background-replaced or foreground-only images? How do standard MixUp and LISA compare on the same original minority examples? These details would help interpret the gap between pooled and minority WGA.

  3. Stability of the results: Does the WGA advantage over standard MixUp persist with longer training and across paired seeds?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.