acceptodds
Under review as a conference paper at ICLR 2027

Known-Shortcut Interventions under Mixed Graph Distribution Shifts: A Diagnostic Study

Abstract

Graph classifiers can exploit spurious correlations in both background topology and node attributes. We study how knowledge of a shift-prone feature block affects robustness under such distribution shifts. We introduce RedHouse, a synthetic graph-classification benchmark with independently controllable correlations between the label and the background graph family or node colors. Under a protocol with 300 labeled training graphs, targeted color-block randomization substantially improves worst-environment accuracy over the evaluated baselines. A prior-matched comparison shows that a simple GIN with the same finite-warmup intervention (T_w=100) achieves 95.7±2.6% worst-environment accuracy across ten seeds, compared with 93.6±4.2% for the full MixedStable framework; these results do not establish an advantage of the full framework. We distinguish persistent randomization from finite warmup followed by shortcut release: the main low-label setting primarily evaluates sustained shortcut suppression, whereas longer post-release training reveals substantial performance decay and seed-dependent failures. Additional experiments on external benchmark variants show heterogeneous outcomes, including failures of the full framework. Our findings identify targeted intervention on a known nuisance block as the main source of the observed RedHouse gains and motivate evaluations that match prior information, training budgets, and checkpoint selection. They do not establish that a short warmup reliably yields lasting invariance or that mixed shifts necessarily cause stronger shortcut dependence than feature shifts alone.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
spurious correlation, graph neural networks, out-of-distribution generalization, shortcut learning, benchmark, prior-matched evaluation, targeted intervention, graph classification

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_5QhvOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how knowledge of a shift-prone feature block affects robustness under distribution shifts. And this paper designs a synthetic graph-classification benchmark REDHOUSE, with independently controllable correlations between the label and the background graph family or node colors. The paper also presents the corresponding experiments.

Critical strengths
  1. The paper presents a detailed design of the experiment.
Critical weaknesses
  1. The paper’s main conclusion is that if tell the model which feature block is problematic and permute it, the shortcut carried by that block can be eliminated, but in REDHOUSE, the color block happens to be a one-hot block whose position is known (and, given the motif, label-independent), and the motif structure fully determines the label. As a result, the 95%+ results are almost predetermined by the benchmark’s construction.
Questions for authors
  1. If the prior is imperfect (e.g., the block is only partially localized correctly, or the nuisance block is coupled with label information), how the paper’s conclusions would change?
Reviewer_nTDQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies the task of graph classification and analyzes the influence of graph topology and node attributes of graphs. It proposes a synthetic benchmark called REDHouse that can control the correlations between the labels and graphs. It shows that a simple GIN with a finite-warmup intervention gets strong performance even in the worst-case setting. The paper further gives a deep analysis of the results to show the real factors.

Critical strengths

The paper builds a new setting with the REDHouse benchmark for graph classification. The benchmark offers much flexibility for conducting experiments. Based on the benchmark, the paper finds that a simple GIN with finite warmup achieves good and robust performance.

Critical weaknesses

The paper is not easy to follow. We cannot even see the full content of Table 3. In the introduction section, the paper does not cite any papers, and we cannot get an overview of the research progress in the field, which affects our assessment of its value.

Questions for authors

It is unclear how important the paper's research problem is. The authors should provide more context (e.g., in the introduction) to show the relationship between this paper and previous papers.

Reviewer_VtvXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces REDHOUSE, a synthetic graph-classification benchmark in which the label is determined by a motif, while the background graph family and node colors are spuriously correlated with the label at independently adjustable strengths. It studies a targeted intervention that randomly permutes the node-color features during training, and evaluates the authors' composite framework MIXEDSTABLE against prior-matched simple baselines.

Critical strengths

From my perspective, this paper is a good and honest experimental analysis report. It clearly documents the authors' findings on targeted interventions against a known spurious feature block: the robustness gains on REDHOUSE come almost entirely from randomizing the known color block, a plain GIN with the same intervention performs on par with the full framework, and the benefit of a finite warmup intervention is not reliably preserved after the original features return. The authors openly report these findings even though they do not favor their own method.

Critical weaknesses
  1. The proposed method appears to be an unsuccessful attempt. MIXEDSTABLE shows no advantage over a plain GIN with the same intervention and fails on some external variants, so the paper essentially reports a negative result.
  2. The proposed dataset appears to be a simple combination of existing ones. REDHOUSE seems to combine the motif-based construction of Spurious-Motif with the node-color shortcut of GOOD-CBAS.
  3. The writing is somewhat hard to follow. In addition, the table on page 7 extends beyond the page margin and is difficult to read.
  4. The proposed framework is never evaluated without the known-color prior or on real-world datasets with unknown spurious features, so there is no evidence that it is effective in the realistic OOD setting.
Questions for authors
  1. Which problems can REDHOUSE address that cannot already be addressed with GOOD-CBAS and Spurious-Motif?

Comments on the paper

1 comment

Hi Reviewers,

We are now near the end of the rebuttal period. If you haven't done so yet, please review the authors' responses and post a reply addressing whether their clarifications or new results resolve your concerns. Even if your score or recommendation remains unchanged, a brief note confirming what did or didn't shift your assessment—and why—is essential for the discussion and final decision. Thank you for your time and engagement.

Best regards,

Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.