- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies how knowledge of a shift-prone feature block affects robustness under distribution shifts. And this paper designs a synthetic graph-classification benchmark REDHOUSE, with independently controllable correlations between the label and the background graph family or node colors. The paper also presents the corresponding experiments.
Critical strengths
- The paper presents a detailed design of the experiment.
Critical weaknesses
- The paper’s main conclusion is that if tell the model which feature block is problematic and permute it, the shortcut carried by that block can be eliminated, but in REDHOUSE, the color block happens to be a one-hot block whose position is known (and, given the motif, label-independent), and the motif structure fully determines the label. As a result, the 95%+ results are almost predetermined by the benchmark’s construction.
Questions for authors
- If the prior is imperfect (e.g., the block is only partially localized correctly, or the nuisance block is coupled with label information), how the paper’s conclusions would change?
Hi Reviewers,
We are now near the end of the rebuttal period. If you haven't done so yet, please review the authors' responses and post a reply addressing whether their clarifications or new results resolve your concerns. Even if your score or recommendation remains unchanged, a brief note confirming what did or didn't shift your assessment—and why—is essential for the discussion and final decision. Thank you for your time and engagement.
Best regards,
Area Chair