- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes RAIN, a framework for improving the generalization of AI-generated image detection by mitigating shortcut learning. It introduces three components: Real-Anchored Alignment, Disagreement-Guided Reweighting, and Leave-Environments-Out optimization. Experiments across multiple benchmarks demonstrate strong generalization to unseen generators and robustness under limited training data.
Critical strengths
-
Interesting perspective on shortcut learning. The paper provides a well-motivated analysis of acquisition, content, and style shortcuts, supported by controlled experiments demonstrating their impact on generalization. Using cross-generator consistency on shared real images as a training signal is an interesting approach that avoids explicitly specifying individual shortcut attributes.
-
Strong empirical performance and comprehensive evaluation. RAIN achieves substantial improvements over competitive few-shot baselines across in-domain, out-of-domain, and strictly held-out generator evaluations. The component ablations and corruption robustness experiments provide additional evidence of the framework's effectiveness, particularly under limited training data.
Critical weaknesses
-
Insufficient justification for the connection between prediction consistency and shortcut elimination. The central assumption is that disagreement among generator-specific classifiers on real images indicates shortcut reliance. However, such disagreement may also arise from legitimate differences in learned decision boundaries, limited training samples, or optimization variability. Conversely, different classifiers may consistently exploit the same spurious features without exhibiting disagreement. Therefore, enforcing cross-generator consistency does not necessarily guarantee shortcut removal. The current analysis demonstrates improved prediction agreement but does not sufficiently establish that the learned representations are genuinely less dependent on spurious correlations.
-
Limited isolation of the proposed mechanism from other contributing factors. Although the component ablations demonstrate performance improvements, the evaluation does not adequately separate the benefits of shortcut suppression from those of the pretrained DINOv3 backbone, multi-source training, and additional regularization. In particular, the controlled shortcut experiments primarily establish that conventional detectors are vulnerable to injected biases, rather than directly measuring whether RAIN eliminates these biases. More targeted interventions and backbone-matched comparisons would strengthen the causal interpretation of the results.
Questions for authors
-
How can the authors establish that cross-generator prediction consistency specifically suppresses shortcuts? Could the authors evaluate RAIN under the controlled acquisition, content, and style shortcut interventions introduced in Section 3, comparing against standard ERM and other invariance-based objectives?
-
Can the authors clarify whether prediction consistency is sufficient to identify shortcut reliance? In particular, how does RAIN handle spurious features shared across multiple generator environments, where different classification heads may agree despite relying on the same shortcut?
-
How much of the improvement comes from the proposed invariance mechanism rather than the pretrained representation? A comparison with backbone-matched ERM and alternative regularization methods under identical training budgets would help clarify the contribution of RAIN.
5 comments
W1.3: Improved agreement does not show that the learned representations rely less on spurious correlations.
A1.3: Thank you for this important point! We measure reliance directly with the Hard AUC, which is independent of cross-head agreement. Under the same setup as A1.1, we compare ERM and RAIN.
| Unseen real | Hard AUC ↑ (ERM) | Hard AUC ↑ (RAIN) |
|---|---|---|
| Without shortcut attribute | 0.838 | 0.982 |
| + Acquisition | 0.352 | 0.979 |
| + Content | 0.336 | 0.878 |
| + Style | 0.377 | 0.926 |
ERM falls below chance once the shortcut is reversed, indicating that it largely decides based on the shortcut. RAIN stays well above chance across all three shortcut families, showing that its decisions rely substantially less on spurious correlations, rather than merely becoming more consistent. We provide per-image disagreement maps in the anonymous repository and will add these results in our revision.
W2.1: The evaluation does not separate the benefit of shortcut suppression from those of the DINOv3 backbone, multi-source training, and additional regularization.
A2.1: Thank you for this constructive suggestion! We isolate each factor under the same data and training budget as follows.
- Pretrained backbone. We replace the encoder and compare RAIN with backbone-matched multi-source ERM. The AUC results are shown below.
| Encoder | ERM | RAIN (ours) | Δ |
|---|---|---|---|
| DINOv3 ViT-L/16 | 0.914 | 0.999 | +0.085 |
| DINOv2 ViT-L/14 | 0.818 | 0.981 | +0.164 |
| DINOv2 ViT-B/14 | 0.803 | 0.988 | +0.185 |
| CLIP ViT-L/14 | 0.936 | 0.972 | +0.036 |
| ResNet-50 | 0.774 | 0.840 | +0.066 |
RAIN consistently improves over the backbone-matched ERM on all five encoders, including the convolutional ResNet-50, so its gain is not tied to DINOv3 representations.
- Multi-source training. We respectfully clarify that Table 4 of our paper already isolates this factor under the same DINOv3 + LoRA backbone.
| Setting | AP ↑ | FNR ↓ |
|---|---|---|
| Baseline 1: ERM on ProGAN data | 71.3 | 45.4 |
| Baseline 2: ERM on the same 5-shot multi-source data as RAIN | 83.7 | 58.6 |
| Baseline 2 + RA | 99.6 | 20.4 |
Multi-source data alone improves AP but also increases FNR. With identical backbone, data, and training budget, adding RA alone brings the major improvement, so the gain beyond Baseline 2 comes from our mechanism rather than from multi-source training.
- Additional regularization. Starting from the multi-source ERM with DINOv3 ViT-L/16, we add generic regularizers and a head-consistency penalty that applies an L2 penalty to the of the heads. The AUC results are shown below.
| Method | AUC ↑ |
|---|---|
| ERM | 0.914 |
| + Dropout (0.5) | 0.916 |
| + Label smoothing | 0.911 |
| + head L2 | 0.904 |
| RAIN (ours) | 0.999 |
Generic regularizers bring negligible changes and the head L2 penalty even degrades performance. Together, these results confirm that RAIN's improvement comes from its invariance mechanism rather than from the pretrained backbone, multi-source training, or extra regularization. We will add these results and discussion in our revision.
W2.2: The controlled shortcut experiments only show that conventional detectors are vulnerable, without directly measuring whether RAIN eliminates these biases.
A2.2: Thank you for this comment! Please kindly refer to A1.3 and A3, where we directly evaluate RAIN under the controlled ACS shortcut interventions against ERM and other invariance-based objectives.
Q1: Can RAIN be evaluated under the controlled ACS shortcut interventions of Sec. 3, against ERM and other invariance-based objectives?
A3: Thank you for this constructive question! Following your suggestion, we further compare RAIN with IRMv1, V-REx, and GroupDRO. All methods use the same encoder, multi-head, 5-shot data from AIGIBench environments with shared real images, and training budget, with the penalties of the baselines applied to the per-environment head risks. Following the operators in Table 6, we inject the same acquisition shortcut (JPEG) into the fake images of k environments (k = 1 or 12), and report the AUC under the shortcut-removal (Equal) and shortcut-reversal (Hard) protocols of Sec. 3.
| k | Method | Equal ↑ | Hard ↑ |
|---|---|---|---|
| 1 | ERM | 0.853 | 0.296 |
| 1 | IRMv1 | 0.854 | 0.343 |
| 1 | V-REx | 0.852 | 0.305 |
| 1 | GroupDRO | 0.853 | 0.297 |
| 1 | RAIN (ours) | 0.979 | 0.891 |
| 12 | ERM | 0.818 | 0.056 |
| 12 | IRMv1 | 0.806 | 0.107 |
| 12 | V-REx | 0.813 | 0.057 |
| 12 | GroupDRO | 0.818 | 0.056 |
| 12 | RAIN (ours) | 0.940 | 0.894 |
Existing invariance objectives behave almost identically to ERM and fall far below chance under reversal, since per-environment training risks quickly approach zero in the few-shot setting, leaving little signal for risk-based penalties. RAIN instead anchors on the predictions of shared real images and remains robust, even when the shortcut is shared by about half of the environments. We will add these results in our revision.
Q2: Is prediction consistency sufficient to identify shortcut reliance, especially when spurious features are shared across environments?
A4: Thank you for this question! Please kindly refer to A1.2.
Q3: How much of the improvement comes from the invariance mechanism rather than the pretrained representation?
A5: Thank you for this question! Please kindly refer to A2.1, where we compare RAIN with backbone-matched ERM and alternative regularization under identical training budgets.
Dear Reviewer #qv6a, we sincerely thank you for your precious time and constructive comments. We are deeply encouraged by your positive recognition of our interesting perspective on shortcut learning, strong empirical performance, and comprehensive evaluation. We sincerely hope the following clarifications and new experiments can address your concerns.
W1.1: Cross-head disagreement on real images may also come from legitimate boundary differences, limited samples, or optimization variability, rather than shortcuts.
A1.1: Thank you for this insightful comment! We fully agree that these factors also cause disagreement. However, shortcut-induced disagreement is much larger, and RAIN is designed to suppress only this part.
From the results, we find that disagreement from limited samples and optimization noise does exist, but shortcuts cause substantially larger disagreement across all three shortcut families. We will add these results and discussion in our revision.
W1.2: Heads may consistently rely on the same spurious feature without any disagreement.
A1.2: Thank you for this observation!
We agree that an attribute shared by all environments with the same intensity cannot be separated from authenticity evidence without external prior knowledge. This limits any environment-based mitigation strategies, since both are equally predictive in every environment.
However, in practice, a shared shortcut rarely has the same intensity across generators, since their pipelines differ. The heads depend on it to different degrees, and the disagreement signal remains.
Following your suggestion, we inject the same shortcut into all training environments under two settings with the same total number of shortcut-carrying samples: uniform (identical intensity across environments) and non-uniform (varying intensity). We report the shortcut reversed AUC.
ERM fully relies on the shared shortcut in both settings. When the shared shortcut differs in intensity across environments, RAIN exploits this difference and largely recovers. When the intensity is perfectly identical, the cross-head signal vanishes and RAIN's gain is limited, although it still outperforms ERM. This is a common boundary of environment-based invariance learning, and we will discuss it as a limitation in our revision.