acceptodds
Withdrawn from ICLR 2027

The Geometry of Fakeness: AI-image detection in the open world

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

AI-generated image detection is commonly formulated as a binary classification problem between real and fake images. We argue that this formulation is fundamentally misaligned with the open-world nature of the task. While AI-generated images originate from a relatively limited family of generators and thus form a compact, learnable distribution, real images arise from diverse cameras, scenes, acquisition pipelines, and post-processing operations, making them too heterogeneous to be faithfully captured by finite training samples. This asymmetry creates a posterior mismatch between the training and deployment distributions, so that even a near-optimal binary classifier can suffer amplified open-world generalization error. We therefore reformulate AI-generated image detection as an out-of-distribution (OOD) detection problem, treating AI-generated images as in-distribution and real images as outliers. Yet OOD learning alone is insufficient, since the learned representation may still rely on unstable generator-specific shortcuts, such as low-level artifacts or compression traces, that fail to transfer across models and domains. Motivated by our theoretical analysis that information bottleneck (IB) can mitigate such bias amplification by filtering out spurious nuisance features, we propose an IB-regularized one-class objective that learns compact representations of AI-generated images while preserving stable generative cues. Experiments on both classical diffusion models and recent generation systems show consistent improvements over strong baselines, including a 12.90% absolute gain in average accuracy over the best prior detector. Our results suggest that AI-generated image detection is better understood as open-world distribution modeling rather than closed-world binary classification.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
AI-generated image detection, one-class learning, information bottleneck, open-world generalization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_C6trOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This research focuses on improved design of fake image detectors. The primary argument is that real image distribution is much more varied than that of fake images, and hence it should be treated as an outlier detection problem. More specifically, the fake images should be considered in-distribution and the real images are considered out of distribution. The authors demonstrate that the training error gets amplified on the test distribution as a function of how misaligned the training and test distributions are. So the idea is to learn only the invariant features, for fake image detection, and provided that it captures those invariant features, the train test mismatch will be minimized and with the invariant features a out of distribution detector can be built by treating the real images as out of distribution. For the learning of these invariant features, the authors use VICReg-based info bottleneck, and use OOD detection techniques such as Deep SVDD and Energy-based techniques. They benchmark on modern Image generators and compare against prominent baselines showing improvements in accuracy/AP.

Critical strengths
  1. The core philosophy of treating the fake images as the in-distribution class and real images as the out of distribution class is logical and resonates with me as opposed to methods which attempt to treat the real images as the in-distribution class.
Critical weaknesses
  1. The paper lacks ablations regarding the OOD and the Info Bottleneck design choices. For instance, what if a classifier were just trained on the output of IB features instead of training an OOD technique. The paper also lacks what happens if the IB-specific self-supervised loss was removed and instead those augmented images are also just used as training data along with the OOD objective. This will further motivate the need for both of those design choices (although the OOD one makes sense to me intuitively). I request the authors to perform a compute equalized experiment in this case (same number of iterations batch size etc and please report these details as well for all these experiments since they are critical)
  2. If the model learns only invariant features (as claimed), and if the real images mainly differ due to nuisance features (lines 240-242), then there I am unable to understand why the OOD formulation still matters. Given that the paper actually claims that lines 987-990 explicitly mention that train and OOD real distributions will be close to each other. In this case, if the nuisance-based differences between them are ignored then even a simple classifier should suffice. I request the authors for more clarity on this regard.
  3. The "Real as ID" baseline (Cai et al) as cited by the authors. I would be interested in learning how this experiment is conducted. Since the appendix b.3 referenced by the authors does not contain any information on how Cai et al method is applied for their setting.
  4. To me there seems to be a substantial disconnect between theory presented in the first 5 pages and the design choices adopted later on. To the best of my understanding, the theory essentially suggests that if you get rid of spurious features, and only capture invariant, predictive features, the method is more likely to generalize better which I find to be very straightforward and not offering a lot of predictive value. Building on this, Information Bottleneck is predicted as a means to do this, but it is not clear how VICReg/other self-supervised learning is the only way to achieve this (echoing my concerns raised in Weakness 1). I wish to hear the authors thoughts on the connection between their design and the theory as well as the novelty in the information bottleneck viewpoint which has not been presented in prior works such as [1].
  5. I would really like to see the authors test the robustness of the detector to post-processing techniques such as JPEG, resizing, WEBP, additive noise, color jitter etc and provide a detailed analysis since robustness to these techniques are very important in practice.

References

  1. Shwartz-Ziv, R., & Tishby, N. (2017). Opening the black box of deep neural networks via information. arXiv preprint arXiv:1703.00810.
Questions for authors

Please refer to weaknesses.

Reviewer_TLEaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on the AI-generated image task under open-world nature setting, where generated images originate from a limited family and real images from diverse settings. To address this issue, the authors reformulate the task as an out-of-distribution (OOD) problem by treating AI-generated images as in-distribution and real images as outliers. Furthermore, to mitigate the bias, an IB-regularized one-class objective is introduced to learn compact representations of AI-generated images while preserving stable generative cues. Extensive experiments on diverse settings demonstrate the effectiveness of the proposed method.

Critical strengths
  1. Well-motivated problem. How to detect AI-generated images in the open world is always a critical issue in this area.
  2. Interesting formulation perspective. The authors treat AI-generated images as in-distribution and real images as outliers, which is different from some existing OOD detection methods that treat AI-generated images as outliers.
  3. Direct theoretical analysis. The authors provide a direct theoretical motivation from posterior mismatch to demonstrate why binary classification fails.
  4. The paper is well-organized and easy to follow.
Critical weaknesses
  1. Open-world setting. The authors define the open-world setting as limited generated images and diverse real images, which is not consistent with the common sense of open-world where there are evolving generators and diverse post-preprocessing. I recommend the authors recheck the definition of open-world and then decide the proper presentation. Besides, I do not quite agree with the authors' statement that fake images share a compact distribution since there is only limited generator family. There are many different generator families already, such as GANs, diffusion models, and flow-based models. More deeper explanations are needed.
  2. How the theoretical analysis motivates method design. The authors present a theoretical proof in sec 2.2 to explain why binary classifier fails under open-world setting, which is direct and coherent with common sense. However, I cannot see how this posterior mismatch motivates the OOD formulation and the objective design. More explanations are needed.
  3. Introduction of IB regularization. Personally, I think the authors treat real images as outliers is interesting. But I don't quite understand why introducing IB regularization as one-class OOD objective. There are many different regularization methods, why choose this one? More justifications are needed. Besides, discussions with related OOD detectors are encouraged to justify the novelty of this paper.
  4. Experiments. This paper reformulates the AI-generated image detection task as OOD problem. Thus, the performance on each subclass (real/fake) should provide more support for the open-world setting. More analysis on these is encouraged. Besides, the DSVDD-based variant consistently outperforms the energy-based variant, so it is better to reorganize the variant selection and results presentation.
  5. Some typos, such as the overlapped numbers in Fig. 3.
Questions for authors

Please see the weakness above. My major concerns focus on the open-world setting, overall logic, and method design. Thus, I currently lean towards negative ratings. I hope the authors can address these issues.

Reviewer_9BsVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies AI-generated image detection in the open world. The authors argue that real images originate from highly diverse sources and imaging pipelines, whereas generated images are more amenable to compact modeling. They therefore treat generated images as in-distribution (ID) and real images as out-of-distribution (OOD). A posterior-mismatch analysis motivates the potential generalization risks of binary classification, while an information bottleneck (IB) perspective is used to explain how nuisance factors might be suppressed. The proposed detector combines a pretrained image encoder, VICReg regularization, and either a multi-center Deep SVDD or an energy-based objective to model generated images in feature space.

Critical strengths
  • The paper addresses the tension between finite training data and the diversity of real images in the open world. Treating generated images as ID, together with observations of distributional asymmetry using two frozen encoders, provides a useful perspective for revisiting conventional binary classification.
  • The authors combine representation regularization with OOD learning and provide both distance-based and energy-based implementations. The inclusion of VICReg also takes into account the risk of representation collapse when generated-image features are encouraged to contract.
  • Under the reported evaluation setting, the main variant achieves 83.48% average accuracy and 90.48% mean AP, outperforming the compared baselines. The paper also includes leave-one-generator-out evaluation, experiments with six synthetic shortcuts, and ablations across encoders and regularization methods, providing initial empirical support for its effectiveness.
Critical weaknesses
  • Assumption 1 requires the invariant-factor distributions of training and deployment real images to be close. Appendix A.4 further explains that this requires sufficient coverage of content categories, approximately matched category proportions, and the absence of content-correlated selection bias. However, open-world shifts may also involve content itself, such as training only on natural landscapes and testing on unseen categories such as animals. Although the authors acknowledge the limitation of insufficient content coverage, the current evidence does not adequately establish the method’s behavior under such shifts. This condition substantially restricts the theoretical guarantee and leaves a gap between the motivation that finite data cannot cover real-image diversity and the types of shift the framework actually addresses.
  • Section 3.2 describes the augmentations as semantic-preserving, whereas Appendix A.5 further assumes that they perturb nuisance factors while preserving generative traces. Preserving semantics does not imply preserving forensic cues: some augmentations may weaken useful subtle traces, while certain generator-specific patterns or semantic shortcuts may persist across both views. The invariance loss cannot distinguish these cases by itself, and the variance and covariance terms do not directly identify useful forensic information either. The six synthetic-shortcut experiments provide indirect support but do not sufficiently establish this selective preservation mechanism. The specific augmentation operations and their parameters also need to be clearly specified.
  • Theorem 1 establishes the existence of training distributions on which an optimal binary classifier incurs a large deployment loss. It does not directly establish that binary classification is generally unsuitable for this task or that OOD modeling is necessarily superior. Theorem 2 concerns a classifier that minimizes training cross-entropy on an ideal IB-optimal representation, whereas the implemented method uses VICReg with a one-class distance objective or an energy-based objective. The paper should more clearly distinguish conditional theoretical motivation from guarantees established for the actual algorithm, and avoid treating representation invariance, mutual-information compression, and the extraction of stable forensic cues as equivalent.
  • A limited number of generator families does not ensure that images from unseen generators fall within compact regions learned from training data; the observations in Figure 1 also depend on the particular data and encoders. The leave-one-generator-out evaluation is useful, but the listed settings include different sampling configurations from the same family, so it should be clarified whether entire generator families are held out. Furthermore, the main experiments use LAION real images and synthetic images generated from their corresponding captions, with limited validation on independent sources of real images. Comparisons with binary classifiers matched in backbone, trainable parameters, augmentations, and training budget are needed under protocols evaluating strictly unseen generators and unseen real-image sources, to better isolate the contributions of OOD modeling and VICReg.
Questions for authors
  • Does Assumption 1 still hold when training and test real images differ substantially in content? Could you provide evaluations on unseen content categories to clarify the method’s applicability?
  • What evidence shows that the augmentations perturb nuisance factors while preserving useful generative traces? Please specify the augmentation strategy and provide supporting validation.
  • Theorem 2 concerns an ideal IB representation and a cross-entropy classifier. How does this result support the actual method combining VICReg with DSVDD or Energy?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.