acceptodds
Withdrawn from ICLR 2027

Reference-Guided Whitening and Contrast for AI-Generated Image Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Advances in image generation have made synthetic imagery more diverse and realistic, increasing the risks of manipulation and misinformation, yet existing methods often overfit to generator-specific artifacts or shortcuts in their training data, performing well within the training generator but degrading sharply on unseen generators. In this paper, we propose CoRe (Content-and-Real Referencing), a reference-guided approach with two components. Real-Referenced Whitening (RRW) whitens features with a covariance estimated from real images only, discounting the high-variance directions of natural variation and emphasizing the low-variance directions where transferable evidence lies. Content-Referenced Fake Contrastive (CRFC) then operates on the whitened features, contrasting each fake against content-similar reals and content-different fakes, so content cannot act as a shortcut. We instantiate CoRe on the Perception Encoder (PE), a recent vision foundation model whose features exhibit particularly favorable anisotropy for this purpose. On the recent and challenging AIGIBench and Chameleon benchmarks, our method achieves state-of-the-art mean accuracy, exceeding the strongest baseline on AIGIBench by 5.7 points, Chameleon (SD v1.4) by 7.9 points.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
AI-generated image detection, deepfake detection

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_1CcpOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CoRe combines a real-reference whitening metric (RRW), estimated from real-image covariance, with a content-referenced fake contrastive objective (CRFC) intended to reduce semantic shortcuts. The authors implement the method on PE and DINOv3 and evaluate it on GenImage, Chameleon, and AIGIBench, with component ablations and image-corruption tests.

Critical strengths
  1. Strong reported results on challenging benchmarks. The PE version achieves 92.3% mean accuracy and 97.7% mean AP on AIGIBench, with a minimum subset accuracy of 74.8%. These results support further investigation of the method.
  2. A motivated design with modest inference overhead. Real-image statistics define the detection metric, while content-based pairing constrains training. The reference bank is used only during training. Table 3 also reports gains from CRFC with the same backbone, particularly on Chameleon.
Critical weaknesses
  1. Missing baselines using the same backbone leave the algorithmic contribution unclear. Table 3 compares only RRW and RRW+CRFC. It omits a standard frozen linear probe and ordinary BCE fine-tuning, so it estimates the benefit of adding CRFC to RRW but does not establish RRW’s contribution over conventional classification. The raw mean-difference direction in Figure 5 is not a substitute for a fully trained linear classifier. Simple modern-VFM detectors and REM are important reference points. Comparisons using the same PE-L or DINOv3-L backbone, training data, and preprocessing are needed to separate the method’s gains from backbone capability and support its claimed advantage.
  2. The link between reducing content dependence and learning transferable synthesis traces remains unverified. Content neighbors are approximate matches. Subtracting their mean can leave semantic differences, source differences, or pipeline traces specific to the training generator. Such traces may be part of the synthesis process, but that does not make them transferable to other generators. Emphasizing low-variance directions in source real images likewise does not ensure that those directions remain stable for a new real-image source. The random-positive ablation in Table 4 may change both the content filter and residual-similarity selection, so it does not isolate the effect of content conditioning. The results support empirical effectiveness, but not the stronger explanation that the method removes content and extracts shared synthesis traces.
Questions for authors
  1. With identical backbones, data, and training budgets, how do frozen linear probing, ordinary BCE fine-tuning, RRW, and full CoRe compare? Does the advantage remain against REM at the same model scale?
  2. If residual-similarity selection is held fixed and only the content filter is changed, how much gain remains from CRFC? Does it persist when real and fake images have matched encoding and post-processing?
  3. What are the size and composition of the reference bank? Anchors use the current encoder while bank features come from the initial encoder. How well does re-whitening alone control representation drift, and how does it compare with re-encoding the bank?
Reviewer_RTg7OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes CoRe, a framework for improving the generalization of AI-generated image detection through Real-Referenced Whitening (RRW) and Content-Referenced Fake Contrastive Learning (CRFC). RRW emphasizes low-variance directions in real-image feature distributions, while CRFC aims to reduce content-related shortcuts using content-matched real references and contrastive learning. Experiments demonstrate strong performance on AIGIBench (92.3% accuracy) and Chameleon (93.5% accuracy), suggesting improved generalization across unseen generators.

Critical strengths

S1. A simple formulation with a useful diagnostic. RRW reduces to a closed-form discriminant direction measured in a real-only metric and adds no test-time cost beyond a linear transform. Building the score from the top- or bottom-variance eigen-subspaces of the real covariance and measuring transfer offers a concrete way to examine where transferable evidence lies in a backbone.

S2. Strong results with the PE instantiation. On AIGIBench under Setting-II, CoRe (PE) reaches 92.3% mean accuracy with a worst-subset accuracy of 74.8%. On Chameleon under the SD v1.4 protocol, it attains 93.5% with balanced real and fake accuracy (96.1% and 90.0%), a regime in which many detectors collapse toward labeling everything as real. The limitations section is also candid about the dependence on the real reference set.

Critical weaknesses

W1. The contribution of RRW is not isolated, and method and backbone effects are conflated. No experiment compares RRW with a linear probe or standard BCE fine-tuning on the same backbone, with pooled-covariance discriminant analysis, or with RRW applied to frozen features. The “Raw” score in Figs. 1 and 5 is the projection onto the unwhitened mean difference, a nearest-centroid rule rather than a trained probe. On the source distribution, the whitened separation is always at least as large by the Cauchy–Schwarz inequality, so the several-fold gain in Fig. 1 is not evidence of transfer. Baseline numbers are transcribed from papers that often rely on older backbones, while the paper itself cites evidence (Zhou et al., 2025) that simple probes on modern foundation models are already strong. The closest same-backbone methods discussed in the related work, Cai et al. (2026) on PE and Choi et al. (2026) on DINOv3, are not compared.

W2. The mechanism claims are only partly supported. Fig. 6 is described as showing that the low-variance tail carries the transferable signal on both targets. On AIGIBench, however, PE’s top-64 and bottom-64 subspaces yield nearly identical AP (0.79 vs. 0.81). The smallest eigenvalues are also the least reliable: the covariance is re-estimated from a 2,000-example probe in a space of roughly comparable dimension, and neither ε value is reported. In the GenImage source, real images are JPEG-compressed photographs while most generated images are stored losslessly, so low-variance real directions may encode compression rather than synthesis traces, and no format-matched control is provided. For CRFC, positives are mined by residual similarity under the initial backbone, which may reinforce the initially dominant residual direction. No measurement shows that the residuals carry less semantic information after training.

W3. Several claims conflict with the paper’s own results. Fig. 5 reports whitened AP of 0.94 and 0.91 for DINOv3 on FaceSwap and SimSwap, whereas Table 5 reports 55.7 and 59.7 AP for CoRe (DINOv3) on the same subsets. The claim that the high worst-case accuracy “follows from the construction rather than from tuning” is contradicted by the DINOv3 instantiation of the same construction. Its worst subset is 52.2%, it is near chance on all five face-swap subsets, and Fig. 3 shows only the PE instantiation. The weakness on ADM is attributed to the generator rather than the representation, yet several baselines in Table 2 exceed 90% on ADM. Finally, the robustness explanation, that a perturbation “shifts the reference along with the image”, is inconsistent with the method, because the reference statistics are fixed from clean source images at test time.

W4. Experimental rigor and presentation. λ appears to be selected on the AIGIBench test subsets (Table 6), and no validation protocol is described for any hyperparameter. All results come from single runs without variance estimates, which makes small differences such as the 0.4-point negative-mining ablation difficult to interpret. Table 5 contains exact duplicates that suggest transcription errors: SD3 and SDXL coincide for DDA and both CoRe variants, and CNNDet’s SD3 and SDXL entries equal its StyleGAN3 and StyleGAN-XL entries. Section 3.3 also contains a duplicated sentence, and the calibration parameters are written inconsistently as (α, b) and (α, β).

Questions for authors

Q1. Figure 5 reports whitened AP of 0.94 and 0.91 for DINOv3 on FaceSwap and SimSwap, while Table 5 reports 55.7 and 59.7 AP for CoRe (DINOv3) on the same subsets. Which real images define the whitening in Figure 5, and was any statistic computed from the target subset? If only source statistics were used, what explains the large drop from the whitened direction in Figure 5 to the full trained detector? This is the question most likely to change my assessment.

Q2. Could the authors report a linear probe and standard BCE fine-tuning of PE and DINOv3 under the same data, resolution, and schedule? Alongside these, it would help to include RRW on frozen features, Fisher discriminant analysis with a pooled covariance, and the same-backbone methods of Cai et al. (2026) and Choi et al. (2026). Together, these comparisons would show how much of the reported gain comes from the backbone, from the real-only covariance, and from fine-tuning under the whitened metric.

Q3. How were λ, the learning rate, the layer-wise decay, and the number of neighbors selected? Table 6 sweeps λ on AIGIBench, the chosen value is the best on that test set, and no validation split is described. Please also report both ε values and the sensitivity of the results to them, since the covariance is re-estimated from about 2,000 examples in a feature space of comparable dimension.

Q4. Can the authors show that the low-variance real directions capture synthesis evidence rather than processing or format differences? One way would be to retrain with real and generated images matched in JPEG compression and file format, and to report real and fake accuracy separately under the JPEG sweep in Figure 4. It would also help to reconcile the claim that bottom-k transfer greatly exceeds top-k transfer with the nearly equal values for PE on AIGIBench in Figure 6.

Q5. For CRFC, could the authors separate the effect of content-based selection from that of residual-similarity mining of positives, and measure how much semantic information the residuals retain before and after training, for instance with a linear probe for content categories? Given that the keys are stored initial-backbone features and the peak backbone learning rate is 5×10⁻⁷, it would also help to quantify how much the backbone representation actually changes during training.

Reviewer_Rjt1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CoRe for generalizable AI-generated image detection. Real-Referenced Whitening (RRW) estimates a feature covariance from source real images and uses the resulting metric to define a detection direction and guide backbone adaptation. Content-Referenced Fake Contrastive learning (CRFC) subtracts content-matched real references from fake features and aligns their residuals with selected, content-different fake references. The method is evaluated with PE and DINOv3 backbones on GenImage, AIGIBench, and Chameleon. With PE, CoRe achieves 92.3% mean accuracy and 97.7% mean AP on AIGIBench, and 93.5% accuracy on Chameleon when trained on SD v1.4.

Critical strengths
  1. The cross-dataset results are strong and cover diverse generation methods. On AIGIBench, CoRe improves over the strongest reported baseline by 5.7 percentage points in mean accuracy and 2.7 points in mean AP. Its gains also extend to the hardest subset, rather than being confined to the average. The Chameleon results provide additional evidence of transfer, with both real and fake accuracies reported. 2.The method has a clear statistical motivation, supported by feature-subspace analyses across multiple backbones. The experiments examine both the effect of adding CRFC and the choice of contrastive pairs, and include tests under compression, blur, and resizing. These analyses provide useful evidence beyond the main benchmark comparisons.
Critical weaknesses
  1. The experiments do not sufficiently establish the benefit of RRW over simpler alternatives using the same backbone. Table 3 starts from RRW rather than a conventional classifier trained on PE or DINOv3 under the same optimization settings. The feature visualizations and subspace analyses support the geometric motivation, but do not replace this controlled comparison. Comparisons with diagonal normalization and a pooled within-class covariance would also help establish whether the real-only covariance is essential. Without these controls, the contribution of RRW relative to the backbone and adaptation procedure remains unclear.
  2. The positive-pair ablation does not isolate the claimed role of content conditioning. The proposed procedure selects positives using both low content similarity and high residual similarity. Replacing these positives with random fakes, as in Table 4, changes both properties. The resulting 2.2-point difference therefore cannot be attributed specifically to selecting content-different examples. A control retaining residual-similarity mining while removing the content filter is needed to support this interpretation.
Questions for authors
  1. The experiments do not sufficiently establish the benefit of RRW over simpler alternatives using the same backbone. Table 3 starts from RRW rather than a conventional classifier trained on PE or DINOv3 under the same optimization settings. The feature visualizations and subspace analyses support the geometric motivation, but do not replace this controlled comparison. Comparisons with diagonal normalization and a pooled within-class covariance would also help establish whether the real-only covariance is essential. Without these controls, the contribution of RRW relative to the backbone and adaptation procedure remains unclear.
  2. The positive-pair ablation does not isolate the claimed role of content conditioning. The proposed procedure selects positives using both low content similarity and high residual similarity. Replacing these positives with random fakes, as in Table 4, changes both properties. The resulting 2.2-point difference therefore cannot be attributed specifically to selecting content-different examples. A control retaining residual-similarity mining while removing the content filter is needed to support this interpretation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.