acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Shortcuts in AI-Generated Image Detection

Abstract

We identify a computational shortcut in AI-generated image detection: linear classifiers on the frozen visual encoders of two trained multimodal large language model (MLLM) detectors recover much of their accuracy without language decoding. Further analysis reveals that these detectors are sensitive to input resolution, suggesting potential dataset shortcuts. For example, matching fake-image aspect ratios to the real-image distribution lowers Deep-VRM's AIGI-Now Semantic accuracy from 93.8% to 63.8%. This sensitivity motivates a systematic metadata-only audit of 12 datasets and training resources, revealing label-correlated differences in file format, encoding, and image geometry. Guided by these findings, we propose an aligned visual learning framework based on image editing. Pairing real photographs with complete text-guided editing outputs under consistent preprocessing suppresses dataset shortcuts and enables joint learning of pixel-level and semantic forgery features in a single visual representation space. Our DINOv3-L detector achieves 91.3% mean accuracy across 12 in-the-wild domains and 92.4% on AIGI-Now, retaining over 90% on both subsets after resolution alignment. It also transfers to full-image synthesis and local manipulation on DailyBench.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
AI-Generated Image Detection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_F9pgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies directly readable detection information in MLLM visual encoders and dataset shortcuts involving format, encoding, and geometry. It audits twelve datasets and training resources, then fine-tunes a single visual encoder using source-paired real and edited Mani-X images. Evaluation covers established benchmarks, real-world images, processing alignment, and recent synthesis and manipulation generators.

Critical strengths

S1: This paper audits twelve datasets and training resources using shallow metadata-only classifiers, finding several cases where format or encoding nearly determines the label. These results provide concrete reasons to revisit existing evaluation protocols.

S2: This paper compares Mani-X, DDA, and B-Free with the backbone, training size, and update count controlled, and observes similar gains with CLIP-L and DINOv2-L. This strengthens the evidence that the training data contributes beyond DINOv3 itself.

Critical weaknesses

W1:This paper extends existing source-aligned training with selected editing data, while retaining a standard DINOv3 encoder, linear head, and BCE loss. Its methodological advance over Aligned, B-Free, and DDA is mainly in the training-data recipe, with limited independent technical novelty.

W2:This paper does not isolate the contribution of semantic cues or distinguish shortcut reliance from information loss in its geometric experiments. Its cross-generator gains also remain uneven, with substantial performance gaps on some recent editors.

Questions for authors

Q1: This paper compares DDA and B-Free. With source images, generators, backbone, and training budget matched, how much does full instruction-based editing improve over reconstruction and inpainting, and is this gain the main methodological contribution?

Q2: Does the model actually rely on semantic anomalies?

Q3: How much of the sensitivity to image geometry is attributable to dataset shortcuts?

Q4: Does the computational shortcut finding apply to training, inference, or both?

Reviewer_Q8T4OpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper's methodology describes a way to find out what inner mechanisms are at work to find AI generated images: entangling the shortcuts used by the model into a) metadata shortcuts and b) visual shortcut (which almost always already contains the information and thus meaning that the visual encoder already detects the "fake signal".

Critical strengths
  • the paper does not merely state that fake detection benchmarks are biased; instead, it measures the bias across many datasets using simple metadata-only tests (file format, JPEG encoding features, aspect ratio, image size/shape); e.g., in some datasets all fake images have the same obvious resolutions
  • the language part may not be necessary for fake image detection
  • proposed training design is simple and broadly evaluated, so real-world use case is possible and likely
Critical weaknesses

Major issues:

  • fundamental reproducibility is to be questioned, as well as proposed license / release. Could you elaborate what parts of code / dataset / models you can / will release and under which license? Also, if you provided anonymous GitHub / anon Huggingface repositories (works for double-blind reviews when not mentioning / disclosing your own institute) the whole pipeline would be easier to "reproduce" mentally - if not possible to provide, at least state compute used, quants, training FLOPs, etc.
  • Tables lack fundamental required statistics: at least report the dataset size within the table, even better providing additional metrics such as SEM or 95% confidence intervals (n is mandatory, the SEM/95%CI optional, because the tables are already quite "cluttered".

Minor issues

  • some tables lack exact model description (e.g., Table 13: Midjourney v6? or v3? which version? Or Table 11: Flux.1 and Flux.2 dev / klein? (It is obvious from the text that you used dev and klein-9B, but this belongs to the table)
  • the shortcut story is plausible and well-supported, but still partly indirect. Stronger evidence would require: matched counterfactual datasets where only metadata differs, probing what specific neurons/features encode resolution, an/or adversarial tests that isolate whether the cue is truly causal

Citation issues

  • potential citation hallucination: you use Qwen 3.5 / VL, and only cite Qwen-2.5-vl? Or is the Newer Qwen hidden somewhere in the other citations I cannot quickly access?
  • you Cite a withdrawn paper (not available on arxiv anymore), and you cite it for Mani-X, but the cited, withdrawn Next-IMDL paper does not make the Mani-X editing dataset explicit, does it?
Questions for authors

This is a diagnostic + practical paper. Its main value is showing that AI-image detectors are often rewarded for metadata and geometry shortcuts, and that visual encoders contain much of the needed detection signal without heavy language decoding.

My further clarification questions for the others are:

  • For resolution alignment, was the same information removal applied to real and fake images, and does the Deep-VRM or Veritas++ drop persist when fake images are matched by class-independent padding, resizing, or equal-area cropping?
  • Are any AIGI-Now, DailyBench, or in-the-wild generators also present in Mani-X training sources? Please state exact train-test overlaps / split and whether Add Text plus Replace Text was selected using held-out results
  • Were baselines evaluated with official weights and cited numbers rather than retrained, and can DDA, B-Free, or leading visual detectors be retrained under the same DINOv3, resolution, data-size, and compute constraints as your method?
Reviewer_qs9UOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper shows that MLLM detectors can be read out linearly from their ViTs and that common benchmarks leak labels through format, JPEG and geometry. It then fine-tunes DINOv3 on text-guided Mani-X edits, showing strong transfer.

Critical strengths

The shortcut problem is prevalent in the domain of AIGI detection, but most researchers do not seem to be aware.

  1. The results in Table 3 is impressive compared to AIGI-Now. The square-fake bias looks real.
  2. Robustness to real-world processing is strong, e.g., 82.4% after RRDataset re-digitization.
Critical weaknesses

W1. Section 3.2's fake-image crops change content and processing alongside geometry. The accuracy drop does not fundamentally isolate shortcut reliance. W2. Limited novelty over B-Free, which already trains on source-paired inpainting with whole-image regeneration and a restored-background variant.

Questions for authors

Q1. How would the performance of a frozen DINOv3-L compare with this work? Q2. Which Mani-X split was used for training and operation selection, and do Mani-X COCO sources overlap with AIGI-Now's COCO reals?

A satisfactory response to Q1 and Q2 would make the reviewer willing to revise the assessment accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.