acceptodds
Under review as a conference paper at ICLR 2027

Residual Family Augmentation for Detecting Images from High-Fidelity Autoencoders

Abstract

AI-generated image (AIGI) detection has advanced rapidly, yet on the newest text-to-image models even robust detectors degrade sharply. In this paper, we diagnose the failure with a Residual Scaling Sweep, which scores a detector as the reconstruction residual is scaled, and identify an overlooked patch-size mismatch. The sweep shows that detectors trained with existing recipes respond only near the magnitude and the sign of their training residual, and they fail to keep up with recent generators. We therefore propose Residual Family Augmentation: the residual is added back to the photograph within a bounded range of magnitudes and at both signs, all labeled fake, plus a second reconstruction pass. The detector then no longer depends on the magnitude or the sign of one residual and responds across that range, which includes the weaker residuals of autoencoders never seen in training. We further observe that the autoencoders of recent generators leave a reconstruction residual, the difference between a photograph and its reconstruction, that is weaker and concentrated at finer spatial scales than the residual of earlier autoencoders, scales that the pretrained filters of vision backbones attenuate. Notably, we find that simply halving the patch of the pretrained backbone, which adds no parameter, restores the residual to scales the backbone retains, and the augmentation reaches its full gain only under the resampled patch. Trained on a single, older autoencoder, our detector leads seven public detectors by five points or more in mean balanced accuracy over eleven sets from the newest generators, while remaining competitive with the best baseline across public benchmarks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
AI-Generated image detection, image forensics, generalization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ScfhOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the effect of the residual (reconstruction error of an autoencoder) on fake image detectors. They observe that both the magnitude as well as the sign of this residual causes detectors to misclassify images. This is practically significant since, most detectors have are trained on generators with a certain residual magnitude which often decreases with future models, making reliance on these cues undesirable. The paper also claims that within vision transformer based detection backbones, the patch size is a major contributor in the detection of these results, more importantly. a larger patch size results in these details being lost. Based on these findings, the authors propose 2 changes, (i) halving the patch size, (ii) residual augmentation which involves using images that have been reconstructed once or twice with the latent diffusion autoencoder and with some probability adding the residual at a different magnitude. The authors report results on several benchmarks, such as X-AIGD, newT2I, GenImage, Synthbuster, AlignedForensics etc reporting a wide array of results demonstrating the promise of their approach.

Critical strengths
  1. The data augmentation style is novel to the best of my knowledge. It also seems to result in good generalization.
  2. The breadth of evaluation is impressive, several recent models, across multiple benchmarks. I also really like the fact that the authors have run the experiment where each baseline is compared to at its original setting as well as when training with their proposed SA-1B. Some of the experiments are also reported over 3 seeds which is a good degree of rigor.
  3. The ablation coverage is also good, demonstrating the effectiveness the design choices (based on balanced accuracy)
Critical weaknesses
  1. I personally feel the evidence behind the smaller patch sizes helping could be more convincing. My main concerns arise from Fig 4, where there is not a huge difference in terms of accuracy between the smaller patch size (around 6,7 and the baseline one, around 12). The difference could be experimental noise. The evidence in Table 2, makes this more convincing to me, plus it also intuitively makes sense to me, however, the original experiment in Fig 4 should be more convincing. I also have concerns with the use of balanced accuracy which also relates to these findings.
  2. The paper makes some crucial arguments using balanced accuracy (Fig 2,4,6 and also Table 2). I feel threshold-free estimate should be made in order to make these conclusions. For instance, in Table 1, baseline methods Stay-Positive, AlignedForensic, PROBE etc show non-trivial improvement (AUROC is good but accuracy is bad) suggesting that some of the differences may be inflated by the choice of threshold. Looking at the tables in the appendix, I do still think the detector developed by the author is comparable/is ahead of the baselines (although the margins become a lot closer in several cases), however, I think unless average precision (AP) or AUROC is used as the fundamental metric for all of the experiments, it could promote incorrect conclusions.
  3. For the experiments with post-processing (JPEG, Blur etc), it would be nice if the authors can clarify as to which of these augmentations are seen during train time and which ones are not. Once again, I think all these experiments should have AP or AUROC reported.

Writing Level Weaknesses

  1. Line 053 should come when the term residual is used for the first time. It is not intuitive to researchers as to what this term means.
  2. Equation-2: W~k\widetilde{W}_k should be defined properly when introduced.
  3. Line 112-113 is vague, what does "positive" evidence even mean.
Questions for authors
  1. Fig 4 shows a "double ascent" kind of phenomenon where the scores shoot up again, at almost twice the so-called optimal patch size, why does this happen?
  2. From Table 2, residual scaling seems to provide a huge jump on detecting GAN generated images (almost like 15 points), is there a particular reason or any intuition that the authors could provide regarding this?
4 comments
AuthorsOpenReview ↗

Response to Reviewer Scfh (1/2)

We thank the reviewer for the careful reading, for recognizing the augmentation and the breadth of the evaluation, and for the detailed suggestions.

W1 / Q1. Patch-size gain and the second rise in Figure 4

Thank you for these questions. We would like to clarify that the pretrained patch is 14 px for DINOv2 and CLIP and 16 px for SigLIP2, so the 12 px point in Figure 4 is a resampled size, not the baseline. We repeated every probe with three seeds and added AUROC (Figure 4b). At block 6 of DINOv2, the 7 px and 12 px probes are within the seed spread (.788 ± .014 against .782 ± .010). However, on all three backbones, half the pretrained size exceeds the pretrained size by more than the spread (by .036 to .127 bAcc).

Fine-tuning separates the sizes more clearly. We fine-tuned three ViT-B backbones at five patch sizes with three seeds each, using the standard recipe without residual scaling (new Table 1):

Recent 11, bAcc / AUROCPretrained size12 pxJust above halfHalf
DINOv2-B (14 px).830 / .938.865 / .966.801 / .938 (8 px).921 / .982 (7 px)
SigLIP2-B (16 px).851 / .970.893 / .983.773 / .928 (10 px).942 / .995 (8 px)
CLIP-B (16 px).966 / .997.974 / .998.963 / .996 (10 px).980 / .999 (8 px)

Half the pretrained size is best on every backbone (standard deviation over seeds .001 to .006), and also on a validation split of the training data that contains no test image.

The second rise comes from two factors. First, a smaller patch covers fewer pixels per token, so the fine residual is averaged less; this effect alone favors every smaller size. Second, the resampled filters reproduce the pretrained tokens only at some sizes. We measure this agreement on DINOv2-B with 400 photographs. We compare the tokens of the pretrained 14 px filters on a 448 px crop with those of the filters resampled to k px on the same crop resized to 32k px; both grids are 32×32, and we report the cosine of matching tokens (Table 13). The agreement is high at 14, 12, 10 and 7 px (at least .87), but low at 8 px (.51) and below .3 at 6 px and smaller. Half the pretrained size is therefore the smallest size with high agreement. Sizes just above half break the agreement, and their accuracy falls below that of the pretrained size even after fine-tuning (table above). The score then rises again toward 12 and 14 px, where the agreement returns.

W2. Threshold-free metrics and the choice of threshold

Thank you for asking for threshold-free metrics. Our detector and the reconstruction-trained baselines learn from reconstructions of real photographs and aim to generalize to generators absent from training. This is the deployment setting: a new generator appears, and no labeled images of this generator exist to tune a threshold. We therefore fixed the threshold at 0.5, as a deployed detector would, and reported bAcc as the main metric. The baselines do the same: AlignedForensics reports accuracy at 0.5, DDA reports bAcc without threshold adjustment, Stay-Positive reports accuracy at 0.5 next to AP, and B-Free reports AUC and bAcc at 0.5, noting that, in a realistic setting, calibration images of the tested generator cannot be guaranteed.

Nevertheless, AUROC, AP and TPR at 1% FPR give a complementary view, so we added these metrics to the revision: TPR@1% in Table 2 (the submission's Table 1), AUROC and TPR@1% in Tables 1 and 4 and Figure 4, and all three for the ablation in Table 21. The sweep (Figure 1) keeps one fixed threshold by design, because the sweep tests whether a single threshold remains valid across residual magnitudes. The sign test of the sweep already uses AUROC (reversed residual at β = −0.5 against the photograph). On the added metrics, our detector leads on the recent generators:

Recent 11bAccFPRAUROCAPTPR@1%
Ours (DINOv2-B/7).949.027.980.984.888
DEAR-r (R50).896.080.958.962.719
PROBE (DINOv2-L/14).853.014.952.955.698
DDA (DINOv2-L/14).827.180.900.907.589
B-Free (DINOv2-B/14).772.035.827.847.507
AlignedForensics (R50).747.001.920.921.562
Stay-Positive (R50).709.002.956.955.590
GenSign (CLIP-L/14).643.245.712.671.057

A gap between bAcc and AUROC, as for AlignedForensics and Stay-Positive, means that a detector ranks the images well, but the scores of many images from new generators fall below 0.5. At 0.5, these two detectors flag almost no photographs but detect less than half of the recent generated images (.49 and .42). Even without a threshold, the AUROC of these two detectors remains below ours, and the margin is larger at a low false-positive rate (TPR@1% .888 against at most .719 for any baseline).

AuthorsOpenReview ↗

Response to Reviewer Scfh (2/2)

W2 (continued). Calibrated thresholds

To measure how much the choice of threshold affects the comparison, we also calibrated every detector under four protocols (new Appendix D.7):

  • (i) one common validation set of held-out SA-1B photographs and their SD 2.1 reconstructions;
  • (ii) data close to the training set of each detector: COCO and LSUN photographs with LDM images for AlignedForensics, Stay-Positive and DEAR-r; COCO photographs with SD 2 reconstructions for DDA, and for B-Free and PROBE, whose training data are not available; GenImage SD 1.4 for GenSign;
  • (iii) one threshold for all 67 test sets, chosen on a disjoint half of the test images;
  • (iv) a separate threshold for each generator, chosen on its own test set, an upper bound that is not available in deployment.
Recent 11 bAcc0.5(i)(ii)(iii)(iv)
Ours (DINOv2-B/7).949.936.936.950.956
DEAR-r (R50).896.814.898.888.919
PROBE (DINOv2-L/14).853.810.848.872.905
DDA (DINOv2-L/14).827.843.837.843.858
B-Free (DINOv2-B/14).772.749.777.780.798
AlignedForensics (R50).747.708.824.818.871
Stay-Positive (R50).709.677.839.881.909
GenSign (CLIP-L/14).643.652.651.653.679

AlignedForensics and Stay-Positive gain the most from calibration, because at 0.5 the scores of many recent generated images fall below the threshold (W2 above). However, our detector remains first under every protocol, and its default threshold of 0.5 is already close to its per-generator optimum (.949 against .956).

W3. Post-processing: seen augmentations and AUROC

Thank you for the suggestion. Table 18 now marks the post-processing that each detector sees in training (J: JPEG, B: blur, R: downscaling), and Table 19 gives the AUROC of every condition. The table below gives the means over the conditions of each group (200 pairs per recent generator):

bAcc / AUROCSeenJPEG (4)Blur (3)Downscale (2)All 9
AlignedForensics (R50)J B.647 / .848.706 / .904.552 / .797.646 / .855
Stay-Positive (R50)J B.618 / .904.644 / .919.539 / .798.609 / .885
B-Free (DINOv2-B/14)J B R.735 / .756.734 / .764.616 / .657.708 / .737
DDA (DINOv2-L/14)J B R.745 / .816.825 / .896.728 / .797.768 / .838
PROBE (DINOv2-L/14)?.790 / .890.799 / .898.641 / .762.760 / .864
DEAR-r (R50)J B.831 / .910.869 / .945.723 / .806.819 / .899
GenSign (CLIP-L/14)–.500 / .460.611 / .663.590 / .620.557 / .563
Ours (DINOv2-B/7)J R.774 / .870.893 / .959.764 / .867.811 / .899
Ours (DINOv2-L/7, one seed)J R.844 / .940.952 / .984.859 / .961.884 / .959

Our detector sees JPEG and downscaling in training but not blur. With DINOv2-B/7, our detector is the most accurate under blur and downscaling and matches DEAR-r in AUROC over the nine conditions, but trails DEAR-r and PROBE under JPEG. With DINOv2-L/7, the backbone size of DDA and PROBE, our detector is the most accurate in every group. We discuss JPEG in General Response §2 and in the reply to 5n62.

Writing

Thank you for these suggestions. We now define the residual where the term first appears in the abstract and the Introduction, and define W~k\widetilde{W}_k with Equation 2. We also replace the vague "positive evidence" with what Stay-Positive does: Stay-Positive constrains the weights of the last layer to be non-negative, so that the score rises only with features that indicate a generated image.

Q2. Intuition for the gain on GAN sets

Thank you for this question. Our intuition is that GAN images carry no autoencoder residual, but the last layers of GAN generators are learned upsamplers that also leave fine high-frequency detail, with a magnitude different from the SD 2.1 residual. A detector trained on one magnitude misses this detail, whereas residual scaling trains the detector to respond across magnitudes. Our results support this intuition. The attenuated part of the family (0 < β < 1) helps most: training with this range alone raises the bAcc on the seven GAN sets of Table 3 (the submission's Table 2) from .509 to .655, more than negative β or β > 1 alone, and the full family reaches .700 (three seeds).

We thank the reviewer again, and we would be glad to clarify any point further.

Reviewer_ScfhOpenReview ↗

Reviewer Response

I would like to thank the authors for taking the time to perform the requested experiments. This gives me confidence that the half-patch size is helpful as well as the results not being inflated due to the reporting of threshold-less metrics. Furthermore, I also feel confident regarding the robustness to post-processing. I thereby keep my score and recommend acceptance to the conference.

We thank the reviewer for showing us that readers also want to see threshold-free metrics such as AUROC. We are also grateful for the detailed comments on the writing, which made the paper clearer. We sincerely appreciate the careful review.

Reviewer_T4EJOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper addresses the poor generalization of reconstruction-based AI-generated image detectors to images from recent generative models. The authors argue that reconstruction residuals vary across generative models, causing detectors to rely on model-specific residual patterns. To address this issue, they propose Patch Resampling and residual family augmentation, which introduce residual signals with different magnitudes and directions and generate additional reconstructed samples. These methods help improve the detector’s generalization ability. Experiments on multiple test sets show promising results.

Critical strengths
  1. The paper addresses the limited generalization of reconstruction-based AI-generated image detectors to recent generative models.
  2. The authors propose residual family augmentation combined with Patch Resampling to improve the detector's ability to handle diverse residual patterns.
  3. The test sets cover multiple generative models, and the paper provides a reasonably comprehensive set of ablation studies.
  4. The method is relatively simple and achieves promising generalization performance.
Critical weaknesses
  1. The proposed method mainly consists of Patch Resampling and residual family augmentation, but some aspects of these designs appear similar to existing methods. The authors should further clarify the novelty of their approach, explain how it differs from prior methods in terms of motivation and mechanism, and better highlight its core contributions.
  2. The method constructs residuals with different magnitudes and directions, labeling all resulting samples as fake to improve generalization. However, it remains unclear whether these artificially constructed residuals can adequately represent the characteristics of real generative models. The authors should further investigate whether the method genuinely improves generalization and analyze which features the detector relies on when making predictions.
  3. The paper mainly focuses on generative models based on latent autoencoders. It remains unclear whether the proposed method is also effective for generative models that do not rely on such architectures. Further validation is needed to establish its broader applicability.
  4. The authors are encouraged to provide ablation studies using different backbones to examine whether Patch Resampling depends on a specific backbone and whether it consistently improves performance across different architectures.
Questions for authors

Please refer to the weaknesses mentioned above and address these concerns in the revised manuscript.

6 comments
AuthorsOpenReview ↗

Response to Reviewer T4EJ (1/4)

We thank the reviewer for the careful review and for noting that the method is simple and generalizes well. We refer to the General Response for the shared points.

W1. Novelty in motivation and mechanism

Thank you for asking us to clarify the novelty. Our method learns from the reconstructions of just a single decoder rather than from many generators. Unlike existing residual-based detectors, the method aims to read the residuals of decoders absent from training. This is possible because we found a property of the residual that prior work does not exploit: across eight autoencoders, the residuals share one pattern and differ mainly in magnitude and spatial scale (Section 3). Because of this difference, a detector that learns the residual of one decoder can rely on what is specific to that decoder, and the proposed Residual Scaling Sweep shows that prior detectors take various shortcuts of this kind. These shortcuts break where decoders differ in magnitude and scale, so a detector needs to respond to the shared pattern at any magnitude and sign, and at the finer scale of recent decoders, a requirement that prior work misses.

A direct way to meet this requirement is to collect many autoencoders. We instead meet it with just a single decoder: because the residual of a single decoder already carries this shared pattern, reusing the scaling of our diagnosis shows the detector this pattern in many variants. As in the Residual Scaling Sweep, scaling this residual by β keeps the pattern while spanning the magnitudes of other decoders and both signs. Following this derivation, we propose two simple yet effective components. The first, Residual Family Augmentation, is a new training scheme that obtains these variants from a single decoder instead of many autoencoders, including amplified and reversed residuals, on which no prior detector trains. The second, Patch Resampling, assists the first: halving the patch lets the finer residual of recent decoders pass the patch embedding, so that the response taught by Residual Family Augmentation also reaches these decoders.

Specifically, we support each step of this derivation with our results below.

Evidence for the property. Over eight autoencoders, the first principal component carries 48% of the residual variance with a positive loading on every autoencoder, and the residuals of recent decoders are weaker (.45 to .64 of the SD 2.1 magnitude) and finer (.31 to .52 of the residual energy in flat regions at periods below four pixels, against .21 to .28 for Stable Diffusion; Figure 3).

Prior detectors. AlignedForensics, Stay-Positive and B-Free train on reconstructions (β = 1 below) against content bias, and DDA blends a photograph with its JPEG-aligned reconstruction (0 ≤ β ≤ 1) against format bias. Each trains on one point or segment of the β range. Standard benchmarks hide the resulting shortcuts, because each generator is scored only at its own residual, whereas the three readings of the sweep (Section 3.2) expose them (Appendix D.3):

Sweep readingShortcutPublic detectorsBlocked by
Decoder spread at matched damagethe specific pattern of the training decoderAlignedForensics, Stay-Positive: .97 to .99 on older decoders, .50 to .66 on recent onesPatch Resampling
Amplitude responsethe magnitude of the residualB-Free rises from .60 to .79 and DDA falls from .77 to .59 on recent decoders as β rises from 1 to 2.8Residual family
Sign readingsharpness as a cue for realDDA, PROBE, DEAR-r rate a photograph sharpened by the reversed residual as more real than the original (AUROC .25, .38, .39)Residual family

Each shortcut costs accuracy on recent generators, whose residuals are weaker and finer. On the recent sets (Table 2), the detectors that rely on the pattern or the magnitude of the training residual (AlignedForensics, Stay-Positive, B-Free) reach .71 to .77, the detectors that rely on sharpness (DDA, PROBE, DEAR-r) reach .83 to .90, and our detector reaches .949.

Gains of each component. Patch Resampling, which halves the pretrained patch as in FlexiViT, closes most of the sweep gap between older and recent decoders (.27 to .09) and raises bAcc on the recent sets from .830 to .921, but leaves the amplitude and sign readings unchanged (Section 3.2), so on its own this step still leaves the hardest recent set at .670 and the other 56 sets nearly unchanged (.816 to .824). Residual Family Augmentation then labels fake the residuals at seven values of β with both signs and a second reconstruction pass. With the same photographs, decoder and loss, this step raises the recent sets further to .949, the hardest one to .802, and the other 56 sets from .824 to .889 (Table 3). The two components work together: with the pretrained patch, Residual Family Augmentation adds only .010 on the recent sets, against .028 with the resampled patch (Tables 3 and 15).

AuthorsOpenReview ↗

Response to Reviewer T4EJ (2/4)

W1 (continued). Relation to DDA

DDA is the closest prior work in form, and the sweep shows that DDA does not read the residual. DDA rates a photograph sharpened by the reversed residual as more real than the photograph itself (AUROC .25). More importantly, DDA misses more fakes as the residual grows (.77 at β = 1 and .59 at β = 2.8 on the recent decoders), although a stronger residual should make a fake easier, not harder, to detect.

The training of DDA explains this. In the released code, DDA labels fake a blend of the photograph x and its JPEG-aligned reconstruction x̃, in pixels or over all DCT coefficients with one weight. The blends align the fakes with the photographs, so that format traces such as JPEG cannot separate the two classes, and serve as hard fakes close to the photograph. As a result, every fake of DDA lies in a narrow band between the photograph and its smoother reconstruction, and DDA reads a deficit of fine detail relative to the photograph, not the residual: an image outside this band, sharper or more strongly damaged, is often called real (Appendix D.3).

Mathematically, the blends are close to our method: because the DCT is linear, both blends equal x + β(x̃ − x) with β between 0 and 1, the attenuated part of our residual family. The goals differ, however. DDA moves the fakes toward the photographs to align the two classes, whereas we scale the residual in both directions and beyond the reconstruction, so that neither the magnitude nor the direction of the change decides the label and the detector has to read the residual itself. This difference matters on real generators: within our recipe, the range 0 < β < 1 alone reaches .922 on the recent sets, against .949 for the full family (Table 4).

In short, the core of this work is a set of novel findings and a simple yet effective solution. The findings are the property of the residual (one shared pattern that differs mainly in magnitude and spatial scale), the shortcuts that this property exposes in prior detectors, and the requirement that follows. The solution meets this requirement with two simple components: Residual Family Augmentation covers the magnitude and the sign, and Patch Resampling assists it at the finer spatial scale of recent decoders. To our knowledge, no prior work measures these shortcuts, and no prior detector trains on amplified or reversed residuals. We revised the Abstract, the Introduction, Section 3 and the Conclusion to state this view.

AuthorsOpenReview ↗

Response to Reviewer T4EJ (3/4)

W2. Constructed residuals and real generators

Thank you for this question. Like most of the baselines, which also train on a single generator or autoencoder, we do not aim to make the training fakes stand in for other generators, for the detector to memorize. The constructed residuals instead tell the detector that generators share one residual pattern but can differ in its magnitude and sign (Section 3). Scaling the residual of a single decoder keeps this shared pattern and varies its magnitude over a range that brackets the recent decoders, and the reversed residuals keep sharpness from marking a photograph as real. The second pass adds a second residual pattern from the same decoder. The detector therefore learns the pattern that generators share, not the specific pattern of its training decoder.

This shows in the accuracy on generators absent from training, including GANs and closed generators (Table 3, General Response §3), and more directly in the feature space of each model (Appendix D.5, Table 20). For each recent generated image, we find the nearest neighbour in a pool of photographs and training-like fakes:

Feature space, recent 11Nearest neighbour is a fakeMissed at the threshold
Reconstruction only (DINOv2-B/14).67.33
+ resampled patch (DINOv2-B/7).90.16
+ residual scaling.89.09
+ second pass (Ours).90.07

The resampled patch moves the recent generated images next to the training fakes. In our final model, their nearest fakes are the reconstructions and the positively scaled residuals, chosen more often than their share of the pool, and almost never the reversed residuals: real generators fall within the magnitudes that the scaling covers. The residual family also cuts the miss rate by more than half at about the same nearest-neighbour rate, moving these images further from the photographs, while the false-positive rate on the recent reals stays low (.027, Table 2), below that of DEAR-r and DDA.

Features the detector relies on. Our detector responds to the decoder residual at every β from −1 to 2.8 (Section 3.2), not only near the residual of its training decoder. Our detector also responds to both signs of the residual that closed generators leave when asked to return a photograph unchanged, whereas the baselines respond mainly to the original residual (Appendix F).

Reviewer_5n62OpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper proposes Residual Family Augmentation for training reconstruction-based AIGI detectors. The authors argue that existing detectors depend too much on the magnitude and signed pattern of the training residual, while recent latent autoencoders leave weaker and finer residuals that are weakened by pretrained patch embeddings. The method resamples the DINOv2 patch filters from 14x14 to 7x7 and creates fake samples using multiple residual magnitudes, both residual signs and a second reconstruction pass. Trained only on SD 2.1 reconstructions, it reaches 0.949 mean balanced accuracy on the recent 11 benchmark, ranking first among seven public detectors.

Critical strengths
  1. The paper has clear novelty. The Residual Scaling Sweep provides a new diagnostic view, and Residual Family Augmentation is directly motivated by the diagnosis. The evaluation is also relatively complete, covering 67 generator sets, seven public detectors and multiple post-processing operations.

  2. The Residual Scaling Sweep attributes the failure of existing detectors to two dependencies: the residual magnitude and the signed pattern. Result shows the residual family gains under the resampled patch from .921 to .947 balanced ACC.

  3. To preserve fine-scale reconstruction residuals that are weakened by pretrained patch embeddings, the authors resample the patch filters to half their original size without adding parameters. The paper reports a clear balanced-accuracy improvement on the recent-11 sets.

Critical weaknesses
  1. Table 2 shows that residual scaling raises balanced accuracy on the recent 11 sets from 0.921 to 0.947, while FPR on the real images of all benchmarks rises from 0.006 to 0.039. Table 3 further shows that adding negative β raises the mean to 0.949, but also raises the FPR to 0.027. I suggest adding AUROC or TPR at FPR=1% to these tables, so that it is clear whether the negative β values and the second pass improve discrimination or only shift the decision threshold.

  2. Post-processing robustness matters for deploying AIGI detectors, but the full comparison is only in Appendix D.4. Figure 8 and Table 16 show that the proposed detector degrades faster under JPEG compression than the other methods. Yet the training already applies JPEG re-encoding at qualities 40–95 (Appendix B). The key robustness comparison should be moved to the main paper, and this degradation needs an explanation.

Questions for authors
  1. I suggest reporting AUROC and TPR at FPR=1% for each row of Table 2 and Table 3, so that the practical gain of each component at a low false-positive rate can be assessed.

  2. The authors attribute the 0.54–0.62 bACC on GauGAN, BigGAN, DALL-E 3 and Firefly to these generators being outside the scope of the work, and the Limitations section says so. I suggest that the authors report detection results on recent mainstream closed-source generators such as GPT-Image-2, Nano-Banana-2.0 and Qwen-2.0-Pro, which would show whether the method also works on closed-source generators.

  3. I would appreciate further explanation of why TPR still drops largely from 0.952 to 0.834 at JPEG quality 95 after training with JPEG re-encoding at quality [60, 95]. And drops from largely 0.952 to 0.703 at JPEG quality 60.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 5n62 (1/2)

We thank the reviewer for the positive assessment and the precise suggestions. We answer each point below.

W1 / Q1. Discrimination gain or threshold shift

Thank you for this suggestion. We added AUROC and TPR at 1% FPR to both tables (now Tables 3 and 4; General Response §1). All values below are means over three seeds of our DINOv2-B/7 detector.

Components (Table 21)Recent bAccRecent AUROCRecent TPR@1%Other 56 bAccOther 56 AUROCOther 56 TPR@1%FPR
Resampled patch.921.982.861.824.917.673.006
+ residual scaling.947.979.883.884.947.750.039
+ second pass (Ours).949.980.894.889.954.750.042

The residual family raises the false-positive rate from .006 to .042, but the gain is not only a shift of all scores toward fake: at a matched 1% FPR, the residual family detects more generated images on both groups, and AUROC on the other 56 sets rises from .917 to .954.

On the recent sets, the resampled patch alone already ranks the images well (AUROC .982), so the further gain there lies mainly at the decision threshold. This gain matters in deployment: a detector must decide at a fixed threshold, because no labeled images of a new generator exist to tune it, and the baselines also report accuracy at 0.5. With a separate best threshold for each generator, the resampled patch would reach .952, but at 0.5 it reaches .921, because the scores of several unseen generators fall on the wrong side of the threshold. The residual family closes this gap (.949 against .956) and halves the spread across generators (standard deviation .102 to .055), so the fakes of unseen generators fall on the fake side of the fixed threshold by themselves.

AUROC on the recent sets also hides the shortcuts that the resampled patch leaves. The sweep shows them (Table 17, bAcc at 0.5 over eight decoders):

Sweep, bAcc at 0.5β = −1β = −0.5β = 0.25β = 1β = 2.8
Resampled patch.500.500.513.925.631
+ residual family (Ours).966.916.775.971.982

The resampled patch alone responds only near the residual of its training decoder: at chance for a reversed or weak residual, and at .631 for an amplified one, the magnitude and sign shortcuts of Section 3.2. These shortcuts remain a risk for generators outside our test sets: a generator whose residual is weaker, stronger or sharper than that of the training decoder would fall where the resampled patch alone fails, even though the resampled patch ranks the current recent sets well. The residual family removes both shortcuts, and the risk is already visible outside the recent sets: on VQDM, whose VQ autoencoder is absent from training, the resampled patch alone reaches .559 bAcc and .691 AUROC, against .890 and .967 with the residual family (one seed).

Training β (Table 4), recent 11bAccFPRAUROCTPR@1%TPR@1%, one shared threshold
none (one pass).921.004.982.861.862
β < 0.904.030.951.785.782
0 < β < 1.922.019.962.845.844
β > 1.934.004.989.887.888
all seven (Ours).949.026.980.894.902

Because decoders differ mainly in the magnitude of the residual (Section 3.1), each range covers a different part of the magnitude range. The model trained only on the range above 1 has the highest AUROC. However, this model responds only to amplified residuals: on the sweep, its bAcc is at most .63 for every β ≤ 0.5. Negative β alone lowers the AUROC, but negative β is the only range that makes the detector respond to the residual with the opposite sign. Negative β also brings gains that the recent mean hides: alone, it raises the hardest recent sets (Krea .670 to .793, Z-Image .789 to .907) and VQDM from .559 to .852, where the attenuated range 0 < β < 1 stays at .534. Combined, the full range gives the highest bAcc and TPR@1%. The last column uses one threshold for all eleven sets, set at 1% FPR, as in deployment. With this shared threshold, the full range detects the most generated images, and the full range also leads on the other 56 sets (.756 against .729 for the range above 1). Per-set AUROC implicitly chooses a new threshold for each generator and therefore hides the dependence on the residual magnitude. Finally, the false-positive rate of the full range remains below that of DEAR-r and DDA.

AuthorsOpenReview ↗

Response to Reviewer 5n62 (2/2)

W2 / Q3. Robustness comparison and the JPEG drop

Thank you for this request. We moved the comparison to the main text as Section 4.4, with the post-processing that each detector sees in training and the AUROC of every condition (General Response §2).

Reconstruction-based detectors, ours included, read the fine residual that a decoder leaves, and JPEG removes fine detail. Augmentation teaches a detector that compressed images can be fake, but augmentation cannot restore the detail that compression has removed. Therefore, every detector is weakest under JPEG on the six recent sets with the finest residuals (the FLUX.1, FLUX.2 and CogView4 autoencoders):

JPEG, mean of four qualities, bAcc / AUROCSix sets with the finest residualOther five sets
AlignedForensics (R50).545 / .774.771 / .936
Stay-Positive (R50).518 / .871.737 / .943
B-Free (DINOv2-B/14).586 / .587.913 / .960
DDA (DINOv2-L/14).647 / .709.863 / .944
PROBE (DINOv2-L/14).654 / .808.954 / .989
DEAR-r (R50).773 / .864.900 / .965
GenSign (CLIP-L/14).475 / .422.529 / .507
Ours (DINOv2-B/7).685 / .804.881 / .949
Ours (DINOv2-L/7, one seed).765 / .901.939 / .987

On these six sets, the result depends on the size of the backbone. With DINOv2-B/7, our detector has the highest bAcc among the transformer detectors but remains below DEAR-r, a ResNet-50 with strong JPEG augmentation. With DINOv2-L/7, the backbone size of DDA and PROBE, our detector has the highest AUROC of all detectors and a bAcc close to that of DEAR-r. Stronger augmentation does not remove this loss: JPEG on half of the training items, close to the setting of DEAR-r, adds only .011 under JPEG and lowers the clean bAcc by .015.

The comparison over all nine conditions follows the same pattern. Our DINOv2-B/7 detector matches DEAR-r in AUROC and leads every transformer detector, and our DINOv2-L/7 detector leads all detectors (General Response §2). Under JPEG, DEAR-r and PROBE lead our DINOv2-B/7 detector, DEAR-r at about three times our false-positive rate on clean images. Moreover, when the threshold of our DINOv2-B/7 detector matches the false-positive rate of DEAR-r, our TPR at quality 95 is higher (.788 against .765).

Q2. Closed-source generators

Thank you for this suggestion. We generated 200 images each with GPT Image 2, Nano Banana 2 and Qwen Image 2.1 Pro, from the X-AIGD prompts alone and at 1024×1024, and paired each image with the photograph of the same prompt. Our detector has the highest bAcc, AUROC and TPR@1% of the eight detectors (mean bAcc .983; General Response §3). Appendix F gives the full protocol and an analysis: when asked to return a photograph unchanged, these generators leave a residual as large as that of SD 2.1. The baselines respond to this residual only near its original magnitude, whereas our detector responds at every magnitude.

We hope these results answer the reviewer's questions, and we would be glad to provide further details.

Thank the authors for the supplementary experiments. The added AUROC and TPR@1% answer my main question. I will raise Soundness to 3 and keep my rating at 3.

The reviewer’s suggestions led us to verify the gain of each component at a low false-positive rate and to examine our detector on closed-source generators. We would like to study the detection of such generators further in future work. The review made the paper more complete, and we sincerely thank the reviewer for the constructive comments.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response: threshold-free metrics, post-processing, and closed-source generators

We thank all reviewers for their careful reading. All three name the residual family as a strength, and 5n62 and Scfh value the breadth of the evaluation. Changes to the manuscript are marked in blue, and table numbers refer to the revised manuscript (the submission's Tables 1, 2 and 3 are Tables 2, 3 and 4).

§1. Threshold-free metrics (Scfh W2/W3, 5n62 W1/Q1)

We use balanced accuracy at the fixed threshold of 0.5 (logit 0 for B-Free) as the main metric, because a deployed detector has no labeled images of a new generator to tune the threshold. Nevertheless, threshold-free metrics give a complementary view, so we added TPR at 1% FPR (TPR@1%) next to AUROC in Table 2, and all three metrics for the ablation (Table 21).

Recent 11bAccAUROCAPTPR@1%
AlignedForensics (R50).747.920.921.562
Stay-Positive (R50).709.956.955.590
B-Free (DINOv2-B/14).772.827.847.507
DDA (DINOv2-L/14).827.900.907.589
PROBE (DINOv2-L/14).853.952.955.698
DEAR-r (R50).896.958.962.719
GenSign (CLIP-L/14).643.712.671.057
Ours (DINOv2-B/7).949.980.984.888

Our detector leads in every metric, and the margin is largest at a low false-positive rate.

§2. Post-processing (5n62 W2/Q3, Scfh W3)

We moved the robustness comparison to the main text (Section 4.4, Figure 7) and added AUROC for every condition (Table 19).

JPEG lowers the accuracy of every reconstruction-based detector, ours included, for one reason: these detectors read the fine residual of a decoder, and JPEG removes fine detail. JPEG augmentation teaches a detector that compressed images can be fake, but augmentation cannot restore the removed detail. Therefore, every detector is weakest under JPEG on the generators with the finest residuals (reply to 5n62).

Seen marks the post-processing in training (J: JPEG, B: blur, R: downscaling, ?: not released; Table 18).

Recent 11, bAcc / AUROCSeenJPEG (4)Blur (3)Downscale (2)All 9
AlignedForensics (R50)J B.647 / .848.706 / .904.552 / .797.646 / .855
Stay-Positive (R50)J B.618 / .904.644 / .919.539 / .798.609 / .885
B-Free (DINOv2-B/14)J B R.735 / .756.734 / .764.616 / .657.708 / .737
DDA (DINOv2-L/14)J B R.745 / .816.825 / .896.728 / .797.768 / .838
PROBE (DINOv2-L/14)?.790 / .890.799 / .898.641 / .762.760 / .864
DEAR-r (R50)J B.831 / .910.869 / .945.723 / .806.819 / .899
GenSign (CLIP-L/14)–.500 / .460.611 / .663.590 / .620.557 / .563
Ours (DINOv2-B/7)J R.774 / .870.893 / .959.764 / .867.811 / .899
Ours (DINOv2-L/7, one seed)J R.844 / .940.952 / .984.859 / .961.884 / .959

The comparison depends on the size of the backbone. With DINOv2-B/7, our detector matches DEAR-r in AUROC over the nine conditions and leads under blur and downscaling. Under JPEG, however, our DINOv2-B/7 detector falls below DEAR-r and PROBE, through missed generated images rather than false positives. With DINOv2-L/7, the backbone size of DDA and PROBE, our detector leads in every group, JPEG included.

§3. Closed-source generators (5n62 Q2)

As 5n62 suggested, we generated 200 images with each of three closed generators from X-AIGD prompts and paired each image with the photograph of the same prompt (Appendix F).

bAcc / AUROCGPT Image 2Nano Banana 2Qwen Image 2.1 ProFPR
AlignedForensics (R50).595 / .897.963 / .992.562 / .897.000
Stay-Positive (R50).545 / .940.992 / 1.000.575 / .954.005
B-Free (DINOv2-B/14).497 / .201.532 / .576.620 / .657.035
DDA (DINOv2-L/14).745 / .839.885 / .955.867 / .946.160
PROBE (DINOv2-L/14).738 / .930.840 / .982.965 / .996.005
DEAR-r (R50).945 / .993.963 / 1.000.950 / .993.075
GenSign (CLIP-L/14).595 / .685.505 / .493.713 / .779.225
Ours (DINOv2-B/7).985 / .999.978 / .997.985 / .998.020

Trained only on SD 2.1 reconstructions, our detector is the most accurate on GPT Image 2 and Qwen Image 2.1 Pro, and the only detector above .97 bAcc on all three generators. Appendix F explains this result: asked to return a photograph unchanged, a closed generator leaves a residual as large as that of SD 2.1, and in a scaling sweep of this residual, only our detector responds across magnitudes and signs. Because their architectures are unknown, closed generators remain an additional test, not part of the main claim.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.