- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper asks whether hidden states of frozen VLMs support detecting AI-generated images, whether the signal transfers to unseen generators and datasets, and whether it can be distilled into small image-only models. The method concatenates last-token hidden states from eight descriptive prompts, fits a PCA plus linear-probe teacher on each of five GenImage generators, compares the teachers' discriminant directions, and distills them into MobileNetV3 and ResNet-18 students. All seven VLMs separate real and generated images well in distribution, the five teachers' directions are partially aligned but distinct, and transfer and distillation gains vary widely across generators and test sets, with students that are much faster but noticeably weaker than the teacher on external benchmarks.
Critical strengths
S1. Beyond asking whether frozen VLM features can detect generated images, the paper asks whether detectors trained on different generators rely on the same discriminant direction. This question bears directly on why single-source training transfers so unevenly.
S2. The external evaluation and model-selection protocol is carefully controlled. External benchmarks play no role in fitting or model selection, the deployment configuration is fixed in advance, each student is compared with a label-only student that shares its data, architecture and seed, and the paper also reports its below-chance results.
Critical weaknesses
W1. The direction differences are confounded with image size The title's conclusion draws largely on comparing the teachers' discriminant directions. In Fig. 3b, SD1.5 and Wukong are close to each other (cosine 0.67), ADM, GLIDE and VQDM are close to each other (0.64 to 0.75), and every cross-group pair is below 0.49. This grouping may reflect the generators themselves, but it also coincides exactly with fake-image resolution in GenImage (512×512 for SD1.5 and Wukong, 256×256 for the other three). Grommelt et al. (2024) showed that detectors trained on GenImage exploit this size difference, as well as the JPEG-versus-PNG gap between real and fake images, which affects all five teachers alike. The limitations section acknowledges that teachers may pick up dataset or compression cues, but no experiment, such as a size-matched comparison, separates generator-specific signal from these cues. Without such a control, the current evidence cannot attribute the direction differences to generator provenance.
W2. The contribution of the language layers and prompts beyond visual features is not tested BPS runs the VLM once per prompt, eight times per image, and concatenates the final-layer last-token states. The representation comparison in Sec. 4.2 covers only different aggregations of language-layer hidden states. There is no comparison with Qwen3-VL vision-encoder, CLIP or DINOv2 features under the same training data and linear probe, and no ablation on the number of prompts. Zhou et al. (2026), cited in the paper, show that linear probes on frozen visual features already generalize well. Without these comparisons, I cannot tell whether the separability and direction structure the paper analyzes come from the language layers and prompts or are inherited from the visual features, or whether eight VLM forward passes per image are needed.
Questions for authors
- Each teacher is fitted only once. If a teacher is refitted on a different sample of training images from the same generator, how high is the cosine between the two directions? If it is comparable to the within-group values among ADM, GLIDE and VQDM in Fig. 3b, the differences among these three may simply reflect sampling noise.
- Sec. 6 states that "Decision-weighted matching addresses this issue", namely that accurate global reconstruction may not preserve the coordinates most relevant to detection. In the component ablation cited in Sec. 4.4 (Table S16), however, adding the weighting changes the mean OOD AUROC by −0.004, logit distillation alone reaches 0.690 against 0.683 for the full method, and Sec. 4.4 itself notes no additional gain from weighting. Could you report the full method with only the weighting removed, so that its contribution can be judged in the configuration actually deployed?
- PROBE-DINO-L, PROBE-RN50, HSIC Bottleneck and Effort in Table 1 are not cited. Do these baselines and UnivFD use released weights, or were they trained on the same SD1.5 data as the students, and what is their latency under the protocol of Table 1(b)? PROBE-RN50 scores 0.13 to 0.19 higher than the ResNet-18 student on the three benchmarks. Where do the students stand relative to detectors of comparable compute trained on the same data, for example NPR [r1]?
- According to Sec. 3.5, in Fig. 4b is computed with label-only students. Would the conclusion in Sec. 4.4 that selecting a teacher for transfer and selecting one for distillation are different decisions still hold with distilled students?
- A few minor issues. I could not find the code, and the reproducibility statement mentions robustness_results.json and source-cache hashes without saying where to obtain them. Panel B of Fig. 1 derives the signature from "geometry + trajectory", whereas Eq. (5) defines as the final-layer concatenation. The retention figures of 81.0% and 77.0% in Sec. 4.4 are raw AUROC ratios. Measured as the fraction of the teacher's margin above chance that the student keeps, they are about 53% and 43%.
[r1] Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection. CVPR, 2024.
4 comments
Thank you for your comments. The revisions are marked in blue.
W1. Section 4.3 and Table 2 (p.7) compare original processing with common-size cropping on the same eligible images. Appendix C.9 and Tables S19–S20 (pp.40–41) give the refitting results and processing details. We revised the title, Introduction, and Limitations to describe source dependence under the tested conditions. The text acknowledges that content, real-image pools, and compression history may still contribute.
W2. Section 4.2 and Table 1 (p.6) add frozen Qwen vision-tower, CLIP, and DINOv2 comparisons. Appendix C.8 and Tables S17–S18 (pp.39–40) report prompt comparisons at the same final dimension. We no longer imply that language layers or eight prompts are necessary.
Q1. Table 2 (p.7) and Appendix C.9/Table S19 (p.40) compare within-source refit similarity with between-source similarity. The appendix separates the original full-cohort results from the paired processing controls.
Q2. Appendix C.10 and Table S21 (p.41) report the requested ablation on both architectures with five paired seeds. Sections 4.4–6 (pp.8–10) revise the claim about weighting to reflect these results.
Q3. Table 3 (p.9) adds baseline citations and measured latency. Appendix B.3–B.4 (pp.20–23) specifies released-weight usage, NPR training, and the timing protocol. Table S21 (p.41) compares matched conventional students, and Table S23 (p.42) reports benchmark-specific baseline scores.
Q4. Figure 4b and Section 4.4 (p.8) use actual full-KD students in place of the label-only proxy. Appendix C.11 and Table S22 (pp.41–42) give the results and uncertainty for each architecture. The deployment choice remains unchanged.
Q5. Figure 1 (p.3) now shows final-layer prompt concatenation as the deployed signature. Section 4.4 (p.9) and Equation 54 in Appendix A.4 (p.17) report retention above chance. Appendix B.3 (p.22) documents the historical teacher and checkpoint checks. The Reproducibility Statement (p.11) describes the code release and regeneration instructions, and the anonymous repository link has been added.
We hope that our clarifications and revisions address your concerns, and we would be glad to continue the discussion if any concerns remain.
Dear Reviewer jmqy,
Thank you for your supportive assessment and detailed suggestions. We added refitting and cropping controls, visual feature and prompt comparisons, and the requested weighting ablation. We also evaluated teacher selection using distilled students, expanded the baseline comparisons, and corrected Fig.1 and the retention metric.
We hope these additions strengthen the paper and address your concerns. As the discussion deadline approaches, we would be happy to clarify any remaining questions. We appreciate your support and consideration of the revised manuscript.
Regards,
Authors
Official Comment by Reviewer jmqy
Thanks for the response and the new experiments. My main concerns are addressed. I will keep my positive score.
Thank you.
W1: Resolution and compression confounds
We refit all five teachers under the original processing and a deterministic 224x224 center crop without prior resizing. Both conditions use the same eligible images within each source, with identical labels, splits, prompts, and classifier settings. Cropping changes the learned directions, but within-source refit similarity exceeds between-source similarity, as shown under Q1.
These results show reproducible differences beyond the original 512-versus-256 grouping. Image content, real-image pools, and compression history may also contribute.
W2: Visual-feature and prompt controls
We added frozen visual-feature probes with the same source data, PCA-112, and classifier settings. The table reports external AUROC differences, language minus visual:
The language representation does not significantly improve over Qwen's vision tower on SD1.5 and performs worse on ADM. These results show that useful detection information is already accessible through visual features.
At the matched dimension, eight prompts outperform a validation-selected single prompt across three sources and underperform on two. For GLIDE, another prompt tied on validation AUROC also exceeds eight prompts externally. Eight passes are therefore not consistently necessary and take 3317.06 ms versus 865.03 ms for one prompt.
Q1: Refitting variability
We refit PCA, standardization, and the classifier on repeated stratified training halves, matching sizes across sources (1000 images per half in the original-data experiment). Within-source refit cosines range from 0.715 to 0.773. The table reports mean within-source cosine minus between-source cosine for pairs with the same original resolution.
Resampling refit repetitions gives significant differences for all four pairs under both conditions (Holm-adjusted p=0.0016), supporting source differences beyond the measured refitting variability.
Q2: Removing only decision weighting
We conducted ablations with five paired seeds by removing only decision weighting from the full objective, keeping all other loss terms and coefficients fixed. Weighting changes external AUROC by -0.0017 for MobileNetV3 and +0.0131 for ResNet-18. The differences are not statistically significant. Logit-only KD also remains competitive. We will revise Section 6 to state that decision weighting emphasizes classifier-relevant coordinates, but its additional benefit over unweighted matching is not established.
Q3: Baseline citations, training data, latency, and NPR
PROBE, HSIC, Effort, and UnivFD use released weights, without retraining on our split. HSIC and Effort checkpoints are trained on SD1.4 data. We will document checkpoint provenance and add citations for PROBE (Cao et al., ICML 2026), HSIC (Yang et al., ICLR 2026), and Effort (Yan et al., ICML 2025).
The latency results (median end-to-end latency, batch size 1, FP32) are:
We trained the official NPR architecture on our SD1.5 split with five seeds and source-validation checkpoint selection. NPR uses random initialization and 50 epochs, whereas the students use ImageNet initialization and 150 epochs.
Against label-only students with identical architecture, initialization, data, and training settings, full KD improves AUROC from 0.6604 to 0.6834 for MobileNetV3 and from 0.6246 to 0.6491 for ResNet-18. We will report the offline teacher cost.
Q4: Teacher selection using distilled students
With full-KD students, Wukong exceeds SD1.5 by 0.0157 AUROC averaged across architectures, although SD1.5 gives the stronger teacher. The difference is +0.0203 for ResNet-18 and +0.0111 for MobileNetV3. We will replace the label-only proxy with these results and qualify the conclusion by architecture. No deployment choice was changed using external results.
Q5: Code, Fig1, and retention
We will provide anonymous code and reproduction instructions, including access to or regeneration of robustness_results.json and source-cache hashes.
Fig1 will be revised to show final-layer prompt concatenation as the deployed signature, with geometry and trajectory labeled as alternatives.
Retention above chance, Retention=AUROCT−0.5AUROCS−0.5, gives 53.4% for MobileNetV3 and 43.5% for ResNet-18 with the SD1.5 teacher. We will report the absolute AUROCs for teachers and students alongside these values.