- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes EFG (Encoder-Free Forensic Guard), an encoder-free vision-language framework for AI-generated image detection and explanation. The authors argue that conventional VLM-based forensic detectors suffer from a visual encoder bottleneck, where semantic-oriented encoders discard low-level forensic artifacts required for reliable detection. To address this issue, EFG removes the external visual encoder and directly converts raw image patches into visual tokens through a lightweight patch embedding module, allowing the language model to reason over both low-level forensic traces and high-level semantics. The model is fine-tuned with LoRA and evaluated on multiple synthetic image detection benchmarks with both detection accuracy and explanation quality. The experimental results show strong performance compared with existing VLM-based forensic detectors and conventional detection approaches.
Critical strengths
- Important and timely problem. The paper addresses an important limitation of current VLM-based synthetic image detectors. Improving both detection accuracy and explanation ability is valuable for trustworthy AI-generated content analysis.
- Interesting encoder-free formulation. The key idea of removing the visual encoder and directly feeding pixel-level tokens into a native VLM is interesting. The paper provides a different perspective from previous approaches that attempt to adapt or fine-tune existing visual encoders.
- Strong empirical evaluation. The paper evaluates EFG on multiple benchmarks, including FakeClue, LOKI, Holmes-Set, and DMImage. The results demonstrate improvements over both conventional detectors and specialized forensic VLMs.
- Practical explainability advantage. Compared with traditional binary classifiers, EFG can provide natural-language explanations describing potential artifacts and inconsistencies, which improves interpretability.
Critical weaknesses
- The contribution of the encoder-free design is not sufficiently isolated. The central claim of the paper is that visual encoders introduce a fundamental bottleneck for forensic detection. However, the current experiments do not fully isolate whether the improvement comes from removing the encoder itself or from other differences, such as the choice of native VLM backbone, tokenization strategy, or training procedure. A more controlled comparison using the same backbone with encoder-based and encoder-free variants would provide stronger evidence.
- Limited methodological novelty beyond applying native VLMs to forensics. Encoder-free or native vision-language architectures have been explored in previous works. The main novelty of this paper comes from applying this paradigm to synthetic image detection rather than introducing a fundamentally new multimodal learning mechanism. The authors should better clarify the forensic-specific contribution compared with existing native VLM architectures.
- Insufficient analysis of how forensic signals are preserved. The paper claims that removing the visual encoder allows preservation of low-level forensic cues. However, there is limited analysis showing which visual tokens or internal representations capture these cues. More detailed visualization or attribution studies would make the proposed mechanism more convincing.
- Explanation evaluation remains incomplete. Although EFG generates better explanations according to automatic metrics, metrics such as ROUGE-L and CSS do not fully measure whether explanations are actually grounded in correct forensic evidence. Human evaluation or artifact-level verification would strengthen the explanation claims.
- Lack more visualization examples. The paper lacks sufficient visualization examples to qualitatively demonstrate the effectiveness of the proposed method. More visual comparisons, failure cases, and intermediate results would be helpful to provide stronger evidence and make the claims more convincing.
Questions for authors
- Can the authors provide a controlled comparison where the only difference is whether the visual encoder is used?
- Which components of the native visual tokens contribute most to forensic detection?
- How reliable are the generated explanations compared with human forensic judgments?
- Could you give more visualization examples to illustrate?
- Could the authors provide robustness evaluations under additional degradations, such as Gaussian noise or resizing?
3 comments
Response to Reviewer YvLA (2/2)
Weakness 4: Explanation evaluation incomplete
Thank you for this comment. As ROUGE-L and CSS measure text similarity rather than grounding, we add a human evaluation.
Five annotators independently evaluate the same 50 images (balanced between real and fake and covering multiple generators) together with the answers of three models (EFG, FakeVLM and AIGI-Holmes). Two key aspects are assessed: (1) content description — whether the description correctly reflects the content of the image (1–5); (2) credibility — whether the annotator finds the judgment and its justification trustworthy (1–5). Annotators also select the most satisfactory answer among the three models.
Table 2: Human evaluation of model answers.
| Criterion | EFG | FakeVLM | AIGI-Holmes |
|---|---|---|---|
| Content description (1–5) | 4.52 | 3.87 | 4.31 |
| Credibility (1–5) | 3.76 | 3.18 | 3.43 |
| Win rate (%) | 40.6 | 27.4 | 32.0 |
EFG obtains the highest content-description and credibility scores and is preferred in 40.6% of the cases (vs. 27.4% and 32.0%). Since the image set is balanced between real and fake, these results cover both classes. We therefore revise the claim from "better explanations" to "more accurate and better-grounded explanations".
Weakness 5: More visualization examples
Thank you for this reminder. We will add more comparison cases and failure cases to the Appendix, together with a dedicated failure-case figure.
Question 1: Controlled comparison
Please see Weakness 1 (Table 1). The backbone, data, LoRA configuration, schedule and resolution are identical; only the visual front-end differs.
Question 2: Contribution of native visual tokens
We compare the image regions to which the fake token responds: no consistent, dominant subset of native visual tokens is found, and the response is distributed over the whole image. This indicates that all regions contribute to the fake prediction, which is consistent with the fact that the entire synthetic image is generated rather than locally manipulated.
Question 3: Reliability versus human judgments
Please see Weakness 4 (Table 2): a five-annotator evaluation of content description, credibility and preference, where EFG is rated highest on all three.
Question 4: More visualization examples
Please see Weakness 5.
Question 5: Robustness to Gaussian noise and resizing
We extend Table 4 of the paper with resizing (0.5–2.0×); the Gaussian blur results are reproduced from Table 4 of our paper.
Table 3: Additional degradation robustness (FakeClue accuracy).
| Method | Gaussian blur σ=1 | Resize (0.5–2.0×) |
|---|---|---|
| CNNSpot | 47.6 | 51.2 |
| UnivFD | 53.3 | 60.1 |
| NPR | 59.3 | 85.6 |
| AIDE | 68.7 | 81.3 |
| FakeVLM | 90.6 | 95.1 |
| AIGI-Holmes | 91.3 | 94.8 |
| Ivy-xDetector | 83.5 | 90.3 |
| EFG | 92.0 | 95.2 |
EFG remains the most accurate under both degradations, and its drop under resizing is smaller than under Gaussian blur.
Summary
Dear Reviewer YvLA,
We are deeply grateful for the time and effort you have invested in reviewing our paper. To conserve your valuable time, here we summarize the key points of our rebuttal to facilitate a quick read.
-
Same-backbone comparison: we add a controlled comparison with frozen ViTs and fine-tuned ViT; EFG outperforms all of them on all four FakeClue metrics. (Please refer to Weakness 1 for details)
-
Contribution reframed: the novelty is the forensic finding, not the architecture. (Please refer to Weakness 2 for details)
-
Mechanism: encoder-based variants lose accuracy and, more sharply, explanation quality. (Please refer to Weakness 3 for details)
-
Explanation evaluation: we add a five-annotator human evaluation; EFG achieves the best content-description, credibility and preference scores. (Please refer to Weakness 4 for details)
-
Robustness: blur and resizing results added. (Please refer to Question 5 for details)
We humbly hope that our responses have adequately addressed your concerns. If you have further concerns, please let us know and we will continue actively responding to your comments and improving our submission.
Thank you for your invaluable time and consideration.
Best regards,
Authors of Paper 33562
Response to Reviewer YvLA (1/2)
Dear Reviewer YvLA,
We thank you for the insightful and valuable comments. We respond to each comment as follows and sincerely hope that our rebuttal could properly address your concerns.
Weakness 1: Encoder-free contribution not isolated
Thank you for this precise comment. To further isolate the encoder variable, we add the controlled comparison: the same NEO-2.2B backbone, training data, LoRA configuration, schedule and resolution — only the visual front-end differs. Specifically, we replace EFG's patch embedding with (i) a frozen CLIP-ViT-L/14, (ii) a CLIP-ViT-L/14 fine-tuned on FakeClue, and (iii) a frozen SigLIP-400M, each connected to NEO through a linear projection.
Table 1: Same-backbone controlled comparison on FakeClue.
All encoder-based variants underperform EFG on all four metrics. Fine-tuning the encoder partially recovers accuracy, consistent with Figure 1, but does not close the gap. Notably, the drop is much larger for ROUGE-L and CSS than for accuracy: adding a ViT introduces an additional feature level that is not aligned with the image-level features, but we have kept all other experimental conditions as consistent as possible.
Weakness 2: Limited methodological novelty
Thank you for this comment. The novelty of this work is a forensic finding: the visual encoder is a bottleneck for synthetic-image detection. Encoder-free design is the instrument to verify this, and it surpasses encoder-based methods in performance. We substantiate this with three observations. (1) We confirm that the encoder affects forgery features: tuning it improves Qwen2-VL-2B from 89.8 to 90.7 and LLaVA-1.6-7B from 90.1 to 91.2 on Holmes-Set (Figure 1). (2) Prior forensic works act on the encoder side — AIGI-Holmes fine-tunes it and Deep-VRM injects its features into deeper layers — and both improve detection. (3) We take this one step further: with the backbone, data and training fixed, removing the encoder improves both detection and explanation quality (Table 1), and the gain carries over to the unseen DMImage (93.3 vs. 91.7 for FakeVLM). This yields, to our knowledge, the first encoder-free forensic VLM.
Weakness 3: How forensic signals are preserved
Thank you for this important request. The forensic cues in EFG are not confined to a small, identifiable subset of tokens; they are distributed across the visual tokens produced by the patch embedding. We support this with three observations.
(1) In the same-backbone controlled comparison (Table 1), fixing the backbone, data, LoRA configuration, schedule and resolution and replacing EFG's patch embedding with a frozen/tuned CLIP-ViT-L/14 or a frozen SigLIP-400M lowers accuracy to 89.6–93.4 and, more sharply, ROUGE-L/CSS to 43.7–49.8 / 64.2–70.6, against 59.1 / 79.0 for EFG: the encoder removes information on which the explanation depends.
(2) Tuning the encoder improves two encoder-based VLMs (Qwen2-VL-2B 89.8 → 90.7, LLaVA-1.6-7B 90.1 → 91.2; Figure 1), confirming that the semantic alignment of the encoder is the limiting factor.
(3) Under low-level degradations (Table 4 of the paper), EFG stays accurate where conventional detectors collapse, and its explanations describe concrete low-level artifacts (texture, saturation, smoothness) that annotators verify match the image (content description 4.52 vs. 3.87 / 4.31).
Together, these results indicate that low-level forensic signals remain accessible at the patch level and are used by the model, rather than being compressed into high-level semantics.