- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether rationale semantics affect AI-generated image detection beyond rationale structure. The authors construct the Custom Gaze benchmark and compare gaze-, anatomy-, and context-focused rationales with decision-only supervision. Experiments on Custom Gaze, COCOAI, held-out inpainting editors, and a second detector architecture show that gaze-focused supervision improves detection performance, particularly on Custom Gaze. However, it remains unclear whether these gains reflect semantic learning or sensitivity to synthetic eye artifacts.
Critical strengths
- Comparing different semantic content under a fixed rationale structure directly addresses whether rationale content matters.
- Tests across COCOAI, multiple generation methods, and a second architecture provide evidence beyond the custom benchmark.
Critical weaknesses
- Because all held-out inpainting examples still contain generated eyes, it is unclear whether the detector learns meaningful gaze-related cues or simply picks up on inpainting artifacts.
- Although fine-tuning changes the model's verbalized rationales, it is unclear whether these explanations reflect the actual basis for its predictions. The improvements on external datasets are also relatively modest.
Questions for authors
- Were the training data, optimization settings, checkpoint selection, and random seeds kept consistent across rationale conditions so that rationale content was the main difference?
- How do the authors distinguish learning meaningful gaze-related cues from picking up on artifacts introduced by eye inpainting?
- Do the reported confidence intervals account for both variation across training runs and uncertainty from test-set sampling?
- What evidence supports the interpretation that gaze-related output cards reflect the model's actual decision process rather than simply correlated verbalizations?
3 comments
Thanks for the response. Almost all of my concerns are addressed. Considering the novelty of this work, I would like to maintain my score.
Thank you; what is new, for the record
Thank you. We are glad that almost all of your concerns are addressed.
Since novelty was not among the points of the review, we state briefly what is new, for the record.
- A content-controlled design: images, labels, rationale structure, and training recipe are fixed and only the rationale content varies, now with three training seeds (your first strength).
- The result that rationale supervision moves three quantities differently: the decision improves (+0.55 to +0.70 BA in all three seeds), the emitted answer becomes less reliable, and the explanation is not the basis of the decision. We are not aware of a prior measurement of these three for explanation-generating detectors.
- Custom Gaze: 23,415 identity-preserving real/fake pairs that differ only inside the eye band.
If a specific prior work already covers these, we would be grateful for the reference.
Reply to Reviewer q6qv
We have measured both of your weaknesses (W1, W2), and the answers are now part of the paper's result: the detector has no gaze-direction shortcut, and its verbalized evidence is not the basis of its decision. Tables R1 and R2 are in the General Response.
If a specific point remains open for your assessment, please let us know and we will address it.