- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces paste-back auditing, which restores authentic source pixels inside a forensic VLM’s reported region and measures the resulting verdict-score change. Oracle controls, released-system case studies, and controlled VLM training experiments show that localization accuracy and intervention response capture different properties. The contribution is a synthesis-free auditing procedure with detailed validation and sensitivity analyses.
Critical strengths
1.Authentic-source restoration addresses an important forensic evaluation confound: synthesized replacement pixels can themselves trigger manipulation detection. This makes the procedure useful when paired authentic sources are available.
2.The positive controls convincingly distinguish localization from verdict response. Extensive analyses of readouts, region definitions, sufficiency, construction effects, and source overlap strengthen the credibility of the narrowly stated conclusions.
Critical weaknesses
1.The released-system findings depend on modified inference configurations. FakeShield’s reported effects use an elicited score, while its native prose changes to “authentic” on 0/50 fully restored pairs. FFAA uses a different system prompt and omits native answer selection. Consequently, alignment with the released pipelines’ actual verdicts remains insufficiently validated.
2.The learned-model results provide limited generalization evidence. Source overlap was addressed after training, leaving only 18/10/15 examples in the strict subsets; one self-citation interval reaches zero. These limitations constrain how broadly the observed dissociation can be interpreted.
Questions for authors
1.Validation: Do the principal intervention effects persist under the exact released inference pipelines, and how well do the graded scores agree with generated verdicts after restoration?
2.Validation: Can the learned self-citation contrast be reproduced with host/donor groups separated before training?
3.Clarification: Please distinguish the FFAA cohorts in §3.3.2. Appendix E′ reports AUC 0.997 and restoration effect 0.709 for eight-part swaps, versus 0.923 and 0.641 for the single-part experiment.