acceptodds
Under review as a conference paper at ICLR 2027

Do Forensic Verdicts Follow Output Regions?

Abstract

Explainable forensic VLMs pair manipulation verdicts with localized evidence, but localization overlap alone cannot establish whether the verdict actually depends on the cited region. Existing deletion and inpainting tests are also problematic because the intervention can itself introduce forensic cues. We introduce paste-back auditing: when a paired manipulation retains its authentic source, we restore authentic pixels only inside the model's output region and measure the resulting verdict change (necessity) alongside localization. Paste-back introduces no synthesized content and is exactly the identity on authentic images, whereas two classical inpainters flip 31.3% of authentic examples for served chat VLMs. In a positive control, a faithful oracle and a correct-but-non-causal oracle have identical IoU, yet necessity is 0.99 versus 0.00, showing that localization and intervention response capture distinct properties. Controlled Qwen2-VL-2B experiments reproduce this dissociation under a JPEG-75 cue. We further audit FakeShield and FFAA as released-system case studies. The audits reveal that output regions can be only partially aligned with the measured verdict response, while narration- grounded interventions can produce stronger effects when the narration explicitly identifies the manipulated region. Paste-back therefore provides a synthesis-free test of verdict–output-region alignment wherever the authentic source is retained, while keeping localization correctness and intervention dependence as separate quantities.

Primary area:
interpretability and explainable AI
Keywords:
forensic VLMs, explainable AI, explanation faithfulness, image forensics, counterfactual evaluation, causal auditing, vision-language models, deepfake detection, image manipulation detection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_GV3TOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces paste-back auditing, which restores authentic source pixels inside a forensic VLM’s reported region and measures the resulting verdict-score change. Oracle controls, released-system case studies, and controlled VLM training experiments show that localization accuracy and intervention response capture different properties. The contribution is a synthesis-free auditing procedure with detailed validation and sensitivity analyses.

Critical strengths

1.Authentic-source restoration addresses an important forensic evaluation confound: synthesized replacement pixels can themselves trigger manipulation detection. This makes the procedure useful when paired authentic sources are available.

2.The positive controls convincingly distinguish localization from verdict response. Extensive analyses of readouts, region definitions, sufficiency, construction effects, and source overlap strengthen the credibility of the narrowly stated conclusions.

Critical weaknesses

1.The released-system findings depend on modified inference configurations. FakeShield’s reported effects use an elicited score, while its native prose changes to “authentic” on 0/50 fully restored pairs. FFAA uses a different system prompt and omits native answer selection. Consequently, alignment with the released pipelines’ actual verdicts remains insufficiently validated.

2.The learned-model results provide limited generalization evidence. Source overlap was addressed after training, leaving only 18/10/15 examples in the strict subsets; one self-citation interval reaches zero. These limitations constrain how broadly the observed dissociation can be interpreted.

Questions for authors

1.Validation: Do the principal intervention effects persist under the exact released inference pipelines, and how well do the graded scores agree with generated verdicts after restoration?

2.Validation: Can the learned self-citation contrast be reproduced with host/donor groups separated before training?

3.Clarification: Please distinguish the FFAA cohorts in §3.3.2. Appendix E′ reports AUC 0.997 and restoration effect 0.709 for eight-part swaps, versus 0.923 and 0.641 for the single-part experiment.

Reviewer_LVZ6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes paste-back auditing for explainable forensic vision-language models. When a paired manipulation retains its authentic source image, the authors restore authentic pixels only inside the model's output region RR and measure the resulting change in the verdict (necessity), reported alongside localization (IoU).

Critical strengths
  1. Paste-back is a useful specialization of established intervention-based explanation testing.
  2. The reviewer appreciates the effort and manual labor put into this 97-page manuscript. Evaluations and ablation studies provide strong controls and reproducibility.
Critical weaknesses

The principal concern is limited methodological novelty. Paste-back uses authentic pixels instead of an AI-generated edit. As Section 5 correctly disclaims novelty of answer change, the question is whether the remaining intervention-level contribution is sufficiently substantial. From the reviewer's opinions, necessity mostly measures how much of the edit a region covers.

Too much negative sentences in the manuscript makes the text hard to comprehend and follow.

Questions for authors

Q1. What does the fragmented control do when no disjoint equal-area placement exists, and how many rows does this affect? Q2. Every global-cue positive sample in the trainset carries a canvas-wide cue. Is it possible that the trained dissociation is a learned / built-in shortcut?

Reviewer_FD84OpenReview ↗
Rating
1(−1)
Confidence
4(+1)
Informativeness
2
Presentation
1(−1)
Soundness
2
Read review
Paper summary

This paper proposes Paste-back Auditing, a synthesis-free intervention method for evaluating whether forensic vision-language models genuinely rely on the image regions they identify as manipulated. By restoring the reported regions with authentic source pixels and measuring changes in manipulation scores, the method assesses verdict–region alignment beyond conventional localization metrics such as IoU. Experiments on controlled models and existing forensic VLMs, including FakeShield and FFAA, reveal that accurate localization does not necessarily imply strong evidence dependence. The paper highlights the importance of evaluating localization accuracy and intervention-based necessity separately when assessing explanation faithfulness.

Critical strengths
  1. The paper highlights an important distinction between localization accuracy and actual evidence dependence.

  2. Paste-back auditing provides a straightforward way to assess verdict–region alignment.

  3. The positive-control experiments help validate the effectiveness of the proposed metric.

  4. The experiments cover both released forensic VLMs and controlled trained models.

Critical weaknesses
  1. No index for all equations and formulas. Not rigorous.

  2. No explaintion of formulas. For example, the one at line 108.

  3. The performance of the grounding significantly impacts the Paste-back. On the one hand, the grounding possibly cannot precisely identify the region mentioned by the text evidence. For example, the evidence "the eyes and nose have been manipulated" mentions one region; however, the grounding model identifies another region. On the other hand, the region where the F-VLM connects to the evidence possibly does not match the region identified by the grounding model. Therefore, even if the grounding model is precise, the Paste-back cannot cover the correct evidence region.

  4. The Paste-back for texture-based evidence remains unclear. Texture cues are often spatially distributed and context-dependent, while local restoration may introduce new statistical inconsistencies. Thus, necessity scores may reflect intervention artifacts rather than genuine evidence dependence. Further evaluation across texture types, mask scales, and boundary treatments is needed.

  5. The proposed necessity metric may underestimate evidence dependence when multiple manipulated regions provide redundant or complementary cues. Restoring a single region may not significantly change the verdict if other evidence remains. Evaluating joint paste-back interventions across multiple regions would help reveal evidence interactions and provide a more comprehensive assessment of explanation faithfulness.

Questions for authors
  1. What is the meaning of the notation Ω\Omega in line 107?

  2. Paste-back seemingly relies on no synthesized contents being introduced. Does this mean that the proposed method is not suitable for diffusion-based forensic tasks?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.