acceptodds
Withdrawn from ICLR 2027

Learning Transferable Anti-Forensic Priors via Adversarial Diffusion Training

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Image Manipulation Detection and Localization (IMDL) models have achieved remarkable progress in identifying and localizing manipulated regions, yet their robustness against anti-forensic attacks remains largely unexplored. Existing adversarial approaches typically rely on victim-specific information or image-by-image optimization, limiting their scalability and practical applicability. Inspired by recent advances that repurpose pretrained diffusion models as visual priors for diverse downstream tasks, we investigate whether such priors can be adversarially activated to acquire generalizable anti-forensic capability. To this end, we propose A-PAIR (Adversarial Prior Activation and Image Refinement), a framework that learns a transferable anti-forensic prior through adversarial adaptation of pretrained diffusion representations. Rather than optimizing perturbations independently for each image, A-PAIR trains a diffusion-based attack branch in competition with a manipulation localization branch. The two branches are optimized alternately: the localization branch learns to recover manipulation masks from generated adversarial images, while the attack branch learns to generate images that induce incorrect localization. Through this alternating competition, the diffusion representation acquires a transferable anti-forensic prior that can directly generate adversarial images across different inputs and localization models. The refinement stage improves image quality while preserving attack effectiveness, and further strengthens attacks when victim queries or gradients are available. Extensive experiments across multiple IMDL benchmarks and localization models demonstrate the advantages of A-PAIR over existing anti-forensic methods.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Image Manipulation Localization, Adversarial Attack

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_54ZQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a two-stage framework named A-PAIR. In Stage I, adversarial training is implemented based on a pre-trained diffusion model, where the attack branch and localization branch are trained alternately to generate coarse adversarial maps. In Stage II, a perturbation refinement module is adopted to improve image quality while preserving attack performance. The paper demonstrates that the proposed method achieves superior cross-model transferability and attack capability across diverse attack scenarios (black-box, gray-box, and white-box), and the resulting adversarial images exhibit lower visual distortion.

Critical strengths
  • This work demonstrates that pre-trained diffusion models can provide anti-forensic priors with generalization capability.

  • A two-stage A-PAIR framework is proposed. It generates adversarial samples via attack-localization interaction, and the refinement stage supports attacker settings of black-box, gray-box, and white-box.

  • Extensive evaluations on multiple datasets verify that A-PAIR outperforms several baseline methods in terms of attack performance, cross-model transferability, and image fidelity.

Critical weaknesses
  • The problem addressed in this paper is not rigorously formulated, and an overview of the threat model is missing. For instance, Section 3.1 states that the target scenario takes a manipulated image x as input to generate an adversarial image x*. However, the subsequent generation and refinement processes actually require the ground-truth mask, rather than enabling blind attacks relying solely on x. Consequently, the applicability of the proposed method is restricted to parties that construct the manipulated image x (if my understanding is correct). These aspects should be rigorously clarified at the outset.

  • Section 3.2 is poorly organized. It fails to specify the source of the diffusion backbone theta, whether it corresponds to a black-box or white-box setting, and how the LoRA parameters theta_b are obtained. Moreover, I do not observe the necessity of employing the diffusion sampling pipeline here. The authors should provide a reasonable justification for adopting the VAE encoder/decoder from diffusion models instead of alternatives such as GAN-trained modules. The current approach appears to retrofit diffusion sampling merely to follow research trends, with weak intrinsic connections to the core methodology.

  • The motivation presented in Section 3.3 lacks valid justification. Why are the adversarial samples produced in the aforementioned Stage 3.2 coarse? Why can Stage II achieve refinement? Preliminary experiments are required to substantiate these claims, rather than relying solely on textual description.

  • The assumption about m in Line 268 is unreasonable. How can m or m_1 be accessible during inference?

  • Finally, reviewing the entire methodology section (Section 3), the authors do not introduce original algorithms or modules; the work merely assembles existing techniques (e.g., diffusion models, adversarial attacks, and iterative update strategies). Therefore, the technical contributions are insufficiently demonstrated.

  • The experimental datasets are outdated. While the inclusion of classic datasets is reasonable, newer datasets (e.g., diffusion-based manipulation datasets) should also be incorporated.

  • The attack algorithms selected for comparison are general-purpose, instead of methods specifically designed against IMLD. Targeted attack baselines such as the following references should be added for comparison: [*1] L. Zhuo, S. Luo, S. Tan, H. Chen, B. Li, and J. Huang, “Evading detection actively: Toward anti-forensics against forgery localization", IEEE TDSC, vol. 22, no. 2, pp. 852–869, 2025. [*2] S. Fang and M. Stamm, “Attacking image splicing detection and localization algorithms using synthetic traces", IEEE TIFS, vol. 19, no. 1, pp. 2143–2156, 2024.

  • Given the numerous issues identified in this manuscript, I do not believe the current version can be adequately revised within a short rebuttal window. The authors are advised to carefully consider the comments above and conduct thorough revisions.

Questions for authors

Please refer to the weaknesses.

Reviewer_fyaKOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes A-PAIR, a diffusion-based framework for generating transferable adversarial images to disrupt image forgery localization. It uses alternating adversarial training between attack and localization branches to learn transferable anti-forensic priors, followed by an adaptive refinement stage to improve visual fidelity. Experiments across multiple forensic datasets and localization models report the effectiveness under black-box, gray-box, and white-box settings.

Critical strengths

(1) The paper is easy to follow.

(2) The experiments cover multiple datasets and image forgery localization models.

Critical weaknesses

(1) I have concerns about the claims made in Lines 40–45. The authors state that there has been “limited attention to their robustness against anti-forensic attacks” and that “existing anti-forensic research has primarily focused on evading image-level forgery detectors.” However, substantial prior research [1][2][3][4][5] has investigated adversarial attacks against pixel-level image forgery localization models. These claims therefore appear to overlook relevant existing literature and significantly weaken the motivation and claimed significance of this work.

(2) The title of this paper explicitly includes “Transferable,” and the ability of the proposed perturbations to disrupt localization across different forgery models is presented as a major contribution. Therefore, black-box attacks should be a primary focus of the evaluation. However, the black-box results reported in Table 3 are far from convincing. The authors do not compare their method against widely used, highly transferable ℓ∞\ell_{\infty}-bounded attack strategies, such as transformation-based attacks [6][7]. The baselines are therefore too weak to convincingly demonstrate the claimed transferability of the proposed method.

(3) There are concerns regarding the use of image forgery localization models in this work. Training protocols vary considerably across different forgery localization studies, and even for the same dataset, there is often no standardized training/test split. Datasets such as NIST16 are also commonly used for training. How do the authors ensure that their test samples were not included in the training sets of the evaluated models?

(4) Pretrained diffusion models typically learn semantic representations rather than forensic traces of image manipulation. Therefore, I find the motivation for using a pretrained diffusion model in image forgery detection and localization insufficiently justified.

[1] Query-Efficient Hard-Label Attacks Against Black-Box Image Forgery Localization Model via Reinforcement Learning, TIFS 2026. [2] Active adversarial noise suppression for image forgery localization, TPAMI 2026. [3] Adversarial attack on deep learning-based splice localization, CVPRW, 2020. [4] Self-adversarial training incorporating forgery attention for image forgery localization, TIFS 2022. [5] Attacking image splicing detection and localization algorithms using synthetic traces, TIFS 2023. [6] A Unified Spatial Alignment Framework for Highly Transferable Transformation-Based Attacks on Spatially Structured Tasks, arXiv 2026. [7] Segtrans: Transferable adversarial examples for segmentation models, TMM 2026.

Questions for authors

(1) How do the authors justify these claims given the substantial prior research on adversarial attacks against pixel-level image forgery localization models?

(2) Can the authors provide comparisons with stronger transferable attack baselines, such as transformation-based attacks, to better support their claims of transferability?

(3) How do the authors ensure that the test samples were not included in the training sets of the evaluated models? The authors could consider evaluating on GreatSplicing [1], which is less commonly used for training, or explicitly clarify how data leakage was prevented for each evaluated model.

(4) Given that pretrained diffusion models primarily learn semantic representations rather than forensic traces, what justifies their use in image forgery localization?

[1] GreatSplicing: A Semantically Rich Splicing Dataset, arXiv 2023.

Reviewer_X951OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents A-PAIR, a two-stage framework for generating transferable anti-forensic attacks against Image Manipulation Detection and Localization (IMDL) models. In the first stage, the framework adapts a pretrained diffusion model through alternating adversarial training of attack and localization branches to learn anti-forensic priors that generalize across images and target models. Building on these learned priors, the second stage introduces an adaptive perturbation refinement module to improve visual fidelity while preserving attack effectiveness. Extensive experiments across multiple benchmark datasets and diverse IMDL architectures demonstrate strong attack effectiveness and cross-model transferability, while ablation studies further validate the contributions of the proposed components.

Critical strengths
  1. The paper proposes a diffusion-based adversarial learning framework that combines alternating training of attack and localization branches with adaptive perturbation refinement, offering a promising approach to learning transferable anti-forensic priors while preserving visual fidelity.
Critical weaknesses
  1. The experimental comparisons lack consistent attack constraints. In the white-box setting, A-PAIR is not subject to the same perturbation budget as the baselines, while the gray-box comparisons use different query budgets and feedback conditions. These inconsistencies limit the fairness of the comparisons and weaken the evidence supporting the reported performance gains.

  2. The evaluation relies primarily on relative F1-score degradation, which does not distinguish between false negatives in manipulated regions and false positives in authentic regions. The absence of absolute localization metrics before and after attacks further limits the interpretation of the reported attack effectiveness.

Questions for authors
  1. In Section 3.2, the ground-truth manipulation mask is specified as part of the conditioning inputs during training, but is unavailable at inference. Could the authors clarify whether the mask is explicitly provided to the attack and localization branches as a conditioning signal or used solely for training supervision? If it serves as a conditioning signal, how is the resulting training–inference discrepancy addressed?

  2. The experimental comparisons involve different perturbation constraints in the white-box setting and varying query budgets and feedback conditions in the gray-box setting. Could the authors provide additional comparisons under matched perturbation budgets and standardized query and feedback conditions to better establish the fairness of the evaluation?

  3. The evaluation primarily relies on RelDrop, which does not distinguish between missed detections in manipulated regions and false positives in authentic regions. Could the authors report absolute F1 scores, precision, and recall before and after attacks to provide a more comprehensive assessment of attack effectiveness and clarify the sources of localization degradation?

Reviewer_zkRROpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses the under-explored problem of robustness against anti-forensic attacks in image manipulation detection and localization methods. It leverages the strong prior provided by pre-trained diffusion models. The method first learns a transferable anti-forensic prior through alternating attacking and localization using two distinct learnable low-rank adapters. Since this may result in severe visual perturbation, the method subsequently refines the resulting image to preserve visual similarity to the original. The method supports white/gray/black-box settings and does not require ground-truth masks during inference.

Critical strengths

The experimental results show that the method performs especially well in black-box settings.

Critical weaknesses

The paper does not consider an “adaptive” localization method. This is analogy the adaptive attacks in images classification robustness area. That is, one (or ideally each) of the nine victim models trained or fine-tuned against the proposed adversarial method.

Questions for authors
  • Did you observe mode collapse during the alternating training of the attack/localization branches?

  • Would you please provide experiments in which the localization method is trained against the proposed ani-forensic attack?

I would be more inclined to increase the score if such experiments are provided.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.