acceptodds
Withdrawn from ICLR 2027

ForensicWCT: Whitening and Coloring Transform in Forensic Latent Space for Robust Localization of AI-Edited Image Region

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The spread of generative image editing tools poses a growing threat to digital safety. Developing forensic models with strong generalization is therefore critical, yet remains challenging due to two key issues: (1) semantic bias, where models rely on spurious correlations between image content and generator artifacts, and (2) the limited diversity of localization data, which restricts generalization to unseen forgery techniques. To address these challenges, we propose to reformulate forgery localization as a forensic style classification problem. We conceptualize generator-specific artifacts as a distinct ”style”, enabling the disentanglement of forensic traces from semantic content. Building on this formulation, we present ForensicWCT, a novel training framework that instantiates this concept. During training, ForensicWCT applies a whitening transform to decompose image features into semantic content and forensic style components. It then performs a style-swapping augmentation by exchanging styles across samples, while dynamically updating the corresponding ground-truth labels. This training strategy, which does not introduce additional computational cost at inference time, forces the model to learn content-agnostic artifact representations. Moreover, by elegantly injecting styles from fully-generated images into arbitrary regions, our approach enables the effective use of large-scale, diverse, fully-generated image datasets to improve localization generalization. Extensive experiments on multiple benchmarks demonstrate the effectiveness of our ForensicWCT.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
ML: Representation Learning, CV: Low-Level & Physics-based Vision

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_X9UXOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses semantic bias and insufficient diversity of localization data in AI-generated image forgery localization. It reformulates the forgery localization task as a forensic style classification task and proposes the ForensicWCT training framework to disentangle image semantics from forgery artifacts. Leveraging dynamic style-swapping augmentation, the framework makes full use of large-scale synthetic images and adopts a contrastive-learning-like paradigm to align diverse forensic styles. Experiments on several benchmarks demonstrate strong generalization performance of the proposed method.

Critical strengths
  1. The work reformulates forgery localization as forensic style classification, treating generative artifacts as an independent forensic style.
  2. The ForensicWCT training framework is designed to disentangle and swap content and style, enabling the learning of content-agnostic forgery features.
  3. Evaluations are conducted across multiple datasets. Results show that the framework is compatible with various existing forensic models. The robustness and limitations of the method are also analyzed.
Critical weaknesses

The motivation is inadequately justified. Simply rephrasing forensic artifacts using the term "style" does not establish the novelty of this concept. Prior studies have demonstrated that the "fake" class is not a semantically compact category; instead, it consists of diverse unnatural artifacts. For instance, each GAN model yields a distinct set of artifacts, and the aggregation of these artifacts constitutes the "fake" class when contrasted with authentic traces such as camera fingerprints. Multiple existing works have attempted to extend the seemingly binary real/fake classification to multi-class tasks under weak supervision. Accordingly, the authors fail to clarify why rebranding artifacts as "style" transforms the task into an entirely new classification problem.

The method is described in an overly complicated manner, which undermines the credibility of its effectiveness. Notably, the pipeline illustrated in Figure 3 incorporates seven loss functions; the mechanism for balancing these losses remains unclear. Furthermore, although content–style disentanglement has been extensively investigated (as mentioned in Section 2.2), the paper does not explain why this specific disentanglement yields forensic styles rather than other types of styles. I argue that high-quality paired images are required to supervise such disentanglement, yet this prerequisite is omitted by the authors (or overlooked by myself). A closely related work is DegAE (DegAE: A New Pretraining Paradigm for Low-Level Vision, CVPR 2023), which performs analogous disentanglement to separate image content from low-level artifacts. Its narrative offers a more rigorous justification, whereas the methodology for forensic styles in this manuscript lacks comparable persuasiveness.

In addition, the experimental section fails to visualize the disentangled forensic styles (e.g., whether they manifest as distinct style representations analogous to the artistic styles of Van Gogh and other painters; visualizations are recommended). Samples generated after swapping real and fake styles are not presented either. Only pixel-level localization results are provided, consistent with conventional baselines. Therefore, I consider the experimental design insufficient to fully validate the methodological contributions of this work.

Given that the aforementioned critical weaknesses cannot be fully addressed within a single rebuttal, I cannot recommend a positive score. I suggest that the authors substantially revise the current manuscript.

Questions for authors

What is the distinction between forensic style and generated style described in Line 52?

The green lines in Figure 3 are illegible; an alternative visualization is recommended.

Reviewer_UERsOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The manuscript proposes ForensicWCT, an image forgery localization framework designed to improve generalizability to unseen forgery techniques. The authors reformulate forgery artifacts as forensic styles, aiming to disentangle forgery-specific artifacts from semantic content. ForensicWCT employs a whitening transformation to separate semantic content and forensic style representations. The authors proposed a style-swapping augmentation strategy with dynamic label updates. Furthermore, the proposed framework enables the integration of fully generated images into localization training to enhance data diversity. Experimental results on multiple benchmarks demonstrate the effectiveness and generalizability of the proposed method.

Critical strengths

The manuscript is well structured and easy to follow. The authors provide extensive discussions and justifications for the model design and loss functions. Furthermore, the experimental results demonstrate that the proposed method achieves promising generalizability for AI-edited image forgery localization.

Critical weaknesses

The authors claim that there are two key challenges in generalizable AI-edited image forgery localization: semantic bias and the limited diversity of localization data. However, these challenges are not sufficiently justified to support the motivation behind the proposed model design. Furthermore, the authors conceptualize generator-specific artifacts as a distinct "style" and aim to disentangle forensic style from image content. However, the methodological novelty of this work is limited, as feature-style disentanglement and style-based feature augmentation have been explored in previous studies on image forensics and domain generalization, such as [A] and [B].

The authors should further clarify the technical differences and novelty of the proposed method compared with these existing approaches. Although the manuscript reports extensive pixel-level localization experiments, additional comparisons with SOTA methods on image-level forgery detection are expected to provide a more comprehensive evaluation of the proposed method. Moreover, the training datasets used by the proposed method and the baseline methods in Table 1 are not aligned, potentially compromising the fairness of the comparisons. Furthermore, additional traditional forgery localization datasets, such as NIST, CASIA, and Columbia, should be included in the evaluation to further assess the generalizability of the proposed method across different forgery types. Overall, the manuscript lacks sufficient methodological novelty and rigorous, fairly aligned experimental comparisons. Therefore, I do not believe that the manuscript meets the acceptance standards of ICLR.

Questions for authors

Please see the critical weaknesses.

Reviewer_CFWPOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ForensicWCT, a training-only framework for localizing AI-edited image regions. It treats authentic and generated traces as two forensic styles, predicts a latent transformation from content and donor second-order feature statistics, transfers the donor style to selected regions, and updates the pixel labels accordingly. The central goals are to reduce correlations between semantic content and forgery labels and to use large image-level generated-image collections as supervision for regional localization. The method is optimized with segmentation, style-matching, style-classification, and feature-clustering losses and is removed at inference. Experiments on six benchmarks report improvements across several backbones, including an increase from 38.2 to 50.4 average IoU for the ConvNeXt baseline on OpenSDID, together with cross-dataset, robustness, and ablation results.

Critical strengths
  1. The regional latent-style exchange with dynamic label updates is a useful mechanism for converting image-level generated data into localization supervision. The same semantic region can receive authentic or generated supervision after style exchange, which directly targets the content–label shortcut described by the paper. This strength is consequential because Table 4 shows that direct joint training with the additional images reaches only 38.8 IoU, whereas swapping operation I reaches 48.7 IoU; the result provides evidence that the proposed use of the extra data matters beyond simply adding full-image labels.
  2. The method produces broad empirical gains without increasing inference cost. Table 4 reports improvements for multiple architectures, including TruFor (36.9 to 45.1 IoU), IML-ViT (32.5 to 43.7), SegFormer (39.0 to 48.6), and ConvNeXt (38.2 to 50.4). The training-only design is consequential to my assessment because it suggests that the idea may be practically reusable across localization models while preserving their deployment cost.
Critical weaknesses
  1. The paper’s whitening-and-coloring interpretation is not established by either the derivation or the implementation. With features defined as having shape HW by C, a channel covariance uses F-transpose times F and the implemented channel transformation right-multiplies the features. Equations (1)–(3) instead use F times F-transpose and left multiplication, which is inconsistent with the stated dimensions and Section 2.3. More importantly, the learned product T = VcVs is not constrained to whiten the content covariance or color it to the donor covariance. The current evidence therefore supports a learned transformation conditioned on second-order statistics, but not a guaranteed WCT. The feature-clustering loss also subtracts the inter-class squared distance without a margin or stated feature normalization, making the displayed objective potentially unbounded; it is undefined as written when an updated mask contains only one class and the inter-class pair set is empty. These issues are consequential because they affect the paper’s title, theoretical contribution, and reproducibility, rather than only its exposition.
  2. The headline comparisons do not cleanly separate the contribution of ForensicWCT from the much larger external-data budget, and the evaluation protocol contains an unresolved metric inconsistency. The proposed model uses CommunityForensics (approximately 2.7 million images from nearly 5,000 generators) plus real images from COCO, LAION, and SA-1B, while Tables 1–3 largely quote baseline results from papers that did not necessarily use the same images, generator coverage, training schedule, or initialization. The submission does not report the actual external-data quota for each benchmark, family-level generator exclusion, sampling balance, or near-duplicate checks. In addition, several SID-Set entries in Table 2, such as 35.7 IoU and 71.3 F1 for PSCC, require an explicit explanation of thresholds, aggregation, sample subsets, and provenance. This weakness is consequential because the claimed state-of-the-art generalization is the main empirical basis for acceptance, yet the present comparison cannot attribute the gains reliably to the proposed method.
Questions for authors
  1. Clarification required for assessment: Please provide a dimensionally consistent version of Equations (1)–(3) and state the exact normalized covariance or Gram definition used in the implementation. What constraint or measured result shows that the learned transformation performs whitening and coloring rather than a general transformation conditioned on second-order statistics?
  2. Clarification required for assessment: What is the exact implemented form of the feature-clustering loss? In particular, how is it evaluated when an updated mask is all authentic or all generated and the inter-class pair set is empty, and what prevents the feature scale from growing without bound?
  3. Evidence required for assessment: For each benchmark, how many distinct external real and generated images and how many generator families are actually sampled? Please specify the generator-family/checkpoint exclusion rule, sampling ratios, and duplicate checks. Can the authors provide a data-budget-matched comparison between direct joint training and ForensicWCT under the same backbone, initialization, external samples, augmentations, steps, and at least three random seeds, together with one strong baseline trained under the same external-data budget? This evidence would determine whether the reported gain can be attributed to ForensicWCT.
  4. Clarification required for assessment: Please define the pixel-IoU and pixel-F1 thresholding, micro/macro or per-image aggregation, empty-mask handling, and model-selection protocol, and verify the SID-Set entries in Table 2 from the original predictions. An explanation of the unusually high F1 values relative to IoU is necessary for judging the validity of the benchmark comparison.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.