- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies Continual Image Forgery Localization (CIFL), where a pixel-level forgery localizer learns sequentially from different manipulation datasets while retaining performance on previously encountered tasks without access to their training data. The authors argue that this setting requires preserving non-semantic discrepancies between authentic and forged regions, which existing continual semantic segmentation methods may not adequately capture. They propose DUNK, a PIM-based framework with three components. 1)DPM stores cluster means and diagonal variances of spatial features and frequency descriptors. 2)DAPR is designed to preserve historical knowledge by aligning the old and current models’ Mahalanobis-based prototype-relation distributions, with spatial and frequency losses weighted according to relation entropy and representation count. 3)MPKD further constrains changes in manipulation-mask predictions, boundary predictions, and features within informative regions.
Critical strengths
-
A practically relevant continual-learning problem. This paper addresses the need to adapt image forensic models to emerging manipulation techniques while retaining localization performance on previously encountered ones. It focus on pixel-level localization without retaining historical raw images introduces meaningful constraints on both knowledge preservation and data storage, making CIFL a worthwhile setting for continual-learning research.
-
An evaluation design that considers both localization performance and forgetting. The experiments cover five datasets and three task orders, with metrics assessing localization accuracy and historical-task retention. Adapting multiple continual-learning baselines to a common PIM backbone also helps isolate differences between knowledge-preservation strategies. Together with the component ablations, this design provides a useful basis for examining the method’s contributions and sensitivity to task order.
Critical weaknesses
Weakness 1: The central dual-space mechanism is not justified, and the frequency branch appears to provide no learning signal.
The main contribution of the paper is the claim that historical forgery knowledge should be consolidated jointly in spatial and frequency spaces. However, the frequency descriptor in Eq. 3 is a fixed, parameter-free function of the input image and the ground-truth mask , and it never passes through the network. As a result, the old-model and new-model distances to the stored frequency prototypes are identical, the two relation distributions in Eq. 7 coincide, and the frequency replay loss is identically zero with zero gradient. The reliability weight is likewise constant with respect to the model parameters and only rescales the spatial replay term.
Despite this, Table 3 attributes a gain of more than 20 IoU points to the "FPP + DAPR" configuration over the base model. Unless a learnable frequency encoder has been left out of the description, this result cannot come from the method as stated. This undermines both the ablation study and the paper's core "dual-space" claim. In addition, the spectrum of a masked image is largely determined by the shape and edges of the binary mask, so it is unclear why this descriptor should capture forensic traces rather than region geometry.
Weakness 2: Internally inconsistent results and insufficient experimental analysis to support the claimed advantages.
Under the paper's own definition of Average Forgetting (Eq. 53), the best historical score on a dataset equals its final score in Table 1 plus its forgetting in Table 2, and it cannot exceed 100. For the Adapter baseline, this gives implied peak scores of 109.5 F1 on AutoSplice, and 104.1 IoU / 110.5 F1 on MMTD-Set, which are impossible. Moreover, several entries report F1 values above the upper bound implied by the corresponding IoU — for example CoGaMiD on IMD2020 (21.9 IoU vs. 42.1 F1, bound ) and Joint-Train on MMTD-Set (79.8 IoU vs. 91.8 F1, bound ). This suggests either an undisclosed difference in how the two metrics are computed or errors in the tables. All results come from single runs without variance, while the reported margins over the strongest continual learning baseline are very large (around 19 IoU points), so these inconsistencies matter.
Beyond this, the experiments do not isolate the contributions of the key design choices. For example, the paper lacks:
- an MPKD-only configuration, which is needed because MPKD is close to standard LwF-style output and feature distillation;
- a comparison between the stored prototypes and a trivial or randomized memory, to show that the gains come from the prototypes rather than from old/new-model consistency alone.
Questions for authors
-
The frequency prototype replay is presented as a core component of DUNK. Could the authors clarify whether there is any learnable component between the input image and the frequency descriptor in Eq. 3? If not, how can produce a non-zero gradient, and what explains the large gain of the "FPP + DAPR" row in Table 3? If there is one, please describe it fully: architecture, descriptor dimension, and how authentic-region descriptors are built.
-
Tables 1 and 2 appear inconsistent under the paper's own definition of Average Forgetting. Could the authors explain exactly how IoU and F1 are computed (per-image or dataset-level, fixed or optimal threshold), account for the Adapter entries whose implied peak scores exceed 100, and provide the full accuracy matrices for all methods?
-
Several datasets contain many authentic images with all-zero masks. Could the authors describe how these images are handled in training and in the IoU/F1 computation?
-
The proposed DUNK framework combines prototype memory, adaptive prototype replay, and multi-perspective knowledge distillation, which appear to build on established continual learning techniques. Could the authors clarify what the core theoretical or methodological innovation of DUNK is beyond integrating these components?
I would be willing to raise my score if the authors can adequately address my concerns.
6 comments
Official Comment by Authors (2/3)
[W2&Q2]: Evaluation Procedures And Additional Experiments
In Tab. 1, our reported image-level IoU and pixel-level F1 use different aggregation strategies. Specifically, IoU is calculated for each test image and then averaged across the dataset, ensuring that each image contributes equally despite variations in manipulated region sizes. In contrast, F1 is computed by aggregating pixel-level true positives (TP), false positives (FP), and false negatives (FN) across the entire dataset, following [1][2][3]. To ensure a consistent and transparent evaluation, we further report pixel-level IoU and F1. As below, DUNK achieves the best IoU/F1 of 59.7%/72.3%, leading the strongest baseline Adapter by 17.3%/15.2%, showing its effectiveness.
| Method | CASIAv2 | IMD2020 | AutoSplice | MMTD-Set | SID-Set | Average |
|---|---|---|---|---|---|---|
| Joint-Train | 59.6/74.7 | 62.5/76.9 | 86.2/92.6 | 84.8/91.8 | 82.8/90.6 | 75.2/85.3 |
| PIM + LwF | 19.0/31.9 | 22.0/36.1 | 37.7/54.8 | 39.2/56.3 | 79.5/88.6 | 39.5/53.5 |
| PIM + PLOP | 18.9/31.8 | 18.7/31.5 | 52.0/68.4 | 54.1/70.2 | 51.6/68.1 | 39.1/54.0 |
| PIM + DKP | 19.5/32.7 | 21.1/34.8 | 46.7/63.7 | 55.8/71.6 | 60.5/75.4 | 40.7/55.6 |
| PIM + Adapter | 20.6/34.2 | 20.0/33.4 | 56.4/72.1 | 60.4/75.3 | 54.4/70.5 | 42.4/57.1 |
| PIM + CoGaMiD | 20.6/34.1 | 26.7/42.1 | 28.1/43.9 | 44.6/61.7 | 67.5/80.6 | 37.5/52.5 |
| DUNK (Ours) | 36.1/53.1 | 30.6/46.9 | 78.3/87.8 | 75.7/86.2 | 77.9/87.6 | 59.7/72.3 |
[1] BusterNet: Detecting Copy-Move Image Forgery with Source/Target Localization [2] Multi-Supervised Encoder-Decoder for Image Forgery Localization [3] IMDL-BenCo: A Comprehensive Benchmark and Codebase for Image Manipulation Detection & Localization
Besides, we sincerely thank you for pointing out our oversight in Tab. 2 and we further provide the complete stage-wise accuracy matrices for DUNK and the CL baselines below. We have carefully re-examined the corresponding results in the revised manuscript.
| Method | Dataset | CASIAv2 | IMD2020 | AutoSplice | MMTD-Set | SID-Set |
|---|---|---|---|---|---|---|
| PIM | CASIAv2 | 0.563/0.715 | — | — | — | — |
| IMD2020 | 0.163/0.289 | 0.639/0.786 | — | — | — | |
| AutoSplice | 0.116/0.206 | 0.242/0.327 | 0.877/0.919 | — | — | |
| MMTD-Set | 0.068/0.151 | 0.314/0.342 | 0.288/0.414 | 0.854/0.919 | — | |
| SID-Set | 0.044/0.091 | 0.157/0.269 | 0.332/0.428 | 0.426/0.513 | 0.661/0.797 | |
| PIM+Adapter | CASIAv2 | 0.756/0.850 | — | — | — | — |
| IMD2020 | 0.374/0.553 | 0.564/0.715 | — | — | — | |
| AutoSplice | 0.319/0.409 | 0.214/0.302 | 0.844/0.912 | — | — | |
| MMTD-Set | 0.197/0.349 | 0.207/0.341 | 0.304/0.452 | 0.894/0.915 | — | |
| SID-Set | 0.184/0.342 | 0.189/0.334 | 0.564/0.721 | 0.604/0.753 | 0.545/0.705 | |
| PIM+CoGaMiD | CASIAv2 | 0.571/0.627 | — | — | — | — |
| IMD2020 | 0.446/0.566 | 0.468/0.694 | — | — | — | |
| AutoSplice | 0.396/0.467 | 0.351/0.527 | 0.433/0.536 | — | — | |
| MMTD-Set | 0.346/0.402 | 0.279/0.459 | 0.358/0.477 | 0.743/0.811 | — | |
| SID-Set | 0.217/0.341 | 0.219/0.421 | 0.308/0.439 | 0.492/0.617 | 0.687/0.806 | |
| DUNK (Ours) | CASIAv2 | 0.734/0.812 | — | — | — | — |
| IMD2020 | 0.466/0.679 | 0.541/0.718 | — | — | — | |
| AutoSplice | 0.449/0.647 | 0.429/0.589 | 0.878/0.960 | — | — | |
| MMTD-Set | 0.333/0.466 | 0.383/0.527 | 0.723/0.836 | 0.887/0.940 | — | |
| SID-Set | 0.387/0.531 | 0.356/0.469 | 0.792/0.878 | 0.764/0.862 | 0.751/0.876 |
We conduct an additional configuration, PIM+MPKD, which incorporates only the proposed multi-perspective knowledge distillation module without DPM and DAPR. We compare this configuration with PIM and PIM+LwF to distinguish the contribution of MPKD from conventional knowledge distillation. As below, PIM+MPKD outperforms PIM+LwF by 3.7%/4.9% in average IoU/F1, suggesting the benefits of jointly preserving mask, boundary, and feature representations.
| Method | Avg. IoU (%) | Avg. F1 (%) |
|---|---|---|
| PIM | 32.5 | 42.0 |
| PIM+LwF | 40.9 | 53.5 |
| PIM+MPKD | 44.6 | 58.4 |
| DUNK (Ours) | 61.0 | 72.3 |
To further investigate the effectiveness of historical prototype construction, we consider two additional memory strategies: Random Feature Memory (RFM), which randomly samples historical features as memory representations, and Simple Mean Memory (SMM), which constructs memory representations through straightforward feature averaging. Both configurations preserve the same memory budget and knowledge distillation mechanism as DUNK. As below, our DUNK outperforms RFM+DAPR+MPKD and SMM+DAPR+MPKD by 8.2%/8.5% and 6.6%/6.1% in average IoU/F1, respectively, showing the effectiveness of our DPM in preserving authentic–forgery discrepancy knowledge in both spatial and frequency domains.
| Configuration | Avg. IoU (%) | Avg. F1 (%) |
|---|---|---|
| RFM+DAPR+MPKD | 52.8 | 63.8 |
| SMM+DAPR+MPKD | 54.4 | 66.2 |
| DUNK (Ours) | 61.0 | 72.3 |
Official Comment by Authors (3/3)
[Q3]: Handling of Authentic Images
During training, authentic images are assigned all-zero ground-truth masks and are treated in the same manner as forged images, allowing the model to learn to distinguish authentic from manipulated regions.
[Q4]: Methodological Innovation
In this paper, we propose a Dual-Space Non-Semantic Knowledge Consolidation (DUNK) framework to address the Continual Image Forgery Localization (CIFL) task. Compared with existing continual learning (e.g., Continual Semantic Segmentation and Person Re-Identification) methods that primarily preserve semantic knowledge, the core methodological innovation of DUNK lies in preserving non-semantic authentic–forgery discrepancy knowledge to maintain historical forgery localization capabilities while adapting to new manipulation patterns.
Specifically, we introduce three complementary modules: (1) Dual-Space Authentic–Forgery Prototype Memory (DPM), which preserves historical authentic–forgery discrepancy knowledge through structured prototypes in both spatial and frequency domains, without storing historical images; (2) Dual-Space Adaptive Prototype Replay (DAPR), which adaptively replays historical prototype relations according to the reliability of spatial and frequency cues, thereby preserving complementary forgery knowledge across different tasks; and (3) Multi-Perspective Knowledge Distillation (MPKD), which maintains the consistency between old and new models from mask, boundary, and feature perspectives to preserve fine-grained forgery localization capabilities.
As shown in Tab. 1, our DUNK achieves 61.0%/72.3% in average IoU/F1, outperforming the strongest continual learning baseline, Adapter, by 19.3%/15.2%, respectively. As shown in Tab. 3, introducing SPP+FPP+DAPR improves the average IoU/F1 by 25.7%/28.6% over the baseline. Furthermore, incorporating MPKD brings an additional 2.8%/1.7% improvement in average IoU/F1, respectively. These results demonstrate the effectiveness of our proposed non-semantic knowledge consolidation framework for CIFL.
Official Comment by Authors (1/3)
[W1&Q1]: Frequency Replay Mechanism
Thank you for pointing this out. We further provide the details of the frequency replay mechanism in Eq. (6) of our latest manuscript(lines 270-278). Our frequency-based knowledge preservation branch provides learning signals through prediction-conditioned frequency prototype relation distillation. Specifically, Eq. (3) is used to extract frequency descriptors for constructing historical prototypes that preserve the distributional characteristics of authentic and forged regions. The resulting frequency descriptors are represented as 512-dimensional vectors. During frequency replay, we incorporate the current model's soft predictions into frequency extraction. Specifically, the descriptors are computed as:
vfgt=TF(g(xi)⊙yi⊙pt),
vautht=TF(g(xi)⊙(1−yi)⊙(1−pt)),
where TF denotes the fixed frequency descriptor extraction operator, yi denotes the ground-truth manipulation mask, and pt represents the current model's predicted forgery probability. As defined in Eqs. (7)–(9) in our latest manuscript, DAPR minimizes the KL divergence between the previous and current models' relation distributions over historical frequency prototypes. Consequently, the frequency replay loss can propagate gradients through the prediction-conditioned frequency descriptors to the prediction head φt(⋅), as follows:
∂θ∂LrepF=∂vt∂LrepF∂pt∂vt∂θ∂pt.
As shown in Tab. 3, Base+FPP+DAPR improves the average IoU/F1 from 32.5%/42.0% to 52.8%/64.6%, as preserving frequency-domain authentic–forgery discrepancy knowledge helps retain subtle manipulation traces, such as spectral perturbations and compression artifacts, that are difficult to capture in the spatial domain alone. Furthermore, by adaptively preserving authentic–forgery discrepancy knowledge across spatial and frequency domains, the combined SPP+FPP+DAPR configuration achieves 58.2%/70.6% in average IoU/F1, demonstrating the effectiveness of complementary dual-space knowledge in improving forgery localization performance.
We clarify the distinction between fixed frequency prototype construction and differentiable prediction-conditioned replay in the revised manuscript.