- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DINOv3‑IML for the IML task. It performs lightweight adaptation on the DINOv3 backbone via QKV‑LoRA without deploying a dedicated forensic feature extractor. The proposed concept of forensic readability gap is defined as follows: frozen DINOv3 features already contain tampering cues, yet these cues cannot be retrieved by cross-image shared linear rules. After LoRA adaptation of the encoder, the feature directions of tampered regions and background become aligned across different images, which substantially improves cross-image linear readability, enabling a shallow head to accomplish localization. Evaluated under two benchmark protocols of CAT-Net and MVSS, the method achieves pixel F1 scores of 0.85 and 0.77 respectively, outperforming existing approaches by a considerable margin.
Critical strengths
- The detector for IML can be efficiently fine-tuned via lightweight QKV-LoRA adaptation of the DINOv3 encoder alone.
- The concept of forensic readability gap is put forward, revealing that even equipped with a powerful decoder, a frozen backbone cannot bridge this gap; instead, internal patch interactions within the encoder require modification.
- It surpasses conventional dedicated tampering localization models by a large margin under the CAT-Net and MVSS evaluation protocols.
Critical weaknesses
-
The adopted QKV-LoRA is not custom-designed for forensic tasks. The proposed framework is theoretically transferable to other dense prediction tasks, with no task-specific operators, artifact extraction modules tailored for image forensics, nor incorporation of forensic priors such as noise patterns and compression artifacts. Despite the prominent performance gains observed in experiments, such improvements stem from representation reshaping of the pre-trained backbone driven by task-specific data, rather than forensics-oriented architectural innovation.
-
The base model is inherently trained for semantic tasks. However, the paper fails to elaborate on how it is transferred to learn forensic features, and what exactly these forensic features correspond to (e.g., boundary noise, JPEG compression artifacts, illumination inconsistency, or other forensic cues), leading to limited interpretability.
-
Although the datasets adopted in experiments are standard benchmarks, they are relatively outdated and do not cover the latest forgery types, such as diffusion-model-based local generation and full-image reconstruction.
Given these notable limitations, I assign a negative score at this stage and look forward to the authors' rebuttal.
Questions for authors
Please refer to the weaknesses.
3 comments
Response (2/2): W2 – forensic cues; W3 – newer forgery types
W2 (which forensic features). "The base model is inherently trained for semantic tasks. However, the paper fails to elaborate on how it is transferred to learn forensic features, and what exactly these forensic features correspond to (e.g., boundary noise, JPEG compression artifacts, illumination inconsistency, or other forensic cues)."
The adapted model uses the noise, compression and illumination cues you mention, and adaptation teaches it to read them as differences between a region and its surroundings. We tested this on 248 authentic photos from COVERAGE and CASIAv1. Each photo receives a region shaped like a real forgery mask, and one trace is applied inside the region, outside it, or to the whole image. In every condition, a 32-pixel gray band is painted over the region's outline, so the trace leaves no visible edge. We report the share of the region's tokens that the model flags as edited.
1. Noise, JPEG compression, blur and resampling are read as differences between a region and its surroundings. For the CAT-Net-protocol model, which flags 0.093 without a trace:
| Trace | Inside only | Outside only | Everywhere |
|---|---|---|---|
| Noise σ = 10 | 0.169 | 0.621 | 0.114 |
| JPEG q50 | 0.211 | 0.186 | 0.114 |
| Blur σ = 2 | 0.502 | 0.179 | 0.110 |
| Resampling to 50% | 0.327 | 0.145 | 0.094 |
A trace on one side raises the flagged share, whether it is applied inside the region or only outside it. In the second case the region itself is untouched; it is flagged because it now differs from its surroundings. The same trace over the whole image barely changes the share.
2. Illumination changes are read mainly at the region's outline. In a separate test on 237 photos, changing the region's brightness with a gamma of 1.4 raises the AP of the model's output for that region from 0.32 to 0.47 when the outline is covered, and from 0.11 to 0.80 when the outline is visible.
3. Reading traces as differences is learned during adaptation. Frozen features respond to the trace itself: with blur, their flagged share rises when the whole image is blurred, but not when only the surroundings are. Both adapted models show the opposite, including the one trained without any augmentation:
| Encoder | Training data | No trace | Inside only | Outside only | Everywhere |
|---|---|---|---|---|---|
| Frozen ViT-L | CASIAv2 | 0.075 | 0.303 | 0.080 | 0.136 |
| ViT-L + LoRA r32 | CASIAv2, no augmentation | 0.022 | 0.390 | 0.183 | 0.013 |
| ViT-L + LoRA r32 | CAT-Net protocol | 0.093 | 0.502 | 0.179 | 0.110 |
Sec. 5 shows how adaptation changes the features: a single linear rule shared across images reads edited tokens with AP 0.501 on frozen features and 0.919 after adaptation, and the edited-versus-authentic direction becomes shared across images, with alignment rising from 0.21 to 0.78. We will add these tests to App. H.
W3 (newer forgery types). "Although the datasets adopted in experiments are standard benchmarks, they are relatively outdated and do not cover the latest forgery types, such as diffusion-model-based local generation and full-image reconstruction."
Evaluated zero-shot, our CAT-Net-protocol model reaches pixel F1 0.684 on CocoGlide and 0.508 on AutoSplice, the highest mean over the two datasets among the compared methods. CocoGlide inpaints one region of each COCO image with the GLIDE diffusion model, and AutoSplice edits local regions with DALL-E 2; we use the Q100 release of AutoSplice. The other rows are from RITA (Zhu et al., 2025), which trains each method under the IMDL-BenCo CAT-Net protocol and tests at 512×512, and from PAL (Zhu et al., NeurIPS 2026).
| Method | Training data | Input | CocoGlide | AutoSplice | Mean |
|---|---|---|---|---|---|
| MVSS-Net | CAT-Net protocol | 512 | 0.443 | 0.385 | 0.414 |
| CAT-Net | CAT-Net protocol | 512 | 0.410 | 0.387 | 0.399 |
| PSCC-Net | CAT-Net protocol | 512 | 0.474 | 0.551 | 0.513 |
| TruFor | CAT-Net protocol | 512 | 0.283 | 0.393 | 0.338 |
| Mesorch | CAT-Net protocol | 512 | 0.450 | 0.402 | 0.426 |
| RITA | CAT-Net protocol | 512 | 0.493 | 0.644 | 0.569 |
| ConvNeXt-Small + PAL | CAT-Net protocol + EditGroup-45K | 256 | 0.538 | 0.483 | 0.510 |
| Ours, ViT-L + LoRA r32 | CAT-Net protocol | 512 | 0.684 | 0.508 | 0.596 |
On CocoGlide, our model is 14.6 points above the next best method. On AutoSplice, it ranks third, behind RITA and PSCC-Net. We will add this table to the appendix.
We hope these answers address your concerns, and we are happy to clarify anything further.
Response (update): W1 – Bayar, SRM and Noiseprint++ results
Here are the results with Bayar, SRM and Noiseprint++ that we promised in our response to W1.
The forensic priors add little beyond the extra parameters of the branch, and switching the forensic branch off barely changes the result.
Setup. We added each extractor to the model of the paper and retrained it with everything else unchanged:
| Model | DINOv3 ViT-L/16 + LoRA r64 + Conv3 head, MVSS protocol, 512×512 input |
| Extractors | Bayar: learnable constrained 5×5 convolution. SRM: three fixed filters. Noiseprint++: TruFor's pre-trained extractor, frozen |
| How they are added | Each extractor feeds its own ResNet-18 (up to stride 16), whose output is concatenated with the DINOv3 features before the head. The new head weights start at zero |
| Control | The same branch fed with the RGB image, to separate the forensic prior from the extra parameters |
| Kept the same | CASIAv2 training data (5,123 images), augmentation, batch size 80 with 2 accumulation steps, learning rate 3e-4, 100 epochs, seed 43 |
Pixel F1 at the best epoch on the test sets:
| Extra branch | Trainable params | CASIAv1 | Columbia | NIST16 | COVERAGE | IMD2020 | Mean |
|---|---|---|---|---|---|---|---|
| None (seed 43) | 12.19M | 0.8976 | 0.8946 | 0.6229 | 0.8699 | 0.6533 | 0.7876 |
| None (seed 44) | 12.19M | 0.8848 | 0.9111 | 0.6415 | 0.8657 | 0.6489 | 0.7904 |
| RGB (control) | 16.15M | 0.8910 | 0.8901 | 0.6399 | 0.8848 | 0.6578 | 0.7927 |
| SRM | 16.17M | 0.9002 | 0.8827 | 0.6434 | 0.8809 | 0.6231 | 0.7861 |
| Noiseprint++ | 16.15M | 0.8901 | 0.9121 | 0.6308 | 0.8859 | 0.6435 | 0.7925 |
| Bayar | 16.15M | 0.8979 | 0.9008 | 0.6494 | 0.8858 | 0.6616 | 0.7991 |
1. The extra parameters alone give as much gain as SRM or Noiseprint++. The branch adds about 4M trainable parameters. Fed with the plain RGB image, which carries no forensic prior, it raises the mean F1 from 0.7876 to 0.7927. SRM (0.7861) and Noiseprint++ (0.7925) do no better; only Bayar (0.7991) is higher, by 0.6 points.
2. Switching off the forensic branch barely changes the result. We set the branch output to zero and evaluated each model again at its best epoch. The forensic models lose less than one point, and Bayar only goes from 0.7991 to 0.7987. Switching off the RGB branch, in contrast, costs about six points:
| Extra branch | Branch on | Branch switched off |
|---|---|---|
| RGB (control) | 0.7927 | 0.7316 |
| SRM | 0.7861 | 0.7776 |
| Noiseprint++ | 0.7925 | 0.7899 |
| Bayar | 0.7991 | 0.7987 |
Response (1/2): W1 – forensic priors
Thank you for the review and for keeping an open mind for our rebuttal. Following your comments, we have added experiments on forensic priors, the cues the model learns and newer forgery types. We answer the three weaknesses in order.
Our contribution is not a forensic module but evidence on whether localization needs one: an adapted general-purpose encoder, with no forensic prior built in, outperforms detectors designed around such priors. Three results support this.
1. Our model outperforms detectors built around the priors you mention. Noise patterns are modelled by Noiseprint++ in TruFor and by the noise branch of MVSS-Net, and compression artifacts by the DCT stream of CAT-Net. Our model has none of these modules and outperforms all three under the CAT-Net protocol, also at a comparable parameter count:
It also leads the strongest baseline of each protocol, by 12.2 points under the CAT-Net protocol and 24.0 points under the MVSS protocol (Tables 1 and 2).
2. Adapting the encoder lets it learn these cues from task data, so no module has to supply them. As you note, the gain comes from adapting the pre-trained encoder: with a similar number of trainable parameters, it is far more effective than a larger head on frozen features (Table 3, all models trained on CASIAv2):
What the encoder learns includes the cues that forensic modules are built to extract. As our answer to W2 shows, the adapted encoder responds to noise and compression differences between a region and its surroundings, much more strongly than frozen features.
3. In our test so far, adding a DCT branch to our model did not improve it. We added a DCT branch to the adapted encoder: fixed 8×8 DCT filters followed by a small CNN, fused with the encoder features by cross-attention. Both models below were trained on CASIAv2:
Experiments with three other common forensic extractors, Bayar convolution, SRM filters, and TruFor's Noiseprint++, are still running, and we will post the results here within two days.