- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2(+1)
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This work constructs the MEDTD dataset for document tampering detection, which incorporates not only textual content but also visual elements commonly embedded in documents, including images, formulas, and tables. Furthermore, the ClueDet model is proposed based on DINO. It performs feature aggregation using visual similarity across different regions within a page to optimize detection queries. Experiments verify the effectiveness of this context-aware contrastive paradigm.
Critical strengths
- This work builds the multi-element document tampering dataset MEDTD, covering diverse content such as texts, figures, tables, and formulas, with labels annotated for four categories of tampering operations.
- The ClueDet detector is proposed to aggregate contextual features via intra-page regional similarity, validating the benefits of page-level contextual information for document tampering recognition.
Critical weaknesses
-
The motivation for constructing the dataset is insufficiently highlighted. Although this dataset additionally introduces editing and tampering of graphics and tables, it bears no fundamental distinction from existing document manipulation/forgery datasets. In particular, as shown in Table 1, the proposed dataset exhibits no distinctive advantages in terms of the counts of original/tampered images, document categories, or manipulation types (CE, CD, CR, CDS). Moreover, the workload invested is not substantial. It is suggested to explore broader forms of adversarial editing, such as reprinting and regional AI generation, to enrich the dataset coverage. Therefore, I argue that the dataset construction is a weak core contribution.
-
The proposed ClueDet (presented in Section 3.2 and Figure 2) lacks sufficient novelty. It essentially assembles existing components (DINO, FFN) without dedicated improvements tailored to document priors, especially forgery clues including CE, CD, CR, and CDS.
Given the two major drawbacks outlined above, I cannot assign a positive score to this manuscript.
Questions for authors
Please refer to the weaknesses.
3 comments
(3) Evidence for the distinction. If MEDTD had no distinction from existing datasets, released forgery localizers should transfer to it. The following table lists the region AP on the tampered test pages and the false-positive rate on the untampered test pages (Table 4).
| Model | Region AP | False-positive rate |
|---|---|---|
| ADCD-Net | 0.99 | 51.12% |
| Co-Transformers | 0.02 | 69.48% |
| TruFor | 0.09 | 100.00% |
| PIXAR | 0.61 | 100.00% |
| DCL-Net | 13.62 | 22.21% |
| DINO | 34.17 | 8.79% |
As shown in the table, the released models obtain below 1 region AP and flag more than half of the untampered pages. Even after training on MEDTD, DCL-Net as the best of seven forensic localizers reaches only 13.62 region AP, compared with 34.17 for DINO. The element type also matters, e.g., in all runs the AP is 42.17–49.71 for duplicated pictures but only 0.11–0.50 for duplicated words, which a text-only dataset cannot show. Finally, in a post-hoc analysis of the nine runs, 94.0–95.2% of the vacated relocation sources are overlapped by an exported box, but only 43.9–53.5% of them receive the relocation label from the highest-scoring box, compared with 87.2–92.9% correct labels for the three other region types we examined.
(4) Workload. We inspected every one of the 46,290 edited pages, corrected errors and reinspected the pages (Sec. 3.1.3), which results in 69,839 region annotations. Table 4 evaluates twenty methods from four families under one protocol, and seven of them are forensic localizers trained on MEDTD. The data volume is also important for training: with fixed optimizer updates, training on the full partition instead of a fixed half improves the operation AP by 12.16 points for DINO and 1.86 for Faster R-CNN, and reduces the false-positive rate of DINO from 28.81% to 8.79% (Tables 5 and 10).
(5) Reprinting and regional AI generation. Thank you for these suggestions. Capture conditions and generated text are the focus of other datasets in Table 1, i.e., FSTS (scans and captures), DanceText (AI-generated and edited text) and AIForge-Doc (generative field edits), which all edit text only, and MEDTD is complementary to them. Following your reprinting suggestion, we nevertheless evaluated a synthetic print–scan degradation (half resolution, blur, gain and noise) on the full test set, for the forensic localizers of Table 4 (region AP).
| Model | Original | Print–scan | Retained |
|---|---|---|---|
| DINO | 34.17 | 4.92 | 14.4% |
| Uniform control | 34.00 | 5.38 | 15.8% |
| ClueDet | 35.13 | 3.86 | 11.0% |
| RTMDet-M | 20.21 | 3.14 | 15.5% |
| Cascade R-CNN | 19.61 | 1.66 | 8.5% |
| RetinaNet | 18.07 | 0.64 | 3.5% |
| Faster R-CNN | 17.24 | 0.58 | 3.3% |
| FRD-Net | 13.16 | 9.71 | 73.8% |
| CAFTB-Net† | 9.90 | 8.79 | 88.8% |
| DCL-Net | 13.62 | 7.00 | 51.4% |
| TIFDM† | 9.13 | 3.90 | 42.8% |
| PSCC-Net | 3.20 | 1.85 | 57.8% |
| CAT-Net v2† | 3.85 | 0.00 | 0.0% |
The results show that all seven box detectors keep only 3–16% of their region AP, while FRD-Net, CAFTB-Net and DCL-Net keep 51–89% and overtake them. Therefore, MEDTD can already be used to study such degradations, and the ranking of the evaluated models changes under them. Physical reprinting and regional AI generation are not part of the current release, and we will name both as limitations.
[W2] What is tailored to forgery clues in ClueDet
The proposed ClueDet (presented in Section 3.2 and Figure 2) lacks sufficient novelty. It essentially assembles existing components (DINO, FFN) without dedicated improvements tailored to document priors, especially forgery clues including CE, CD, CR, and CDS.
We would like to clarify that the FFN is only the final projection of the added module. Before it, ClueDet pools the features inside each predicted box, looks for similar regions on the same page, and passes the difference between a region and their weighted mean to the decoder query (Sec. 3.2, Fig. 2). This design follows from the operations you mention. A duplication leaves a second copy on the page, a relocation leaves a vacated source and a displaced copy, and a splicing inserts content from another page. In each case the evidence lies in how a region relates to other regions, which is what the module compares. The training keeps the classification and box losses of DINO and uses no contrastive loss.
The effect of this comparison can be tested with controls of matched size. In addition to the uniform-aggregation control in the submission, we trained a query-only MLP without any region input and a local-feature refinement that uses the region feature but makes no comparison. All models use the same training setting at the submitted seed.
| Model | Operation AP | ClueDet minus model |
|---|---|---|
| DINO | 33.02 | +1.09 |
| Query-only MLP | 33.51 | +0.60 |
| Local-feature refinement | 32.38 | +1.73 |
| Uniform aggregation | 32.56 | +1.55 |
| ClueDet | 34.11 |
The results show that, with the same regions, similarity weighting is better than uniform averaging and better than using the local region feature without any comparison. However, the query-only MLP is not separated from ClueDet, so we do not attribute the gain over DINO to the regional comparison itself.
We also trained DINO and ClueDet with two additional seeds. The DINO and ClueDet columns are operation AP.
| Seed | DINO | ClueDet | Operation AP difference | Region AP difference |
|---|---|---|---|---|
| 0 | 33.02 | 34.11 | +1.09 | +0.96 |
| 1 | 32.17 | 32.65 | +0.48 | +0.62 |
| 2 | 33.22 | 35.22 | +2.00 | +1.75 |
We appreciate your comments and hope this clarification of the distinctions and contributions addresses your concerns. If you know an existing dataset with edited pictures, formulas, and tables, or with annotated duplication and relocation sources on full document pages, we would be glad to compare it with ours. We would be grateful if you would reconsider the dataset contribution and the paper's soundness based on what MEDTD annotates, what it covers, and what it measures.
We thank Reviewer nXdg for the feedback and comments. We respectfully disagree that MEDTD bears no fundamental distinction from existing datasets, and we would like to clarify the distinction with the evidence below.
[W1] Motivation and distinction of the dataset
(1) Motivation. A mask of modified pixels cannot represent the unchanged source of a duplication, and it cannot tell whether a blank region was erased or its content was moved to another place. Therefore, MEDTD annotates both regions of duplication and relocation events and evaluates the operation assigned to each box. Source and target localization has a precedent in Forensim (Sec. 2), and MEDTD brings such targets to full document pages.
(2) Comparison in Table 1. The following table compares MEDTD with the seven previous datasets on the columns of Table 1.
As shown in the table, MEDTD is the only dataset whose edits go beyond text and the only one that reports document identities, which allows us to keep all pages of a recipient document in one split. All 28 combinations of operation and element are present in every split. Besides, its pages come from six source domains, and the domain matters: in all full-data DINO-family runs reported, the operation AP is highest on laws and regulations (41.83–46.78) and scientific articles (38.38–41.75) and stays at 19.83–31.36 on financial reports, manuals, patents and government tenders. We agree that the four operations are standard. The distinction lies in which content is edited and what is annotated, i.e., both regions of duplication and relocation events with their operation, together with 25,145 untampered pages for false-positive evaluation.