- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes TraceTok, an MLLM-based framework for fine-grained tampered-text localization. The method uses autoregressive mask-token generation to produce pixel-level tampering masks along with textual responses. It introduces two main components: TAMR for adaptive-length mask reconstruction and EAGF for integrating forensic expert features into the visual encoder. Experiments on TFR and RTM show improvements over several existing methods, particularly for small and irregular tampered-text regions.
Critical strengths
-
The paper addresses an interesting problem of fine-grained tampered-text localization. Using pixel-level masks instead of bounding boxes makes sense, especially when the manipulated regions are small, irregular, or spread across multiple characters.
-
The idea of combining forensic expert features with an MLLM is reasonable. The proposed TAMR and EAGF modules help incorporate low-level forensic information that may not be captured well by the visual encoder of a general-purpose MLLM.
-
The experiments cover several challenging settings, including cross-image-style and cross-tampering-method evaluation. The improvements on TFR are encouraging, and the separate mask reconstruction experiments are useful for understanding the effect of adaptive token lengths.
Critical weaknesses
-
The novelty of the proposed method is not entirely clear to me. ALToLLM has already explored adaptive-length mask-token generation, while ForensicsTok considers autoregressive mask generation with forensic expert features. TraceTok seems to build on these existing ideas, and I am not fully convinced that the proposed TAMR and EAGF modules introduce substantial methodological novelty beyond these works.
-
I am not sure how much the MLLM itself contributes to the localization performance. The method relies on a pretrained SegFormer forensic expert, and Table 4 shows that adding EAGF leads to a considerable improvement. The authors also mention that the standalone SegFormer expert performs better on the Test and CIS subsets. This makes it difficult to understand whether the improvement mainly comes from the forensic expert or the proposed mask-token generation framework.
-
The paper emphasizes generating both textual explanations and localization masks, but the experiments only evaluate localization and mask reconstruction. There is no evaluation of the generated textual responses or whether they correctly describe the manipulated content. Since this is one of the main motivations for using an MLLM, I think this part of the work needs more experimental support.
-
The computational complexity is another concern. The framework uses an 8B-parameter MLLM, a separate forensic expert, and multiple training stages. However, the paper does not provide a detailed analysis of inference time, memory consumption, or computational overhead. It would be useful to understand whether the additional complexity is justified, particularly when the improvements over task-specific methods are relatively small.
-
The presentation could be improved in a few places. Figures 2 and 3 contain several small labels that are difficult to read. The qualitative examples in Figure 4 are also quite small, making it difficult to observe the differences between the predicted masks, especially for very small tampered regions.
Questions for authors
-
What is the main technical novelty of TraceTok compared with ALToLLM and ForensicsTok? Since adaptive-length mask generation and forensic feature fusion have already been explored, I would like to understand what is fundamentally different in TAMR and EAGF.
-
How much of the improvement actually comes from the MLLM rather than the pretrained forensic expert? Since the standalone SegFormer performs better on some subsets, could the authors compare the complete framework with an expert-only model using a simpler segmentation head?
-
Could the authors provide separate ablations for MLLM pretraining and EAGF? In Table 4, both components are enabled together in Row 3, making it difficult to understand their individual contributions. Would a simpler interpolation-based fusion approach achieve comparable results?
6 comments
Response to Reviewer sY1r (Part 2)
Response to [W2&Q2]
Thank you for the suggestion. We added a configuration with only MLLM PT enabled and considered the standalone SegFormer results to clarify the roles of pretraining, expert feature injection, and mask reconstruction in localization performance.
Table A. Ablations of TAMR, MLLM PT, and EAGF.
| No. | TAMR | MLLM PT | EAGF | Test F1 (%) | Test Mask IoU (%) | CIS F1 (%) | CIS Mask IoU (%) | CTM F1 (%) | CTM Mask IoU (%) |
|---|---|---|---|---|---|---|---|---|---|
| 1 | - | - | - | 22.3 | 18.9 | 21.6 | 15.2 | 37.9 | 27.3 |
| 2 | ✓ | - | - | 25.7 | 22.7 | 23.5 | 19.8 | 38.6 | 27.9 |
| 3 | - | ✓ | - | 44.1 | 38.2 | 39.7 | 32.6 | 50.3 | 39.9 |
| 4 | - | ✓ | ✓ | 54.6 | 47.6 | 62.2 | 53.3 | 75.5 | 69.5 |
| 5 | ✓ | - | ✓ | 55.7 | 49.3 | 50.4 | 42.0 | 61.2 | 53.0 |
| 6 | ✓ | ✓ | ✓ | 59.3 | 53.4 | 68.0 | 60.2 | 78.8 | 73.1 |
1. Roles of expert features and the generative framework. As shown in Table A, enabling MLLM PT improves Mask IoU over row 1 by 19.3, 17.4, and 12.6 percentage points on Test, CIS, and CTM, respectively (row 3). Adding EAGF yields further gains of 9.4, 20.7, and 29.6 points (row 4). With MLLM PT and EAGF fixed, adding TAMR provides another 5.8, 6.9, and 3.6 points (row 6). These paired comparisons quantify the performance gains under the corresponding settings: expert feature injection contributes substantially, while pretraining and mask reconstruction also affect the final localization quality.
2. Expert-only comparison and the value of joint outputs. For the expert-only comparison, Supplement S6 already reports the localization results of standalone SegFormer-MiT-B2, providing a reference for direct segmentation without the MLLM or mask-token generation. Under the same TFR Mask Localization Protocol, standalone SegFormer achieves Mask IoUs of 61.6%, 61.0%, and 55.1% on Test, CIS, and CTM, respectively, compared with 53.4%, 60.2%, and 73.1% for the complete TraceTok framework. The standalone expert performs better on Test, the two are close on CIS, and TraceTok is 18.0 points higher on CTM. Thus, the standalone expert is a strong segmentation model, while TraceTok achieves better localization on CTM alongside joint generation of forensic text and masks.
Although a standalone expert may achieve higher segmentation performance, a segmentation mask alone does not explain the specific anomalies or how they support a tampering assessment. For example, the NFSTC guide illustrates an altered check whose leading amount digit was changed from “1” to “9” and whose recipient name was added; differences in the inks’ infrared responses revealed the alterations [c]. This example shows how identifying the affected fields and supporting observations helps explain the forensic significance of a localized region. The ENFSI guideline likewise recommends reporting significant findings, their evaluation, and conclusions, with a demonstrable process leading to those conclusions [d]. To support such needs, TraceTok generates forensic text and fine-grained masks in the same autoregressive response, linking textual analysis to spatial locations that a reviewer can inspect. EAGF integrates local forensic features into the visual representation, while TAMR improves the reconstruction of small regions and boundaries, preserving detailed regional information alongside the text for human review.
We will include the additional MLLM PT ablation and contribution analysis in the revised paper. Together with the standalone expert comparison, these results will clarify the complete framework’s localization performance and the role of jointly generating forensic text and masks.
References
[c] National Forensic Science Technology Center (NFSTC). A Simplified Guide to Forensic Document Examination. 2013. Section “Applications: Fraudulent Checks.” https://www.forensicsciencesimplified.org/docs/why.html. Accessed November 18, 2026.
[d] European Network of Forensic Science Institutes (ENFSI). ENFSI Guideline for Evaluative Reporting in Forensic Science. Approved version 3.0, 2015. https://enfsi.eu/wp-content/uploads/2016/09/m1_guideline.pdf.
Response to Reviewer sY1r (Part 3)
Response to W3
Thank you for pointing out the lack of evaluation of textual responses. We have added classification accuracy (Cls.) and explanation response scores (Res.) to assess image-category predictions and agreement between generated and reference explanations, respectively. The results are reported in Table B.
Table B. TFR classification accuracy and explanation response scores. Values are percentages (%).
| Method | Test Cls. | Test Res. | CIS Cls. | CIS Res. | CTM Cls. | CTM Res. |
|---|---|---|---|---|---|---|
| InternVL3-8B | 78.6 | 41.7 | 70.8 | 33.8 | 67.7 | 32.3 |
| Qwen2.5-VL-7B | 79.1 | 42.9 | 71.1 | 35.7 | 73.6 | 36.2 |
| FakeShield | 70.5 | 35.6 | 62.0 | 29.2 | 57.5 | 21.4 |
| SIDA | 71.2 | 35.7 | 62.2 | 29.4 | 57.5 | 21.5 |
| TextShield-R1* | 85.4 | 50.29 | 73.0 | 55.84 | 87.9 | 50.69 |
| TraceTok | 95.6 | 60.88 | 78.6 | 62.32 | 88.1 | 54.22 |
TextShield-R1 denotes our evaluation based on official weights of TextShield-R1
Following the TextShield-R1 scoring procedure, Res. averages cosine similarity, ROUGE-L, and BLEU between the generated natural-language explanation and its reference on generated and fake samples. Incorrectly classified samples receive a score of zero. Mask tokens are excluded from text scoring, and pixel-level localization is evaluated separately using mask metrics.
TraceTok achieves Res. scores of 60.88, 62.32, and 54.22 on Test, CIS, and CTM, respectively, exceeding TextShield-R1 evaluated with its official weights by 10.59, 6.48, and 3.53 percentage points, while also achieving higher classification accuracy. These results extend the original evaluation to both category prediction and explanation responses, showing higher response scores under this automatic scoring protocol.
Our methodological improvements focus on TAMR and EAGF. TAMR supports compact mask representation and reconstruction, while EAGF integrates expert forensic features into the visual representation, allowing the model to jointly generate forensic text and fine-grained masks in a single autoregressive response. The additional text evaluation provides experimental evidence for this joint output. We will incorporate the classification and textual explanation evaluations into the revised paper, including the scoring protocol and comparative results.
Response to Reviewer sY1r (Part 1)
Response to [W1&Q1]
Thank you for raising this question. ForensicsTok is an arXiv preprint released about three months before our submission. The ICLR 2027 reviewer FAQ states that authors are not required to compare with arXiv-only papers and that the lack of such comparisons cannot be a basis for rejection. Our Table 1 includes this comparison. ForensicsTok combines token-splatting decoding with hierarchical expert fusion; we address mask-specific token-budget learning and evidence-preserving alignment for fine-grained text tampering.
Table E. Localization comparison under the same TFR Mask Localization Protocol. Mask IoU (%).
† denotes retraining by us under the corresponding protocol.
1. TAMR: Guiding reconstruction training with a learned length prior.
ALToLLM and ForensicsTok address general-purpose mask generation and image-tampering localization, respectively, without specifically optimizing the representation and recovery of small tampered-text regions. Small text edits may comprise several scattered regions with irregular boundaries, so their token requirements depend on size, number, and shape. We therefore propose using the first-stage TLP estimate L to define a mask-specific sampling interval:
L∼UZ[max(1,L−12), min(32,L+4)].
This makes the learned estimate guide further reconstruction training, after which TLP is refined against the updated reconstructor (Section 3.2, Eq. (2), Figure 2). Forensic features also condition reconstruction, supplying image evidence for small regions and boundaries. In Table D, TLP guidance reduces mean token counts on Test/CIS/CTM from 13.3/14.2/16.1 to 12.9/13.2/14.8; full TAMR reaches reconstruction IoUs of 90.2/93.6/97.9. Table F shows end-to-end Mask IoU gains of 5.8/6.9/3.6 points.
Table F. TAMR ablation with MLLM PT and EAGF enabled. F1 and Mask IoU (%).
2. EAGF: Aligning features while retaining local forensic evidence.
ForensicsTok’s HEF explicitly resizes expert features to match the visual features; ViT-Adapter also evaluates resize-and-add fusion in its ablation [a]. Text forensics requires sensitivity to subtle manipulation traces, which may occupy only a few characters and need not follow semantic object boundaries. When expert features are mapped to a coarser visual grid, bilinear interpolation can attenuate isolated responses, while max pooling retains peaks but discards other evidence within each cell.
Prior work supports the complementary value of average and maximum statistics [b]. With this in mind, we design EPA to maintain spatial correspondence while combining complementary evidence within each target cell: average responses summarize distributed evidence, while maxima retain concentrated peaks. We apply this idea to forensic feature alignment by retaining an interpolated geometric baseline and learning gated residual corrections from cell-wise averages and peaks (Section 3.3, Figure 3). Both distributed and concentrated responses can thus contribute to the aligned representation. EAGF then fuses expert stages at corresponding positions and injects the result into the visual branch. Table C reports EPA gains over bilinear interpolation of 8.7/9.1/11.8 Mask IoU points on Test/CIS/CTM, and over max pooling of 1.4/2.9/7.2 points.
Table C. Alignment-operator comparison. F1 and Mask IoU (%).
[a] Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao. Vision transformer adapter for dense predictions. In The Eleventh International Conference on Learning Representations, 2023.
[b] Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. CBAM: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19, 2018.