acceptodds
Under review as a conference paper at ICLR 2027

TraceTok: Fine-Grained Text Tampering Localization via Native Mask Token Generation

Abstract

Text images are pervasive in financial, legal, and administrative records, where text tampering can undermine the credibility of the underlying information. Existing forensic multimodal large language models (MLLMs) can localize suspicious regions and provide natural-language analysis. However, they localize tampering through textual bounding boxes or tokens coupled to segmentation decoders, which are coarse for irregular shapes and fragmented regions not aligned with object-level semantic boundaries. Moreover, MLLM visual encoders are less sensitive to subtle manipulation traces than dedicated forensic experts. To address these limitations, we propose TraceTok, a forensic-enhanced MLLM that formulates fine-grained tampered text localization as native mask-token generation and jointly produces textual forensic responses and pixel-level tampering masks. The Tamper-Aware Adaptive-Length Mask Reconstruction module (TAMR) conditions mask reconstruction on image-side forensic features and uses Token Length Predictor (TLP)-guided retained-length selection, enabling compact token prefixes to preserve small and irregular tampered text structures. Evidence-Preserving Adaptive Grid Fusion (EAGF) aligns multistage expert features with the current visual-token grid before fusing and injecting them into the visual branch, thereby strengthening low-level forensic perception across varied input resolutions. On TFR-Test, CIS, and CTM, TraceTok attains 62.2 average Mask IoU, exceeding our reevaluated TextShield-R1 baseline by 12.7 points; on RTM it matches the strongest task-specific localizers, and its reconstructor reaches comparable or better F1 with fewer retained tokens than previous methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
text tampering localization, image forensics, multimodal large language models, mask token generation, forensic feature fusion, fine-grained localization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_sY1rOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes TraceTok, an MLLM-based framework for fine-grained tampered-text localization. The method uses autoregressive mask-token generation to produce pixel-level tampering masks along with textual responses. It introduces two main components: TAMR for adaptive-length mask reconstruction and EAGF for integrating forensic expert features into the visual encoder. Experiments on TFR and RTM show improvements over several existing methods, particularly for small and irregular tampered-text regions.

Critical strengths
  • The paper addresses an interesting problem of fine-grained tampered-text localization. Using pixel-level masks instead of bounding boxes makes sense, especially when the manipulated regions are small, irregular, or spread across multiple characters.

  • The idea of combining forensic expert features with an MLLM is reasonable. The proposed TAMR and EAGF modules help incorporate low-level forensic information that may not be captured well by the visual encoder of a general-purpose MLLM.

  • The experiments cover several challenging settings, including cross-image-style and cross-tampering-method evaluation. The improvements on TFR are encouraging, and the separate mask reconstruction experiments are useful for understanding the effect of adaptive token lengths.

Critical weaknesses
  • The novelty of the proposed method is not entirely clear to me. ALToLLM has already explored adaptive-length mask-token generation, while ForensicsTok considers autoregressive mask generation with forensic expert features. TraceTok seems to build on these existing ideas, and I am not fully convinced that the proposed TAMR and EAGF modules introduce substantial methodological novelty beyond these works.

  • I am not sure how much the MLLM itself contributes to the localization performance. The method relies on a pretrained SegFormer forensic expert, and Table 4 shows that adding EAGF leads to a considerable improvement. The authors also mention that the standalone SegFormer expert performs better on the Test and CIS subsets. This makes it difficult to understand whether the improvement mainly comes from the forensic expert or the proposed mask-token generation framework.

  • The paper emphasizes generating both textual explanations and localization masks, but the experiments only evaluate localization and mask reconstruction. There is no evaluation of the generated textual responses or whether they correctly describe the manipulated content. Since this is one of the main motivations for using an MLLM, I think this part of the work needs more experimental support.

  • The computational complexity is another concern. The framework uses an 8B-parameter MLLM, a separate forensic expert, and multiple training stages. However, the paper does not provide a detailed analysis of inference time, memory consumption, or computational overhead. It would be useful to understand whether the additional complexity is justified, particularly when the improvements over task-specific methods are relatively small.

  • The presentation could be improved in a few places. Figures 2 and 3 contain several small labels that are difficult to read. The qualitative examples in Figure 4 are also quite small, making it difficult to observe the differences between the predicted masks, especially for very small tampered regions.

Questions for authors
  1. What is the main technical novelty of TraceTok compared with ALToLLM and ForensicsTok? Since adaptive-length mask generation and forensic feature fusion have already been explored, I would like to understand what is fundamentally different in TAMR and EAGF.

  2. How much of the improvement actually comes from the MLLM rather than the pretrained forensic expert? Since the standalone SegFormer performs better on some subsets, could the authors compare the complete framework with an expert-only model using a simpler segmentation head?

  3. Could the authors provide separate ablations for MLLM pretraining and EAGF? In Table 4, both components are enabled together in Row 3, making it difficult to understand their individual contributions. Would a simpler interpolation-based fusion approach achieve comparable results?

6 comments
AuthorsOpenReview ↗

Response to Reviewer sY1r (Part 1)

Response to [W1&Q1]

Thank you for raising this question. ForensicsTok is an arXiv preprint released about three months before our submission. The ICLR 2027 reviewer FAQ states that authors are not required to compare with arXiv-only papers and that the lack of such comparisons cannot be a basis for rejection. Our Table 1 includes this comparison. ForensicsTok combines token-splatting decoding with hierarchical expert fusion; we address mask-specific token-budget learning and evidence-preserving alignment for fine-grained text tampering.

Table E. Localization comparison under the same TFR Mask Localization Protocol. Mask IoU (%).

MethodTestCISCTM
ForensicsTok†37.029.940.7
TraceTok53.460.273.1

† denotes retraining by us under the corresponding protocol.

1. TAMR: Guiding reconstruction training with a learned length prior.

ALToLLM and ForensicsTok address general-purpose mask generation and image-tampering localization, respectively, without specifically optimizing the representation and recovery of small tampered-text regions. Small text edits may comprise several scattered regions with irregular boundaries, so their token requirements depend on size, number, and shape. We therefore propose using the first-stage TLP estimate L^\widehat L to define a mask-specific sampling interval:

L∼UZ[max⁡(1,L^−12), min⁡(32,L^+4)].L\sim\mathcal U_{\mathbb Z}\left[\max(1,\widehat L-12),\ \min(32,\widehat L+4)\right].

This makes the learned estimate guide further reconstruction training, after which TLP is refined against the updated reconstructor (Section 3.2, Eq. (2), Figure 2). Forensic features also condition reconstruction, supplying image evidence for small regions and boundaries. In Table D, TLP guidance reduces mean token counts on Test/CIS/CTM from 13.3/14.2/16.1 to 12.9/13.2/14.8; full TAMR reaches reconstruction IoUs of 90.2/93.6/97.9. Table F shows end-to-end Mask IoU gains of 5.8/6.9/3.6 points.

Table F. TAMR ablation with MLLM PT and EAGF enabled. F1 and Mask IoU (%).

ConfigurationTest F1Test IoUCIS F1CIS IoUCTM F1CTM IoU
Without TAMR (ALToLLM reconstructor)54.647.662.253.375.569.5
Full TAMR59.353.468.060.278.873.1

2. EAGF: Aligning features while retaining local forensic evidence.

ForensicsTok’s HEF explicitly resizes expert features to match the visual features; ViT-Adapter also evaluates resize-and-add fusion in its ablation [a]. Text forensics requires sensitivity to subtle manipulation traces, which may occupy only a few characters and need not follow semantic object boundaries. When expert features are mapped to a coarser visual grid, bilinear interpolation can attenuate isolated responses, while max pooling retains peaks but discards other evidence within each cell.

Prior work supports the complementary value of average and maximum statistics [b]. With this in mind, we design EPA to maintain spatial correspondence while combining complementary evidence within each target cell: average responses summarize distributed evidence, while maxima retain concentrated peaks. We apply this idea to forensic feature alignment by retaining an interpolated geometric baseline and learning gated residual corrections from cell-wise averages and peaks (Section 3.3, Figure 3). Both distributed and concentrated responses can thus contribute to the aligned representation. EAGF then fuses expert stages at corresponding positions and injects the result into the visual branch. Table C reports EPA gains over bilinear interpolation of 8.7/9.1/11.8 Mask IoU points on Test/CIS/CTM, and over max pooling of 1.4/2.9/7.2 points.

Table C. Alignment-operator comparison. F1 and Mask IoU (%).

Alignment operatorTest F1Test IoUCIS F1CIS IoUCTM F1CTM IoU
Bilinear interpolation46.840.640.132.951.741.2
Adaptive max pooling53.247.947.639.156.045.8
EPA55.749.350.442.061.253.0

[a] Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao. Vision transformer adapter for dense predictions. In The Eleventh International Conference on Learning Representations, 2023.

[b] Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. CBAM: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19, 2018.

AuthorsOpenReview ↗

Response to Reviewer sY1r (Part 2)

Response to [W2&Q2]

Thank you for the suggestion. We added a configuration with only MLLM PT enabled and considered the standalone SegFormer results to clarify the roles of pretraining, expert feature injection, and mask reconstruction in localization performance.

Table A. Ablations of TAMR, MLLM PT, and EAGF.

No.TAMRMLLM PTEAGFTest F1 (%)Test Mask IoU (%)CIS F1 (%)CIS Mask IoU (%)CTM F1 (%)CTM Mask IoU (%)
1---22.318.921.615.237.927.3
2✓--25.722.723.519.838.627.9
3-✓-44.138.239.732.650.339.9
4-✓✓54.647.662.253.375.569.5
5✓-✓55.749.350.442.061.253.0
6✓✓✓59.353.468.060.278.873.1

1. Roles of expert features and the generative framework. As shown in Table A, enabling MLLM PT improves Mask IoU over row 1 by 19.3, 17.4, and 12.6 percentage points on Test, CIS, and CTM, respectively (row 3). Adding EAGF yields further gains of 9.4, 20.7, and 29.6 points (row 4). With MLLM PT and EAGF fixed, adding TAMR provides another 5.8, 6.9, and 3.6 points (row 6). These paired comparisons quantify the performance gains under the corresponding settings: expert feature injection contributes substantially, while pretraining and mask reconstruction also affect the final localization quality.

2. Expert-only comparison and the value of joint outputs. For the expert-only comparison, Supplement S6 already reports the localization results of standalone SegFormer-MiT-B2, providing a reference for direct segmentation without the MLLM or mask-token generation. Under the same TFR Mask Localization Protocol, standalone SegFormer achieves Mask IoUs of 61.6%, 61.0%, and 55.1% on Test, CIS, and CTM, respectively, compared with 53.4%, 60.2%, and 73.1% for the complete TraceTok framework. The standalone expert performs better on Test, the two are close on CIS, and TraceTok is 18.0 points higher on CTM. Thus, the standalone expert is a strong segmentation model, while TraceTok achieves better localization on CTM alongside joint generation of forensic text and masks.

Although a standalone expert may achieve higher segmentation performance, a segmentation mask alone does not explain the specific anomalies or how they support a tampering assessment. For example, the NFSTC guide illustrates an altered check whose leading amount digit was changed from “1” to “9” and whose recipient name was added; differences in the inks’ infrared responses revealed the alterations [c]. This example shows how identifying the affected fields and supporting observations helps explain the forensic significance of a localized region. The ENFSI guideline likewise recommends reporting significant findings, their evaluation, and conclusions, with a demonstrable process leading to those conclusions [d]. To support such needs, TraceTok generates forensic text and fine-grained masks in the same autoregressive response, linking textual analysis to spatial locations that a reviewer can inspect. EAGF integrates local forensic features into the visual representation, while TAMR improves the reconstruction of small regions and boundaries, preserving detailed regional information alongside the text for human review.

We will include the additional MLLM PT ablation and contribution analysis in the revised paper. Together with the standalone expert comparison, these results will clarify the complete framework’s localization performance and the role of jointly generating forensic text and masks.

References

[c] National Forensic Science Technology Center (NFSTC). A Simplified Guide to Forensic Document Examination. 2013. Section “Applications: Fraudulent Checks.” https://www.forensicsciencesimplified.org/docs/why.html. Accessed November 18, 2026.

[d] European Network of Forensic Science Institutes (ENFSI). ENFSI Guideline for Evaluative Reporting in Forensic Science. Approved version 3.0, 2015. https://enfsi.eu/wp-content/uploads/2016/09/m1_guideline.pdf.

AuthorsOpenReview ↗

Response to Reviewer sY1r (Part 3)

Response to W3

Thank you for pointing out the lack of evaluation of textual responses. We have added classification accuracy (Cls.) and explanation response scores (Res.) to assess image-category predictions and agreement between generated and reference explanations, respectively. The results are reported in Table B.

Table B. TFR classification accuracy and explanation response scores. Values are percentages (%).

MethodTest Cls.Test Res.CIS Cls.CIS Res.CTM Cls.CTM Res.
InternVL3-8B78.641.770.833.867.732.3
Qwen2.5-VL-7B79.142.971.135.773.636.2
FakeShield70.535.662.029.257.521.4
SIDA71.235.762.229.457.521.5
TextShield-R1*85.450.2973.055.8487.950.69
TraceTok95.660.8878.662.3288.154.22

TextShield-R1 denotes our evaluation based on official weights of TextShield-R1

Following the TextShield-R1 scoring procedure, Res. averages cosine similarity, ROUGE-L, and BLEU between the generated natural-language explanation and its reference on generated and fake samples. Incorrectly classified samples receive a score of zero. Mask tokens are excluded from text scoring, and pixel-level localization is evaluated separately using mask metrics.

TraceTok achieves Res. scores of 60.88, 62.32, and 54.22 on Test, CIS, and CTM, respectively, exceeding TextShield-R1 evaluated with its official weights by 10.59, 6.48, and 3.53 percentage points, while also achieving higher classification accuracy. These results extend the original evaluation to both category prediction and explanation responses, showing higher response scores under this automatic scoring protocol.

Our methodological improvements focus on TAMR and EAGF. TAMR supports compact mask representation and reconstruction, while EAGF integrates expert forensic features into the visual representation, allowing the model to jointly generate forensic text and fine-grained masks in a single autoregressive response. The additional text evaluation provides experimental evidence for this joint output. We will incorporate the classification and textual explanation evaluations into the revised paper, including the scoring protocol and comparative results.

Reviewer_3hwdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TraceTok addresses fine-grained tampered-text localization by generating discrete mask tokens alongside textual responses. It introduces an expert-conditioned adaptive-length mask reconstructor (TAMR) and visual-grid-aligned forensic feature fusion (EAGF), supported by staged training. Experiments cover box and mask localization on TFR, mask reconstruction, and localization on RTM. The reported results show substantial improvements over MLLM baselines on TFR and competitive performance on RTM. The demonstrated contribution is primarily improved localization; the quality of generated explanations is not evaluated.

Critical strengths
  • The localization improvement is substantial and supported by relevant comparisons. On TFR, average Mask IoU increases from 49.5 for reevaluated TextShield-R1 to 62.2. Gains also hold under the box protocol, and the comparisons include both forensic MLLMs and mask-token baselines (Table 1). This provides useful evidence for the proposed adaptation to text forensics.
  • The ablations support an incremental contribution from TAMR. With MLLM pretraining and EAGF fixed, adding TAMR improves Test/CIS/CTM Mask IoU by 5.8/6.9/3.6 points (Table 4, rows 3 and 5). The separate reconstruction evaluation also reports improved fidelity with fewer retained tokens (Table 2). These results strengthen the case beyond a comparison of complete systems alone.
Critical weaknesses
  • The novelty is mainly in the forensic adaptation, and the evidence for the specific internal designs is incomplete. Table 4 establishes the value of the combined modules but does not isolate EPA alignment from simpler fusion with the same expert, or TLP-guided sampling from the other TAMR changes. Consequently, the results support the complete design more strongly than the explanations of why each new mechanism is needed.
  • The empirical advantage is less pronounced on RTM: 20.9 IoU/34.6 F1 versus TVSIP’s 20.6/34.2 (Table 3). Since the paper reports a single training run per setting (p. 14), this comparison establishes competitiveness but provides limited evidence of a further performance advance. This tempers the breadth of the empirical contribution without undermining the larger TFR gains.
Questions for authors
  • Do the existing analyses distinguish the benefit of EPA from simpler alignment/fusion under the same expert, and the benefit of TLP-guided sampling from the other TAMR changes? This would help assess the contribution of the specific designs.
  • Please clarify which training splits were used for the retrained tokenizer/reconstructor baselines in Table 2, and what “on the corresponding subsets” means. In particular, were all reported reconstruction results obtained on held-out images and masks? Please point to the relevant details in S1–S6 if already documented there.
2 comments
AuthorsOpenReview ↗

Response to Reviewer 3hwd (Part 1)

Response to [W1&Q1]

Thank you for the suggestion. We have added two sets of internal ablations to distinguish EPA from simpler alignment under the same expert, and to examine the contributions of forensic conditioning and TLP-guided sampling within TAMR.

  1. EPA versus simpler alignment under the same expert. In Table C, all three configurations enable TAMR and disable MLLM PT. They use the same expert, dataset, training epochs, and remaining settings; only the alignment operator changes.

Table C. Ablation of alignment operators. TAMR is enabled and MLLM PT is disabled in all three configurations.

AlignmentTest F1 (%)Test Mask IoU (%)CIS F1 (%)CIS Mask IoU (%)CTM F1 (%)CTM Mask IoU (%)
EPA55.749.350.442.061.253.0
Adaptive max pooling53.247.947.639.156.045.8
Bilinear interpolation46.840.640.132.951.741.2

EPA augments bilinear alignment with gated residuals from average and max pooling to preserve regional responses and local peaks, reducing the loss of fine-grained forensic cues during alignment. It improves Mask IoU by 8.7/9.1/11.8 percentage points over bilinear interpolation on Test/CIS/CTM, and by 1.4/2.9/7.2 points over adaptive max pooling. This comparison supports EPA’s localization benefit over both simpler alignment operators.

  1. Separate contributions of forensic conditioning and TLP-guided sampling. Table D removes each design separately and evaluates reconstruction from encoded ground-truth masks. #Tok denotes the mean number of retained mask tokens.

Table D. TAMR reconstruction results and internal ablations. IoU and F1 are computed per image and averaged within each subset.

ReconstructionTest #TokTest IoU (%)Test F1 (%)CIS #TokCIS IoU (%)CIS F1 (%)CTM #TokCTM IoU (%)CTM F1 (%)
ALToLLM†13.886.690.614.191.394.215.997.298.3
Without forensic conditioning13.087.792.512.991.094.715.297.298.5
Without TLP-guided sampling13.388.092.614.293.295.916.198.599.2
Full TAMR12.990.293.813.293.696.214.897.998.9

Forensic conditioning supplies local image evidence for reconstruction. Retaining it improves reconstruction IoU by 2.5/2.6/0.7 percentage points on Test/CIS/CTM. TLP-guided sampling restricts training prefixes to a bounded interval around the predicted length to balance reconstruction quality and sequence length. Compared with the unguided configuration, full TAMR reduces the mean token count from 13.3/14.2/16.1 to 12.9/13.2/14.8 while improving IoU by 2.2 and 0.4 points on Test and CIS. On CTM, it trades 0.6 IoU points for a shorter sequence. These results are consistent with its goal of preserving reconstruction quality within a compact token budget.

We will include these ablations in the revised paper and expand the analysis of the individual contributions of EPA, forensic conditioning, and TLP-guided sampling.

AuthorsOpenReview ↗

Response to Reviewer 3hwd (Part 2)

Response to W2

Thank you for your careful assessment of the RTM results. TraceTok improves over TVSIP by 0.3 percentage points in IoU and 0.4 in F1. With one training run per setting, we agree that these results mainly indicate comparable localization performance. However, localization scores alone do not fully reflect how the two methods differ in forensic analysis.

TVSIP passes expert-predicted tampered regions and their visual and textual cues to an MLLM for subsequent forensic analysis. This may cause the MLLM to accept the expert’s region cues without questioning them: when the cues are wrong, the model may still produce plausible explanations about the wrong regions, leading to forensic hallucinations. We observed this phenomenon in another study submitted to IEEE Transactions on Information Forensics and Security (TIFS). We expect that work to be published in 2027. TraceTok instead uses EAGF to integrate expert features into visual representations and generates forensic text and mask tokens together in a single autoregressive response. It does not first treat expert-predicted regions as established conclusions and then explain them.

TraceTok thus brings region prediction and explanation into the same model process. The benefit goes beyond reducing processing stages: it changes the expert’s role in reasoning, from supplying a prior tampering conclusion to providing forensic features that inform the model’s judgment. On RTM, this design jointly produces forensic text and pixel-level masks while achieving localization performance comparable to TVSIP. The larger localization gains on TFR provide further evidence of its effectiveness.

Response to Q2

Thank you for your question. The retrained tokenizer/reconstructor baselines in Table 2 use the TFR training split for the TFR experiments and the RTM training split for the RTM experiments. “On the corresponding subsets” means training on the corresponding dataset’s training split and reporting results on its evaluation splits, rather than training separately on TFR Test, CIS, or CTM. We will replace this wording with “trained on the training split of the corresponding dataset.”

All reported reconstruction results were obtained on held-out images and their masks. Table 2 evaluates mask encoding and reconstruction: each evaluation mask is encoded into tokens, reconstructed, and compared with the original mask. Using the ground-truth mask as the encoding input is part of the reconstruction task and does not mean that the sample was used for training.

Section S4 describes the reconstruction evaluation procedure, and Section S5.1 introduces the dataset splits. However, neither explicitly states that the reconstruction evaluation images and masks are disjoint from the training data. We will add this statement and specify the training and evaluation splits for each baseline in Section S5.1, and clarify the corresponding text and caption of Table 2.

Reviewer_d15iOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper applies an MLLM to tampered-text localization. Instead of a <SEG> token or a textual box, the model generates a short sequence of discrete mask tokens that a de-tokenizer converts into a pixel mask. TAMR conditions the de-tokenizer on features from a frozen SegFormer forensic expert and samples the retained length around the Token Length Predictor’s estimate. EAGF aligns the expert’s stage-3/4 features with the visual-token grid and injects them into the visual branch. Experimental results demonstrate the effectiveness of the proposed method.

Critical strengths
  1. The paper is generally easy to follow.
  2. The formulation is well motivated. Tampered text regions are thin, fragmented, and rarely axis-aligned, so supervising the mask directly through token labels is a better fit than boxes or an externally decoded <SEG> token, and the gains over MLLM baselines are large.
Critical weaknesses
  1. The frozen expert may be doing most of the work. Sec. 3.3 states that the SegFormer-MiT-B2 expert is trained on the corresponding dataset and frozen; its features feed both the visual branch and the mask decoder. Sec. 4.2 then acknowledges that this expert alone outperforms TraceTok on Test and CIS, and on RTM Table 3 lists 19.4 IoU for SegFormer versus 20.9 for TraceTok. The paper never clarifies how much localization comes from the MLLM rather than the expert, or why routing the expert’s features through TAMR and EAGF ends up below its own dense prediction.
  2. The textual forensic explanation is not evaluated. The abstract highlights joint text-and-mask generation, yet all reported metrics concern masks or boxes, and the Limitations section admits that neither classification nor response quality was assessed.
  3. The ablation does not isolate EAGF. Table 4 has no configuration with MLLM pre-training enabled and EAGF disabled; row 1 versus row 3 switches both on together. All numbers come from a single run with no variance. The paper also does not state whether ALToLLM† and ForensicsTok† received the same expert features.
  4. EAGF is motivated by varying input resolutions (Abstract, Sec. 3.3), yet no experiment varies the resolution.
  5. Sec. 3.3 claims plain bilinear alignment attenuates traces and max pooling is too coarse, but neither variant is ablated.
Questions for authors
  1. Were ALToLLM† and ForensicsTok† given the same frozen SegFormer features? If not, what does ALToLLM achieve with expert conditioning alone?
  2. Please report the standalone expert’s mask IoU, P, and R on all three TFR subsets alongside Table 1, and explain why it stays ahead of the full system on Test and CIS.
  3. Please add Table 4 rows for pre-training only and for TAMR + pre-training without EAGF.
  4. How does TraceTok compare with TextShield-R1 and TVSIP on the textual explanation itself?
5 comments
AuthorsOpenReview ↗

Response to Reviewer d15i (Part 1)

Response to [W1&Q2]

Thank you for this suggestion. We have added an ablation with only MLLM PT enabled, shown in row 3 of Table A.

1. Contributions of the MLLM and expert features. With EAGF and TAMR disabled, enabling MLLM PT increases Mask IoU from 18.9/15.2/27.3 to 38.2/32.6/39.9 on Test/CIS/CTM. Adding EAGF (rows 3 and 4) provides another 9.4/20.7/29.6 percentage points, followed by 5.8/6.9/3.6 points from TAMR (rows 4 and 6). These results show that MLLM learning contributes to localization, while expert features injected through EAGF further improve localization accuracy.

Table A. Ablations of TAMR, MLLM PT, and EAGF.

No.TAMRMLLM PTEAGFTest F1 (%)Test Mask IoU (%)CIS F1 (%)CIS Mask IoU (%)CTM F1 (%)CTM Mask IoU (%)
1---22.318.921.615.237.927.3
2✓--25.722.723.519.838.627.9
3-✓-44.138.239.732.650.339.9
4-✓✓54.647.662.253.375.569.5
5✓-✓55.749.350.442.061.253.0
6✓✓✓59.353.468.060.278.873.1

2. Why the standalone expert stays ahead on Test and CIS. TraceTok’s Mask IoU is lower than standalone SegFormer on Test, close on CIS, and higher on CTM. We recognize the standalone expert’s strength in dense segmentation. Three factors help explain this difference.

  • Tampered regions are smaller in Test and CIS. These subsets include many small character-level edits in documents and receipts, whereas CTM commonly involves word-level edits in scene text, such as shop signs and road signs. The median tampered-area fractions are 0.43%/0.39%/2.10% on Test/CIS/CTM. The larger, more prominent text targets in CTM are more favorable for MLLM perception and generative localization. Small edits in Test and CIS depend more on local forensic cues and precise pixel-wise discrimination, where the standalone expert has an advantage.

  • Mask reconstruction loses more detail on Test and CIS. Decoding compact mask tokens can lose small foreground regions and boundaries, reducing localization accuracy. Even with ground-truth masks as input, TAMR reaches IoU scores of 90.2/93.6 on Test/CIS, below 97.9 on CTM (Table 2 of the paper).

  • Dense-text localization in Test and CIS is affected by spatial errors. TextShield-R1 also notes this problem and uses OCR rectification. Under the OCR-rectified box protocol, TraceTok achieves IoU scores of 58.2/61.6/85.0 on the TFR Test/CIS/CTM subsets. Numerically, Test approaches the expert’s IoU of 61.6, while CIS exceeds its 61.0.

Although standalone experts may achieve higher segmentation accuracy, a segmentation mask alone does not explain specific anomalies or how they support a tampering assessment. Analysis assisting forensic examination also requires a basis for reviewing the conclusion. TraceTok jointly generates forensic text and fine-grained masks, allowing the described anomalies and corresponding regions to be examined together.

We will incorporate the new MLLM PT ablation and contribution analysis into the paper, add the standalone expert’s Mask IoU, P, and R on all three TFR subsets to Table 1 of the main paper, and explain these differences in supplementary Section S6.

AuthorsOpenReview ↗

Response to Reviewer d15i (Part 2)

Response to [W2&Q4]

Thank you for pointing out the lack of evaluation of classification and textual forensic explanations. We have added classification accuracy (Cls.) and explanation response scores (Res.), as reported in Table B.

Table B. TFR classification accuracy and explanation response scores. Values are percentages (%).

MethodTest Cls.Test Res.CIS Cls.CIS Res.CTM Cls.CTM Res.
InternVL3-8B78.641.770.833.867.732.3
Qwen2.5-VL-7B79.142.971.135.773.636.2
FakeShield70.535.662.029.257.521.4
SIDA71.235.762.229.457.521.5
TextShield-R1*85.450.2973.055.8487.950.69
TraceTok95.660.8878.662.3288.154.22

TextShield-R1 denotes our evaluation based on official weights of TextShield-R1.

Following the TextShield-R1 scoring procedure, Res. is evaluated on generated and fake samples. It averages cosine similarity, ROUGE-L, and BLEU between the generated natural-language explanation and the reference explanation, with incorrectly classified samples receiving a score of zero. Mask tokens are excluded from text scoring, and pixel-level localization is evaluated separately using mask metrics.

TraceTok achieves Res. scores of 60.88, 62.32, and 54.22 on Test, CIS, and CTM, respectively, exceeding our rerun of TextShield-R1 by 10.59, 6.48, and 3.53 percentage points, while also achieving higher classification accuracy. TAMR supports compact mask representation and reconstruction, while EAGF integrates expert forensic features into visual representations, enabling the model to jointly generate textual forensic explanations and fine-grained masks within a single autoregressive response. These additional results show that TraceTok achieves higher explanation response scores alongside improved localization, complementing the evaluation of joint text-and-mask generation in the original paper.

Evaluation of TVSIP using the same text scoring procedure is not yet complete, and we will provide the comparison once available. For now, please refer to the comparison between TraceTok and TextShield-R1. We will also incorporate the classification and textual explanation evaluations into the revised paper.

AuthorsOpenReview ↗

Response to Reviewer d15i (Part 3)

Response to [W3&Q1&Q3]

Thank you for your suggestions. We have added the pre-training-only configuration and clarify below which expert features the baselines received.

  1. Additional ablation configurations and EAGF’s contribution (W3, Q3). Row 3 of Table A enables MLLM PT while disabling TAMR and EAGF, providing the requested pre-training-only configuration. Adding EAGF to this configuration (rows 3–4) improves Mask IoU by 9.4/20.7/29.6 percentage points on Test/CIS/CTM. With TAMR enabled and MLLM PT disabled, adding EAGF (rows 2–5) also yields gains of 26.6/22.2/25.1 points. These paired comparisons isolate EAGF’s contribution under two configurations.

The TAMR + pre-training configuration without EAGF has not yet been completed and will be added in subsequent experiments. All current results come from a single run, so variance across random seeds is not yet available. We will state this in the revised paper and add experiments with multiple random seeds.

Table A. Ablations of TAMR, MLLM PT, and EAGF. F1 (%) and Mask IoU (%) are reported in separate columns.

No.TAMRMLLM PTEAGFTest F1 (%)Test Mask IoU (%)CIS F1 (%)CIS Mask IoU (%)CTM F1 (%)CTM Mask IoU (%)
1−−−22.318.921.615.237.927.3
2✓−−25.722.723.519.838.627.9
3−✓−44.138.239.732.650.339.9
4−✓✓54.647.662.253.375.569.5
5✓−✓55.749.350.442.061.253.0
6✓✓✓59.353.468.060.278.873.1

Note: Row 3 uses pretrained weights with prior expert-fusion exposure; EAGF is disabled during the current fine-tuning and inference stages.

  1. Expert features used by the baselines (Q1). ForensicsTok† and TraceTok receive the same frozen SegFormer forensic features through their respective fusion modules. ALToLLM†’s original architecture lacks alignment and injection modules for these forensic features, so SegFormer features were not additionally incorporated in our reproduction. One concrete adaptation uses EAGF to inject expert features into the MLLM visual branch and TAMR to incorporate forensic conditioning during reconstruction. Row 5 of Table A provides an empirical reference for this adaptation: with MLLM PT disabled and TAMR and EAGF enabled, Mask IoU is 49.3/42.0/53.0 on Test/CIS/CTM. We will specify each baseline’s expert configuration in the revised paper.

Response to W4

Thank you for the suggestion. Our TFR training set contains 51,690 images spanning 359 distinct resolutions, with the longer side ranging from 560 to 2,352 pixels; 30.24% exceed 1,024 pixels. Square, landscape, and portrait images account for 69.24%, 17.90%, and 12.86%, respectively. During training, dynamic tiling adjusts the visual-token layout according to the input image dimensions and aspect ratio. EAGF then aligns expert features with the corresponding visual-token grid to accommodate these spatial differences. We will add these statistics and clarify the dynamic tiling settings and EAGF’s alignment procedure in the revised paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of responses and manuscript revisions

Dear Area Chair and Reviewers,

Thank you for your careful reading and constructive feedback. We have revised the manuscript and supplementary material to incorporate the additional experiments and clarify the experimental settings discussed in our responses. The main updates are:

  • Component contributions: Table 4 now includes the TAMR + MLLM PT configuration without EAGF. We have expanded the contribution analysis and clarified the prior expert-fusion exposure of the PT-only initialization.
  • Localization comparisons and internal ablations: We include the standalone SegFormer expert under the corresponding localization protocols, clarify the expert configurations of the baselines, and report additional EPA and TAMR ablations.
  • Classification and textual explanations: Tables 5 and 6 report comparisons with TextShield-R1 and TVSIP, respectively. The supplementary material explains their different scoring protocols.
  • Computational cost and presentation: Table S7 reports inference time and GPU memory usage. We have also clarified training and evaluation settings, updated the limitations, corrected references, and improved Figures 2 and 4.

In the revision-marked PDFs, changes to existing material are shown in blue, while additions are shown in red, with short vertical lines in the margin.

We appreciate your time and welcome any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.