acceptodds
Withdrawn from ICLR 2027

ForensicZip: More Tokens are Better but Not Necessary in Forensic Vision-Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal Large Language Models (MLLMs) enable interpretable multimedia forensics by generating textual rationales for forgery detection. However, processing dense visual sequences incurs high computational costs, particularly for high-resolution images and videos. Visual token pruning is a practical acceleration strategy, yet existing methods are largely semantic-driven, retaining salient objects while discarding background regions where manipulation traces such as high-frequency anomalies and temporal jitters often reside. To address this issue, we introduce ForensicZip, a training-free framework that reformulates token compression from a forgery-driven perspective. ForensicZip models temporal token evolution as a Birth-Death Optimal Transport problem with a slack dummy node, quantifying physical discontinuities indicating transient generative artifacts. The forensic scoring further integrates transport-based novelty with high-frequency priors to separate forensic evidence from semantic content under large-ratio compression. Experiments on deepfake and AIGC benchmarks show that at 10% token retention, ForensicZip achieves speedup and over 90% FLOPs reduction while maintaining state-of-the-art detection performance. Code is available in the appendix and will be released on GitHub.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Deepfake

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XjTUOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

ForensicZip is a training-free visual token selection method designed for forensic MLLMs. For videos, tokens are scored based on a novelty metric derived from optimal transport—specifically, the "birth/death" dynamics between adjacent frames—while for single images, the score is based on the distance between a token and the average embedding. These scores are further weighted by a high-frequency prior, and the top-K tokens are retained for input into the LLM. The paper reports that at a 10% retention rate, accuracy drops only slightly (by 0.9–2.8 percentage points across eight settings) while achieving a 2.97× speedup.

Critical strengths
  1. The motivation is sound. Semantic token selectors often discard boundary and background patches—precisely where forgery traces typically reside—a point clearly illustrated in Fig. 1b.
  2. The method is computationally efficient, can be integrated into existing models, and has been evaluated on several forensic backbones.
Critical weaknesses
  1. It is unclear how these figures were derived. According to Sec. 3.3.1 and Algorithm 1, single-image processing does not utilize OT, and Table 19 lists "FakeClue" as the single-image baseline. However, OT hyperparameters clearly alter FakeClue's accuracy (Fig. 4; Table 11, 94.50–97.74). For SID-Set—also an image dataset—performance varies by up to 13 points across different OT variants (Table 5). Fig. 16(a) even reports non-zero average birth evidence (approximately 0.05–0.09) across all eight image datasets. This contradicts the condition b=0b=0 when T=1T=1 and exceeds the paper's own upper bound of bj≤1/Nb_j \le 1/N (Prop. A.4). The only video-related results appear in Table 8 (video-level AUC on CDF-v2/DFDC/DFD), yet the table fails to specify the number of frames used, whether OT was enabled, or include a pruning baseline. Consequently, the core module has never been compared against baselines in the setting for which it was designed.

  2. Efficiency comparisons are unfair. VisionTrim operates using masking (Table 23), whereas the baselines show almost no reduction in VRAM usage (Table 1); thus, the disparity appears largely attributable to implementation differences. ForensicZip's own VRAM reduction (dropping from 26.7 GB to 19.8 GB at a 25% ratio; Table 2) is difficult to explain solely by the ~0.3 GB KV cache. Similarly, LLaVA-Scissor performs pruning prior to LLM input but yields no VRAM savings in Table 1. VisionZip appears only in the figures.

  3. Some figures are inconsistent. At a 50% retention rate, the method outperforms the vanilla baseline by exactly +0.05 on six of the eight datasets (Table 1). The "Avg. Drop" column in Table 14 and the "Ret." column in Table 17 do not match the values ​​in the respective table cells. Variance figures were not reported at all.

Questions for authors
  1. For a single image, how are the OT hyperparameters incorporated into the score? If they aren't, how should Fig. 4 and Tables 5, 10, and 11 be interpreted?

  2. Could you have the baseline perform actual token dropping before entering the LLM, and then re-run the comparison using the same timing script?

  3. Were the results in Table 8 obtained using multi-frame input with the OT pathway enabled? How many frames were used?

Reviewer_ajtLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ForensicZip, a training-free framework that accelerates MLLM-based AIGC detection through visual token compression. It combines spatial novelty for images or Birth–Death optimal transport across video frames with a Laplacian high-frequency prior to prioritize forensic evidence over semantic saliency. At 10% target token retention, ForensicZip achieves 97.74% accuracy on FakeClue. Across eight backbone–dataset settings, the paper reports a 2.97× speedup based on mean end-to-end latency and an 85.4% reduction in mean FLOPs.

Critical strengths
  1. The paper is generally well written and easy to follow.
  2. The results are promising 2.97 speedup only at minor performance decline.
Critical weaknesses
  1. The motivation and task-specific design are unclear. Why is optimal transport particularly suitable for accelerating forensic VLMs? Does the proposed method incorporate any design tailored to forensic cues, or could it be applied equally well to general-purpose VLMs? As presented, the approach appears to combine visual token compression with forensic detection without sufficiently explaining or demonstrating the adaptations needed for this task.

  2. The evaluation provides limited evidence of out-of-distribution generalization. The benchmarks used to evaluate FakeVLM and FakeShield appear closely aligned with their training distributions. However, performance on in-the-wild benchmarks is more relevant to practical deployment. The authors should include evaluations on benchmarks such as Chameleon to assess whether the reported benefits extend to out-of-distribution scenarios.

Questions for authors
  1. Which components of the optimal transport formulation are specifically tailored to forensic cues, and what evidence shows that they preserve these cues better than generic visual token compression methods?

  2. Can the authors provide out-of-distribution results on an in-the-wild benchmark such as Chameleon to establish whether the reported accuracy–efficiency trade-off holds beyond the training distributions?

Reviewer_QCUCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This submission studies visual token compression for forensic multimodal language models. It argues that general-purpose pruning ranks tokens by semantic salience while forgery evidence tends to live in semantically flat regions, so semantic pruning removes the cues a forensic model relies on. The proposed ForensicZip is a training-free selector that scores patch tokens by temporal novelty, obtained from an entropic optimal transport plan augmented with a dummy node that absorbs unmatched mass between adjacent frames, and by a Laplacian high-frequency prior used as a multiplicative gate. Single images instead use embedding deviation from the frame mean, and the selected tokens are physically removed before the language model. Experiments across forensic benchmarks and several backbones report high accuracy retention at aggressive retention ratios together with lower end-to-end latency, FLOPs and KV cache.

Critical strengths
  • Priority maps and an evidence-recall curve show that semantic pruning removes the low-salience regions where forgery traces reside, which grounds ForensicZip's design in a task-specific analysis.
  • ForensicZip is a simple, training-free plugin, and its benefits are measured end to end in latency, FLOPs, memory and KV cache.
Critical weaknesses
  • The video transport is the headline mechanism, yet no per-benchmark modality or sequence dimensions are reported and the headline accuracy comes from a benchmark the paper calls single-image, so a video-only evaluation is needed to show what the transport contributes.
  • Interpretability motivates the work, yet the evaluation only parses free-form responses into binary predictions, so an explanation-quality or localization metric is needed to support that claim.
Questions for authors
  • Beyond detection accuracy, should the evaluation also assess the quality of the generated textual explanations?
  • The main experiments appear to be image-only; should video forgery benchmarks also be evaluated?
  • Since LLaVA-OneVision is listed as a primary backbone, should experiments with it also be added?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.