- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
ForensicZip is a training-free visual token selection method designed for forensic MLLMs. For videos, tokens are scored based on a novelty metric derived from optimal transport—specifically, the "birth/death" dynamics between adjacent frames—while for single images, the score is based on the distance between a token and the average embedding. These scores are further weighted by a high-frequency prior, and the top-K tokens are retained for input into the LLM. The paper reports that at a 10% retention rate, accuracy drops only slightly (by 0.9–2.8 percentage points across eight settings) while achieving a 2.97× speedup.
Critical strengths
- The motivation is sound. Semantic token selectors often discard boundary and background patches—precisely where forgery traces typically reside—a point clearly illustrated in Fig. 1b.
- The method is computationally efficient, can be integrated into existing models, and has been evaluated on several forensic backbones.
Critical weaknesses
-
It is unclear how these figures were derived. According to Sec. 3.3.1 and Algorithm 1, single-image processing does not utilize OT, and Table 19 lists "FakeClue" as the single-image baseline. However, OT hyperparameters clearly alter FakeClue's accuracy (Fig. 4; Table 11, 94.50–97.74). For SID-Set—also an image dataset—performance varies by up to 13 points across different OT variants (Table 5). Fig. 16(a) even reports non-zero average birth evidence (approximately 0.05–0.09) across all eight image datasets. This contradicts the condition when and exceeds the paper's own upper bound of (Prop. A.4). The only video-related results appear in Table 8 (video-level AUC on CDF-v2/DFDC/DFD), yet the table fails to specify the number of frames used, whether OT was enabled, or include a pruning baseline. Consequently, the core module has never been compared against baselines in the setting for which it was designed.
-
Efficiency comparisons are unfair. VisionTrim operates using masking (Table 23), whereas the baselines show almost no reduction in VRAM usage (Table 1); thus, the disparity appears largely attributable to implementation differences. ForensicZip's own VRAM reduction (dropping from 26.7 GB to 19.8 GB at a 25% ratio; Table 2) is difficult to explain solely by the ~0.3 GB KV cache. Similarly, LLaVA-Scissor performs pruning prior to LLM input but yields no VRAM savings in Table 1. VisionZip appears only in the figures.
-
Some figures are inconsistent. At a 50% retention rate, the method outperforms the vanilla baseline by exactly +0.05 on six of the eight datasets (Table 1). The "Avg. Drop" column in Table 14 and the "Ret." column in Table 17 do not match the values in the respective table cells. Variance figures were not reported at all.
Questions for authors
-
For a single image, how are the OT hyperparameters incorporated into the score? If they aren't, how should Fig. 4 and Tables 5, 10, and 11 be interpreted?
-
Could you have the baseline perform actual token dropping before entering the LLM, and then re-run the comparison using the same timing script?
-
Were the results in Table 8 obtained using multi-frame input with the OT pathway enabled? How many frames were used?