- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper addresses video anomaly understanding with MLLMs under a fixed visual-token budget. It keeps all temporal units and compresses only spatial redundancy within each unit, and introduces two metrics for interval coverage and dispersion. The core idea "never sacrificing temporal resolution" is sound and the module is lightweight, but the method is an incremental combination of existing token-merging and Mamba aggregation, and the experiments do not support the localization claims.
Critical strengths
- The efficiency problem is real, and the temporal-coverage vs. spatial-detail trade-off is a sensible framing of it.
- This paper points out that for generative video anomaly understanding, models suffer from interval collapse in their predicted anomaly temporal intervals
Critical weaknesses
-
This paper shows limited innovation and mainly assembles existing techniques. Specifically, AnomMerge repurposes established modules including STAC‑derived Mamba aggregation, token-merging-style spatial merging, statistical‑driven MLP thresholds, and the straight‑through estimator. It adapts these off‑the‑shelf components for VAU via engineering tweaks rather than proposing fundamentally new algorithmic or modeling paradigms.
-
The paper repeatedly claims temporal merging corrupts localization evidence. Yet it provides no quantitative statistics identifying which anomaly categories fail under temporal merging baselines, such as short‑duration anomalies or spatially tiny anomalies. Sample‑level statistical support is absent.
-
The same configuration gives different numbers in different tables. For example, Table 1(a) ("no merging", 100% retention, Qwen2.5-VL-3B, 1 FPS) reports Acc 0.938 / F1 0.933, while Table 6's "Qwen2.5-VL-3B (SFT), 1 FPS, 100%" reports Acc 0.879 / F1 0.856. The 0.059–0.077 gap is unexplained, and which row is the true uncompressed reference.
-
IDC is not a localization metric, yet it carries the headline result in this paper. It compares dataset-level endpoint dispersion, so it is silent on whether any individual interval is correct. In Table 6, models with mIoU 0.253 and 0.270 have IDC = 0.000. It is also computed over a model-dependent subset V, so abstaining shifts it, and as a single scalar it cannot support a significance test.
-
This paper lacks sufficient ablation for \lambda involved in CA‑tIoU. It does not analyse metric behaviours under different \lambda values or discuss \lambda’s generalisability across datasets and videos with varying anomaly ratios. Correlation between CA‑tIoU scores and human annotations remains unexamined. It is unclear whether CA‑tIoU aligns with human‑perceived localisation quality.
-
IDC measures distributional shrinkage of predicted interval endpoints and only captures dataset‑level statistics rather than direct localisation accuracy. Though the paper acknowledges this limitation, reduced IDC is heavily promoted as a key result. IDC can yield favourable values even for fully incorrect predictions.
-
The proposed framework heavily relies on Qwen‑VL backbones. Experiments are conducted only on Qwen‑family models. No transfer evaluations are performed on other mainstream multimodal large models such as Intern‑VL and LLaVA‑NeXT‑Video. This makes it difficult to demonstrate the broad generalisability of the compression framework.
Questions for authors
- The experimental design has notable flaws with unfair and insufficient comparisons. For example,
- Baselines like FastVID, VidCom2 and FlashVID are training‑free, while AnomMerge requires joint fine‑tuning of context modules, threshold heads and LoRA adapters. Direct comparison mixes trainable and training‑free setups, leaving performance gains ambiguous. Control experiments with LoRA applied to baselines are missing.
- The other question is that AnomMerge degrades heavily on real‑world surveillance subsets of VANE‑Bench versus full‑token VAU‑R1. The authors only blame hard surveillance scenes, without diagnosing whether spatial merging destroys small anomaly cues or adaptive thresholds fail. Error and failure‑case analyses are absent.
- The temporal‑merging ablation uses merely one hyperparameter configuration. It cannot prove temporal merging is universally worse than intra‑frame spatial merging. Temporal merging may deliver competitive results under suitable token budgets.
- See the above weakness of other questions.
5 comments
Response to Reviewer C297 (2)
W7. Transfer across model families
Thank you for highlighting this limitation. We added InternVL3.5-2B and GLM-4.1V-9B-Thinking to Table 5, providing additional evidence of AnomMerge's generalizability across model families.
Table 5. AnomMerge across model sizes and backbone families.
| Backbone | Size | Acc ↑ | F1 ↑ | CA-tIoU ↑ | IDC ↓ | mIoU ↑ |
|---|---|---|---|---|---|---|
| Qwen2.5-VL | 3B | 0.932 | 0.925 | 0.383 | 0.142 | 0.722 |
| Qwen2.5-VL | 7B | 0.941 | 0.935 | 0.420 | 0.118 | 0.748 |
| Qwen3-VL | 2B | 0.916 | 0.905 | 0.426 | 0.056 | 0.746 |
| Qwen3.5 | 2B | 0.932 | 0.922 | 0.431 | 0.117 | 0.765 |
| InternVL3.5 | 2B | 0.927 | 0.918 | 0.404 | 0.099 | 0.745 |
| GLM-4.1V-Thinking | 9B | 0.948 | 0.942 | 0.427 | 0.098 | 0.760 |
To accommodate InternVL's frame count within our experimental context budget, we use a 108-frame cap, compared with 768 for Qwen and GLM, with uniform sampling across the full video when the cap is exceeded.
Q1.1. Adaptation fairness
Thank you for pointing this out. FastVID, VidCom2, FlashVID and OTT-Vid are applied to Qwen2.5-VL-3B fine-tuned with LoRA, using the same data, 1 FPS sampling and LoRA settings as AnomMerge. Here, "training-free" describes the compression algorithms; the model has already been adapted to VAU. AnomMerge learns compression during task fine-tuning, and the results in Table 7 suggest that this joint adaptation can bring further gains.
Q1.2. VANE failures
Thank you for requesting a diagnosis of the Avenue/Ped1 performance drop. We evaluated AnomMerge and the independently trained no-merge model from Table 1(a) on the same 21 clips (63 questions), with Mamba and LoRA retained in both models. We compared accuracy and changes in answer correctness by activity (Table 15; Appendix D.1).
Table 15. Analysis by activity category in the Avenue and Ped1 subsets of VANE-Bench.
| Source | Activity | Count | VAD-R1 (Acc %) | VAU-R1 (Acc %) | Merge (Acc %) | No merge (Acc %) | Recovered | Lost |
|---|---|---|---|---|---|---|---|---|
| Avenue | Bag throwing | 18 | 100.00 | 94.44 | 72.22 | 88.89 | 4 | 1 |
| Avenue | Paper throwing | 9 | 77.78 | 88.89 | 77.78 | 77.78 | 0 | 0 |
| Avenue | Dancing | 6 | 16.67 | 50.00 | 33.33 | 33.33 | 0 | 0 |
| Ped1 | Skateboarding | 9 | 55.56 | 66.67 | 33.33 | 33.33 | 0 | 0 |
| Ped1 | Wheelchair use | 6 | 33.33 | 33.33 | 33.33 | 66.67 | 3 | 1 |
| Ped1 | Bicycles/carts | 15 | 80.00 | 86.67 | 80.00 | 86.67 | 1 | 0 |
Recovered counts AnomMerge errors corrected by the no-merge model; Lost counts AnomMerge-correct questions answered incorrectly by the no-merge model.
The no-merge model improves accuracy on bag throwing from 72.22% to 88.89% and wheelchair use from 33.33% to 66.67%. However, dancing and skateboarding remain at 33.33%, so merging does not explain all failures.
Case analysis. Figure 10 illustrates failures involving fine-grained posture and object-interaction cues in low-resolution surveillance frames. Figure 11 shows recovered bag-throwing and wheelchair answers, consistent with sensitivity to spatial compression. The no-merge model was trained independently, so this comparison does not isolate the merging operation from adaptive-threshold effects.
Q1.3. Temporal merging budgets
Table 2 reports the comparison with manually adjusted k and approximately matched retention. At higher budgets, temporal merging has slightly higher CA-tIoU, while spatial merging generally favors recognition; recognition is similar around 62% retention. Neither method improves monotonically with more tokens. As discussed in Reviewer HQ4S Q4, our learned spatial thresholds achieve higher F1 with fewer tokens than the tested fixed-k settings.
Thank you again for your valuable questions and comments. We have added detailed experiments and made corresponding revisions to address your concerns. As the discussion period is nearing its end, we would be happy to provide further clarification if any concerns remain.
Response to Reviewer C297
W1. Contribution
As discussed in General response G1, we identify interval collapse and introduce CA-tIoU/IDC to complement overlap metrics. Reviewer HQ4S also recognizes the importance of balancing temporal coverage and spatial detail. We address this issue with learned thresholds and constrained spatial grouping, preserving sampled temporal units and their positions while reducing spatial redundancy.
Mamba, LoRA and surrogate gradients are established tools. With similar retention, our merging achieves F1 0.8455 versus 0.8274 for pooling without Mamba.
W2. Temporal merging
Thank you for this suggestion. We added category, duration and case analyses of temporal versus spatial merging in Appendix C.3.
Table 14. Recognition and localization by anomaly category.
For the tested configurations, Table 14 shows higher CA-tIoU and recall with spatial merging for traffic accidents, shooting, wrong-way driving and riot. Improvements cover both short and long events, but short falling-object events have lower CA-tIoU. Thus, the results show no consistent duration pattern. The annotations lack spatial anomaly regions, so we cannot quantify performance by spatial anomaly size.
Case analysis. Figure 8 shows that temporal merging misses the four-second event in RoadAccidents091 and predicts an overly broad interval in ECVA/713; spatial merging produces an interval closer to GT in the latter case.
W3. Reference rows
Table 1(a) is the uncompressed reference for AnomMerge's merging ablation: it retains spatiotemporal Mamba and LoRA with all merging disabled. The SFT row in Table 7 is the plain LoRA-finetuned backbone without Mamba. Both retain all visual tokens. We have clarified this SFT configuration in blue in the revised paper.
The reported gains of 0.059 in Acc and 0.077 in F1 therefore come from adding spatiotemporal Mamba during task fine-tuning.
W4. IDC subsets and uncertainty
Thank you for these comments. General response G3 clarifies IDC's role in diagnosing interval collapse.
(1) In Table 7, IDC=0 indicates no endpoint-dispersion contraction; varied but inaccurate boundaries can still yield low mIoU.
(2) Model-dependent subsets affect comparisons, but IDC still diagnoses collapse within each evaluated subset. A single IDC value does not establish statistical significance.
W5. CA-tIoU penalty sensitivity
Thank you for this suggestion. We added sensitivity experiments at λ=0, 1, 5, 10 (Appendix C.2). By source, Vad-Reasoning comprises UCF-Crime, XD-Violence, TAD, ShanghaiTech, UBnormal, ECVA and additional Internet videos (NEW).
(1) Across sources, increasing λ changes the AnomMerge/VAD-R1 ranking on UCF-Crime, XD-Violence and ECVA, while AnomMerge remains higher on TAD and NEW. Thus, the effect of penalizing excess coverage varies across sources.
(2) Across anomaly fractions, VAD-R1 shows a larger proportional score decline at low occupancy. AnomMerge remains higher in the low- and middle-occupancy groups, while the high-occupancy ranking reverses at λ=10. We retain λ=1 as an interpretable unit-weight penalty and report mIoU on abnormal videos alongside CA-tIoU.
(3) Regarding human annotations, CA-tIoU directly compares predictions with reference temporal boundaries through overlap and excess coverage. This assesses agreement with annotated intervals; correlation with independent human ratings of localization quality has not been measured.
W6. IDC scope
Thank you for highlighting this distinction. We will report IDC as a supplementary diagnostic alongside recognition and localization metrics, and will not use its reduction alone to claim improved VAU performance.