acceptodds
Withdrawn from ICLR 2027

Where to Verify, How to Correct: Selective Semantic Verification for Weakly Supervised Video Anomaly Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Weakly Supervised Video Anomaly Detection (WSVAD) aims to localize frame-level anomalies using only video-level supervision, yet visually dynamic normal activities can still receive high anomaly scores and cause false alarms. Language-guided verification provides complementary semantic evidence for reassessing such elevated responses. However, applying semantic verification to every snippet is costly for long videos and does not necessarily improve anomaly localization. We propose Semantic-Temporal Reprojection (STR), a training-free framework for selective semantic verification of frozen WSVAD detectors. STR retains dense baseline scoring and applies semantic verification to selected elevated responses. Adaptive temporal anchor mining allocates queries across distinct temporal contexts. Structured semantic verification produces a bounded semantic score, while a video-adaptive perception veto limits corrections at extreme baseline responses. Local Gaussian reprojection constrains the remaining corrections to nearby temporal locations. The resulting one-sided correction operates on candidates exposed by the frozen detector and retains or attenuates existing anomaly scores. Experiments on UCF-Crime, XD-Violence, and MSAD show consistent improvements across multiple frozen WSVAD baselines, achieving 90.80% AUC, 89.08% AP, and 88.40% AUC, respectively. Under matched query budgets, adaptive mining outperforms uniform, random, and score-based Top-K selection. Compared with dense verification, STR uses far fewer semantic queries and achieves higher detection performance with lower verification-stage runtime. Code is available at https://anonymous.4open.science/r/STR.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Weakly Supervised Video Anomaly Detection, Selective Semantic Verification, Vision-Language Models, Temporal Localization, Training-Free Inference

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_YmtfOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents STR, a training‑free post‑processing framework for weakly‑supervised video anomaly detection. It performs selective semantic verification only on high‑score candidate snippets using multimodal large models and applies local score correction via Gaussian reprojection. The framework works on top of pre‑trained detectors, reduces semantic query overhead, and yields consistent gains across three benchmarks. However, the novelty is limited, several experimental limitations exist, the applicability boundary is insufficiently discussed, and key comparisons are incomplete.

Critical strengths
  1. The whole pipeline functions as inference‑time post‑processing. It requires no weight updates and can be directly applied to multiple frozen WSVAD baselines.

  2. Comprehensive ablation studies validate individual components, including anchor mining, semantic verification, perception veto, and Gaussian reprojection. Robustness across different vision‑language models is also examined.

Critical weaknesses
  1. The core idea is selecting high‑score segments for LLM‑based verification, followed by local score suppression. The overall novelty is limited. This work is purely a test‑time post‑processing scheme instead of a fundamental video anomaly detection architecture.

  2. The proposed approach fully depends on baseline anomaly scores. True anomalies assigned low scores by the original detector will never be selected as anchors and receive no semantic correction. This limitation is mentioned without any mitigation attempts.

  3. Comparisons against other training‑free VAD approaches are inadequate. Only overall numbers are listed in the main tables, lacking controlled experiments for LLM‑driven post‑processing competitors

  4. The perception veto protects a small fraction of top‑scoring positions within each video. Some false positives produced by the baseline can remain untouched. Only parameter sensitivity is provided, with little analysis of this practical drawback.

  5. The Gaussian reprojection width is manually fixed. Without adapting to video frame‑rate or event duration, performance may become unstable for anomalies of varying lengths.

  6. One unified set of hyperparameters is used across all datasets. Although basic sensitivity checks are included, deeper discussion about performance variation with dataset‑specific hyper‑parameter tuning is missing.

Questions for authors

see the above weakness

Reviewer_T4wTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

A training-free plug-and-play model for any frozen WSVAD detector. To avoid the expensive costs of running a vision-language model on every snippet of every video, it picks a few high-scoring candidate moments, has a VLM and an LLM judge whether they are truly anomalous, and lowers the detector’s scores around candidates judged normal.

Contributions:

A training-free framework for selective verification that controls both where semantic checks are spent and how scores are corrected. Adaptive anchor mining, corrections based on a structured verification protocol, a per-video veto threshold and local Gaussian reprojection. Consistent gains across benchmarks, detectors, features and VLMs, with a good set of ablations.

Evaluation:

Data: UCF-Crime and MSAD (AUC), XD-Violence (AP). Detectors: Applied to frozen DeSC, VadCLIP, DMU, RTFM and MGFN. Comparison and transfer: SOTA comparison, transfer across detectors and across VLMs. Budget and cost: Matched-budget comparison of anchor-selection strategies. Dense versus selective verification in accuracy and runtime. Ablations: Components, aggregation, veto and Gaussian width, boundary rules, shuffled-score control, recall at 5% false-positive rate.

Critical strengths

The paper is well written and the figures are mostly informative and clear. The idea is interesting and useful, and the experiments showing that it translates to different frozen baseline VAD models are well designed.

Critical weaknesses

The main weakness of the paper is the experimental section. The authors compare their proposed STR, applied to DeSC, and show that it achieves sota on UCF-Crime and XD-Violence, while they show sota on MSAD with STR applied to RTFM. Then, they compare the gains due to STR on different frozen VAD models and datasets in Table 3. However, the more convincing comparison for STR would be against other TTA methods, which are currently underexplored in VAD. A further, interesting evaluation would be to find shared charateristics of the frozen VAD models on which applying STR results in bigger or smaller gains.
Another interesting evaluation would be an ablation over the language part of the method. Currently, the authors have only shown the results by using Llama 3.1-8B with a fixed prompt that lists examples of normal and anomalous behaviours. This has two issues:

  • How important are the LLM capabilities to generate the refined score? Does STR cope with smaller, less perfoming LLMs such as the quantized ones that could more likely be deployed on edge devices, where the VAD models (should) run?
  • How does STR behave when examples of normal and anomalous behaviours are not provided to the LLM? In the prompt presented in Appendix B.2, the examples overlap with classes of UCF-Crime and XD-Violence, which could leak some information that, under the WSVAD paradigm, is not available.

The paper would also benefit from exploring further the mechanistic reasons for why such approach works. For example, Table 9 shows that shuffling semantic scores results in STR slightly underperfoming compared to the full STR, which naturally leads to investigate what leads to such low sensitivity, if it depends on the selection of the anchors, etc...

Questions for authors

Intuitively, using mean and standard deviation of the cosine similarity of snippets in a video in Eq. 3 implies that the threshold that produces the local segments depends on the length of the video. For example, a video with some anomalous snippets and mostly static content would result in mean and standard deviation very close to the mean and standard deviation of the static-only content. However, trimming static snippets would progressively change that, resulting in mean and standard deviation getting closer to the ones of the anomaly-only snippets. Therefore, the local segments C\mathcal{C} are dependant on the length and the "staticity" of the video.

  • Is my understanding of this mechanism correct?
  • Can the authors provide an ablation or qualitative result of the behavior of the adaptive temporal anchor mining when the amount of normal snippets in anomalous videos change?
Reviewer_TgTUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Semantic-Temporal Reprojection (STR), a training-free framework for improving weakly supervised video anomaly detection through selective semantic verification. The method first identifies candidate temporal locations using feature variations and baseline anomaly scores. It then employs vision-language and language models to assess the semantic evidence of anomalies at selected locations. Finally, a perception veto mechanism and Gaussian-based local correction are used to refine the original anomaly scores. Experiments on three benchmarks demonstrate improvements across multiple frozen baseline detectors.

Critical strengths
  1. The paper addresses a practical problem in weakly supervised video anomaly detection: visually dynamic normal activities may produce high anomaly scores and lead to false alarms. Introducing selective semantic verification to refine existing detection results is a reasonable and potentially useful direction.
  2. The authors conduct experiments on three benchmark datasets with multiple baseline detectors. The evaluation includes comparisons with existing methods, cross-baseline transfer, query efficiency analysis, ablation studies, and qualitative results, providing empirical support for the proposed framework.
Critical weaknesses
  1. Although the paper addresses a meaningful problem, the Introduction does not present the research motivation and core challenges in a sufficiently clear and concise manner. In particular, the paragraphs beginning with "These considerations motivate selective semantic verification" and "To address this problem" contain overlapping descriptions of the proposed approach. The distinction between the limitations of existing methods, the specific problem to be addressed, and the proposed solution is not clearly established. I suggest reorganizing the Introduction to better highlight the research motivation, technical challenges, and main contributions while reducing repetitive explanations.
  2. The overall pipeline in Figure 2(a) does not clearly correspond to the detailed illustrations in Figures 2(b)–(d). For example, adaptive anchor mining relies on both temporal features and baseline anomaly scores, but the corresponding feature dependency is not explicitly represented in the main pipeline. Moreover, the figure contains numerous icons and graphical elements, while the key technical operations and information flow remain difficult to follow. I suggest improving the consistency between the overall pipeline and its submodules, and emphasizing the actual technical mechanisms rather than decorative elements.
  3. According to Eq. (9), the perception veto sets the correction strength to zero for anchors whose baseline scores exceed the adaptive threshold. However, Eq. (10) propagates Gaussian corrections from other anchors to neighboring temporal locations. Consequently, a high-confidence anomaly location protected by the veto may still be attenuated by the Gaussian correction originating from a nearby unprotected anchor. How does the proposed mechanism ensure that high-confidence anomaly responses are preserved after Gaussian reprojection? Further clarification or experimental analysis would help establish the effectiveness of this protection mechanism.
  4. The proposed framework uses Qwen3-VL-8B to generate visual descriptions and Llama-3.1-8B-Instruct to produce structured anomaly scores. However, the paper does not clearly explain why this two-stage design is preferable to directly generating anomaly scores with a single vision-language model. Since modern VLMs can perform both visual understanding and structured reasoning, it would be helpful to include a comparison between the proposed two-model pipeline and a single-VLM verification approach.
Questions for authors

Please see the "Weaknesses".

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.