acceptodds
Withdrawn from ICLR 2027

Just Imagine Real-World Anomalies! Class-Specialized Vision-Free Training for Video Anomaly Detection and Understanding

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Real-world anomalies are difficult to collect: they are rare, unsafe, and often occur in privacy-sensitive environments, yet most video anomaly detection (VAD) methods require training videos. We ask: can a detector recognize anomalies it has only imagined through language? Building on evidence that text-trained detectors can transfer to video, we introduce NOVAD, a video-free framework for class-specialized anomaly detection and understanding. An LLM imagines normal and anomalous events as narratives ordered into beginning, following, climax, and resolution, with controlled variation over locations, objects, and participant counts. Encoded with a frozen vision–language text encoder, these narratives train a gated mixture of category-specialized experts, and their known temporal structure provides pseudo-labels for localization. At inference, the paired image encoder projects video frames into the same space. The predicted scores then guide multimodal event segmentation, and a video-language model explains only the resulting segments. Experiments on UCF-Crime, XD-Violence, and MSAD show that language-imagined events enable real-world anomaly localization without domain videos during detector training. On Vad-R1, detection-guided segments improve zero-shot reasoning over the untrimmed video, and a model trained with supervised fine-tuning only surpasses the full Vad-R1 model when fed with these segments.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Anomaly Detection, Video Anomaly Understanding, Text-Driven, Vision Free, Computer Vision, Synthetic Dataset

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ESEWOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work proposes NOVAD, a video‑free framework for video anomaly detection. It trains category‑specialized expert models solely on structured textual narratives generated by an LLM. A detector‑guided segmentation pipeline invokes video‑language models only on candidate abnormal segments to produce anomaly explanations. The strengths lie in the end‑to‑end video‑free training pipeline, structured event prompting and comprehensive empirical evaluations. Major weaknesses include persistent text‑video modality gap, uneven cross‑dataset generalization, limited architectural novelty and non‑negligible performance drops in certain settings.

Critical strengths
  1. The anomaly detector does not require real target‑domain training videos, which tackles the practical difficulty of collecting and annotating rare real‑world anomaly footage.

  2. Events are structured into four sequential phases: beginning, following, climax and resolution. Controlled sampling of locations, participants and objects diversifies synthetic textual training material.

Critical weaknesses
  1. The novelty of this paper is limited. The mixture‑of‑experts backbone is directly adapted from GS‑MoE, and few original insights are contributed to core model design.

  2. The proposed NOVAD underperforms TD‑VAD on several anomaly classes such as explosion and arson. Only shallow analysis is offered and no actionable fixes are proposed for these failure scenarios.

  3. The proposed NOVAD outperforms TD‑VAD but still falls behind real‑video‑trained baselines, and no practical mitigation strategy is provided in the paper.

  4. Qualitative outputs reveal obvious unsupported contextual hallucinations from the VLLM module, and the paper does not present any approach to ease this hallucination issue.

  5. The reproduced baseline of TD-VAD's result deviates from the originally reported value, and this circumstance weakens the reliability of comparative experiments.

  6. Cross‑dataset generalization delivers inconsistent performance, and the results degrade heavily under certain transfer setups, and the authors do not discuss any concrete improvement plans. Knowledge transfer from UCF‑Crime to XD‑Violence works well while the reverse direction yields poor results, so could you further explain the underlying root causes?

Questions for authors

see the above weakness

Reviewer_vhbzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents NOVAD, a video anomaly detector trained solely on LLM-generated narratives. Contextual variations and four-stage narratives provide semantic diversity and temporal pseudo-supervision. Category-specific experts transfer from text to video through a frozen vision–language model. A subsequent detector-guided VLLM stage scores and explains events. Evaluation covers detection on UCF-Crime, XD-Violence, and MSAD, and understanding on Vad-R1.

Critical strengths
  1. Practical and clearly qualified setting. Avoiding target-domain training videos addresses collection and privacy constraints. The paper appropriately describes its setting as ontology-supervised and closed-set.

  2. Effective contextual diversification. Table 4 shows that introducing contextual factors improves UCF-Crime AUC from 76.59 to 82.78, supporting the value of controlled narrative generation.

  3. Useful supplementary analysis. The trained detector outperforms a prototype-similarity baseline across three datasets. The paper also acknowledges Phase C’s trade-off between improved abnormal-video localization and reduced overall detection AUC.

Critical weaknesses
  1. Category specialization is insufficiently isolated. Without shared-detector or capacity-matched ensemble baselines, improvements cannot be clearly attributed to category-specific experts rather than increased capacity.
  2. Temporal supervision relies on a restrictive assumption. Fixing the anomaly peak at the start of the Climax stage may encourage artificial positional patterns. Ablations are needed to establish the transferability of this supervision.
Questions for authors
  • How does NOVAD compare with a shared detector or capacity-matched ensemble?
  • How does performance change when anomaly-peak positions vary or the TGS loss is removed?
  • What gains remain without detector guidance under an otherwise matched VLLM pipeline? Please also report total inference costs, including VLLM scoring.
Reviewer_LZiLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes NOVAD, a video anomaly detection framework trained entirely on LLM-generated textual descriptions without using real videos during detector training. The method generates temporally structured event narratives with diverse contexts, trains category-specific experts with temporal pseudo-supervision, and uses detection-guided video segments for VLM-based anomaly understanding. Experiments on multiple benchmarks show consistent improvements over the text-supervised baseline TD-VAD.

Critical strengths
  • The paper addresses a practical problem: learning video anomaly detection without access to real training videos, which can be difficult to collect due to privacy and safety concerns.

  • The method achieves consistent improvements over TD-VAD across three benchmarks. The cross-dataset evaluation further supports its generalization capability.

  • Using detected anomaly segments rather than full videos for VLM reasoning is a reasonable design choice, and the improvements on Vad-R1 demonstrate its effectiveness.

Critical weaknesses
  • The technical novelty is relatively limited. The framework largely combines existing ideas from TD-VAD, GS-MoE, and EventVAD. While the integration is reasonable, the paper does not introduce a substantially new learning paradigm or architecture.

  • The temporal pseudo-supervision relies on the assumption that anomalies peak at the climax of LLM-generated narratives. This is a rather strong heuristic, and it is unclear how well such artificial temporal structures correspond to real video anomalies.

  • The ablation studies are insufficient to isolate the contributions of the main components. In particular, the paper does not clearly establish how much improvement comes from richer text generation, category-specific experts, or temporal pseudo-supervision.

  • The overall pipeline is fairly complicated, particularly the anomaly understanding stage, which involves multiple feature extractors, multimodal segmentation, and VLM inference. The additional complexity is not fully justified by the reported improvements.

Questions for authors
  • How much of the performance gain comes from the proposed training strategy versus the stronger category-specific MoE architecture? A controlled comparison using the same backbone and training data would help clarify this.

  • How sensitive is the method to the temporal pseudo-label design? Would simpler alternatives, such as uniform labels or different Gaussian centers and widths, achieve comparable results?

  • Is the four-stage narrative structure actually necessary? It would be useful to compare against unstructured descriptions with matched text diversity and training data size.

  • What is the computational overhead of the complete pipeline, especially Phase C? Given that multimodal segmentation reduces the overall detection AUC on UCF-Crime, it would be helpful to better understand the trade-off between complexity, detection accuracy, and reasoning quality.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.