acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Open-World Video Anomaly Detection: Diagnosing Definition Blindness

Abstract

Open-world video anomaly detection (OWVAD) aims to localize anomalous events in videos according to a user-specified definition. An OWVAD detector should therefore use the supplied definition to decide what is anomalous. We call this behavior definition following, and its failure definition blindness. However, existing temporal evaluation protocols may obscure this distinction, making strong generic anomaly detection appear to reflect definition following. For example, Drift@5 and the customizable VAD protocol treat both normal frames and anomalies outside the current definition as negatives. Across the three benchmarks used in our study, normal frames account for 78.6%-96.4% of these negatives. As a result, the reported score can be driven largely by target-versus-normal separation rather than target-versus-other-anomaly separation. Consistent with this diagnosis, a LaGoVAD branch that never receives the anomaly definition nearly matches the full model under Drift@5. We therefore introduce three complementary definition-conditioned evaluations that separate definition following from generic anomaly detection. Applying them to VAD systems and general vision-language models on UCF-Crime, XD-Violence, and MSAD reveals that strong performance under existing evaluation can coexist with weak definition following. Motivated by these results, we introduce Definition-Contrastive Scoring (DeCoS), which reduces anomaly evidence shared across competing definitions and improves definition following across the three benchmarks. Our results show that generic anomaly detection and definition following are distinct capabilities and should be evaluated separately in OWVAD.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video anomaly detection, Open-world video anomaly detection, Anomaly detection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_bvVKOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents an in-depth investigation of Open-World Video Anomaly Detection (OWVAD), focusing on the issue of definition blindness. The authors provide a thorough analysis supported by compelling empirical evidence, revealing the limitations of existing evaluation protocols. Furthermore, they introduce three complementary evaluation metrics to assess definition following and propose a novel method, Definition-Contrastive Scoring (DeCoS), to mitigate definition blindness.

Critical strengths

This paper identifies and investigates an important yet potentially overlooked issue in Open-World Video Anomaly Detection. The problem of definition blindness is highly relevant to real-world applications, making this study both meaningful and practically valuable. The authors provide a thorough and systematic analysis, supported by theoretical derivations and convincing empirical evidence. Furthermore, the proposed evaluation metrics offer valuable insights into the limitations of existing evaluation protocols.

Critical weaknesses

I did not identify any critical weaknesses. However, I have two minor concerns:

  1. Semantic overlap between anomaly categories.

Both the proposed evaluation metrics and DeCoS largely rely on an important assumption that anomaly categories are mutually exclusive. However, in practice, some categories may share substantial semantic or visual similarities. For example, Assault and Arrest may both involve physical confrontation or restraint.

For the three proposed metrics, such overlap may lead to valid predictions being penalized when semantically related anomalies are treated as negatives or non-matching definitions. For DeCoS, the contrastive scoring mechanism may suppress evidence that is genuinely relevant to multiple definitions, while the single-label training objective may encourage unnecessary competition between related categories.

Nevertheless, I acknowledge that this may represent a broader challenge in open-world anomaly detection rather than a limitation specific to this work.

  1. Limited evaluation on natural multi-event videos.

Although ΔDC-Sel addresses an important and meaningful problem, its evaluation on only 43 natural multi-event videos seems insufficient for a comprehensive investigation. I understand that the lack of suitable benchmarks is a practical limitation, and I find the proposed metric reasonable. However, I feel that presenting this as a contribution may be somewhat premature, as it would benefit from a more thorough and systematic study.

Questions for authors

I have a question regarding the design of the proposed evaluation protocol. Why do the proposed metrics focus on single-category anomaly definitions, given that users may want to detect multiple anomaly categories simultaneously in practical scenarios? Even with single-category definitions, the evaluation already involves numerous combinations of target categories and input definitions. Extending the protocol to definitions containing two, three, or more anomaly categories would substantially increase the number of possible combinations, potentially making a comprehensive evaluation impractical. It would be helpful if the authors could discuss how the proposed evaluation framework could accommodate multi-category definitions while keeping the evaluation manageable.

I would also appreciate further clarification on how the three proposed metrics should be interpreted and used in practice, particularly DC-Disc and ΔDC-Det. Since these two metrics evaluate different aspects of definition following, a model may perform well on one but poorly on the other. How should such conflicting results be interpreted, and should one metric be prioritized over the other when assessing a model's overall definition-following capability? Some additional discussion or practical guidance would make the proposed evaluation framework more useful.

Reviewer_WpkwOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper argues that current open-world video anomaly detection (OWVAD) evaluations can reward models that localize generic anomalies without following the supplied text definition. It calls this failure "definition blindness," analyzes Drift@5, introduces DC-Disc, ΔDC-Det\Delta\mathrm{DC\text{-}Det}, and ΔDC-Sel\Delta\mathrm{DC\text{-}Sel}, and proposes Definition-Contrastive Scoring (DeCoS). DeCoS combines centered vision-language similarities, a temporal residual, and a frozen definition-agnostic anomaly gate.

Critical strengths
  1. The decomposition of Drift@5 clearly shows how normal negatives can dominate a target-versus-rest score. The query-independent LaGoVAD branch is also a useful counterexample: it retains much of the conventional performance without receiving the queried definition.

  2. The study covers three datasets and several model families, with semantic, ablation, and multi-event analyses. These experiments make the distinction between generic anomaly localization and definition-conditioned selection easy to understand.

Critical weaknesses
  1. The novelty and independent value of the three measures are not established. Query neglect is not specific to OWVAD; temporal grounding work such as QD-DETR has already studied models that underuse video--query relevance [Moon et al., 2023]. The proposed quantities also rely on familiar statistics: DC-Disc is anomaly-only, query-conditioned macro one-vs-rest AUROC/AP, while ΔDC-Det\Delta\mathrm{DC\text{-}Det} is a matched-minus-mismatched query margin.

    In the paper's symmetric two-event AUROC protocol, let

    Amatched=12[AUC⁡(sa;A,B)+AUC⁡(sb;B,A)].A_{\mathrm{matched}}=\frac{1}{2}\left[\operatorname{AUC}(s^a;A,B)+\operatorname{AUC}(s^b;B,A)\right].

    Then

    ΔDC-SelAUROC=2Amatched−1.\Delta\mathrm{DC\text{-}Sel}_{\mathrm{AUROC}}=2A_{\mathrm{matched}}-1.

    Thus, in this setting ΔDC-Sel\Delta\mathrm{DC\text{-}Sel} gives exactly the same model ordering as matched-query selection AUC. This equality does not generally extend to AP or arbitrary multi-event settings, but it applies to the main controlled experiment. Difference scores can also improve by reducing mismatched-query performance rather than improving matched-query detection. The paper does not compare with a full query--class matrix, macro one-vs-rest AUROC/AP, Top-1/MRR, Recall@KK, or temporal mAP@tIoU/HIT@1 [Lei et al., 2021], nor does it show ranking disagreements that would demonstrate unique information. The main contribution appears to be a useful query-swap protocol rather than three independent new metrics.

  2. The experiments do not support the broader open-world claim. DeCoS is trained with definition-selection loss and zero-sum centering, which is closely aligned with the proposed difference metrics. Its centered score is

    Mt,c=cos⁡(vt,ec)−1C∑k=1Ccos⁡(vt,ek),M_{t,c}=\cos(v_t,e_c)-\frac{1}{C}\sum_{k=1}^{C}\cos(v_t,e_k),

    so the score for a fixed video and query changes when other candidate definitions are added or removed. With a single anomaly candidate and no auxiliary competitor, this term is zero. The method therefore behaves as S(V,z;Z)S(V,z;\mathcal{Z}), although the task is introduced as scoring one arbitrary user definition.

    Moreover, the final output is

    St,c=gtQt,c,S_{t,c}=g_tQ_{t,c},

where gtg_t comes from a frozen definition-agnostic detector. If that gate misses an unknown event, DeCoS cannot recover it. The "unseen definition" experiment removes names from the task-specific vocabulary but still uses benchmark-grounded concepts; it tests semantic reassignment more directly than discovery of new anomalies. The main selection experiment also uses synthetic two-event splices, while the natural-video check contains only 43 videos.

[Moon et al., 2023] W. Moon et al. "Query-Dependent Video Representation for Moment Retrieval and Highlight Detection." CVPR 2023. https://openaccess.thecvf.com/content/CVPR2023/html/Moon_Query-Dependent_Video_Representation_for_Moment_Retrieval_and_Highlight_Detection_CVPR_2023_paper.html

[Lei et al., 2021] J. Lei et al. "Detecting Moments and Highlights in Videos via Natural Language Queries." NeurIPS 2021. https://arxiv.org/abs/2107.09609

Questions for authors
  1. What information do the three measures provide beyond standard query-conditioned classification, retrieval, and localization reports? How do the authors justify ΔDC-Sel\Delta\mathrm{DC\text{-}Sel} as a separate AUROC metric given the affine equivalence above?

  2. Please report the absolute matched and mismatched components and the full query-by-event matrix. Do the gains remain when matched-query performance is held fixed?

  3. How can DeCoS detect an unknown anomaly that the frozen gate gtg_t misses? A held-out-concept or cross-dataset localization experiment would be useful.

  4. How stable is DeCoS when only one query is supplied, or when unrelated, duplicate, synonymous, or overlapping definitions are added to the candidate set? Please also provide temporal score curves on natural multi-event videos.

Reviewer_FbbNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates the problem of definition blindness in open-world video anomaly detection (OWVAD), where models may detect anomalous events without properly following user-specified anomaly definitions. The authors identify limitations in existing evaluation protocols and introduce three definition-conditioned metrics to better assess this capability. They further propose Definition-Contrastive Scoring (DeCoS) to improve definition following by contrasting anomaly evidence across different definitions. Extensive experiments on three benchmarks demonstrate the limitations of existing evaluation protocols and the effectiveness of the proposed approach.

Critical strengths
  1. The paper identifies a potentially important limitation in existing OWVAD evaluation protocols. Distinguishing generic anomaly detection from definition following is a meaningful research direction that could contribute to more reliable evaluation of OWVAD systems.
  2. The authors conduct experiments on three benchmark datasets and compare several categories of methods, including VAD models and general vision-language models. The evaluation also includes ablation studies and additional analyses, providing considerable empirical evidence to support the proposed approach.
Critical weaknesses
  1. The paper is difficult to follow, particularly in the Abstract and Introduction. The research problem and motivation are not presented clearly enough, and the concept of definition blindness is not explained in a sufficiently intuitive manner. Moreover, the paper does not clearly distinguish its main contributions, particularly the proposed evaluation metrics and the DeCoS method. Both sections place excessive emphasis on experimental observations, numerical results, and analyses of existing evaluation protocols, making them read more like experimental result analyses than a clear introduction to the research problem and proposed solution. I suggest substantially reorganizing these sections to better explain the motivation, core ideas, and contributions.
  2. The experimental validation of open-world generalization remains insufficient. Although the paper evaluates unseen event names and name-free natural-language definitions, these experiments are still largely based on predefined anomaly categories from existing benchmarks. It remains unclear whether DeCoS can effectively follow arbitrary user-specified definitions or generalize to genuinely novel anomaly concepts, rather than primarily improving discrimination among existing anomaly categories.
  3. Insufficient qualitative visualization and analysis. Although Figure 1 provides an illustrative example, the paper lacks comprehensive qualitative comparisons to demonstrate how DeCoS improves definition following in different scenarios. I suggest including more visualization results showing frame-level anomaly scores under different user-specified definitions, together with comparisons against existing methods.
Questions for authors

Please see the Weaknesses.

Comments on the paper

2 comments

Dear reviewers,

Thank you for your reviews and constructive feedback.

The authors have submitted their rebuttal. Please take a moment to review it and consider whether your concerns have been adequately addressed. It would be helpful if you could briefly indicate:

  • Whether the rebuttal addresses your main concerns;

  • Whether any significant issues remain unresolved;

  • Whether your overall recommendation has changed.

If appropriate, please also update your review and recommendation in the system.

Thank you for your time and valuable contributions.

Best regards,

AC

Area_Chair_BoP4OpenReview ↗

AC update

Dear authors,

Thank you for your time and contributions to the discussion.

I understand that you may be concerned if some reviewers have not provided follow-up comments or explicitly acknowledged how their concerns have been addressed in the rebuttal and revised manuscript. Please be assured that the AC will carefully consider all available information, including the original reviews, author rebuttal, revised manuscript, and discussion comments, as part of the final assessment.

The absence of further reviewer comments does not mean that the authors’ responses and revisions have been overlooked. I will carefully examine the issues raised, assess the authors’ clarifications and revisions, and take the discussion into account when forming my recommendation. I will also consult with the reviewers as appropriate to ensure that the relevant issues are fully considered.

Thank you again for your constructive engagement and contributions to the review process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.