- Rating
- 4
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents an in-depth investigation of Open-World Video Anomaly Detection (OWVAD), focusing on the issue of definition blindness. The authors provide a thorough analysis supported by compelling empirical evidence, revealing the limitations of existing evaluation protocols. Furthermore, they introduce three complementary evaluation metrics to assess definition following and propose a novel method, Definition-Contrastive Scoring (DeCoS), to mitigate definition blindness.
Critical strengths
This paper identifies and investigates an important yet potentially overlooked issue in Open-World Video Anomaly Detection. The problem of definition blindness is highly relevant to real-world applications, making this study both meaningful and practically valuable. The authors provide a thorough and systematic analysis, supported by theoretical derivations and convincing empirical evidence. Furthermore, the proposed evaluation metrics offer valuable insights into the limitations of existing evaluation protocols.
Critical weaknesses
I did not identify any critical weaknesses. However, I have two minor concerns:
- Semantic overlap between anomaly categories.
Both the proposed evaluation metrics and DeCoS largely rely on an important assumption that anomaly categories are mutually exclusive. However, in practice, some categories may share substantial semantic or visual similarities. For example, Assault and Arrest may both involve physical confrontation or restraint.
For the three proposed metrics, such overlap may lead to valid predictions being penalized when semantically related anomalies are treated as negatives or non-matching definitions. For DeCoS, the contrastive scoring mechanism may suppress evidence that is genuinely relevant to multiple definitions, while the single-label training objective may encourage unnecessary competition between related categories.
Nevertheless, I acknowledge that this may represent a broader challenge in open-world anomaly detection rather than a limitation specific to this work.
- Limited evaluation on natural multi-event videos.
Although ΔDC-Sel addresses an important and meaningful problem, its evaluation on only 43 natural multi-event videos seems insufficient for a comprehensive investigation. I understand that the lack of suitable benchmarks is a practical limitation, and I find the proposed metric reasonable. However, I feel that presenting this as a contribution may be somewhat premature, as it would benefit from a more thorough and systematic study.
Questions for authors
I have a question regarding the design of the proposed evaluation protocol. Why do the proposed metrics focus on single-category anomaly definitions, given that users may want to detect multiple anomaly categories simultaneously in practical scenarios? Even with single-category definitions, the evaluation already involves numerous combinations of target categories and input definitions. Extending the protocol to definitions containing two, three, or more anomaly categories would substantially increase the number of possible combinations, potentially making a comprehensive evaluation impractical. It would be helpful if the authors could discuss how the proposed evaluation framework could accommodate multi-category definitions while keeping the evaluation manageable.
I would also appreciate further clarification on how the three proposed metrics should be interpreted and used in practice, particularly DC-Disc and ΔDC-Det. Since these two metrics evaluate different aspects of definition following, a model may perform well on one but poorly on the other. How should such conflicting results be interpreted, and should one metric be prioritized over the other when assessing a model's overall definition-following capability? Some additional discussion or practical guidance would make the proposed evaluation framework more useful.
Dear reviewers,
Thank you for your reviews and constructive feedback.
The authors have submitted their rebuttal. Please take a moment to review it and consider whether your concerns have been adequately addressed. It would be helpful if you could briefly indicate:
Whether the rebuttal addresses your main concerns;
Whether any significant issues remain unresolved;
Whether your overall recommendation has changed.
If appropriate, please also update your review and recommendation in the system.
Thank you for your time and valuable contributions.
Best regards,
AC