- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes that Anomaly is a relative concept instead of the objective concept where the label is sometimes depends on the scene rules/relations. Thus, it reveals the disadvantages of current VAD method where the zero-shot surveillance judgments remain coarse or inconsistent. They may recognize entities yet miss the interaction or rule that makes an event abnormal.
This paper aims to deal with AnomalyRAG method , a weakly supervised framework that equips a frozen VLM with Relation-Centric Dual-Bank Evidence Retrieval.
Learned object–relation alignment retrieves matched precedents for a frozen VLM, whose Structured Evidence-Grounded Reasoning Chain-of-Thought (SEGR-CoT) contrasts rule satisfaction and violation.
Critical strengths
- The method designed is reasonable and easy to follow. It aims in designing the fine-graiend relation with GNN method which is neat and effective and it reaches the SOTA in ucf-crime and XD-violence dataset.
Critical weaknesses
-
The backbone is a bit old-fashioned. In the paper, it uses qwen3-vl and InternVL2 while I believe the latest version of qwen3.5 and InternVL3 has released for a while which should be tested.
-
In the section of relation-centric dual-bank evidence retrieval, it has a lot of hyper-parameters such as numbers in evidence reliability. However, in the ablation study, there is no much experiments and discussion on it.
-
Moreover, the optimization process and details are missing.
Questions for authors
-
Some of the papers related to context-based VAD is missing.
-
Although the ucf-crime and XD-violence are great fundamental benchmarks, I believe there are various advanced and comprehensive benchmarks should be tested such as Vad-R1[1].
Huang C, Wang B, Wang W, et al. Vad-r1: Towards video anomaly reasoning via perception-to-cognition chain-of-thought[J]. Advances in neural information processing systems, 2026, 38: 118486-118518.
4 comments
Response to Reviewer Q5eY (2/2): Training, related work, and Vad-Reasoning
C3: Optimization procedure and reproducibility
Reply:
-
Training has three stages: construct the evidence banks from training data, train the relation retriever, and calibrate bank scores and fusion parameters on validation data. The detector, OpenCLIP visual encoder, and VLM remain frozen. Training and evidence examples come from the training split; test videos are used only as queries.
-
Each training query uses a same-task, same-label positive from another source and three opposite-label hard negatives. The objective combines dual-bank discrimination, contrastive retrieval, pairwise ranking, rule compatibility, and visual anchoring, with respective weights 1, 1, 0.5, 0.2, and 0.1. These terms train evidence selection and alignment around the anomaly criterion.
-
We use AdamW with learning rate and weight decay 1e-4, batch size 64, gradient clipping at 1, and at most 50 epochs. We retain the checkpoint with the lowest validation loss and use early-stopping patience 5. Calibration and fusion settings are selected on validation data and then fixed for testing.
Revision: Main-paper Section 4.1 now states the training sequence and optimization setup. Section 3.3 defines the objective, and Supplementary Sections 3.2-3.3 provide the detailed sampling and optimization procedure.
Q1: Related work on contextual anomaly detection
Reply:
-
Section 2 and Supplementary Tables 10-11 cover contextual VAD from several perspectives. AnomalyRuler derives normality rules; PANDA retrieves scene-conditioned rules; SlowFastVAD retrieves normal and derived abnormal pattern text. VADER encodes object interactions within a video, while SARD identifies rare scene-graph relations. The revised discussion explains how AnomalyRAG uses this contextual information: it aligns objects and relations across the query and retrieved normal/abnormal images, then compares rule satisfaction and violation at the object level.
-
We added Vad-R1, which learns perception-to-cognition anomaly reasoning through supervised fine-tuning and reinforcement learning. AnomalyRAG keeps the judgment VLM frozen and learns retrieval and calibration to provide instance-level visual evidence. The comparison clarifies their different ways of supporting anomaly reasoning.
Revision: Section 2 now includes Vad-R1 and the more explicit comparison of contextual reasoning mechanisms.
Q2: Evaluation on Vad-Reasoning
Reply:
- We added the requested benchmark to complement the detection and explanation evaluations in main-paper Tables 1-2. The released Vad-Reasoning test file contains 439 videos and evaluates classification, temporal localization, and generated reasoning and answers. Source videos in this test set are excluded from training and both evidence banks.
Table H. Vad-Reasoning results. Metrics range from 0 to 1; higher is better. Vad-R1 values are from its published Table 2.
| Metric | Vad-R1 | AnomalyRAG |
|---|---|---|
| Classification accuracy | 0.875 | 0.891 |
| Classification F1 | 0.862 | 0.882 |
| Temporal mIoU | 0.713 | 0.698 |
Temporal [email protected] | 0.770 | 0.761 |
Temporal [email protected] | 0.706 | 0.699 |
| BLEU-2 | 0.233 | 0.208 |
| METEOR | 0.406 | 0.411 |
| ROUGE-2 | 0.194 | 0.223 |
-
We use the official evaluation scripts. Classification treats abnormality as the positive class. Temporal metrics compare predicted and reference intervals; classification errors receive zero localization credit, and normal videos use the full interval. Text metrics compare the reasoning and answer after format tags are removed.
-
Vad-R1 uses fine-tuned Qwen2.5-VL-7B; AnomalyRAG uses frozen Qwen3-VL-8B-Instruct with retrieved evidence. In this comparison, AnomalyRAG improves classification accuracy by 1.6 percentage points and F1 by 2.0 points, and obtains higher METEOR and ROUGE-2. The table also reports all three localization metrics and BLEU-2, extending the evaluation to the temporal and textual dimensions you requested.
Revision: We added Section 4.5 and Table 5 to the main paper, with the complete protocol in Supplementary Section 7.7.
We hope the newer-backbone results, sensitivity experiments, training details, and broader evaluation address your concerns. We would appreciate your reconsideration of the rating in light of these additions.
Follow-up on Our Rebuttal and Manuscript Revisions
Dear Reviewer Q5eY,
Thank you again for your constructive suggestions. We have updated the experiments and manuscript to address the points you raised:
- Newer VLM backbones. Supplementary Table 14 now includes InternVL3-8B and Qwen3.5-9B, with the evidence banks and retriever held fixed when replacing the judgment backbone.
- Parameter sensitivity. Supplementary Table 29 tests the evidence-reliability weights, confidence threshold, and fusion strength. Section 3 also clarifies which parameters are fixed, learned, or selected on validation data.
- Training and related work. Section 4.1 and the supplement now detail the training stages, objectives, sampling, and optimization settings. Section 2 expands the contextual anomaly-detection discussion and includes Vad-R1.
- Broader reasoning evaluation. New Section 4.5 and main-paper Table 5 report Vad-Reasoning results alongside Vad-R1, covering classification, temporal localization, and generated reasoning and answers.
These updates are included in the revised manuscript and supplement, with revisions marked in blue. Could you let us know whether they address your concerns, or if there are remaining questions we can help clarify?
Thank you for your time and consideration.
Answering my Answer
The authors partially answer my questions and I will consider my final score.
Response to Reviewer Q5eY (1/2): Newer backbones and parameter sensitivity
Thank you for recognizing the relation modeling and detection results. We added the newer backbones and sensitivity tests you requested, clarified training, and extended the evaluation to Vad-Reasoning.
C1: Evaluation with newer VLM backbones
Reply:
Table F. Detection across judgment backbones. All metrics are percentages. VERA is from main-paper Table 1; the last two rows are newly added experiments.
Revision: We extended Supplementary Table 14 and its accompanying description with both new backbones and the controlled replacement protocol.
C2: Reliability parameters and sensitivity
Reply:
Supplementary Tables 15-16 already examine bank size, candidate count, retained evidence count, relation-score scaling, admission thresholds, and graph size. We added Table G to test the reliability weights, confidence threshold, and fusion coefficient more directly.
These parameters have distinct roles. Seven fixed cue weights form the evidence-confidence score. The confidence threshold determines whether evidence can modify a prediction, and the fusion coefficient determines the size of that correction. The threshold and coefficient are selected on validation data; retrieval branch weights are learned during training. This separates evidence reliability from learned candidate ranking.
Table G changes one setting at a time. The default threshold is 0.50 and the default fusion coefficient is 0.25. Evidence use is the fraction of queries that pass both the confidence threshold and verification, enabling an evidence-based correction. Queries without an admitted correction retain their evidence-free predictions.
Table G. Evidence use and parameter sensitivity.
Revision: Section 3.2 now distinguishes fixed reliability weights from learned retrieval weights. Supplementary Section 7.6 and Table 29 add these sensitivity tests; Supplementary Tables 4-5 document the defaults and gate diagnostics.