acceptodds
Withdrawn from ICLR 2027

The Decoder Is the Temporal Model: Verdict-First Reasoning for Online Video Anomaly Understanding

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Accurately localizing anomalous moments in videos typically relies on additional classification heads or temporal detection modules, which are effective for fast and precise anomaly localization. However, in scenarios that also require semantic interpretability, existing approaches often introduce separate detection components, making it difficult to fully exploit the inherent video understanding and language reasoning capabilities of vision-language models (VLMs). We observe that the causal context modeling and fixed vocabulary projection of a language model can directly provide a continuous online anomaly score. Based on this insight, we propose Vigil, a VLM-native framework for online video anomaly detection that directly turns a vision-language model into a window-level anomaly detector while preserving its capability to analyze anomalous events. To accommodate streaming-video scenarios, Vigil optimizes the prefill stage of LLM inference and jointly encodes historical frames and current-window frames into a unified model context, without introducing an additional temporal aggregator or recurrent cross-window hidden state. During supervised fine-tuning and inference, the model predicts Normal or Anomaly at the first generation position, and the corresponding token logits are normalized to produce a continuous anomaly score, thereby eliminating the need for a separate detection head. Under a predefined 20-second context setting, Vigil achieves low-latency real-time inference while outperforming the strongest offline baselines on both UCF-Crime and XD-Violence. These results suggest a simple and reproducible pathway toward video anomaly detection systems that retain both efficient online detection and semantic understanding, particularly for edge deployment scenarios.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Anomaly Detection, Video Anomaly Understanding

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_QKdaOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work presents Vigil, a VLM‑based online video anomaly understanding system. It uses decoder causal attention for temporal modeling and a verdict‑first paradigm to get anomaly scores directly from token logits, paired with KV‑cache reuse for streaming inference. Though practically motivated, the work has incremental novelty, relies heavily on dense annotations, leaves critical failure modes unaddressed and uses unfair comparisons.

Critical strengths
  1. The work points out a real pain point of existing two‑stage video anomaly understanding pipelines: detection and explanation come from different model states, which causes inconsistency. It provides a simple way to unify scoring and description within one single forward pass.

  2. KV‑cache reuse for streaming video is practically useful, it reduces redundant computation for overlapping frames and supports low‑latency inference for edge‑side surveillance scenarios.

Critical weaknesses
  1. This paper mostly assembles existing components for the target task and delivers limited innovation. For example, reading classification scores from LLM logits, utilizing decoder self‑attention for temporal modeling and KV‑cache reuse for multimodal streaming are all known techniques.

  2. SOTA performance depends heavily on fine‑grained window‑aligned labels from HIVAU‑70k. Under video‑level weak‑supervision, performance degrades substantially. Cross‑domain transfer results also show poor generalization without target‑domain fine‑tuning, limiting real‑world deployment.

  3. Two key failure patterns are only discussed but not mitigated. False alarms triggered by large normal visual motion, and missed subtle anomalies such as tiny low‑resolution weapons. These are critical pain‑points for surveillance applications.

  4. Most compared online baselines are zero‑shot training‑free methods, while Vigil is fully supervised fine‑tuned. There are no controlled experiments running competing fine‑tuned VAU approaches on the identical backbone under matching online setup. It is hard to quantify how much gain comes from verdict‑first design instead of supervised finetuning or base model capacity.

  5. The performance‑latency trade‑off brought by rotated KV‑cache reuse is not systematically quantified with ablations.

  6. Despite the title referencing reasoning, the model only generates post‑hoc captions without explicit reasoning chains linking historical frames and current anomaly judgment.

Questions for authors
  1. This paper emphasizes reasoning, but the model only generates descriptive captions after prediction, without reasoning steps linking history frames and current anomaly. How do you define reasoning in this manuscript? Why do you avoid inserting intermediate reasoning steps especially for low‑confidence windows?

  2. Table13 shows description‑first setting yields worse result than verdict‑first. Is the performance drop mainly caused by hallucinations from self‑generated text, or other factors? Have you tried inserting short fixed prompt reasoning text between visual input and verdict instead of letting model generate open‑ended reasoning content?

  3. See the above weakness about tther questions.

3 comments

Response to Reviewer QKda

Thank you for recognizing the practical value of unified detection and explanation and streaming cache reuse. We take your concerns about novelty, supervision fairness, and generalization seriously. We clarify our contribution as a unified design for online anomaly detection and on-demand explanation, supported by its empirical results.

1. Novelty and contribution

We do not claim logits-based classification, decoder self-attention, or KV caching as individually new techniques. Vigil's contribution is their integration into a single model for online anomaly understanding: the existing decoder processes historical and current visual observations; first-position Normal/Anomaly logits provide a continuous anomaly score; descriptions can be generated from the same model state when needed; and streaming cache reuse reduces repeated computation over overlapping history.

This design supports routine detection without an additional detection head, temporal aggregator, or full text generation, while retaining on-demand explanation capability. We limit the performance claim to competitiveness under specified supervision and evaluation conditions, rather than novelty of each component or universal superiority over other approaches.

2. Fairness with respect to supervision, backbone, and target-domain fine-tuning

Our main contribution is a single model that combines competitive anomaly detection with on-demand explanations, achieving the best reported results among the methods compared in Table 2 under their respective settings. Beyond optimizing detection alone, Vigil retains the language-generation capability needed to describe anomalous events while using the same decoder for detection.

A pretrained decoder offers strong semantic understanding, but its pretraining is not specifically optimized for the temporal discrimination required by anomaly detection. Task-specific fine-tuning helps adapt these representations to anomaly scoring. We view this as a plausible explanation for the benefit of fine-grained supervision, rather than evidence that decoder-based temporal modeling is inherently inferior to I3D.

Under video-level supervision, Vigil still achieves 82.12 AP on XD-Violence, compared with 77.71 AP for online REWARD, showing that its utility is not restricted to fine-grained labels. We acknowledge that the cross-domain gap limits deployment without adaptation. Our claim is therefore a unified model offering strong detection performance and explanation capability under the stated training conditions, rather than domain-independent detection or supervision-free superiority.

Description training primarily aims to preserve explanation and language-generation capabilities, while freezing the vocabulary projection preserves the pretrained output space. Our experimental observation is that description supervision did not noticeably change detection performance, but this does not constitute a complete attribution study. A frame-supervised I3D comparison is currently running; we do not report unfinished results or claim that a fully matched backbone and training comparison has already been completed.

3. Cross-domain generalization and deployment scope

Table 6 reports transfer without training on the target domain: training only on UCF-Crime yields 72.49 AP on XD-Violence, while training only on XD-Violence yields 85.99 AUC on UCF-Crime. The corresponding target-domain training results are 91.00 AP and 90.42 AUC, respectively, showing the substantial impact of target-domain data.

We acknowledge that this limits direct deployment in new environments. Vigil does not solve open-world anomaly detection: the anomaly definitions and visual distributions in its training data affect its decisions. Joint training achieves 89.91 UCF AUC and 90.54 XD AP, showing that one model can cover these two benchmarks, but not establishing generalization to arbitrary unseen domains. We will qualify the manuscript's interpretations of transfer and pretrained capability and discuss adaptation to the deployment domain as a practical consideration and future direction.

4. Failure cases and possible mitigation

False alarms on normal scenes with large motion and missed detections of small or low-resolution weapons are important limitations. The former suggests possible confusion between motion patterns and learned anomaly cues; the latter suggests insufficient local visual evidence. We do not present these possible explanations as experimentally established mechanisms.

Adding targeted normal scenes with substantial motion and improving local-region or resolution handling are plausible future directions, but their benefits have not been validated in this work. We will present these cases as limitations and distinguish observed evidence from proposed improvements rather than imply that the failure modes have been resolved.

5. Meaning of reasoning, historical evidence, and explanation faithfulness

By reasoning, we refer to forming an anomaly judgment from historical and current visual context, rather than explicitly generating a verified step-by-step reasoning trace. Natural-language descriptions follow the verdict and express the event content. We will clarify this usage to avoid suggesting that causal explanations or faithful chains of thought have already been established.

In the supplemental history experiments, using 8-second, 4-second, and 2-second histories reduces UCF AUC by 0.92, 1.23, and 1.90 percentage points relative to the default configuration; perturbing history reduces it by 1.74 points. These results support the contribution of historical information to judgment, but do not prove that generated descriptions expose every factor behind a decision.

Description quality is evaluated on the same fixed set of windows for all methods, rather than on different sets selected by each method's detections. This measures description capability on common inputs, not explanation faithfulness. We agree that a shared model state is not itself proof of faithfulness and will qualify the corresponding claims.

6. Verdict-first prediction, fixed-text prompts, and reasoning on uncertain windows

In Table 13, verdict-first achieves AP/AUC of 91.05/96.95 on XD-Violence, compared with 88.82/96.04 for description-first. Both settings include descriptions for normal windows, preventing label leakage through description presence. We acknowledge that the comparison does not fully separate content, length, verdict position, and hallucination, and will describe text interference as a possible mechanism rather than the sole established cause.

The direct motivation for verdict-first prediction is that routine anomaly scores can be obtained from first-position logits without decoding text beforehand. A fixed reasoning prompt is a useful additional control, but we have not conducted that independent experiment. We therefore draw no conclusion about its effectiveness and do not use the current results to reject all forms of prompting before the verdict.

Triggering additional reasoning for uncertain windows is also a reasonable extension and is discussed in the paper's future-work section. It requires a triggering policy and an evaluation of its accuracy and latency trade-off, and is not implemented in the current system. This work focuses on the basic unified path for detection and on-demand description.

7. Cache positional correction and the accuracy–efficiency trade-off

Section 3.4 describes an initial full prefill followed by reuse of overlapping visual KV states with positional updates and processing of newly arriving frames. The task suffix and generated responses are excluded from subsequent reuse, the cache is reset between videos, and a full refresh is performed every 20 steps. The main table uses this approximate cache configuration.

In the supplemental comparison, full prefill changes UCF AUC by −0.02 and XD AP by +0.08 relative to reuse on the percentage scale. Processing time per window decreases from 445 ms to 160 ms, approximately a 2.78× speedup. This supports the practical accuracy–computation trade-off of the default configuration. The 20-step interval is an empirical preset rather than an established optimum, and close aggregate metrics do not imply identical scores for every window.

Thanks for the authors' rebuttal, but my concerns have not been addressed, including the incremental novelty, reliance on dense annotations, and the use of unfair comparisons. Thus, I maintain the original score.

Reviewer_TYTeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Vigil, a unified framework for online video anomaly detection and understanding. The method leverages the MLLM decoder to integrate temporal information from historical and current video windows, eliminating the need for a separate temporal aggregation module or detection head. A verdict-first strategy is introduced to derive anomaly scores directly from the first-position logits of the “Normal” and “Anomaly” tokens, while semantic descriptions are generated only when necessary. The framework further employs approximate KV-cache reuse for efficient streaming inference. Experiments on multiple benchmarks demonstrate competitive anomaly detection, temporal grounding, and semantic description performance with low inference latency.

Critical strengths
  1. Unified and efficient framework. The paper presents a simple approach that integrates anomaly detection and semantic understanding within a single MLLM. By directly utilizing the decoder's first-position token logits for anomaly scoring, the method eliminates additional task-specific detection heads and reduces unnecessary autoregressive generation. This design is particularly relevant for online anomaly detection.

  2. Competitive performance with low inference cost. Vigil achieves strong results on UCF-Crime and XD-Violence while maintaining low per-window inference latency. The combination of verdict-first prediction and streaming cache reuse demonstrates promising potential for computationally efficient online video anomaly understanding.

Critical weaknesses
  1. Insufficient validation of the decoder's temporal modeling capability. The central claim of the paper is that the MLLM decoder can replace dedicated temporal modeling modules. However, the current experiments do not sufficiently demonstrate whether the decoder actually captures temporal dependencies or primarily relies on static visual cues and pretrained semantic knowledge. Critical ablations involving temporal frame shuffling, historical context length, and current-window-only inputs are missing. Furthermore, no direct comparison with dedicated temporal aggregation modules under matched visual backbones and supervision settings is provided. This substantially weakens the experimental support for the paper's central claim.

  2. Inconsistent supervision granularity in comparisons with existing methods. In Table 2, Vigil is compared with several methods trained using video-level weak supervision, whereas the full framework utilizes temporally localized anomaly annotations and textual descriptions. Table 9 shows that temporal supervision alone leads to a substantial performance improvement on XD-Violence, from 82.12 AP to 91.00 AP. Therefore, the reported performance gains cannot be fully attributed to the proposed architecture and training strategy. More comparisons under matched supervision conditions are needed to establish the actual contribution of the proposed method.

Questions for authors
  1. Temporal modeling: Can the authors provide controlled ablations using shuffled frame orders, different historical context lengths, and current-window-only inputs? How much of the reported improvement can be attributed specifically to temporal modeling within the decoder?
  2. Supervision fairness: Can the authors provide additional comparisons against representative baselines trained with equivalent temporal annotations and description supervision, in order to better isolate the contribution of the proposed framework?
  3. Verdict-first strategy: Tables 1 and 13 support verdict-first prediction, but what causes the performance degradation when descriptions are introduced? Can the authors distinguish semantic interference from the effects of input length and judgment token position through controlled experiments?
  4. KV-cache reuse: How does approximate KV-cache reuse affect anomaly scores and detection accuracy compared with full-context prefill? What motivates the refresh interval of 20 reuse steps, and how sensitive is performance to this setting?
  5. Online detection effectiveness: Table 8 reports an event recall of 48.1% for the default 4-second window under a constrained false-alarm budget. How does event-level performance change under different false-alarm budgets, and what is the actual delay between anomaly onset and the first correct alarm? Minor presentation issues:
  6. In Figure 1, the vertical arrow connecting the conventional VAD module to the proposed MLLM decoder may misleadingly imply a direct data flow. The authors are encouraged to clarify that it represents functional replacement.
  7. Several typographical errors should be corrected: “groundig” to “grounding” in Table 3, and “Comparision” to “Comparison” and “differebr” to “different” in Table 9.
2 comments

Response to Reviewer TYTe

Thank you for recognizing the online potential of first-position verdicts, on-demand semantic outputs, and cache reuse. We have added results on historical context, causal score assignment, cache accuracy and efficiency, and event-level alarms with validation-selected thresholds. We also clarify the scope of our claims about mechanisms and performance attribution.

1. Does the model use temporal context?

On UCF-Crime, shortening history to 8, 4, and 2 seconds reduces AUC by 0.92, 1.23, and 1.90 percentage points, respectively, relative to the default history configuration. Perturbing historical frames reduces AUC by 1.74 points. The progressive decrease with shorter history and the effect of history perturbation support the use of historical context in anomaly judgment.

We limit this evidence to the contribution of historical context and do not extrapolate it into superiority over all specialized temporal encoders. We have not added a completely history-free experiment. A frame-supervised I3D comparison is currently running, and we do not cite results before they are available. The design contribution is to use the existing decoder for context integration and anomaly scoring without learning an additional temporal aggregation module.

2. Supervision granularity and performance attribution

We agree that finer-grained training supervision is an important factor in the Full setting. Specifically, the change from 82.12 AP with MIL to 91.00 AP with Full on XD-Violence changes both temporal localization supervision and description supervision. The entire difference therefore cannot be interpreted as the isolated benefit of temporal labels. We will revise the corresponding attribution.

Description supervision is primarily intended to preserve explanation and language-generation capabilities, consistent with freezing the pretrained vocabulary projection to retain its language output space. Our experimental observation is that description supervision did not noticeably change detection performance, but we do not present this observation as a quantified isolated ablation. The anomaly score comes from the first-position verdict, so inference does not need to wait for description generation. This ordering property should be distinguished from the effect of training losses on shared parameters.

We will qualify comparisons by supervision level, backbone, and online/offline setting rather than attribute the entire Full-versus-weak-supervision or Full-versus-zero-shot gap to architecture.

3. Benefits and possible mechanisms of verdict-first prediction

Table 13 in Appendix A.9 supplies descriptions for normal windows under both output orders, avoiding label leakage through the presence or absence of a description. On XD-Violence, verdict-first achieves AP/AUC of 91.05/96.95, compared with 88.82/96.04 for description-first. This supports the empirical benefit of verdict-first prediction in the evaluated setting.

Its direct computational advantage is that the anomaly score can be read from the first-position logits without first generating text. Regarding the accuracy mechanism, we agree that this comparison does not fully separate text content, generation errors, length, and verdict position. We will describe interference from self-generated descriptions as a possible explanation, rather than the sole cause established after ruling out other factors. Our claim is that this order is effective in the evaluated setting and suitable for detection with low generation overhead, not that generating descriptions before a verdict is always inferior.

4. Accuracy–efficiency trade-off of cache reuse

The main table uses approximate KV-cache reuse with a refresh every 20 steps. Relative to reuse, full prefill changes UCF AUC by −0.02 and XD AP by +0.08 on the percentage scale, showing that the default reuse configuration yields aggregate detection metrics close to full recomputation.

Processing times per window are 445 ms for full prefill and 160 ms for reuse, approximately a 2.78× speedup. The 20-step interval is an empirical preset rather than an optimum found through systematic search. These results evaluate the default configuration and do not establish equivalence across refresh intervals or identical scores for individual windows.

5. Event recall and delay with validation-selected thresholds

To address limited event recall and the need for multiple operating points, we have added test results with thresholds tuned on the validation set:

DatasetTest false alarms/hourEvent recallMedian time to first alarm
UCF-Crime1.0012.2%4.0 s
UCF-Crime2.0051.9%4.0 s
XD-Violence1.0024.3%5.4 s
XD-Violence2.0036.8%4.0 s

These results reveal a clear false-alarm–recall trade-off, including limited recall at low false-alarm rates. We acknowledge this deployment limitation and report recall alongside delay rather than use short delay alone to claim reliable alerting. The original 48.1% in Table 8 belongs to the original operating-point setting and is not a substitute for recall after validation-based calibration. We will distinguish the original and supplemental results explicitly.

We also evaluate each frame using the score of the most recently completed window. With 4-second windows, the three-seed mean UCF AUC decreases from 89.87 to 87.81, and XD AP from 90.83 to 85.66. With 1-second windows at seed 0, UCF AUC decreases from 89.71 to 89.28, and XD AP from 90.79 to 89.16. We acknowledge the difference between assigning a score to its own window and using a score available from a completed window. The 1-second results support shorter windows as a way to reduce scoring lag; they do not imply that every anomaly is detected within one window.

6. Figures and typographical corrections

We will clarify that the vertical arrows in Figure 1 denote functional replacement rather than data flow. We will also correct “groundig” to “grounding” in Table 3, and “Comparision” to “Comparison” and “differebr” to “different” in Table 9. Thank you for these specific suggestions.

Reviewer_HpN7OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes "Vigil" a online video anomaly detection framework that focuses on solving high inference costs and dependence on additional temporal modules, including SOTA results on UCF-Crime (90.42%) and XD-Violence (91.00%). Additional experiments on MSAD provide an effectiveness evaluation for anomaly understanding and real-time deployment.

Critical strengths
  1. Simple unified framework for VAD and VAU task, keeping on-demand anomaly descriptions capacity while without additional temporal aggregator.
Critical weaknesses
  1. Although Vigil demonstrate the SOTA performence in Table 2, and it is lack of fair comparision under same supervision settings. most offline method mention in the Table is weakly supervised method, only Vigil and Holmes-VAU are using frame-level label(other methods are video-level), which are hard to distingush the performence contribution.
  2. Limited evidence that the proposed components contribute to the performance improvement. The title claims the decoder itself serves as the temporal model, but there is no ablation on the length of the historical context (fixed to 16s in Sec. 3.4, e.g., no history vs. 16s of history). also, it is unclear which setting produces the results in Table 2.
  3. The comparison of decision cost and detection delay in Table 5 is not convincing, because the methods are not evaluated on the same device. The value of MoniTor and REWARD are directly report in original MoniTor paper. And MoniTor paper conduct all experiment on 2*RTX4090 which is different to A100 or RTX4060 Ti, whcih is unfair comparision.

Minor

  • "Discription" in fig 2. is wrong spelling.
Questions for authors
  1. Are the results in Table 2 obtained with full prefill or with the approximate KV-cache reuse in Sec. 3.4?
  2. How much performence gain does the historical context contribute? Please report results with the current window only versus the default 16s history.
  3. Why does Table 9 missed pi-VAD, RefineVAD and Ex-VAD, these video-level methods in Table 2?
1 comment

Response to Reviewer HpN7

Thank you for recognizing Vigil's unified design for anomaly detection and on-demand descriptions. We address your concerns about supervision fairness, historical context, the inference configuration, and efficiency below.

1. Comparisons across supervision settings

Our contribution is to combine strong anomaly detection and on-demand explanations within a single model, achieving the best reported results among the methods in Table 2 under their respective settings. Vigil preserves semantic description capabilities while adapting the pretrained decoder to temporal anomaly discrimination through fine-tuning.

Crucially, Vigil operates online. Under video-level supervision, Vigil MIL achieves 87.17 UCF AUC and 82.12 XD AP, compared with 86.94 and 77.71 for online REWARD. These results demonstrate competitive detection under online constraints even without fine-grained labels. We will make supervision and online/offline distinctions more prominent; the frame-supervised I3D comparison is ongoing.

2. Contribution of historical context

We have added history-length and history-perturbation results on UCF-Crime. Relative to the default history configuration, 8-second, 4-second, and 2-second histories reduce AUC by 0.92, 1.23, and 1.90 percentage points, respectively. Perturbing historical frames reduces AUC by 1.74 points.

The progressive decrease with shorter history supports the contribution of historical context to the current judgment. We use these results to establish the observed trend. We have not added a completely history-free control, so we do not report a numerical gain over current-window-only input or claim superiority over all temporal modules.

3. Inference configuration in Table 2 and the cost of cache reuse

Table 2 uses approximate KV-cache reuse, with a full refresh every 20 steps. We have added a full-prefill comparison: relative to cache reuse, full prefill changes UCF AUC by −0.02 and XD AP by +0.08 on the percentage scale. The aggregate detection metrics are therefore very close under the evaluated default configuration.

Measured processing time per window is 445 ms for full prefill and 160 ms for cache reuse, approximately a 2.78× speedup. This supports the computational benefit of reusing overlapping history. The 20-step refresh interval is an empirical preset; we neither describe it as optimal nor claim that the approximation is strictly lossless.

4. Efficiency comparisons across hardware

We chose the RTX 4060 Ti to evaluate Vigil's performance on consumer-grade hardware representative of resource-constrained edge deployment. We will additionally compare inference speeds on the same device for a controlled evaluation. Given Vigil's direct first-position scoring and reuse of overlapping context, we expect it to retain an efficiency advantage over methods that require text generation for routine detection; this will be verified by the same-device measurements.

5. Missing weakly supervised methods in Table 9

The key distinction is that Vigil MIL operates online, making each decision from the current window and past observations, without access to future frames. π-VAD, RefineVAD, and Ex-VAD are evaluated offline in the reported comparisons. Matching video-level supervision alone therefore does not make their inference settings equivalent to ours.

These methods already appear in Table 2. For completeness, we will also include them in Table 9 as a separate offline reference group, with the online/offline distinction prominently marked:

MethodSettingUCF AUCXD AP
Ex-VADOffline, video-level supervision88.2986.52
RefineVADOffline, video-level supervision88.9288.66
π-VADOffline, video-level supervision90.3385.37
Vigil MILOnline, video-level supervision87.1782.12

The relevant online, video-supervised comparison is REWARD: Vigil MIL achieves 87.17 UCF AUC and 82.12 XD AP, versus 86.94 and 77.71 for online REWARD. In particular, the XD AP improvement is 4.41 percentage points. Our claim concerns competitive detection under online constraints while retaining a unified model for detection and on-demand descriptions, rather than an unrestricted ranking against offline methods.

6. Typographical correction

Thank you for pointing out the typo in Figure 2. We will change “Discription” to “Description” in the revision.

Reviewer_PRsaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, authors propose Vigil, an online video anomaly recognition framework. Vigil fine-tunes Qwen3-VL-2B using LoRA (visual encoder and vocabulary projection frozen) and normalizes the two tokens by reading the window-level anomaly scores from the normal/anomaly logit of the first answer position. Each judgment uses a 4-second current window and a 16-second causal history (4fps), and reduces streaming costs by roughly reusing overlapping visual key-value caches through position correction and performing a full refresh every 20 steps. Window labels are constructed based on the temporal annotations of HIVAU-70k (positive if more than 50% of the frames are anomaly), and training combines judgment token language loss, binary loss for the two-token margin, and explanation loss applied only to positive windows with aligned clip descriptions. Descriptions are decoded as needed after judgment. This paper reports achieving a frame-level AUC of 90.42 on the UCF-Crime dataset and an AP of 91.00 on the XD-Violence dataset under frame-level ("full") supervision and online protocols, and the authors claim that these results outperform all online and offline baselines when using a window of 184 ms per 4 seconds on an RTX 4060 Ti. Additionally, authors evaluate MSAD temporal grounding, description quality, rejection based on judgment order, binary loss, window length, prompt, LoRA rank, and video-level (MIL) supervision variants.

Critical strengths

This design is concise. Detection and on-demand explanations share a single model and a single hidden state. Since scores are read from the normal/abnormal logit of the first answer position, there is no need to train the detection head or decode explanations in normal windows. If explanations are required, decoding continues while the scores are generated. Table 5 shows computation times of 184ms per 4-second window and 46ms per second of video, which means real-time deployment is possible on a single consumer GPU. This is important for my evaluation as it eliminates the two-stage detection-MLLM pipeline used in most VAU systems. However, sharing states does not guarantee that explanations are faithful to the evidence behind the scores.

Through the elimination process, the design choices on which this method is based can be separated. Table 13 compares the verdict-first and explanation-first approaches, including cases where explanations are attached even in normal windows, so that labels are not leaked due to the presence of explanations. On the XD-Violence dataset, the verdict-first approach demonstrated superior performance with an AP of 91.05 versus 88.82. The video-level MIL variant (Table 9) also had a higher AP than the online REWARD (82.12 vs. 77.71, well outside the seed noise AP range of 0.65 reported by the authors; the AUC difference of 87.17 vs. 86.94 is within the 1.39 AUC noise floor range). Therefore, although this method performs lower than the offline VadCLIP and RefineVAD even under video-level supervision, not all performance improvements are attributed to frame-level supervision.

Critical weaknesses
  1. It cannot be concluded that the major performance improvements are due to the proposed temporal modeling design. Although Table 2 shows the supervised level, it compares Vigil trained with HIVAU-70k temporal annotations and explanations with video-level or untrained methods; therefore, the table alone cannot demonstrate the effectiveness of a design that uses the decoder as a temporal model. The video-level variant (Appendix A.3) scored 87.17/82.12, which is lower than the full version (90.42/91.00) and several offline weakly supervised learning methods. Since this removal eliminates both temporal label and explanation loss, the score drop of 3.25/8.88 cannot be interpreted as being solely due to the influence of temporal labels. The claim that "the backbone does not explain these differences" is not supported by the fact that the model sizes match, as the more relevant comparison of +1.46/+3.32 pp against Holmes-VAU(Full) differs from the backbone (InternVL2-2B vs. Qwen3-VL-2B) and training recipes. Furthermore, the authors reported a seed noise floor of 1.39 AUC, which exceeds a 0.09 pp margin relative to π-VAD. It is not specified whether Table 2 represents the average checkpoint or the optimal checkpoint. To isolate the value of the design, historical context must be removed or modified, and it must be compared with controlled external time-domain heads under the same supervised and backbone learning. Without such comparisons, the paper's core claims are based on differences in supervised learning and recipes, which is the primary reason I gave a low score.

  2. The online evaluation protocol is insufficiently specified, and the test data is partially leaked. The false alarm threshold in Table 8 was selected based on 150 normal test videos, and the default 4-second window was also selected based on those comparison results; therefore, these results cannot be interpreted as distribution performance based on settings fixed to the validation data (this does not affect the AUC/AP without thresholds in Table 2). Additionally, it is unclear how the score calculated at the end of the 4-second window is assigned to the frames within that window. If allocated retrospectively, the frame score utilizes frames for up to 4 seconds following that frame; therefore, while online performance is valid at the window level, the frame-level AUC has no causal relationship at the frame level. Finally, since the approximate KV reuse in Section 3.4 (including position correction and updates every 20 steps) results in no loss of accuracy compared to full recalculation, it does not demonstrate that the efficiency figures in Table 5 are obtained without loss of accuracy. While these points do not prove a non-causal implementation, they are necessary to verify the two claims upon which the paper's position depends: online performance and real-time performance.

Questions for authors
  1. Under the same backbone and frame-level supervision, can you (a) ablate or corrupt the 16-second history, and (b) compare the decoder readout against an external temporal head on the same features? Please also report Full SFT with only the description loss removed (the MIL variant removes both temporal labels and the description loss), per-seed results, and whether Table 2 reports a seed mean or a best checkpoint, given the stated noise floor of 1.39 AUC.
  2. To which frames is the score produced at window end t assigned? Does any interpolation or smoothing use later window scores? Do the conclusions hold if each frame is scored with the latest window completed at or before that frame?
  3. Table 8 selects thresholds on the 150 normal test videos and the default 4-second window from that comparison. What are the false-alarm rate and event recall when both are fixed on the held-out validation videos mentioned in §4.1 before touching the test set?
  4. Please describe the positional correction and the 20-step refresh, and compare approximate KV reuse against full-context recomputation at every decision in accuracy (AUC/AP) and latency (mean and p95/p99). Do the Table 5 figures include video decoding, refresh, and description generation, and were the baseline timings measured on the same hardware as Vigil’s?
  5. Does Table 4 evaluate all methods on the same ground-truth anomalous windows under the same (online) protocol? If Vigil is scored only on windows it predicts as anomalous, how are missed anomalous windows and false-positive descriptions handled? Reporting the number of evaluated windows and detection coverage per method would make the comparison interpretable.
2 comments

Response to Reviewer PRsa

Thank you for recognizing the unified detection and explanation design and for your detailed suggestions on attribution, evaluation, and statistical reliability. We have added history ablations, results across three seeds, a comparison of score-assignment protocols, event-level results with validation-selected thresholds, and a comparison between full prefill and cache reuse. We address your concerns below.

1. Supervision fairness and the role of description supervision

We agree that the difference between Full and MIL cannot be attributed entirely to temporal localization labels. MIL also removes window-level description supervision, so this comparison measures the overall difference between supervision configurations rather than the isolated contribution of temporal labels. We will revise the corresponding interpretation in Appendix A.3.

Description supervision is primarily intended to preserve the model's explanation and language-generation capabilities during detection fine-tuning. This is consistent with our decision to freeze the pretrained vocabulary projection and retain the existing language output space. Our experimental observation is that description supervision did not noticeably change detection performance; however, we do not report an isolated numerical gain or loss from this observation. At inference, the anomaly score is read directly from the first-position verdict logits, without generating a description first. Thus, scoring does not depend on the subsequently generated text. Description loss can still affect shared parameters during training, and we distinguish this from the inference-time ordering property.

Our main contribution is a single model that combines strong anomaly detection with on-demand explanations, achieving the best reported results in Table 2 under the respective comparison settings. Task-specific fine-tuning adapts the decoder’s semantic representations to temporal anomaly discrimination while preserving its language-generation capability. Under video-level supervision, Vigil still achieves 82.12 AP on XD-Violence, compared with 77.71 for online REWARD. We therefore emphasize this unified capability while distinguishing supervision, backbone, and inference settings, rather than attributing every performance difference to the temporal architecture. The frame-supervised I3D comparison is ongoing.

2. Does the decoder use historical context?

We have added history-length and history-perturbation experiments on UCF-Crime:

SettingAUC decrease from the default history configuration (percentage points)
8-second history0.92
4-second history1.23
2-second history1.90
Perturbed historical frames1.74

Performance decreases progressively as history becomes shorter, and perturbing the historical input also reduces performance. These results support a contribution from historical context to anomaly judgment. We limit our conclusion to this observed trend: these are not measurements of the gain over a completely history-free model, nor do they establish superiority over all specialized temporal modules. Our central claim is that the existing decoder can process historical and current observations within a unified model, without an additional temporal aggregator.

3. Main-table results and statistics across seeds

The original main table reports seed 0. We now provide results for three random seeds, summarized as mean ± standard deviation. All metrics below are on the percentage scale.

MetricSeed 0Seed 1Seed 2Mean ± SD
UCF-Crime frame-level AUC90.4289.7089.4889.87 ± 0.49
XD-Violence frame-level AP91.0090.5790.9390.83 ± 0.23
XD-Violence frame-level AUC96.9796.8796.9696.93 ± 0.06

The difference between the main-table values and the supplemental means therefore reflects a single-seed result versus an aggregate across three seeds. We agree that the 0.09-point margin over π-VAD should not be presented as a robust advantage. We will describe the results as competitive detection performance and explicitly identify the reporting convention of each table. Existing statements about variability will also be tied to their corresponding experimental configurations rather than used interchangeably.

4. Window score assignment and causal frame-level evaluation

In the original protocol, a score is obtained when a window ends and assigned to the frames within that window, without smoothing. We accept the distinction you raised: an online window-level output is not equivalent to a score already available at every frame's timestamp. To quantify this difference, we additionally evaluate each frame using the score of the most recently completed window.

WindowRuns summarizedMetricOriginal assignmentMost recently completed window
4 sThree-seed meanUCF AUC89.8787.81
4 sThree-seed meanXD AP90.8385.66
1 sSeed 0UCF AUC89.7189.28
1 sSeed 0XD AP90.7989.16

The change in assignment causes a substantial decrease with 4-second windows. With 1-second windows, the decreases are 0.43 and 1.63 percentage points, respectively, supporting the use of shorter windows to reduce the effect of scoring lag. Since the 1-second and 4-second results summarize different numbers of seeds, we do not present this as a matched multi-seed significance comparison. We will revise the overly strong statements about frame-level causality and distinguish window waiting time from computation time.

5. Validation-selected thresholds and event-level alarms

To address the use of test data to select operating points in the original Table 8, we have added test results with thresholds tuned on the validation set:

DatasetTest false alarms/hourEvent recallMedian time to first alarm
UCF-Crime1.0012.2%4.0 s
UCF-Crime2.0051.9%4.0 s
XD-Violence1.0024.3%5.4 s
XD-Violence2.0036.8%4.0 s

These results characterize the false-alarm–recall trade-off after validation-based threshold selection. We will not present the original test-selected operating points as validation-calibrated performance. Recall remains limited at low false-alarm rates, so we report recall together with delay rather than using delay alone to characterize alarm reliability. This evaluation is separate from the threshold-free AUC/AP results in the main table.

6. Cache approximation, refresh, and efficiency

Table 2 uses approximate KV-cache reuse with a refresh every 20 steps. Relative to cache reuse, full prefill changes UCF AUC by −0.02 and XD AP by +0.08 on the percentage scale. Processing time per window decreases from 445 ms with full prefill to 160 ms with reuse, approximately a 2.78× speedup. These results support a small change in aggregate detection metrics alongside a substantial reduction in computation for the default configuration; they do not imply identical scores for every window.

As described in Section 3.4, the first window uses full prefill, followed by reuse of overlapping visual KV states with positional updates. The task suffix and generated responses are excluded from the cache reused by subsequent windows, and the cache is reset between videos. The 20-step interval is an empirical preset, not an optimum established by systematic search. The present results evaluate this default configuration rather than all refresh intervals or tail latency.

For Table 5, we agree that timings quoted across different hardware do not establish a controlled speed ranking. We will explicitly label hardware and sources and qualify the corresponding comparisons. Routine detection does not require description generation, whose additional cost should be distinguished from detection-only processing.

7. Description evaluation and explanation faithfulness

For a fair comparison, all methods in Table 4 are evaluated on the same fixed set of windows, rather than on separate sets selected by their own detections. Thus, the description comparison does not change its evaluation set according to each method's detection coverage. BLEU, METEOR, ROUGE-L, and CIDEr measure description quality on these common windows; detection and event-level metrics separately assess missed detections and false alarms. We will explicitly distinguish this fixed-window evaluation from on-demand description generation during deployment.

Common evaluation windows do not imply identical inference conditions: Holmes-VAU operates offline, whereas Vigil operates online, and we will keep this distinction explicit. We also agree that sharing model states does not by itself establish explanation faithfulness. We treat the shared state as an architectural property and do not present text-quality metrics as a validation of faithfulness.

Comments on the paper

1 comment

Dear reviewers,

Thank you for your reviews and constructive feedback.

The authors have submitted their rebuttal. Please take a moment to review it and consider whether your concerns have been adequately addressed. It would be helpful if you could briefly indicate:

  • Whether the rebuttal addresses your main concerns;

  • Whether any significant issues remain unresolved;

  • Whether your overall recommendation has changed.

If appropriate, please also update your review and recommendation in the system.

Thank you for your time and valuable contributions.

Best regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.