acceptodds
Under review as a conference paper at ICLR 2027

Anomaly-Guided Observation and Two-Level Semantic Feedback for Training-Free Video Anomaly Analysis

Abstract

Training-free language-based video anomaly detection (VAD) uses frozen vision– language models (VLMs) and large language models (LLMs) to localize abnormal events without task-specific model-weight optimization. Existing approaches in- creasingly exploit broader temporal and semantic context, yet generic context does not necessarily foreground abnormal events or specify how video-level seman- tics should modify local anomaly scores. We introduce a training-free semantic feedback framework that uses VAD-guided active observation for video anomaly understanding (VAU) and returns the resulting anomaly-focused description to refine temporal detection. Starting from an upstream anomaly-score curve, we adapt reasoning-based iterative frame selection: high-score regions seed the ob- servations, and anomaly scores together with VLM relevance assessments guide subsequent temporal exploration. A frozen video-language model generates a sin- gle open-ended anomaly description from the selected frames. The description is returned through two complementary feedback levels: video-level feedback de- termines whether the score distribution should be suppressed, preserved, or sharp- ened, with selective visual rechecking, while local feedback retrieves event-related windows and promotes scores only after target-local visual verification. All model weights remain frozen, and feedback is applied once without regenerating the de- scription. On the HIVAU video-summary evaluations of UCF-Crime and XD- Violence, VAD-guided observation improves GPT-rated reasonability, detail, and consistency over the matched CLIP-guided variant. For VAD, semantic feedback improves the re-evaluated upstream detector from 84.36% to 86.25% AUC on UCF-Crime and from 68.07% to 72.58% AP on XD-Violence, where the final AUC is 91.74%.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Anomaly Detection, Video Anomaly Understanding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_prQfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a training-free framework combining video anomaly detection and understanding through active observation and two-level semantic feedback. Starting from URF-HVAA predictions, frozen vision-language models describe adaptively selected frames. These descriptions guide video-level score modulation and visually verified local refinement. Experiments report improvements on several VAD benchmarks.

Critical strengths
  • Separating global semantic interpretation from locally verified refinement addresses the risk of applying descriptions without sufficient temporal evidence.
  • The framework integrates pretrained models and existing detector predictions without additional model-weight optimization.
Critical weaknesses
  1. Temporal localization benefits require more direct evaluation. Table 4 reports additional AP gains from local feedback, and Table 5 shows increased separation between normal and anomalous regions. However, these results do not fully distinguish improved temporal localization from score recalibration. Per-video localization metrics or event-level temporal overlap analysis would more directly substantiate the claimed benefit.

  2. Dependence on initial predictions. Local verification operates at the window level, while anchor promotion favors existing high scores. The minimum retrieval score of 0.50 may exclude missed anomalies and leave incorrectly high-scoring normal anchors eligible for reinforcement.

  3. Unclear detection benefit of active observation. Although description quality improves, URF-HVAA descriptions yield comparable detection results, particularly on UCF-Crime. A comparison with one-shot ATS descriptions under identical feedback would better isolate the benefit of adaptive observation.

  4. Unclear parameter-selection protocol. Appendix D provides sensitivity analyses but does not specify how final parameters and rules were selected. The absence of weight training alone does not establish independence from evaluation annotations.

Questions for authors
  • Were annotations or evaluation-set results used to select feedback parameters, lexical rules, or thresholds?
  • Can local feedback recover anomalous intervals below the initial retrieval threshold?
  • Could tied scores introduced by Eq. (4)’s peak-score floor cause Eq. (5)’s quantile-based promotion to reinforce more anchors than intended?
  • How does one-shot ATS compare with active observation under the same feedback pipeline?
2 comments

Temporal localization versus score recalibration

Per-video localization metrics or event-level temporal overlap analysis would more directly substantiate the claimed benefit.

We measured temporal overlap separately in each test video at the same score threshold (0.5) for both methods. We computed IoU separately for normal and anomalous frames, averaged the two IoUs within each video, and gave every video equal weight, including normal-only videos. Full feedback increased temporal mIoU from 72.69% to 72.90% on UCF-Crime and from 63.88% to 64.74% on XD-Violence. Thus, the predicted regions overlap the ground truth more closely at this threshold. Appendix A (p. 14) of the revised paper gives the metric definition.

Retrieval thresholds, missed anomalies, and incorrect promotion

Can local feedback recover anomalous intervals below the initial retrieval threshold?

Local feedback can increase scores within an anomalous interval even when individual anchors initially score below 0.50. The 0.50 threshold filters candidate windows. If a window passes retrieval and visual verification, anchors inside it can be promoted when their current scores meet the local quantile criterion. An interval outside all such windows receives no direct local update.

To test whether verification limits incorrect reinforcement, we compared local feedback with verification disabled and enabled:

DatasetGround-truth regionPromoted anchors: disabled → enabled (reduction)Mean score increase: disabled / enabled
UCF-CrimeNormal842 → 525 (37.65%)0.3812 / 0.2533
UCF-CrimeAnomalous432 → 385 (10.88%)0.1892 / 0.1745
XD-ViolenceNormal3,282 → 2,519 (23.25%)0.2434 / 0.1796
XD-ViolenceAnomalous6,441 → 5,997 (6.89%)0.1500 / 0.1443

Verification reduced promotion counts more in normal than in anomalous regions: 37.65% versus 10.88% on UCF-Crime and 23.25% versus 6.89% on XD-Violence. Figure 4 of the revised paper (Appendix A, p. 13) compares frame-level precision and recall. On both datasets, the full method achieves higher precision than the URF-refined base over broad portions of the recall range.

Active observation under identical feedback

Unclear detection benefit of active observation. Although description quality improves, URF-HVAA descriptions yield comparable detection results, particularly on UCF-Crime. A comparison with one-shot ATS descriptions under identical feedback would better isolate the benefit of adaptive observation.

How does one-shot ATS compare with active observation under the same feedback pipeline?

To evaluate the detection benefit of active observation, we compared feedback using one-shot ATS and active-observation descriptions on UCF-Crime, with feedback parameters and verifier settings held fixed.

Table 1. Downstream detection on UCF-Crime using ATS and active-observation descriptions. Scores are percentages; bold and underlined values indicate the best and second-best results, respectively.

ConfigurationUCF AUC ↑UCF AP ↑
URF-refined base84.35636.164
Feedback with ATS descriptions84.456‾\underline{84.456}37.015‾\underline{37.015}
Feedback with active-observation descriptions86.24638.363

Under identical feedback settings, active-observation descriptions outperform one-shot ATS by 1.790 percentage points in AUC and 1.348 in AP on UCF-Crime. This shows that the benefit of active observation extends beyond description quality to downstream anomaly detection.

Use of evaluation annotations and results

Unclear parameter-selection protocol. Appendix D provides sensitivity analyses but does not specify how final parameters and rules were selected. The absence of weight training alone does not establish independence from evaluation annotations.

Were annotations or evaluation-set results used to select feedback parameters, lexical rules, or thresholds?

Test-set results were used to select the URF-HVAA feedback configuration shared across all evaluated datasets. We constructed the anomaly vocabulary from UCF-Crime class names, using ChatGPT to expand them with synonyms and related expressions. The vocabulary was not adjusted based on evaluation results.

To assess whether the improvements depend on the exact selected values, we subsequently conducted one-parameter-at-a-time sensitivity analyses. For example, AUC and AP on UCF-Crime and XD-Violence exceed the URF-refined base at every tested value of the peak minimum mm from 0.25 to 0.80, with the other parameters held fixed. These results show that the gains are not confined to the exact reported configuration. The sensitivity results are summarized in the General response under “Parameter selection and sensitivity”.

Quantile ties and number of promoted anchors

Could tied scores introduced by Eq. (4)’s peak-score floor cause Eq. (5)’s quantile-based promotion to reinforce more anchors than intended?

Yes. At a quantile of 0.85, the tie-inclusive rule promotes 29.13% of anchors on UCF-Crime and 33.68% on XD-Violence, exceeding the nominal 15%.

To assess whether resolving these ties improves detection, we evaluated a variant that breaks ties using the scores before peak-score flooring. The table reports changes relative to the original tie-inclusive rule in percentage points.

Table 2. Effect of breaking floor-induced ties using the original scores. Changes are tie-breaking minus the original rule, in percentage points.

DatasetΔ AUCΔ AP
UCF-Crime+0.015-0.427
XD-Violence-0.028-0.826

Breaking ties lowers AP on both datasets, while producing only small changes in AUC. These results support retaining the original tie-inclusive promotion rule. We will clarify that the quantile defines a score threshold rather than a fixed promotion quota.

Reviewer_iSmoOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a semantic feedback framework for training-free video anomaly analysis, aiming to feed anomaly-focused descriptions generated by Video Anomaly Understanding (VAU) back to refine the temporal anomaly score sequences in Video Anomaly Detection (VAD). Specifically, the method leverages initial anomaly scores from an upstream detector (URF-HVAA) to guide active frame observation (adapted from A.I.R.) and utilizes a frozen Multimodal Large Language Model (MLLM) to generate open-ended anomaly descriptions. Subsequently, these descriptions refine the initial scores through a two-level semantic feedback mechanism: video-level feedback combines lexical parsing with selective visual verification to suppress, maintain, or sharpen the overall score distribution; local feedback retrieves candidate temporal windows based on event hypotheses and boosts the anomaly scores of corresponding anchors after passing local visual verification. The authors conduct quantitative evaluations and ablation studies for both VAD and VAU across UCF-Crime, XD-Violence, UBNormal, and MSAD datasets, validating the improvements brought by this feedback mechanism on detection and description metrics.

Critical strengths
  1. Addresses a practical issue in training-free VAD with a structured feedback design. Segment-level local scoring lacks video-level context, while generic context information does not necessarily focus on anomalous events. The authors propose a concrete mechanism to feed anomaly-focused VAU descriptions back into the score sequence, effectively decoupling video-level score modulation from visually verified local score boosting.
  2. Solid experimental validation. Under the zero-shot, training-free setting, the proposed method achieves state-of-the-art detection performance among comparable methods on three real-world datasets (UCF-Crime, XD-Violence, and MSAD), while also generating superior anomaly descriptions on the VAU task compared to existing training-free approaches. The ablation studies provide controlled comparisons for the observation strategy and the two-level feedback, supplemented by parameter sensitivity analyses that effectively demonstrate the contribution of each component.
Critical weaknesses
  1. Limited standalone contribution and high inference overhead. The entire pipeline employs three different MLLMs (Qwen2.5-VL-7B as the candidate frame analyzer, VideoLLaMA3-7B as the description generator, and Qwen2-VL-7B as the verifier) and relies on the complete URF-HVAA upstream pipeline to provide captions, scores, suspicious phrases, and spatial annotations. Furthermore, the overhead report in the appendix explicitly excludes the upstream generation stage and omits end-to-end latency. Essentially, the proposed method functions as a post-processing plugin on top of URF-HVAA rather than a standalone VAD or VAU method.
  2. Excessive manually tuned hyperparameters and lack of a clear parameter selection protocol. As a training-free method, the framework incorporates a large number of hyperparameters requiring tuning, including seven feedback parameters and multiple thresholds. The paper fails to specify which data split was used to determine these parameters, and all sensitivity analyses are reported directly on the test sets.
Questions for authors
  1. The video-level parsing in Module 2 relies on keyword vocabularies for matching, but the paper neither discloses the exact vocabulary content nor explains the basis for its construction. Please clarify: Was the vocabulary designed with reference to the class names of target datasets such as UCF-Crime and XD-Violence? If so, this essentially injects target dataset class priors into the inference pipeline, which conflicts with the claimed zero-shot setting.
  2. String-matching-based parsing struggles to cover synonymous expressions and cannot handle negation or contrast (e.g., "no sign of fighting" would hit both vocabulary categories simultaneously, be judged as mixed, and falsely trigger SHARPEN). Please explain how such cases are handled and report the false judgment rate of the lexical parser. Additionally, how would the performance and computational overhead change if an LLM were used for semantic parsing instead?
  3. The initial scores, captions, and suspicious phrases in this work are all derived from URF-HVAA, which itself contains extensive surveillance-specific prompts and multi-stage processing. Please validate the generalizability of the proposed method on other upstream detectors, such as Qwen or CLIP-based zero-shot baselines. If the performance gains drop significantly when replacing the upstream detector, does it imply that the contribution of this method heavily relies on the specific outputs of URF-HVAA?
4 comments

End-to-end inference cost

Furthermore, the overhead report in the appendix explicitly excludes the upstream generation stage and omits end-to-end latency.

To quantify the processing cost of our modules alongside the upstream pipeline, we profiled all stages, including upstream generation, on all 290 UCF-Crime and 800 XD-Violence test videos using two NVIDIA RTX 3090 GPUs. Table 1 reports the mean processing time per video.

Table 1. Mean processing time in seconds per video. Group headings report subtotals; the Share column gives each stage or group's percentage of the total measured time across all 1,090 videos. Times exclude model loading and measured warm-up; totals sum separately timed stages.

StageUCF-Crime (s)XD-Violence (s)All videos (s)Share of total (%)
Upstream: URF-HVAA processing465.34605.08567.9095.34
Clip caption generation399.74527.27493.3482.82
Initial VAD scoring49.7853.2452.328.78
Anomaly tag extraction5.186.936.471.09
Gated VAD refinement4.284.164.190.70
Bounding box generation and overlay6.3613.4711.581.94
Ours: VAD-guided VAU11.5917.2615.752.64
AIR active frame selection7.7310.759.941.67
Final VAU generation3.876.515.810.97
Ours: VAU-to-VAD feedback5.5214.3412.002.01
Total measured processing time482.45636.68595.65100.00

Upstream processing accounts for 95.34% of the measured time, whereas our VAD-guided VAU and VAU-to-VAD feedback together require 27.75 seconds per video (4.66%). The runtime is therefore dominated by the upstream pipeline, with our modules adding anomaly-guided understanding and semantic feedback at a small fraction of the measured processing cost.

Parameter complexity and selection

The paper fails to specify which data split was used to determine these parameters, and all sensitivity analyses are reported directly on the test sets.

The reported feedback parameters were selected using test-set results. We will clarify this selection procedure in the paper. A single URF-HVAA configuration was used across UCF-Crime, XD-Violence, UBnormal, and MSAD.

To examine whether the improvements depend on the exact selected values, we subsequently varied one parameter at a time on UCF-Crime and XD-Violence, keeping the others fixed. Every tested value of α, ff, and λ improves both AUC and AP on both datasets over the URF-refined base. Improvements also persist at all tested values of mm within 0.25–0.80 and the local quantile within 0.70–0.95, including the reported settings of 0.45 and 0.85.

These results provide evidence that the gains extend beyond the single reported configuration. The sensitivity results are summarized in the General response under “Parameter selection and sensitivity”.

Vocabulary construction and dataset class priors

Was the vocabulary designed with reference to the class names of target datasets such as UCF-Crime and XD-Violence? If so, this essentially injects target dataset class priors into the inference pipeline, which conflicts with the claimed zero-shot setting.

Yes. We constructed the anomaly vocabulary from UCF-Crime class names, using ChatGPT to expand them with synonyms and related expressions. It therefore incorporates UCF-Crime category priors when parsing generated descriptions. The vocabulary was used unchanged across all four evaluated datasets and was not adjusted based on evaluation results. The exact vocabularies are:

Anomaly vocabulary (32 entries): anomaly, anomalous, abnormal, suspicious, crime, criminal, illegal, violence, fight, fire, flame, smoke, explosion, blast, steal, theft, shoplift, burglary, robbery, arrest, shoot, gun, weapon, assault, abuse, accident, collision, crash, vandalism, arson, attack, threat.

Normal vocabulary (13 entries): no unusual; no anomal; no abnormal; no suspicious; normal and uneventful; mundane; ordinary; normal events; without any unusual; without any suspicious; nothing unusual; calm scene; routine.

Category semantics are also used in prior work described as zero-shot VAD. AnyAnomaly uses dataset anomaly classes as text queries and takes the maximum score across queries (Sunghyun Ahn et al., AnyAnomaly: Zero-Shot Customizable Video Anomaly Detection with LVLM, WACV 2026, Sec. 4.6). ASK-HINT constructs class-wise prompts from anomaly class labels for a frozen VLM (Shu Zou et al., Unlocking Vision-Language Models for Video Anomaly Detection via Fine-Grained Prompting, WACV 2026, Secs. 3.1–3.3). These provide precedents for category priors, although their use differs from our description parsing.

Lexical parsing reliability and the LLM alternative

String-matching-based parsing struggles to cover synonymous expressions and cannot handle negation or contrast (e.g., "no sign of fighting" would hit both vocabulary categories simultaneously, be judged as mixed, and falsely trigger SHARPEN). Please explain how such cases are handled and report the false judgment rate of the lexical parser. Additionally, how would the performance and computational overhead change if an LLM were used for semantic parsing instead?

To evaluate whether semantic parsing improves our feedback pipeline, we replaced the lexical parser with Qwen2-VL-7B-Instruct in text-only mode. As shown in Tables 2 and 3, the LLM improves detection on XD-Violence, but reduces UCF-Crime AUC and AP by 1.616 and 1.674 percentage points, respectively, while adding approximately 0.226 seconds per description.

Table 2. Downstream detection performance with lexical and LLM semantic parsing (percentages).

ParserUCF AUCUCF APXD AUCXD AP
Lexical parser86.24638.36391.73872.584
LLM semantic parser84.63036.68992.08673.324

Table 3. Lexical and LLM parsing time per description.

Parsing time per descriptionSeconds
Lexical parser0.000292
LLM semantic parser0.225936
Additional parsing time0.225644

To examine how the parsers differ in their feedback decisions, we compared their initial actions with video ground-truth labels on UCF-Crime (Table 4). The lexical parser assigns SUPPRESS to 3/140 anomalous videos (2.14%), compared with 58/140 (41.43%) for the LLM parser.

Table 4. Initial feedback actions by video ground truth on UCF-Crime.

Video ground truthActionLexicalLLM
Anomalous (140 videos)SUPPRESS3/140 (2.14%)58/140 (41.43%)
Normal (150 videos)SHARPEN90/150 (60.00%)18/150 (12.00%)

When neither vocabulary matches, lexical parsing selects KEEP and leaves the video-level scores unchanged. In some inspected examples, Qwen instead assigns SUPPRESS to descriptions that report activities without explicitly establishing normality. For instance, “no significant movement” describes motion but does not establish the absence of an anomaly. In these cases, KEEP preserves the upstream scores during video-level feedback.

Table 5 reports precision for SUPPRESS and SHARPEN and recall for KEEP, using GPT-6 Astra's text-only judgments as the reference. We use precision to assess the textual support for score-changing decisions, and KEEP recall to assess whether descriptions without a committed normal or abnormal interpretation retain their scores during video-level feedback.

Table 5. Action-specific diagnostics for initial parsing on UCF-Crime (percentages). Reference judgments flagged as ambiguous were excluded.

MetricLexical parserLLM parser
SUPPRESS precision86.6789.44
SHARPEN precision41.5791.11
KEEP recall48.150.00

SUPPRESS attenuates the entire score sequence, so its precision measures how often this video-wide reduction is supported by the description. The lexical parser's SUPPRESS precision is 86.67%, compared with 89.44% for the LLM parser.

The lexical parser has lower SHARPEN precision. It does not interpret negation in context: in the reviewer's example, “no sign of fighting” matches “fight” and initially triggers SHARPEN. When both vocabularies match, the parser labels the description as mixed and also initially assigns SHARPEN. Selective visual rechecking may subsequently change this action to KEEP or SUPPRESS. For a retained SHARPEN action, video-level score increases are restricted to anchors satisfying both a per-video quantile threshold and an absolute minimum.

The lexical parser also selected KEEP in 48.15% of cases where the text reference indicated KEEP. The tested LLM parser never selected KEEP, although it was an available output.

The action counts also show more frequent SHARPEN assignments to normal videos with lexical parsing. We retain lexical parsing as the default: it preserves upstream scores for some descriptions that the text reference assigns KEEP, requires no model inference, and replacing it with Qwen does not consistently improve downstream detection.

Contribution and dependence on the upstream detector

The entire pipeline employs three different MLLMs (Qwen2.5-VL-7B as the candidate frame analyzer, VideoLLaMA3-7B as the description generator, and Qwen2-VL-7B as the verifier) and relies on the complete URF-HVAA upstream pipeline to provide captions, scores, suspicious phrases, and spatial annotations. [...] Essentially, the proposed method functions as a post-processing plugin on top of URF-HVAA rather than a standalone VAD or VAU method.

The initial scores, captions, and suspicious phrases in this work are all derived from URF-HVAA, which itself contains extensive surveillance-specific prompts and multi-stage processing. Please validate the generalizability of the proposed method on other upstream detectors, such as Qwen or CLIP-based zero-shot baselines. If the performance gains drop significantly when replacing the upstream detector, does it imply that the contribution of this method heavily relies on the specific outputs of URF-HVAA?

Our framework uses upstream anomaly scores to guide frame selection for video anomaly understanding (VAU). The resulting description guides video-level score updates and local score increases in visually verified windows. On the same URF-refined base, combining the two feedback levels improves UCF-Crime AUROC by 1.89 percentage points and XD-Violence AP by 4.51 percentage points.

To test whether the framework requires outputs specific to URF-HVAA, we applied it to VADTree on UCF-Crime and XD-Violence. We used VADTree's own frame scores and coarse and fine captions for VAU generation and feedback, without reusing URF-HVAA bounding boxes, anomaly tags, or VAU descriptions.

Table 6. Detection performance with URF-HVAA and VADTree upstream outputs.

MethodUCF-Crime AUROC (%)UCF-Crime AP (%)XD-Violence AUROC (%)XD-Violence AP (%)
URF-HVAA84.3636.1691.3268.07
+ Ours86.2538.3691.7472.58
VADTree84.7442.0890.4567.93
+ Ours84.8342.3490.5068.66

Feedback improved AUROC and AP on both datasets using VADTree's native outputs. The gains were smaller than with URF-HVAA: AP increased by 0.26 percentage points on UCF-Crime and 0.73 on XD-Violence.

We kept both quantile levels, the lexical parser and vocabularies, the candidate-selection rules, and the visual-verification thresholds unchanged. We selected the other five numerical parameters through a post-hoc grid search using evaluation-set labels, considering AUROC and AP jointly across both datasets. We used one VADTree configuration for both datasets (Table 7).

Table 7. Feedback parameter configurations for each upstream detector.

Initial VADαβfmGlobal quantileλτℓτ_{ℓ}
URF-HVAA0.400.800.850.450.700.950.85
VADTree0.950.900.300.200.700.250.85

The absolute threshold m and floor f act on the upstream score scale. They also affect the scores passed to local feedback, where λ moves selected scores toward 1, so we calibrated these three parameters jointly. The larger α and β values retain more of VADTree's original scores. In particular, SUPPRESS multiplies every score in a video by α. If a description is incorrectly interpreted as normal, α = 0.95 therefore limits the score reduction to 5%, compared with 60% at α = 0.40.

To test whether the description source contributes to this difference, we held VADTree's scores and captions fixed on UCF-Crime. We compared VAU descriptions generated from VADTree inputs with those generated from URF-HVAA inputs, which include anomaly tags and, when enabled, bounding-box overlays. We tested both suppression strengths with each description source, keeping the remaining feedback parameters fixed (Table 8).

Table 8. UCF-Crime detection by VAU description source and suppression strength, with VADTree scores and captions fixed. Bold indicates the better of the two suppression settings for each description source and metric.

VAU description sourceVADTree inputsURF-HVAA inputs
AUROC (%)AP (%)AUROC (%)AP (%)
Strong attenuation (α = 0.40)84.57242.07285.44543.379
Weak attenuation (α = 0.95)84.82742.33685.25742.916

Descriptions generated from URF-HVAA inputs yielded higher AUROC and AP at both suppression strengths. At VADTree's reported setting (α = 0.95), changing only the description source raised AP from 42.34 to 42.92. With VADTree's own descriptions, weaker suppression (α = 0.95) yielded higher AUROC and AP than stronger suppression (α = 0.40).

URF-HVAA itself uses anomaly tags and bounding boxes. With the same URF-refined scores and feedback settings, our active-observation descriptions yielded higher AP than the released URF-HVAA descriptions on UCF-Crime (38.36 vs. 38.16) and XD-Violence (72.58 vs. 71.49; Appendix A, Table 6 of the paper). To isolate the role of observation, we compared active observation with one-shot ATS using the same VAD prior, spatial conditioning, and 16-frame input budget for the description generator. With the verifier and feedback parameters fixed, active-observation descriptions improved UCF-Crime AUROC by 1.790 percentage points and AP by 1.348 over ATS. (prQf-Table.1)

Active observation thus made the available upstream evidence more useful for downstream detection. Across all 1,090 UCF-Crime and XD-Violence test videos, our observation, description generation, and feedback stages together took 27.75 seconds per video on average, accounting for 4.66% of the total measured processing time with URF-HVAA. (iSmo-Table.1)

Reviewer_XSpLOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a training-free framework that uses video anomaly understanding (VAU) to refine temporal video anomaly detection (VAD). Starting from URF-HVAA anomaly scores, the framework iteratively selects frames based on anomaly scores and VLM relevance assessments. A frozen video-language model generates an anomaly description from 16 selected frames, which guides two-level semantic feedback. Video-level feedback adjusts the score distribution, while local feedback refines event-related scores after visual verification. All model weights remain frozen, and the description is not regenerated during feedback. The authors evaluate the framework on four detection benchmarks and HIVAU video-summary evaluations on UCF-Crime and XD-Violence. Reported results show improved description quality and detection performance, with UCF-Crime AUC increasing from 84.36% to 86.25% and XD-Violence AP from 68.07% to 72.58%.

Critical strengths
  1. The separation between video-level score modulation and local score promotion is well motivated. A video-level anomaly judgment does not by itself establish where scores should be increased. Requiring direct abnormal evidence inside the target clip provides a sensible criterion for authorizing local promotion and connects the VAU description to temporal detection.
  2. The evaluation considers both VAU description quality and downstream VAD performance. The cumulative ablations show additional improvements when local feedback is applied after video-level feedback. These results support the usefulness of the local feedback stage as a whole under the evaluated configuration.
Critical weaknesses
  1. My main concern is the dependence of both feedback levels on a single VAU description generated from 16 selected frames. If frame selection misses the abnormal event, an incorrect normal-scene description can trigger suppression and disable local promotion. Selective visual rechecking is only eligible when the description contains anomaly expressions, so it does not directly address this failure case. Appendix A compares description sources, but does not establish robustness to missed events or incorrect descriptions. Since the same description guides both video-level score modulation and event-conditioned retrieval, such errors could affect both feedback levels.
  2. The cumulative ablations show improvements from adding the feedback stages, but do not isolate several central design choices. Separate ablations are missing for selective visual rechecking, target-local visual verification, and local feedback applied without video-level feedback. These comparisons are needed to establish how much the gains depend on visual verification and whether both feedback levels are necessary. The experiments also use a fixed Qwen2-VL-7B-Instruct verifier without testing alternative verifiers or model scales, leaving the dependence on verifier capability unclear. Together, these omissions limit the evidence supporting the proposed two-level semantic feedback design.
Questions for authors
  1. How often are GT-anomalous videos assigned a normalish interpretation of the VAU description, and how does semantic feedback affect VAD performance on these videos relative to the URF-refined base? Is there any recovery mechanism when these descriptions are not eligible for selective visual rechecking?
  2. Could the authors provide separate results for video-level feedback without selective visual rechecking, local feedback without target-local visual verification, and local feedback applied directly to the URF-refined base without video-level feedback? Keeping the VAU descriptions and shared feedback parameters fixed would help determine whether the visual verification steps and the combination of both feedback levels are necessary for the reported gains.
1 comment

Missed events and recovery from normalish descriptions

How often are GT-anomalous videos assigned a normalish interpretation of the VAU description, and how does semantic feedback affect VAD performance on these videos relative to the URF-refined base? Is there any recovery mechanism when these descriptions are not eligible for selective visual rechecking?

To quantify this failure mode and its effect on event detection, we examined ground-truth anomalous videos whose VAU descriptions were interpreted as normal by the video-level lexical parser (normalish). We compared the URF-refined base with full feedback, counting an event as detected when at least 10% of its annotated frames met the score threshold of 0.50.

Table 1. Normalish interpretations and event detection in the affected anomalous videos.

MeasureUCF-CrimeXD-Violence
Anomalous videos interpreted as normal3/140 (2.14%)3/500 (0.60%)
Videos judged anomalous by the local parser0/32/3
Videos with scores increased by local feedback0/30/3
Events detected by the URF-refined base0/30/21
Events detected after full feedback0/30/21

The first row covers all ground-truth anomalous videos. Subsequent rows concern only the three affected videos per dataset, containing three UCF-Crime events and 21 XD-Violence events. The local parser's anomaly judgment enables candidate-window retrieval and visual verification; local score increases require visual verification and anchor selection.

Normalish interpretations occurred in 2.14% of GT-anomalous UCF-Crime videos and 0.60% of XD-Violence videos. Under the stated detection criterion, neither the URF-refined base nor full feedback detected the events in these videos: 0/3 events on UCF-Crime and 0/21 on XD-Violence.

Local feedback is assessed separately from video-level rechecking. A video excluded from video-level rechecking can still receive local score increases if a candidate window passes retrieval and visual verification and its anchors satisfy the promotion criterion. None of the six videos triggered video-level rechecking, but the local parser judged two XD-Violence videos anomalous.

Isolating feedback and verification components

Could the authors provide separate results for video-level feedback without selective visual rechecking, local feedback without target-local visual verification, and local feedback applied directly to the URF-refined base without video-level feedback?

We held the VAU descriptions and feedback parameters fixed. Video-level feedback alone improved all four metrics without rechecking. Verified local feedback alone also improved all four when applied directly to the base. Adding local feedback without local verification to the rechecked video-level scores raised AP on both datasets, but lowered UCF-Crime AUC. Enabling local verification then improved all four metrics, including AP gains of 0.522 points on UCF-Crime and 0.505 on XD-Violence. Combining the two verified stages outperformed either stage alone on all four metrics. Video-level rechecking primarily benefited UCF-Crime.

The complete ablation table and matched component comparisons are provided in the General response under “Contribution of feedback and visual verification”.

Dependence on the verifier model and scale

The experiments also use a fixed Qwen2-VL-7B-Instruct verifier without testing alternative verifiers or model scales, leaving the dependence on verifier capability unclear.

Our framework uses Qwen2-VL-7B-Instruct as its default verifier. To address the reviewer’s concern, we conducted additional experiments with off-the-shelf, open-weight VLMs from the widely used Qwen and InternVL families. Their publicly available checkpoints support reproducible comparisons. Specifically, we evaluated Qwen2.5-VL-7B and InternVL2-8B, choosing alternatives with similar parameter counts to the default verifier. We additionally evaluated Qwen2.5-VL-3B to examine performance at a smaller model scale through comparison with its 7B counterpart within the same model family. All metrics are percentages.

Table 2. Detection performance with alternative verifier models (percentages).

VerifierUCF AUCUCF APXD AUCXD AP
URF-refined base84.35636.16491.32368.071
Qwen2-VL-7B-Instruct86.24638.36391.73872.584
Qwen2.5-VL-3B84.34637.60791.78572.546
Qwen2.5-VL-7B85.22937.90191.84672.735
InternVL2-8B85.73037.63691.77572.318

At comparable model sizes, Qwen2.5-VL-7B and InternVL2-8B each improve all four metrics over the URF-refined base. Within the Qwen2.5-VL family, the 7B verifier scores higher than the 3B verifier on all four metrics. The 3B verifier still improves AP on both datasets and AUC on XD-Violence over the base.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General Response to the Reviewers

We thank the reviewers for their feedback. We added two analyses to Appendix A and highlighted them in blue. On both datasets, feedback yielded higher frame-level precision than the URF-refined base over much of the recall range (Fig. 4, p. 13) and higher temporal mIoU averaged across videos at a fixed score threshold of 0.5 (p. 14).

AuthorsOpenReview ↗

Contribution of feedback and visual verification [iSmo, XSpL]

To isolate the contributions of the two feedback levels and their visual verification steps, we evaluated component combinations on the same URF-refined base. We kept the VAU descriptions and all feedback parameters fixed across these variants. All metrics are percentages; bold and underlined values indicate the best and second-best results in each metric column, respectively.

Table 1. Controlled ablation of feedback levels and visual verification. UCF = UCF-Crime; XD = XD-Violence.

RowVideo feedbackSelective video recheckingLocal feedbackTarget-local verificationUCF AUC ↑UCF AP ↑XD AUC ↑XD AP ↑
A————84.35636.16491.32368.071
B✓———84.61736.87691.65071.376
C✓✓——86.225‾\underline{86.225}37.56791.64171.366
D——✓✓84.46136.87691.45368.978
E✓—✓✓84.67237.69991.74772.597
F✓✓✓—86.03537.841‾\underline{37.841}91.66772.079
G (Full)✓✓✓✓86.24638.36391.738‾\underline{91.738}72.584‾\underline{72.584}

Complementary feedback (A, C, D, G). Video-level feedback with rechecking (C) and local feedback with verification (D) each improve all four metrics over the base (A). Combining both feedback levels and their verification steps (G) further improves all four metrics over either C or D. Adding local feedback with verification to configuration C (C → G) increases UCF-Crime and XD-Violence AP by 0.796 and 1.218 percentage points, respectively.

Target-local verification (F → G). With both feedback levels active, adding target-local verification improves all four metrics, including AP gains of 0.522 and 0.505 percentage points on UCF-Crime and XD-Violence. This comparison isolates the benefit of visually confirming an anomaly in the target window before increasing selected scores.

Video-level rechecking (B → C; E → G). Adding video-level rechecking primarily benefits UCF-Crime. In E → G, UCF-Crime AUC and AP increase by 1.574 and 0.664 percentage points, respectively, while XD-Violence performance remains nearly unchanged.

AuthorsOpenReview ↗

Parameter selection and sensitivity [iSmo, prQf]

Selection procedure. For the URF-HVAA experiments, we selected a single feedback configuration by considering overall test-set performance across datasets and metrics. The same configuration was used for UCF-Crime, XD-Violence, UBnormal, and MSAD.

We constructed the anomaly vocabulary from UCF-Crime class names, using ChatGPT to expand them with synonyms and related expressions. The vocabulary was not adjusted based on evaluation results.

Sensitivity analysis. After selecting the configuration, we expanded the one-parameter-at-a-time sensitivity analysis to uniform grids to assess whether the gains depend on the exact selected values. Table 2 reports the mean over all tested values in each grid, with the other parameters fixed at the main configuration. Scores are percentages; the grid column gives start/end/step.

Table 2. Mean performance over each one-parameter uniform grid.

ParameterGrid (start/end/step)UCF AUCUCF APXD AUCXD AP
URF-refined base—84.35636.16491.32368.071
Main configuration—86.24638.36391.73872.584
Normal scale α0.10/0.90/0.1085.82238.22091.73672.586
Low-score scale β0.10/0.90/0.1085.22739.29991.28772.990
Global quantile0.10/0.90/0.1086.18536.43291.63371.892
Peak minimum mm0.05/0.80/0.0585.32135.47691.09171.186
Peak-score floor ff0.50/1.00/0.0586.23038.58691.65271.758
Local strength λ0.05/0.95/0.0586.25338.09291.71572.122
Local quantile τℓτ_ℓ0.05/0.95/0.0586.04735.64491.01568.087

Across these sweeps, every tested value of α, ff, and λ improves all four metrics over the URF-refined base. For mm and τℓτ_ℓ, improvements on all four metrics persist across the tested ranges 0.25–0.80 and 0.70–0.95, respectively, with the remaining parameters fixed. The reported values, m = 0.45 and τℓ=0.85τ_ℓ = 0.85, lie inside these contiguous ranges. Thus, the gains persist across multiple alternative settings and do not require the exact reported configuration. Lowering the peak minimum mm can allow lower-scoring anchors to be sharpened. Lowering the local quantile τℓτ_ℓ can increase the number of anchors promoted within a visually verified window. Since an anomaly may occupy only part of that window, a higher quantile concentrates promotion on anchors with higher current scores.

Area_Chair_5NLHOpenReview ↗

AC update

Dear authors,

Thank you for your time and contributions to the discussion.

I understand that you may be concerned if some reviewers have not provided follow-up comments or explicitly acknowledged how their concerns have been addressed in the rebuttal and revised manuscript. Please be assured that the AC will carefully consider all available information, including the original reviews, author rebuttal, revised manuscript, and discussion comments, as part of the final assessment.

The absence of further reviewer comments does not mean that the authors’ responses and revisions have been overlooked. I will carefully examine the issues raised, assess the authors’ clarifications and revisions, and take the discussion into account when forming my recommendation. I will also consult with the reviewers as appropriate to ensure that the relevant issues are fully considered.

Thank you again for your constructive engagement and contributions to the review process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.