acceptodds
Under review as a conference paper at ICLR 2027

Seeing Twice: Hierarchical Re-Observation for Training-Free Video Anomaly Detection

Abstract

Training-free video anomaly detection relies on large pretrained models to detect abnormal events without task-specific training. However, most existing methods inspect model-generated descriptions only once, making them vulnerable to implicit and ambiguous anomaly cues. Specifically, context-dependent anomaly cues may remain hidden at a single temporal granularity and become discernible only through multi-granularity observation, while semantically ambiguous cues may persist when the initial visual description is incomplete or unreliable. In this paper, we propose Seeing Twice, a hierarchical re-observation framework that uncovers implicit cues at the observation level and resolves semantic ambiguity at the reasoning level. Our method constructs a hierarchical event representation with intervals at multiple temporal granularities. At the observation level, to recover implicit cues, the framework selectively revisits intervals insufficiently observed at a single granularity, recovering context-dependent evidence by jointly examining them with their hierarchical companions. At the reasoning level, to resolve semantic ambiguity, the framework selects between the initial and recovered descriptions according to their agreement with the visual evidence. It then reconciles the corresponding anomaly scores through hierarchical score propagation to produce coherent frame-level predictions. Seeing Twice achieves 87.12% ROC-AUC on UCF-Crime and 75.93% AP on XD-Violence, surpassing existing methods by a clear margin. These results demonstrate the effectiveness of the proposed framework for training-free video anomaly detection. Moreover, our method provides interpretable, human-readable cues for anomalous event analysis.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video anomaly detection, Vision-language models, Video understanding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_VFepOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Seeing Twice, a training-free video anomaly detection framework that addresses missing visual evidence and ambiguous video descriptions through selective hierarchical re-observation. The method constructs coarse- and fine-grained event intervals, identifies intervals requiring further observation using visual–text mismatch and hierarchical score disagreement, and revisits selected intervals together with their parent or child companions. Recovered descriptions are admitted through visual evidence agreement and local claim support before anomaly rescoring. However, this work contains some obvious AI generated content, making it less convincing.

Critical strengths
  1. The paper addresses a meaningful limitation of description-based anomaly detection with a coherent framework. The separation between selective evidence acquisition and evidence-based description selection is well motivated.

  2. The framework provides inspectable descriptions and prediction revisions. By retaining initial descriptions, recovered descriptions, and corresponding score changes, the method allows individual prediction revisions to be traced to specific changes in the textual evidence.

Critical weaknesses
  1. The experiments do not sufficiently isolate the contributions of the central hierarchical re-observation mechanisms. Although the event hierarchy is used throughout the pipeline, the paper does not directly compare EfficientGEBD-based construction with uniform coarse–fine segmentation or fixed-length multiscale windows under comparable observation budgets. Furthermore, the two observation stages use different prompts, with the second explicitly emphasizing subtle actions and object interactions. A single-observation baseline using this more targeted prompt is missing. These comparisons are important for determining whether the improvements require the proposed hierarchical two-stage design or can largely be achieved through simpler sampling and prompting strategies.

  2. The parameters in experiments does not adequately support a fair comparison of system complexity. And the default mixed-model configuration lacks a convincing cost–benefit justification. Table 1 reports only the largest individual VLM, listing Seeing Twice as 8B. Although this convention is explicitly disclosed in the footnote, it omits the additional 7B initial observer, the separate 14B scoring and claim-extraction LLM, and the feature and boundary encoders. The three main generative models alone therefore have a combined size of approximately 29B. Reporting only the largest component obscures differences between systems that reuse one model and those requiring multiple distinct models. A fair resource comparison should provide a component-wise breakdown and consistently account for all required models across methods, supplemented by peak memory and end-to-end inference time. Moreover, CHAI -> CHAI performs only 0.03/0.07 percentage points below the default LLaVA -> CHAI configuration on UCF/XD. Without paired uncertainty estimates and resource measurements for these alternatives, the practical benefit of introducing an additional observer remains unsubstantiated.

  3. The evidence-grounding mechanism lacks direct validation of description factuality and temporal consistency. Figure S2.C establishes that the admission mechanism improves downstream detection metrics, but does not directly demonstrate that accepted revisions are more factually accurate or contain fewer unsupported claims. Moreover, Equation (6) matches each claim independently to its best-supporting local window, while Appendix A.4 represents each window using averaged frame features. This verification does not explicitly enforce temporal ordering or consistency across claims. Individual claims may find support in different windows without jointly supporting the described event. This limitation is relevant to the paper’s emphasis on subtle actions and temporal changes. A manually annotated subset evaluating claim correctness and accepted versus rejected revisions would help establish whether the mechanism reliably recovers missing evidence and resolves ambiguity, beyond improving aggregate detection performance.

  4. The terminology does not precisely match the described failure modes. “Implicit” is not equivalent to missing or omitted evidence, while “ambiguous” is not equivalent to an incorrect or hallucinated description. Several motivating examples appear to concern omitted actions or erroneous event recognition rather than implicit cues or genuine semantic ambiguity. The authors should clarify these distinctions and use terminology that more accurately characterizes the problems addressed.

Minor Issues,

  1. Figure S2 is incorrectly referenced as Table 2.
Questions for authors
  1. Can the gains be attributed to hierarchical re-observation rather than to a larger observation budget and a more targeted second-stage prompt. How does the method compare with a single-observation baseline using that same prompt and with equal-budget uniform or multiscale sampling or more observations, and do annotated checks show that accepted revisions are genuinely more factual and temporally consistent?

  2. Since the pipeline actually uses three generative models (~29B nominal) plus encoders, why does Table 1 report only the largest 8B model, and what component-wise parameters, peak memory, and end-to-end latency justify adding a separate observer when CHAI -> CHAI trails the default by only 0.03/0.07 points without paired uncertainty estimates?

Other questions please refer to the weakness.

8 comments

We thank the reviewer for your thoughtful and constructive reviews of our manuscript, and provide individual responses to your concerns below.

Q1: Figures 1 and 2 appear to have been generated using AI tools.

A1: Figures 1 and 2 were manually drawn in Microsoft PowerPoint, not generated using AI tools. Every label is an editable text object rather than rendered pixels, and the original PPTX files are provided in the supplementary material.

Both figures were designed around our methodology. Taking Figure 2 as an example, we explain its design at three levels:

1. Globally. Figure 2 follows the core idea of hierarchical re-observation. The left-to-right layout organizes the pipeline from observation to reasoning, while the top-to-bottom symmetry highlights the differences between the two observation stages.

2. Locally. We emphasize three core modules—hierarchical observation, evidence-gap routing, and evidence-grounded disambiguation—and connect them according to the data flow, reflecting their roles and interactions in our framework.

3. In detail. We use light blue and warm orange to visually distinguish the initial observation and re-observation stages. This color coding is consistently applied throughout their respective data flows.

We appreciate that the figures' 3D-inspired visual style, including soft shading and depth cues, may appear blurred under magnification, particularly after image compression. We have refined certain aspects of the figures' visual style and replaced both figures in the revised manuscript. If the reviewer could identify any specific regions that remain unclear, we would be happy to refine them further.


Q2: Whether the improvements require the proposed hierarchical two-stage design or can largely be achieved through simpler sampling and prompting strategies?

A2: Our controlled experiments demonstrate that hierarchical sampling and two-stage re-observation perform substantially better than simpler sampling and prompting strategies.

1. Hierarchical sampling. Under matched observation budgets, our EfficientGEBD-based sampling outperforms uniform coarse–fine and fixed multiscale sampling in both event alignment and detection accuracy, achieving a 2.09-point AUC improvement over fixed multiscale sampling.

Table R1. Comparison of sampling strategies on UCF-Crime.

Sampling strategymIoU ↑Single-stage AUC ↑Two-stage AUC ↑
Uniform coarse–fine0.3982.8583.58
Fixed multiscale0.4283.1285.03
EfficientGEBD (Ours)0.4784.8687.12

2. Prompt strategies. We compare different prompts across observation stages. Prompt 1 is the original initial-observation prompt, while Prompt 2 is the original re-observation prompt, which places greater emphasis on subtle actions and object interactions. Using Prompt 2 in a single-stage observation achieves 85.01% AUC, while all two-stage prompt configurations perform better than single-stage observation, with Prompt 1→Prompt 2 achieving the best result of 87.12% AUC.

Table R2. Prompt configurations across observation stages on UCF-Crime (ROC-AUC, %).

ProtocolAUC ↑
Single-stage Prompt 184.86
Single-stage Prompt 285.01
Two-stage Prompt 1→Prompt 186.79
Two-stage Prompt 2→Prompt 287.10
Seeing Twice Prompt 1→Prompt 287.12

Q3: The parameter reporting does not adequately support a fair comparison of system complexity, and the default mixed-model configuration lacks a convincing cost–benefit justification.

A3: We provide a complete model-parameter comparison and a cost–benefit analysis of different model configurations to address both concerns.

1. Complete parameter accounting. We report deduplicated parameters for all VLMs, LLMs, and auxiliary encoders. Compared with existing VLM+LLM methods, our method achieves higher detection accuracy at comparable parameter scales. Compared with VLM-only methods, our method achieves a clear performance margin, while further reducing the reasoning LLM from 14B to 8B has little impact on accuracy.

Table R3. Deduplicated parameters (B) and UCF-Crime ROC-AUC (%) across different methods.

MethodVLMLLMAuxTotalAUC ↑
EventVAD≈8.420≈0.15≈8.5882.03
AnyAnomaly≈8.500≈0.15≈8.6580.70
ReCI≈8.100—≥8.1081.51
LAVAD-UCF≈35.6113≈1.20≈49.8180.28
VADTree≈8.03≈14.77≈1.23≈24.0384.74
Seeing Twice (Default)≈16.80≈14.77≈1.23≈32.7987.12
Seeing Twice (CHAI→CHAI, 14B LLM)≈8.77≈14.77≈1.23≈24.7687.09
Seeing Twice (CHAI→CHAI, 8B LLM)≈8.77≈8.03≈1.23≈18.0286.95

Default: LLaVA-Video→CHAI with a 14B reasoning LLM; CHAI→CHAI variants use a 14B or 8B reasoning LLM. ≈: approximate; —: unavailable.

2. Cost–benefit justification. We evaluate the cost of different configurations in terms of peak GPU memory and end-to-end inference time on 2×RTX 4090. Because the VLM for re-observation is reloaded rather than kept resident alongside the initial observer, the default mixed-model and CHAI→CHAI/14B configurations show comparable peak VRAM, inference time, and accuracy. These results indicate that our framework is insensitive to the choice and combination of observation VLMs.

Table R4. Resource comparison on 2×RTX 4090.

ConfigurationPeak VRAM/GPU (GB) ↓E2E (min/video) ↓AUC ↑
Seeing Twice (LLaVA-CHAI, Default)22.118.187.12
Seeing Twice (CHAI→CHAI)22.318.287.09

Q4: The evidence-grounding mechanism lacks direct validation of description factuality and temporal consistency.

A4: We manually annotated 200 accepted and 200 rejected revisions, each containing 100 samples from UCF-Crime and 100 from XD-Violence. Three independent annotators compared initial and re-observed descriptions against the original videos, with anomaly scores and admission decisions hidden. The results directly support the factual reliability of our evidence-grounding mechanism and show that re-observation preserves temporal consistency.

1. Factuality and admission reliability. Manual assessment shows that 82.0% of accepted revisions improve description factuality, compared with only 13.0% of rejected revisions. The annotations demonstrate substantial agreement (Fleiss' κ = 0.76), supporting the admission mechanism's ability to distinguish beneficial from unreliable revisions.

Table R5. Manual assessment of accepted versus rejected revisions.

Annotator judgmentAccepted (200)Rejected (200)
Re-observed better82.0%13.0%
Tie (initial = re-observed)10.5%12.0%
Initial better7.5%75.0%

2. Temporal consistency. We further assess temporal-order correctness and cross-claim event consistency on the 200 accepted pairs. Re-observation slightly improves temporal-order correctness while maintaining comparable cross-claim event consistency. These results indicate that re-observation does not disrupt temporal relationships while recovering more faithful descriptions.

Table R6. Manual assessment of temporal consistency (%).

MetricInitialRe-observed
Temporal-order correctness ↑81.382.5
Cross-claim event consistency ↑84.783.2

Q5: The terminology does not precisely match the described failure modes.

A5: The terminology (implicit and ambiguous) used in our paper follows the definitions in the Oxford Advanced Learner's Dictionary and accurately reflects the described failure modes.

1. Implicit. In our paper, implicit refers to anomalous cues suggested by visual evidence but not explicitly captured in the initial description. This is consistent with the dictionary definition of implicit: “suggested without being directly expressed.” In Fig. 1(a), the initial description mentions only item handling, while the video suggests the anomalous act of moving goods from a display into a bag. Considering both the visual evidence and textual description, the anomaly cue is implied but not explicitly expressed.

2. Ambiguity. In our paper, ambiguity refers to multiple possible interpretations of the same visual event, which can affect anomaly judgments. This corresponds to the dictionary definition of ambiguous: “that can be understood in more than one way.” In Fig. 1(b), the scene shows a person standing in front of an elevator with his hands behind his back. Across different observation rounds, the VLM may interpret the scene as either restraint or waiting for an elevator. These competing interpretations illustrate semantic ambiguity and can lead to different anomaly judgments.

Reviewer_Zja9OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes Seeing Twice, a training-free VAD framework that adds selective re-observation to description-based anomaly scoring. The framework re-describes intervals with inconsistent descriptions within a coarse–fine event hierarchy and accepts a revised description only if visual evidence better supports it. Scores are then reconciled across the hierarchy to produce frame-level predictions. They evaluate the method on UCF-Crime and XD-Violence, with ablations, observer comparisons, and a cost analysis.

Critical strengths

The paper targets a real limitation of description-based training-free VAD: a single pass can omit decisive actions or produce unsupported interpretations. It addresses this with a selective, interpretable mechanism. The experiments are carefully controlled. Gains over single observation hold with the same model for both observations, and a query-matched random second look performs clearly worse, showing that interval selection rather than extra computation drives the improvement.

Critical weaknesses

The framework relies on many hand-set thresholds, admission margins, and propagation and smoothing settings, but the paper does not describe how they were chosen. Since only the official test splits are used, it is unclear whether these settings were tuned on test data. The authors partly mitigate this by sharing one configuration across both datasets and reporting a budget sweep, leave-one-out ablations and repeated scoring runs, but a sensitivity analysis of the key thresholds is missing. In addition, the disambiguation ablations show aggregate gains, but whether individual admitted revisions are correct is shown only qualitatively. This is a concern because admission relies on ImageBind’s coarse visual–text alignment to verify fine-grained actions.

Questions for authors

Clarifications:

  1. How were the thresholds, admission margin, and propagation and smoothing settings selected, and was any test data involved?. How sensitive are the results to the key thresholds, particularly the routing threshold, admission margin and local-support threshold?
  2. How reliably does ImageBind-based claim support distinguish fine-grained actions, such as those illustrated in Figure 1?

Suggestions: Add a limitations section with failure cases, such as admitted revisions that hurt detection.

4 comments

We appreciate the reviewer's constructive feedback and recognition of our selective and interpretable design, carefully controlled experiments, and effective interval selection. The individual responses to your concerns are provided below.


Q1: How were the thresholds, admission margin, and propagation and smoothing settings selected, and was any test data involved? How sensitive are the results to the key thresholds?

A1: No test-set annotations or ROC-AUC/AP results were used to select parameters. All settings were fixed based on the roles of their components and applied unchanged to UCF-Crime and XD-Violence.

1. Parameter selection. For normalized evidence scores, we use τr = τq = 0.50 as natural midpoint thresholds to provide balanced criteria for routing and claim support. For revision admission, we set δ = 0.02 as a small positive margin to prevent unnecessary changes caused by minor differences in visual agreement. For hierarchical re-observation, we use K = 2 to balance complementary observations against query cost. For temporal smoothing, we use Gaussian scales Σ = {10, 25, 50, 100, 200} with σ₀ = 25, following a similar smoothing strategy to Kim et al. [1]. The full settings are reported in Appendix Table S3.

2. Parameter sensitivity. We vary each parameter independently while keeping the others fixed. The results show that τr = τq = 0.50 provides balanced thresholds for routing and claim support, as both lower and higher values lead to performance degradation. Detection performance is relatively insensitive to the admission margin δ. K = 2 achieves a favorable balance between detection accuracy and computational efficiency.

Table R1. Routing-threshold sensitivity (τr; UCF ROC-AUC / XD AP, %).

τrUCFXD
0.3086.0775.18
0.4086.7475.33
0.50 (default)87.1275.93
0.6086.8975.76

Table R2. Local-support-threshold sensitivity (τq; UCF ROC-AUC / XD AP, %).

τqUCFXD
0.3086.1475.31
0.4086.4175.36
0.50 (default)87.1275.93
0.6087.0175.65

Table R3. Admission-margin sensitivity (δ; UCF ROC-AUC / XD AP, %).

δUCFXD
0.02 (default)87.1275.93
0.0487.0375.03
0.0687.1075.85
0.0886.9975.76

Table R4. Companion-budget sensitivity (K; UCF ROC-AUC / XD AP, %).

KUCFXD
186.4174.98
2 (default)87.1275.93
387.1075.89

References

[1] Jaehyun Kim, Seongwook Yoon, Taehyeon Choi, and Sanghoon Sull. Unsupervised Video Anomaly Detection Based on Similarity with Predefined Text Descriptions. Sensors, 23(14):6256, 2023. DOI: 10.3390/s23146256.


Q2: How reliably does ImageBind-based claim support distinguish fine-grained actions, such as those illustrated in Figure 1?

A2: Our manual evaluation supports the reliability of ImageBind-based claim support within our evidence-grounded admission mechanism.

1. Manual evaluation. Three independent annotators compared 200 accepted and 200 rejected revision pairs against their videos with anomaly scores and admission decisions hidden. 82.0% of accepted revisions were more factual, compared with 13.0% of rejected revisions; 92.5% of accepted revisions were non-worse, with substantial inter-annotator agreement (Fleiss' κ = 0.76).

Table R5. Manual evaluation of accepted and rejected revisions.

Annotator judgmentAccepted (200)Rejected (200)
Re-observed better82.0%13.0%
Tie10.5%12.0%
Initial better7.5%75.0%

2. Fine-grained examples. Using the condensed descriptions in Fig. 1, (a) the initial observation "Retrieve or place an item?" is refined through re-observation to "Items moved into a bag." (b) the initial observation "Handcuffed or waiting?" is refined to "Waiting for the elevator." Re-observation thus recovers the missing anomaly cue in Fig. 1(a) and corrects the unsupported anomalous interpretation in Fig. 1(b).


Q3: Add a limitations section with failure cases, such as admitted revisions that hurt detection.

A3: One example is Normal_Videos_010 from UCF-Crime, frames 237–546. The footage is dimly lit, with the person partially occluded. The initial description depicts a person standing near a doorway and examining an object in their hands. However, the admitted revision incorrectly describes the person as “engaged in a physical altercation with another person,” despite the footage being normal and showing no such interaction. This raises the node anomaly score from 0.1 to 0.7.

This failure highlights the difficulty of fine-grained action verification under low-light conditions, and partial occlusion. Improving verification under challenging visual conditions is a direction for future work.

We will add a Limitations section with a visualization and discussion of this failure case in the revised manuscript.


We sincerely appreciate the detailed feedback, which has helped us clarify our parameter choices and better characterize the limitations of our method.

Reviewer_Zja9OpenReview ↗

Post Rebuttal

I appreciate the authors’ detailed response. Please also incorporate the new experiments and implementation details into the final manuscript. Having reviewed the additional results, I will keep my score accordingly.

Thank you for your time and thoughtful feedback. We have incorporated the additional experiments and implementation details into the revised manuscript. We sincerely appreciate your careful consideration of our work.

Reviewer_qzpdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Seeing Twice, a training-free hierarchical re-observation framework for video anomaly detection (VAD). It identifies two challenges: implicit anomaly cues (coarse intervals lack decisive action detail; fine intervals lack context) and semantic ambiguity (descriptions may misread actions or temporal relations). At the observation level, evidence-gap routing selects intervals and hierarchical companions (parent/children) for a second look. At the reasoning level, visual evidence agreement and local claim support decide whether a recovered description should replace the initial one before rescoring. Hierarchical score propagation and temporal smoothing produce frame-level predictions. The method reports 87.12% ROC-AUC on UCF-Crime and 75.93% AP on XD-Violence, with query-budget sweeps, five-seed repeats, ablations, and cost analysis.

Critical strengths
  1. Clear problem decomposition: missing evidence and ambiguous evidence are handled at observation and reasoning levels, respectively.
  2. The pipeline is interpretable: routing, companion selection, evidence agreement, and claim support all have explicit operational meanings.
  3. Truly training-free: no fine-tuning of VLMs, prompts, or prediction heads; results hold across LLaVA/CHAI observer combinations.
  4. Broad evaluation: two benchmarks, five scoring seeds, routing/companion/disambiguation ablations, budget sweep, GPU cost accounting, and qualitative cases.
  5. Cost-benefit analysis is useful: default 20% seed budget re-observes about 28.6%/30.8% of intervals, adds ~3.5 GPU minutes per video, and yields 2.26/7.91 point gains; exhaustive re-observation saturates.
  6. Ablations are reasonably controlled: routing fixes seed counts, companion tests fix seeds, and disambiguation reuses the same candidate descriptions.
Critical weaknesses
  1. Local claim support with min is overly strict. A single unsupported claim rejects the entire description. This may systematically penalize information-rich recovered descriptions and favor vague ones. Mean or soft-min alternatives should be discussed.
  2. ImageBind as the claim verifier is not justified. Claim support relies entirely on ImageBind visual-text similarity, which is not designed for fine-grained action/object alignment. No comparison with CLIP, BLIP, or direct VLM verification is provided.
  3. Selective comparison in the main table. VadCLIP reaches 88.02% ROC-AUC on UCF-Crime, higher than the proposed 87.12%, but the text only highlights gains over RTFM, UR-DMU, and MGFN. The claim should be explicitly scoped to training-free methods, and the gap to stronger weakly supervised methods should be discussed.
  4. Inference cost remains high. Each video costs about 0.273 GPU hours and requires LLaVA-Video-7B, CHAI-SFT-8B, DeepSeek-R1-14B, and ImageBind. For a training-free method, comparison against a simpler multi-prompt/multi-description voting baseline is needed to isolate the benefit of “seeing twice.”
Questions for authors
  1. Did you try CLIP or direct frozen-VLM verification instead of ImageBind for claim support? How sensitive is performance to the verifier?
  2. Compared with a simpler training-free baseline (e.g., multiple prompts/descriptions per interval plus voting/self-consistency), how much of the gain comes from re-observation itself versus routing and admission?
  3. VadCLIP has higher UCF-Crime ROC-AUC than Seeing Twice. Why is this not discussed? Can you explicitly scope the claim to “best among training-free methods”?
  4. Can you provide quantitative failure analysis, such as oracle routing upper bounds or admission precision/recall?
2 comments

We appreciate the reviewer's constructive feedback and recognition of our framework's novelty, interpretability, training-free design, as well as its comprehensive experimental evaluation. The individual responses to your concerns are provided below.


Q1: Compared with simpler multi-prompt/multi-description voting or self-consistency baselines, how much of the gain comes from re-observation itself versus routing and admission?

A1: Our method substantially outperforms simpler multi-prompt/multi-description baselines under the same query budget. Controlled ablations further demonstrate the contributions of hierarchical re-observation and evidence-grounded admission.

1. Simpler baselines. Under the same budget of 11,076 VLM queries (8,613 initial + 2,463 additional), random multi-description averaging and multi-prompt self-consistency achieve 84.78% and 85.07% AUC, respectively. Adding evidence-gap routing to the averaging baseline improves AUC to 85.43%. All configurations use the same frozen observers, scorer, and downstream temporal refinement.

2. Our hierarchical re-observation. Seeing Twice selects intervals with evidence gaps, re-observes them alongside hierarchical companions, and admits revisions only when supported by visual evidence. Seeing Twice achieves 87.12% AUC, substantially outperforming these simpler baselines.

Table R1. Budget-matched multi-description baselines on UCF-Crime.

ConfigurationVLM QueriesROC-AUC (%) ↑
Single observation8,61384.86
Random multi-description + averaging11,07684.78
Random multi-prompt + self-consistency11,07685.07
Routed multi-description + averaging11,07685.43
Seeing Twice (Full)11,07687.12

Settings: Single: one description per interval, Random + Averaging: random re-observation with score averaging, Multi-Prompt + Self-Consistency: different prompts focusing on actions and context, with median score aggregation, Routed + Averaging: evidence-gap routing and hierarchical companions with score averaging, Ours: evidence-grounded admission and hierarchical score reconciliation. All additional-observation methods use the same frozen models and 11,076 VLM queries.


Q2: How reliably does ImageBind verify fine-grained claims, and how sensitive is performance to the verifier?

A2: Manual evaluation supports the reliability of our admission decisions, and verifier substitutions show limited performance sensitivity. ImageBind provides an effective accuracy–efficiency trade-off.

1. Reliability of ImageBind-based Verification. Three independent annotators compared 200 admitted and 200 rejected revisions against the original videos, blinded to anomaly scores and admission decisions. 82.0% of admitted revisions were better grounded and 92.5% were non-worse, whereas 75.0% of rejected revisions favored the initial description (Fleiss' κ = 0.76).

Table R2. Manual assessment of description revisions.

Annotator judgmentAdmitted (200)Rejected (200)
Re-observed better82.0%13.0%
Tie (re-observed = initial)10.5%12.0%
Initial better7.5%75.0%

2. Verifier sensitivity. We conducted a sensitivity analysis by replacing ImageBind with CLIP and a frozen VLM. The results show that verifier choice has only a minor impact on detection performance. Since direct frozen-VLM verification incurs substantial additional GPU computation, ImageBind offers a favorable balance between accuracy and efficiency.

Table R3. Claim verifier sensitivity on UCF-Crime.

Claim verifierROC-AUC (%) ↑
ImageBind (Default)87.12
CLIP ViT-B/3287.07
CLIP ViT-L/1486.85
Frozen LLaVA-Video-7B87.23

Q3: Can you provide quantitative failure analysis, including oracle routing and admission precision/recall?

A3: Under the same query budget, our routing approaches a ground-truth-guided oracle reference, while achieving strong admission precision and recall.

1. Oracle routing. We use ground-truth anomaly annotations to guide the selection of re-observation intervals through an offline search, while keeping all other settings unchanged. Seeing Twice achieves 87.12% AUC, only 0.43 points below the GT-guided reference (87.55%), demonstrating that our routing approaches the oracle routing upper bound.

Table R4. Oracle routing on UCF-Crime.

ConfigurationVLM QueriesROC-AUC (%) ↑
Seeing Twice11,07687.12
GT-guided routing11,07687.55

2. Admission precision/recall. Using the independent human judgments of admitted and rejected revisions in Table R2, we compute admission precision and candidate-pool-weighted recall. Precision reaches 84.0% / 80.0% and recall 82.7% / 81.4% on UCF-Crime / XD-Violence. Most admitted revisions are visually better grounded, supporting the reliability of our admission mechanism.

Table R5. Manual assessment of admission precision and recall.

MetricUCF-Crime (%)XD-Violence (%)
Admission Precision84.080.0
Admission Recall82.781.4

Q4: Does the minimum in Eq. (6) penalize information-rich descriptions and favor vague ones?

A4: The min aggregation in Eq. (6) is a conservative claim-support constraint, not a preference for vague descriptions. The inner max allows each claim to find support in any local temporal window; the outer min prevents a single unsupported claim from being masked by supported claims. A recovered description must also improve visual evidence agreement to be admitted (Eqs. (7)–(8)), so passing the claim-support check alone does not favor a vague revision.

To test whether min is overly strict for information-rich recovered descriptions, we compare min, mean, and soft-min under identical settings. Min achieves 87.12% AUC, exceeding soft-min (86.87%) and mean (86.39%) by 0.25 and 0.73 points, respectively. These results support retaining the conservative constraint to limit unsupported claims in anomaly predictions.

Table R6. Local claim aggregation on UCF-Crime.

AggregationROC-AUC (%) ↑
Mean86.39
Soft-min86.87
Min (Ours)87.12

Q5: VadCLIP has higher UCF-Crime ROC-AUC than Seeing Twice. Why is this not discussed? Can you explicitly scope the claim to “best among training-free methods”?

A5: We will explicitly discuss the comparison with VadCLIP in the revised manuscript. VadCLIP optimizes task-specific components using video-level labels, whereas Seeing Twice requires no labeled VAD training videos or task-specific optimization of weights, prompts, or prediction heads. Although our method achieves lower performance than VadCLIP, its training-free design offers greater flexibility across detection tasks while providing interpretable evidence for anomaly decisions.

We will revise Section 4.2 to explicitly scope our performance claim to “best among the training-free methods” and added a discussion of the performance gap with VadCLIP.


We hope our responses and additional experiments have addressed your concerns. Thank you again for your constructive feedback, which has helped strengthen our paper.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary and General Response

Dear Reviewers and AC,

We sincerely thank all reviewers for their constructive feedback and recognition of the significance of the problem (VFep), novelty and interpretability of our framework (qzpd, Zja9), and comprehensive experimental evaluation (qzpd, Zja9).

We have conducted additional experiments and revised the manuscript accordingly. Below, we summarize the main improvements.

1. Figure Originality (Reviewer VFep)

We clarified the design and manual construction of both figures, provided the original editable PPTX files, and replaced both figures with refined versions. (VFep-Q1)

2. Effectiveness of Hierarchical Re-observation (Reviewers VFep, qzpd)

Budget-matched experiments demonstrate the advantages of hierarchical sampling and two-stage re-observation over simpler sampling, prompting, and multi-description strategies, as well as the proximity of our routing performance to the oracle reference. (VFep-Q2, qzpd-Q1, qzpd-Q3)

3. Reliability of Evidence-Grounded Admission (Reviewers VFep, qzpd, Zja9)

We conducted manual evaluation of 400 description revisions, verifier sensitivity analysis, and claim aggregation ablations, supporting the reliability and effectiveness of our evidence-grounded admission mechanism. (VFep-Q4, qzpd-Q2, Zja9-Q2)

4. Model Complexity and Parameter Sensitivity (Reviewers VFep, Zja9)

We provided complete parameter accounting, computational cost comparisons, and hyperparameter sensitivity analyses, clarifying the accuracy–efficiency trade-offs and supporting our parameter choices. (VFep-Q3, Zja9-Q1)


Detailed responses are provided under each review. We hope these additional experiments and revisions address the reviewers' concerns. Thank you again for your time and consideration.

Best regards,

Authors

Area_Chair_xqeaOpenReview ↗

Please join the discussion and reply to the authors if you still have concerns

Dear reviewers,

Thank you for your contribution to the ICLR review, and thanks to the reviewers who have joined the discussion.

The Paper Discussion phase is coming to an end very soon. Please could you take a look at the authors' reply, if you have not yet, and let the authors know if you still have any questions or concerns. Reviewer qzpd and Reviewer Zja9, please can you check the authors' response and see if you have any concerns by replying to it?

Thanks, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.