- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Seeing Twice, a training-free video anomaly detection framework that addresses missing visual evidence and ambiguous video descriptions through selective hierarchical re-observation. The method constructs coarse- and fine-grained event intervals, identifies intervals requiring further observation using visual–text mismatch and hierarchical score disagreement, and revisits selected intervals together with their parent or child companions. Recovered descriptions are admitted through visual evidence agreement and local claim support before anomaly rescoring. However, this work contains some obvious AI generated content, making it less convincing.
Critical strengths
-
The paper addresses a meaningful limitation of description-based anomaly detection with a coherent framework. The separation between selective evidence acquisition and evidence-based description selection is well motivated.
-
The framework provides inspectable descriptions and prediction revisions. By retaining initial descriptions, recovered descriptions, and corresponding score changes, the method allows individual prediction revisions to be traced to specific changes in the textual evidence.
Critical weaknesses
-
The experiments do not sufficiently isolate the contributions of the central hierarchical re-observation mechanisms. Although the event hierarchy is used throughout the pipeline, the paper does not directly compare EfficientGEBD-based construction with uniform coarse–fine segmentation or fixed-length multiscale windows under comparable observation budgets. Furthermore, the two observation stages use different prompts, with the second explicitly emphasizing subtle actions and object interactions. A single-observation baseline using this more targeted prompt is missing. These comparisons are important for determining whether the improvements require the proposed hierarchical two-stage design or can largely be achieved through simpler sampling and prompting strategies.
-
The parameters in experiments does not adequately support a fair comparison of system complexity. And the default mixed-model configuration lacks a convincing cost–benefit justification. Table 1 reports only the largest individual VLM, listing Seeing Twice as 8B. Although this convention is explicitly disclosed in the footnote, it omits the additional 7B initial observer, the separate 14B scoring and claim-extraction LLM, and the feature and boundary encoders. The three main generative models alone therefore have a combined size of approximately 29B. Reporting only the largest component obscures differences between systems that reuse one model and those requiring multiple distinct models. A fair resource comparison should provide a component-wise breakdown and consistently account for all required models across methods, supplemented by peak memory and end-to-end inference time. Moreover, CHAI -> CHAI performs only 0.03/0.07 percentage points below the default LLaVA -> CHAI configuration on UCF/XD. Without paired uncertainty estimates and resource measurements for these alternatives, the practical benefit of introducing an additional observer remains unsubstantiated.
-
The evidence-grounding mechanism lacks direct validation of description factuality and temporal consistency. Figure S2.C establishes that the admission mechanism improves downstream detection metrics, but does not directly demonstrate that accepted revisions are more factually accurate or contain fewer unsupported claims. Moreover, Equation (6) matches each claim independently to its best-supporting local window, while Appendix A.4 represents each window using averaged frame features. This verification does not explicitly enforce temporal ordering or consistency across claims. Individual claims may find support in different windows without jointly supporting the described event. This limitation is relevant to the paper’s emphasis on subtle actions and temporal changes. A manually annotated subset evaluating claim correctness and accepted versus rejected revisions would help establish whether the mechanism reliably recovers missing evidence and resolves ambiguity, beyond improving aggregate detection performance.
-
The terminology does not precisely match the described failure modes. “Implicit” is not equivalent to missing or omitted evidence, while “ambiguous” is not equivalent to an incorrect or hallucinated description. Several motivating examples appear to concern omitted actions or erroneous event recognition rather than implicit cues or genuine semantic ambiguity. The authors should clarify these distinctions and use terminology that more accurately characterizes the problems addressed.
Minor Issues,
- Figure S2 is incorrectly referenced as Table 2.
Questions for authors
-
Can the gains be attributed to hierarchical re-observation rather than to a larger observation budget and a more targeted second-stage prompt. How does the method compare with a single-observation baseline using that same prompt and with equal-budget uniform or multiscale sampling or more observations, and do annotated checks show that accepted revisions are genuinely more factual and temporally consistent?
-
Since the pipeline actually uses three generative models (~29B nominal) plus encoders, why does Table 1 report only the largest 8B model, and what component-wise parameters, peak memory, and end-to-end latency justify adding a separate observer when CHAI -> CHAI trails the default by only 0.03/0.07 points without paired uncertainty estimates?
Other questions please refer to the weakness.
8 comments
Q2: Whether the improvements require the proposed hierarchical two-stage design or can largely be achieved through simpler sampling and prompting strategies?
A2: Our controlled experiments demonstrate that hierarchical sampling and two-stage re-observation perform substantially better than simpler sampling and prompting strategies.
1. Hierarchical sampling. Under matched observation budgets, our EfficientGEBD-based sampling outperforms uniform coarse–fine and fixed multiscale sampling in both event alignment and detection accuracy, achieving a 2.09-point AUC improvement over fixed multiscale sampling.
Table R1. Comparison of sampling strategies on UCF-Crime.
| Sampling strategy | mIoU ↑ | Single-stage AUC ↑ | Two-stage AUC ↑ |
|---|---|---|---|
| Uniform coarse–fine | 0.39 | 82.85 | 83.58 |
| Fixed multiscale | 0.42 | 83.12 | 85.03 |
| EfficientGEBD (Ours) | 0.47 | 84.86 | 87.12 |
2. Prompt strategies. We compare different prompts across observation stages. Prompt 1 is the original initial-observation prompt, while Prompt 2 is the original re-observation prompt, which places greater emphasis on subtle actions and object interactions. Using Prompt 2 in a single-stage observation achieves 85.01% AUC, while all two-stage prompt configurations perform better than single-stage observation, with Prompt 1→Prompt 2 achieving the best result of 87.12% AUC.
Table R2. Prompt configurations across observation stages on UCF-Crime (ROC-AUC, %).
| Protocol | AUC ↑ |
|---|---|
| Single-stage Prompt 1 | 84.86 |
| Single-stage Prompt 2 | 85.01 |
| Two-stage Prompt 1→Prompt 1 | 86.79 |
| Two-stage Prompt 2→Prompt 2 | 87.10 |
| Seeing Twice Prompt 1→Prompt 2 | 87.12 |
Q3: The parameter reporting does not adequately support a fair comparison of system complexity, and the default mixed-model configuration lacks a convincing cost–benefit justification.
A3: We provide a complete model-parameter comparison and a cost–benefit analysis of different model configurations to address both concerns.
1. Complete parameter accounting. We report deduplicated parameters for all VLMs, LLMs, and auxiliary encoders. Compared with existing VLM+LLM methods, our method achieves higher detection accuracy at comparable parameter scales. Compared with VLM-only methods, our method achieves a clear performance margin, while further reducing the reasoning LLM from 14B to 8B has little impact on accuracy.
Table R3. Deduplicated parameters (B) and UCF-Crime ROC-AUC (%) across different methods.
| Method | VLM | LLM | Aux | Total | AUC ↑ |
|---|---|---|---|---|---|
| EventVAD | ≈8.42 | 0 | ≈0.15 | ≈8.58 | 82.03 |
| AnyAnomaly | ≈8.50 | 0 | ≈0.15 | ≈8.65 | 80.70 |
| ReCI | ≈8.10 | 0 | — | ≥8.10 | 81.51 |
| LAVAD-UCF | ≈35.61 | 13 | ≈1.20 | ≈49.81 | 80.28 |
| VADTree | ≈8.03 | ≈14.77 | ≈1.23 | ≈24.03 | 84.74 |
| Seeing Twice (Default) | ≈16.80 | ≈14.77 | ≈1.23 | ≈32.79 | 87.12 |
| Seeing Twice (CHAI→CHAI, 14B LLM) | ≈8.77 | ≈14.77 | ≈1.23 | ≈24.76 | 87.09 |
| Seeing Twice (CHAI→CHAI, 8B LLM) | ≈8.77 | ≈8.03 | ≈1.23 | ≈18.02 | 86.95 |
Default: LLaVA-Video→CHAI with a 14B reasoning LLM; CHAI→CHAI variants use a 14B or 8B reasoning LLM. ≈: approximate; —: unavailable.
2. Cost–benefit justification. We evaluate the cost of different configurations in terms of peak GPU memory and end-to-end inference time on 2×RTX 4090. Because the VLM for re-observation is reloaded rather than kept resident alongside the initial observer, the default mixed-model and CHAI→CHAI/14B configurations show comparable peak VRAM, inference time, and accuracy. These results indicate that our framework is insensitive to the choice and combination of observation VLMs.
Table R4. Resource comparison on 2×RTX 4090.
| Configuration | Peak VRAM/GPU (GB) ↓ | E2E (min/video) ↓ | AUC ↑ |
|---|---|---|---|
| Seeing Twice (LLaVA-CHAI, Default) | 22.1 | 18.1 | 87.12 |
| Seeing Twice (CHAI→CHAI) | 22.3 | 18.2 | 87.09 |
Q4: The evidence-grounding mechanism lacks direct validation of description factuality and temporal consistency.
A4: We manually annotated 200 accepted and 200 rejected revisions, each containing 100 samples from UCF-Crime and 100 from XD-Violence. Three independent annotators compared initial and re-observed descriptions against the original videos, with anomaly scores and admission decisions hidden. The results directly support the factual reliability of our evidence-grounding mechanism and show that re-observation preserves temporal consistency.
1. Factuality and admission reliability. Manual assessment shows that 82.0% of accepted revisions improve description factuality, compared with only 13.0% of rejected revisions. The annotations demonstrate substantial agreement (Fleiss' κ = 0.76), supporting the admission mechanism's ability to distinguish beneficial from unreliable revisions.
Table R5. Manual assessment of accepted versus rejected revisions.
| Annotator judgment | Accepted (200) | Rejected (200) |
|---|---|---|
| Re-observed better | 82.0% | 13.0% |
| Tie (initial = re-observed) | 10.5% | 12.0% |
| Initial better | 7.5% | 75.0% |
2. Temporal consistency. We further assess temporal-order correctness and cross-claim event consistency on the 200 accepted pairs. Re-observation slightly improves temporal-order correctness while maintaining comparable cross-claim event consistency. These results indicate that re-observation does not disrupt temporal relationships while recovering more faithful descriptions.
Table R6. Manual assessment of temporal consistency (%).
| Metric | Initial | Re-observed |
|---|---|---|
| Temporal-order correctness ↑ | 81.3 | 82.5 |
| Cross-claim event consistency ↑ | 84.7 | 83.2 |
Q5: The terminology does not precisely match the described failure modes.
A5: The terminology (implicit and ambiguous) used in our paper follows the definitions in the Oxford Advanced Learner's Dictionary and accurately reflects the described failure modes.
1. Implicit. In our paper, implicit refers to anomalous cues suggested by visual evidence but not explicitly captured in the initial description. This is consistent with the dictionary definition of implicit: “suggested without being directly expressed.” In Fig. 1(a), the initial description mentions only item handling, while the video suggests the anomalous act of moving goods from a display into a bag. Considering both the visual evidence and textual description, the anomaly cue is implied but not explicitly expressed.
2. Ambiguity. In our paper, ambiguity refers to multiple possible interpretations of the same visual event, which can affect anomaly judgments. This corresponds to the dictionary definition of ambiguous: “that can be understood in more than one way.” In Fig. 1(b), the scene shows a person standing in front of an elevator with his hands behind his back. Across different observation rounds, the VLM may interpret the scene as either restraint or waiting for an elevator. These competing interpretations illustrate semantic ambiguity and can lead to different anomaly judgments.
We thank the reviewer for your thoughtful and constructive reviews of our manuscript, and provide individual responses to your concerns below.
A1: Figures 1 and 2 were manually drawn in Microsoft PowerPoint, not generated using AI tools. Every label is an editable text object rather than rendered pixels, and the original PPTX files are provided in the supplementary material.
Both figures were designed around our methodology. Taking Figure 2 as an example, we explain its design at three levels:
1. Globally. Figure 2 follows the core idea of hierarchical re-observation. The left-to-right layout organizes the pipeline from observation to reasoning, while the top-to-bottom symmetry highlights the differences between the two observation stages.
2. Locally. We emphasize three core modules—hierarchical observation, evidence-gap routing, and evidence-grounded disambiguation—and connect them according to the data flow, reflecting their roles and interactions in our framework.
3. In detail. We use light blue and warm orange to visually distinguish the initial observation and re-observation stages. This color coding is consistently applied throughout their respective data flows.
We appreciate that the figures' 3D-inspired visual style, including soft shading and depth cues, may appear blurred under magnification, particularly after image compression. We have refined certain aspects of the figures' visual style and replaced both figures in the revised manuscript. If the reviewer could identify any specific regions that remain unclear, we would be happy to refine them further.