acceptodds
Under review as a conference paper at ICLR 2027

When Anomalies Are Too Small to See: Organizing evidence for Video Anomaly Detection

Abstract

Video anomaly detection (VAD) aims to identify events that deviate from normal activity patterns specific to a scene. Given the open-set and scene-dependent nature of anomalies, recent work leverages semantic priors from pretrained vision-language models (VLMs) for detection without target-scene training. These methods have achieved strong performance in frame-level anomaly detection. However, they struggle with small-object anomalies in complex scenes, primarily because subtle anomaly cues from small objects are easily overwhelmed by the dominant normal background. This subproblem remains largely underexplored. To address this gap, we present SAVOR, a framework for detecting small-object anomalies from three complementary perspectives: temporal modeling, spatial refinement, and cross-frame association. Specifically, we propose Temporal Compression Mosaic (TCM) to adaptively select keyframes and highlight motion-salient objects. Then, Spatial Event Reasoning (SER) consolidates spatially sparse trajectory observations into persistent object events, enabling joint reasoning over complementary global and local anomaly evidence. Finally, to mitigate tracking errors, Anchor Prompt Retargeting (APR) realigns unreliable anchor prompts via mask feedback to guide bidirectional SAM2 propagation, preserving object identity and mask stability. The pipeline requires no in-domain training or parameter updates. Comprehensive experiments on ShanghaiTech and UBnormal demonstrate that our method outperforms existing zero-shot and training-free approaches at both pixel and object levels. Additional evaluations on SOAL, a newly constructed small-object anomaly benchmark curated from standard VAD datasets, confirm substantial improvements in spatial localization and temporal tracking.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Anomaly Detection, Anomaly Localization, Vision-Language Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Yyn2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates the challenge of small-object anomaly localization in complex scenes and proposes SAVOR, a training-free framework that organizes fragmented anomaly evidence through temporal modeling (TCM), global-local spatial event reasoning (SER), and cross-frame mask propagation refinement (APR). The authors also construct SOAL, a small-object anomaly localization benchmark derived from ShanghaiTech and UBnormal. Experiments on standard VAD benchmarks and SOAL demonstrate competitive localization performance, particularly for small-object anomalies. Ablation studies and additional analyses generally support the effectiveness of the proposed components. However, some implementation details and additional experimental analyses are still needed to better justify certain design choices.

Critical strengths
  1. The paper proposes SAVOR, a training-free video anomaly detection framework that improves small-object anomaly localization through temporal evidence sampling, global-local spatial reasoning, and cross-frame mask propagation refinement. It achieves competitive localization performance on standard VAD benchmarks and substantial improvements on the small-object anomaly localization benchmark.

2.The paper is clearly written, and the proposed methodology is easy to follow. The experimental evaluation is systematic, with ablation studies and targeted analyses supporting the effectiveness of the main components and providing empirical evidence for the paper's key contributions.

Critical weaknesses
  1. Trajectory construction is fundamental to the proposed framework and plays an important role in subsequent anomaly localization. Although the overall approach is reasonable, the trajectory-level cues qi,ℓq_{i,\ell} introduced in the Temporal Evidence Accumulation module require further clarification. What specific motion and track-quality metrics are used, and what are their corresponding weights wℓw_\ell? How were these metrics and weights determined? Were they manually specified based on empirical experience or determined through other procedures?

  2. The temporal sampling strategy used to construct the Global Context Mosaic also affects anomaly localization performance. The current design divides each video into three macro temporal intervals and selects three frames from each interval. However, the paper does not clearly explain how these intervals are determined or why this configuration was chosen. It would also be useful to evaluate different sampling budgets, such as selecting two or four frames per interval, to examine their effects on performance and further justify the design choices.

  3. The sampling strategy for constructing the Motion Candidate Strip requires further clarification. Specifically, how are the five temporally distributed observations selected from each candidate trajectory? Additionally, an analysis of how the number of sampled observations affects performance would help justify the current configuration.

  4. The paper primarily focuses on improving small-object anomaly localization, and Table 2 reports performance across different object sizes. Meanwhile, the Abstract and Introduction emphasize the complementary roles of TCM, SER, and APR. Therefore, it would be beneficial to extend the component ablation study in Table 3 by reporting results across different object-size ranges. Such an analysis would provide more direct experimental evidence of how each component contributes to small-object anomaly localization and further support the paper's main claims.

Questions for authors
  1. Evaluation on other spatially annotated benchmarks: SOAL is constructed exclusively from ShanghaiTech and UBnormal, although other VAD benchmarks, such as Street Scene and ComplexVAD, also provide spatial annotations for anomalous objects. Did the authors consider using these datasets to evaluate small-object anomaly localization? If so, why were they not included?

2.Construction and quality control of semantic annotations: The paper introduces a unified event-level semantic annotation layer containing 140 semantic events across 19 anomaly types. Could the authors clarify how these annotations were constructed? Were they manually annotated, derived from existing dataset labels, or obtained through a combination of both? Additionally, what quality-control procedures, such as manual verification, were used to ensure annotation reliability?

5 comments

We thank the reviewer for recognizing the contributions and clarity of SAVOR. We appreciate the detailed and constructive comments on our manuscript. The individual responses to your concerns are presented below.

Q1: Clarification of trajectory-level cues and their weighting strategy.

(i) Trajectory-level cues and weighting strategy.

We organize eight motion and track-quality cues into four groups, prioritizing reliable object-supported motion, followed by local motion contrast, background interference suppression, and auxiliary burst evidence. The resulting group and per-cue weights are summarized in Table R1, where rgr_g, ngn_g, WgW_g and wℓw_\ell denote the priority rank, number of cues, group weight, and per-cue weight, respectively.

Table R1. Trajectory-level cue groups and weighting scheme.

Grouprgr_gngn_gCuesWgW_gwℓw_\ell
Reliable tracks (R)13Track quality, displacement consistency, object support+0.63+0.21
Local contrast (L)22Kinematic contrast, motion contrast+0.32+0.16
Background suppression (P)32Dynamic-background penalty, non-translational motion−0.21-0.11
Burst evidence (B)41Motion bursts+0.05+0.05

(ii) Weight determination.

Inspired by Rank-Sum ordinal weighting [1], we assign each group a rank score sg=5−rgs_g=5-r_g, yielding (4,3,2,1)(4,3,2,1). Each cue inherits the group score, giving unnormalized group scores ngsg=(12,6,4,1)n_gs_g=(12,6,4,1).

We normalize the positive evidence to a unit budget, with Z+=12+6+1=19Z_+=12+6+1=19, while retaining background interference as an explicit negative contribution. The group weights follow: Wg=σgngsgZ+,Z+=19,W_g=\sigma_g\frac{n_gs_g}{Z_+}, Z_+=19, where σg=+1\sigma_g=+1 for supporting evidence and σg=−1\sigma_g=-1 for background suppression. In particular, the background-suppression group has WP=−4/19W_P=-4/19, where the numerator 4=nPsP=2×24=n_Ps_P=2\times2 is derived from the same rank-scoring rule. Within each group, individual cues share the same weight wℓ=Wg/ngw_\ell=W_g/n_g. All weights are fixed and consistently applied across datasets.


Q2: Temporal sampling configuration and budget.

(i) Temporal partitioning and configuration.

Each video is divided into three equal-duration intervals, providing effective coverage of the entire video. Within each interval, we select two separated evidence peaks to avoid relying on a single strong motion event, together with a coverage anchor farther from the peaks to preserve broader temporal context.

Under the same nine-frame budget, we further compare four routing strategies: uniform sampling uses fixed temporal spacing, evidence-focused routing prioritizes high-evidence frames, segmented coverage emphasizes temporal coverage, and TCM combines evidence peaks with coverage anchors. As shown in Table R2 (see also Table 10 of the initial draft), TCM achieves the best balance between informative observations and temporal coverage.

Table R2. Comparison of temporal routing strategies under the same nine-frame budget.

Routing emphasisAnomalous frames (%)Interval coverage (%)
Uniform44.4477.32
Evidence-focused57.5369.59
Segmented coverage55.2478.35
TCM (Ours)57.6379.90

(ii) Sampling Budget Sensitivity.

We additionally evaluate two, three, and four samples per interval under otherwise identical settings (Table R3). Two and three yield comparable performance, while four provides no consistent gains. Hence, SAVOR is relatively insensitive to the sampling budget, and we retain three samples as a balanced default across metrics.

Table R3. Sampling-budget sensitivity on ShanghaiTech.

Mosaic Samples(R×C)AUROCAPAUPROF1
2 (3×2)76.2045.8851.8759.01
3 (3×3, Ours)76.3745.6551.3058.88
4 (3×4)75.4842.9550.0259.21

Q3: Motion Candidate Strip Sampling and Observation-Count Sensitivity.

We select five approximately evenly spaced observations from the primary identity-consistent segment of each candidate trajectory. Additional experiments with 3, 5, and 7 observations show only modest gains beyond five. We therefore retain five to balance temporal evidence and visual input overhead.

(i) Sampling strategy.

We prioritize the identity-consistent primary segment of each trajectory, and uniformly sample five observations from the temporally ordered track samples. If the primary segment contains fewer than five observations, we supplement from the same parent trajectory or repeat available observations as needed. Each observation is cropped with surrounding context and resized to a consistent scale.

(ii) Sensitivity analysis.

We further evaluate 3, 5, and 7 observations per trajectory on ShanghaiTech under otherwise identical settings (Table R4). Increasing from five to seven observations yields only marginal improvements, despite requiring more visual input. This makes five a practical balance between temporal coverage and input efficiency.

Table R4. Sensitivity to the number of observations per trajectory on ShanghaiTech.

ObservationsAUROCAPAUPROF1
375.6245.1649.6857.87
5 (Ours)76.3745.6551.3058.88
776.5345.9852.2458.98

Q4: Extend Table 3 with component ablation results across different object-size ranges.

As shown in Table R5, we extend the component ablation in Table 3 by reporting Macro Pixel-AP on ShanghaiTech across the five object-size ranges defined in Table 2. These results demonstrate the complementary contributions of TCM, SER, and APR, with full SAVOR achieving the best performance across all five ranges. We will incorporate these scale-wise results into Table 3 of the revised manuscript.

Table R5. Scale-wise Macro Pixel-AP for component ablations on ShanghaiTech test set.

TCMSERAPR≤0.25%0.25–0.5%0.5–1%1–2%>2%
0.662.199.1918.0732.36
✓1.714.6611.5520.3634.10
✓✓2.045.3812.1323.1834.76
✓✓5.3210.9119.2827.2837.92
✓✓✓6.7611.7524.4030.7539.42

Q5: Evaluation on other spatially annotated benchmarks.

We considered several spatially annotated VAD benchmarks during SOAL construction, including Street Scene, ComplexVAD and UCF-Crime.

Since SOAL targets fine-grained and scale-stratified small-object localization, we prioritized datasets with official pixel-level anomaly ground truth to maintain a consistent evaluation protocol.
Street Scene and ComplexVAD provide bounding-box annotations, so including them would require additional mask annotation, verification, and baseline re-evaluation. We also attempted manual pixel-level annotation for UCF-Crime, but low image quality made anomalous-object boundaries difficult to delineate.

For these reasons, we selected ShanghaiTech and UBnormal, covering six real-world scenes and 19 synthetic scenes. SOAL harmonizes their spatial annotations and adds 140 semantic events across 19 anomaly types, making it particularly suitable for evaluating small-object localization across different size ranges and anomaly categories.

Q6: Construction and quality control of semantic annotations.

Our event-level semantic annotations were manually constructed, with reference to the original dataset annotations and relevant prior work. Three annotators independently reviewed and annotated the data, with inconsistent judgments resolved through manual review.

(i) Annotation construction.

  • For ShanghaiTech (43 videos), we identified distinct anomaly events and their temporal intervals using the official frame-wise masks and video content. Three annotators independently described each event. We consolidated their descriptions, merged equivalent categories, and aligned the resulting labels with the C-ShT categories from AnyAnomaly where applicable (e.g., cycling and bicycle as bicycle). Cases not covered by C-ShT were manually reviewed.
  • For UBnormal (47 videos), three annotators independently assigned semantic categories to the official anomaly-object tracks, referencing the 12 categories provided in its test set.

(ii) Quality control and event associations.

After manual verification, we assigned a shared event_id to anomaly trajectories belonging to the same semantic event. For 160 ShanghaiTech frames where anomalous regions from different categories overlap and cannot be reliably separated, we preserved the original spatial ground truth and excluded these frames from category-wise spatial evaluation. Harmonizing the categories across all 90 videos yielded the unified SOAL annotation layer with 19 anomaly types and 140 semantic events.

(iii) Annotation format.

Each semantic event is represented by its video ID, event ID, category, and temporal intervals:

{
  "video_id": "shanghaitech_02_0128",
  "event_id": "shanghaitech_02_0128_E001",
  "category": "bicycle",
  "frame_ranges": [
    [start_frame, end_frame]
  ]
}

We will add the above annotation construction and quality-control procedures to Appendix A of the revised manuscript.

We sincerely appreciate the detailed feedback, which has helped us clarify our design choices and strengthen the supporting analyses.

References

[1] Zhor Chergui and Antonio Jiménez-Martín. On ordinal information-based weighting methods and comparison analyses. Information, 15(9), 2024. ISSN 2078-2489. doi: 10.3390/info15090527.

Reviewer_HtecOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SAVOR, a training-free framework for small-object video anomaly detection and localization. It consists of three components: Temporal Compression Mosaic (TCM) for motion-guided keyframe selection, Spatial Event Reasoning (SER) for global-local anomaly reasoning, and Anchor Prompt Retargeting (APR) for improving SAM2-based mask propagation. The authors also introduce SOAL, a benchmark specifically designed for small-object anomaly localization. Experiments on UCSD Ped2, ShanghaiTech, and SOAL demonstrate competitive localization performance, achieving 55.25% pixel-level AP on SOAL and 45.65% AP on ShanghaiTech.

Critical strengths

1 The paper addresses small-object anomaly localization, an important challenge that is insufficiently explored in existing VAD methods. The analysis of fragmented anomaly evidence across temporal, spatial, and cross-frame dimensions provides a clear motivation. 2 The method achieves encouraging localization results, particularly on small-object anomalies. The newly introduced SOAL benchmark also enables more targeted evaluation across different object sizes and anomaly categories.

Critical weaknesses

1 The framework builds upon established techniques, including motion-based evidence selection, global-local VLM reasoning, and SAM2-based tracking. While the proposed components introduce task-specific refinements, their methodological advances over closely related pipelines such as GridVAD and TAO are not sufficiently established. 2 Although SAVOR achieves promising results, it does not consistently outperform existing methods across all evaluation metrics, with some results falling below competing approaches. Furthermore, the ablation studies show relatively modest improvements from APR, raising questions about its practical contribution to the overall framework.

Questions for authors

1 What are the main methodological differences between SAVOR and existing pipelines such as GridVAD and TAO, beyond the integration and refinement of established components? 2 Could the authors provide more controlled comparisons with relevant baselines under matched foundation models and inference budgets, where applicable, to better isolate the contributions of TCM, SER, and APR? 3 What are the end-to-end runtime, number of VLM calls, and computational costs per video? How do these compare with existing training-free methods?

5 comments

Thank you for the careful review and for raising questions. We provide further clarifications and additional experimental analyses to address each point below.

Q1 (weakness 1 & question 1): The main methodological differences between SAVOR and existing pipelines such as GridVAD and TAO.

A1: The key methodological distinction is how fragmented small-object evidence is organized and coupled across stages. We clarify the distinctions in the following three aspects: (i) Motion-guided sampling vs. Random sampling in evidence organization, (ii) Semantic reasoning vs. Feature matching in anomaly identification, and (iii) Mask feedback vs. Direct prompting in fine-grained tracking.

(i) Motion-guided sampling vs. Random sampling.

  • SAVOR uses motion-supported trajectories to guide temporal selection, and aggregate repeated observations into persistent object-event evidence.
  • GridVAD forms stratified grids by randomly sampling frames, and consolidates recurring proposals before grounding.

SAVOR links sparse anomalous observations through motion-guided sampling, while random sampling may fragment their temporal evidence.

(ii) Semantic reasoning vs. Feature matching.

  • SAVOR performs training-free semantic reasoning with a frozen VLM, integrating global scene context and trajectory-conditioned local evidence.
  • TAO relies on local feature matching, using a partially fine-tuned anomaly detector to score object boxes before SAM2.

SAVOR better understands anomalous objects through semantic reasoning, especially small ones overlooked by feature matching.

(iii) Mask feedback vs. Direct prompting.

  • SAVOR evaluates the initial SAM2 mask and retargets unreliable anchors before propagation.
  • GridVAD and TAO directly use grounded or detected prompts to initialize SAM2 propagation.

Ours improves prompt reliability and limits error accumulation during propagation.

These distinctions are supported by our existing component ablations and analyses (Tables 3-6 of the initial draft). TCM, SER and APR sequentially improve AP by +1.81, +6.02 and +1.98 points, while APR improves immediate-mask IoU/F1 by +7.77/+8.24 points.

Q2: Performance across evaluation metrics and the practical contribution of APR.

A2: SAVOR achieves the best results on most evaluation metrics, particularly those reflecting anomaly detection quality. APR primarily targets mask quality rather than overall anomaly-detection performance.

(i) Metric-wise Performance.

RBDC/TBDC use permissive region- and track-level matching (IoU ≥ 0.1), so high scores may coexist with substantial false object-level detections. We therefore report one-to-one object metrics in Table R1 (see also Table 12 of the initial draft), where unmatched and duplicate predictions count as false positives.

SAVOR achieves the highest O-AP and lowest O-FDR on both datasets while retaining over 80% O-Recall (Table R1), demonstrating substantially fewer false object-level detections alongside strong object-level detection performance.

Table R1. One-to-one object-level detection performance on UCSD Ped2 and ShanghaiTech.

DatasetMethodO-AP ↑O-Recall ↑O-FDR ↓
Ped2AnomalyRuler24.7156.3356.12
Ped2TAO62.7191.9555.16
Ped2SAVOR67.5482.3317.99
ShanghaiTechAnomalyRuler38.5557.8933.41
ShanghaiTechTAO37.9786.2365.67
ShanghaiTechSAVOR40.9581.7924.43

(ii) APR Contribution.

APR is primarily designed to improve mask quality rather than directly boost anomaly detection performance. By retargeting only unreliable segmentation anchors, its benefits are concentrated on a subset of predictions and diluted when metrics are aggregated over the full test set. Nevertheless, APR consistently improves all four pixel-level metrics in both controlled comparisons (Table R2).

Table R2. Controlled APR ablation on ShanghaiTech (see also Table 3 of the initial draft).

Base settingAPRAUROCAPAUPROF1
TCM×70.1437.6550.7946.46
TCM✓70.3238.3151.0947.04
TCM + SER×76.2343.6751.0757.61
TCM + SER✓76.3745.6551.3058.88

To further assess APR, Table R3 compares the immediate SAM2 masks before and after retargeting. The gains confirm that APR effectively corrects the unreliable anchors it is designed to handle.

Table R3. Immediate-mask quality before and after APR on ShanghaiTech (see also Table 6 of the initial draft).

APR subsetMetricOriginalRetargetedΔ
All attemptsIoU36.7744.54+7.77
F148.2356.47+8.24
APR-acceptedIoU25.4950.68+25.19
F134.1961.41+27.22

Q3: Controlled Comparison and Component Ablations on UCSD Ped2.

A3: We conducted the controlled comparison by replacing the original VLM in SAVOR with Qwen3-VL, the foundation model used by GridVAD. SAVOR still outperformed GridVAD on most metrics. Component ablations further support the complementary contributions of the proposed components.

(i) Comparison with GridVAD.

We conducted additional experiments on UCSD Ped2 by replacing the VLM in SAVOR with Qwen3-VL, the foundation model used by GridVAD. With the foundation model held fixed, SAVOR achieves the best results on three of the four pixel-level metrics (Table R4).

Table R4. Controlled comparison on UCSD Ped2 with Qwen3-VL.

ExpPixel-AUROCPixel-APPixel-AUPROPixel-F1
Direct Qwen3-VL70.9927.3539.4135.10
GridVAD (Qwen3-VL)77.5938.5366.8242.09
SAVOR (Qwen3-VL)82.0241.9854.5254.88

(ii) Component ablations.

The ablations conducted using Qwen3-VL also show that all three components contribute to the performance, with SER providing the largest overall gains (Table R5).

Table R5. SAVOR ablation on UCSD Ped2 with Qwen3-VL.

TCMSERAPRPixel-AUROCPixel-APPixel-AUPROPixel-F1
---70.9927.3539.4135.10
✓--73.7631.6344.5240.81
✓✓-80.1041.7846.1849.14
✓-✓74.1633.5745.1042.04
✓✓✓82.0241.9854.5254.88

Q4: Per-video runtime, VLM calls and computational cost compared with existing training-free methods.

A4: On UCSD Ped2, SAVOR averages 27.51 s, 6.50 VLM calls, and 6,704 tokens per video. Table R6 summarizes the per-video efficiency and localization performance.

GridVAD uses grid-sampled frames for proposal generation and consolidation, while SAVOR processes grounded instances individually, with costs scaling according to their number and temporal extent. Such fine-grained analysis enables SAVOR to achieve stronger localization than GridVAD across all metrics reported in Table R6, while requiring substantially fewer VLM calls than uniform sampling.

Table R6. Per-video efficiency and localization performance on UCSD Ped2.

MethodVLM callsTime (s)TokensPixel-AUROCPixel-APRBDCTBDC
Uniform sampling${}^{\dagger}$16.7553.38-----
GridVAD${}^{\dagger}$3.5823.06-77.5938.5338.9637.70
SAVOR (Qwen3-VL)6.5027.516,70482.0241.9846.5862.75

†{}^{\dagger} denotes results based on the original GridVAD paper [1].


We hope that the above responses address your concerns. We remain happy to discuss any points that may require further explanation.

References

[1] Mohamed Eltahir, Ahmed O. Ibrahim, Obada Siralkhatim, Tabarak Abdallah, and Sondos Mohamed. GridVAD: Open-set video anomaly detection via spatial reasoning over stratified frame grids. arXiv preprint arXiv:2603.25467, 2026. doi: 10.48550/arXiv.2603.25467.

Reviewer_UmTjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SAVOR, a training-free framework for small-object video anomaly detection and localization. The method aims to address the challenge of fragmented anomaly evidence across temporal, spatial, and cross-frame dimensions. Specifically, Temporal Compression Mosaic (TCM) selects informative keyframes while preserving temporal coverage, Spatial Event Reasoning (SER) organizes global scene context and local trajectory evidence for anomaly reasoning, and Anchor Prompt Retargeting (APR) improves the reliability of SAM2-based mask propagation. The framework leverages pretrained vision-language and segmentation models without additional training. Experiments on standard video anomaly detection benchmarks and the proposed SOAL benchmark demonstrate promising performance, particularly for small-object anomaly localization.

Critical strengths
  1. The paper addresses an important and challenging problem in video anomaly detection. Small-object anomalies are easily overlooked in complex scenes, and organizing fragmented visual evidence across temporal and spatial dimensions is a meaningful research direction.
  2. The proposed method achieves competitive performance on standard benchmarks and substantial improvements on the small-object anomaly localization benchmark. The experimental evaluation is relatively comprehensive, including comparisons with existing methods, component ablations, and analyses across different object scales.
  3. The paper is generally well written and logically organized. The motivation, methodology, and experimental findings are clearly presented, making the overall research objective and technical approach relatively easy to follow.
Critical weaknesses
  1. Dependence on motion-supported candidate trajectories. The proposed framework relies on motion information to generate candidate trajectories and organize visual evidence. However, small anomalous objects may exhibit subtle motion or lack sufficiently reliable motion trajectories due to their limited spatial resolution. This raises a concern about whether the method is particularly effective only when anomalous objects have detectable and consistent motion patterns. Although the framework includes a semantic rescue mechanism, its effectiveness when anomalous objects are not adequately covered by the motion candidates remains unclear. It would be valuable to analyze the candidate trajectory recall and detection performance under different motion conditions, particularly for weakly moving or nearly static anomalous objects.
  2. Insufficient comparison with direct VLM-based inference. Since the proposed framework relies on a pretrained VLM for anomaly reasoning, it would be important to compare its performance against directly using the same VLM to identify and localize anomalies, followed by the same SAM2-based segmentation procedure. Although the existing ablations demonstrate the benefits of global-local evidence organization, they do not fully establish the advantages over a simpler VLM-based inference pipeline. Furthermore, for medium-sized or larger anomalous objects that are already clearly visible in the original frames, direct VLM inference might achieve comparable performance with lower computational overhead. Additional comparisons across object scales, together with inference efficiency analyses, would help clarify the necessity and practical benefits of the proposed framework.
  3. Overly complex framework visualization. Figure 2 contains a large amount of information, including multiple processing stages, intermediate representations, and connections between components. Some text and annotations are too small to read comfortably, making the overall framework difficult to follow. Simplifying the figure, highlighting the main processing pipeline, and moving secondary implementation details to supplementary illustrations would improve its clarity and readability.
Questions for authors

Please see the "Weaknesses".

2 comments

We thank the reviewer for the thoughtful and constructive reviews. We appreciate the recognition of our motivation and the strength of our experimental evaluation. Below, we provide responses to the remaining concerns.

Q1: Dependence on motion-supported candidate trajectories.

A1: SAVOR is not limited to anomalous objects with consistently strong motion. We further analyze this from two perspectives.

To quantify motion dependence, we evaluate 173 small-object target trajectories in the SOAL dataset and measure motion using camera-compensated GT displacement: d_{i,t}=\frac{\left\\|(\mathbf{c}_{i,t+2}-\mathbf{c}_{i,t})-\mathbf{g}_{t,t+2}\right\\|_2}{\sqrt{W^2+H^2}}.

(i) Whole-track motion analysis.

Based on the whole-track motion score mi=mediant(di,t)m_i=\mathrm{median}_{t}(d_{i,t}), we divide all trajectories into three groups: Low, Medium, and High motion. As shown in Table R1, Candidate Recall decreases with weaker motion, whereas Detection Recall is less affected. This gap arises because summarizing the entire event with mim_i overlooks that low- and high-motion phases can coexist within the same trajectory, whereas SAVOR exploits informative phases to organize the complete anomalous event.

Table R1. Whole-track motion analysis.

MotionTargetsCandidate Recall ↑Detection Recall ↑Target-macro AP ↑
Low5821/58 (36.21%)46/58 (79.31%)45.15%
Medium5828/58 (48.28%)48/58 (82.76%)59.80%
High5737/57 (64.91%)51/57 (89.47%)50.85%

(ii) Phase-wise motion analysis.

To further examine the above point, we split the 173 trajectories into two groups based on whether they contain a High-motion phase. Table R2 shows that, despite weak overall motion mim_i, a trajectory may contain a locally strong phase that provides informative evidence for recovering the complete anomalous trajectory. Anomalies without High-motion phases may still be recognizable from appearance, interaction, or contextual cues, as in smoking, theft, or arson.

Table R2. Motion-phase analysis.

Motion PhaseTargetsCandidate Recall ↑Detection Recall ↑Target-macro AP ↑
Without High379/37 (24.32%)28/37 (75.68%)45.25%
Contains High13677/136 (56.62%)117/136 (86.03%)53.80%

Q2: Direct VLM comparison, object-scale performance, and inference efficiency.

A2: Under the same VLM and SAM2 backbone, SAVOR consistently outperforms direct VLM inference across all object scales, with the largest gains on tiny and small anomalies. The inference efficiency comparison is reported separately in Table R4.

(i) Direct VLM + SAM2 vs. SAVOR (Ours).

We compare both methods on ShanghaiTech and UCSD Ped2 test videos using the same VLM and SAM2 propagation.

  • The direct baseline inputs a uniformly sampled mosaic and its nine frames separately to the VLM, which identifies anomalies and predicts boxes for SAM2.
  • SAVOR uses TCM-selected frames with global-local reasoning through SER and APR to predict boxes for SAM2.

Table R3 shows that SAVOR improves all metrics under this controlled setting, confirming the benefit of structured evidence organization and grounding.

Table R3. Direct VLM + SAM2 vs. SAVOR on ShanghaiTech and UCSD Ped2.

MethoddatasetAUROC ↑AP ↑AUPRO ↑F1 ↑RBDC ↑TBDC ↑
Direct VLM + SAM2shanghaitech68.7935.8447.9143.9139.9547.92
SAVORshanghaitech76.3745.6551.358.8857.3175.46
Direct VLM + SAM2Ped270.3435.0539.9345.6344.5961.69
SAVORPed281.0948.4460.2965.3971.3680.98

(ii) Object-scale analysis and inference efficiency.

Setting Description:

  • Object-scale. We stratify ShanghaiTech anomaly regions by normalized bounding-box area, grouping the five original size ranges into tiny (≤0.5%), small (0.5–2%), medium (2–4%), and large (>4%). The original >2% group is further split at 4%, close to its median of 3.61%.
  • inference efficiency. Since VLM input resolution is fixed, inference cost is independent of object scales, depending instead on the number and temporal extent of grounded instances. We therefore report scale-wise Macro Pixel-AP alongside per-video average inference time and VLM calls in Table R4.

Result Analysis:

  • SAVOR achieves higher Macro Pixel-AP at every scale, with the largest gains on tiny and small anomalies, supporting the value of structured evidence organization for fine-grained localization.
  • At a comparable inference time, SAVOR achieves stronger localization, demonstrating more effective use of the inference budget.

We will include this analysis in the revised manuscript.

Table R4. Scale-wise Macro Pixel-AP and inference efficiency on ShanghaiTech test set using Qwen3.6-plus.

SettingTiny (≤0.5%)Small (0.5, 2%]Medium (2, 4%]Large (>4%)Time (s)VLM Calls
Direct VLM2.2318.1426.4830.5685.7910.00
SAVOR11.3034.5533.7635.0584.765.71

Q3: Simplifying the framework diagram in Figure 2 to improve clarity and readability.

A3: We have revised Figure 2 to highlight the main processing pipeline and simplify secondary implementation details, as follows:

  • Motion Evidence Mining: Replaced detailed video examples with schematic illustrations, clarifying how motion cues are organized into candidate trajectories.
  • TCM: Replaced the nine-frame mosaic with numbered schematic tiles, using filled and hollow circles to distinguish evidence peaks from coverage anchors and avoid duplication with SER's global-context mosaic.
  • SER: Reduced the number of displayed candidate trajectories.
  • Grounding and Propagation: Simplified cross-component connections and visual symbols, emphasizing the main processing sequence.
  • Overall readability: Enlarged key labels, removed nonessential annotations, and eliminated the bottom legend to reduce visual clutter.

The revised Figure 2 will replace the original, and has also been uploaded separately as supplementary material.


We hope the above responses have addressed your concerns. Thank you again for your constructive feedback and guidance. The suggested modifications certainly enhance our paper.

Comments on the paper

3 comments

Dear reviewers,

Thank you for your reviews and constructive feedback.

The authors have submitted their rebuttal. Please take a moment to review it and consider whether your concerns have been adequately addressed. It would be helpful if you could briefly indicate:

  • Whether the rebuttal addresses your main concerns;

  • Whether any significant issues remain unresolved;

  • Whether your overall recommendation has changed.

If appropriate, please also update your review and recommendation in the system.

Thank you for your time and valuable contributions.

Best regards,

AC

AuthorsOpenReview ↗

Summary and General Response

Dear reviewers and AC,

We sincerely thank all reviewers for their constructive feedback and recognition of the importance of small-object anomaly localization (UmTj, Htec, Yyn2), well-motivated framework (UmTj, Yyn2), and comprehensive experimental evaluation (UmTj, Htec, Yyn2).

We have conducted additional experiments and revised the manuscript accordingly, with updates highlighted in blue. Below, we summarize the main improvements.

1. Motion Robustness (Reviewer UmTj)

We analyzed candidate and detection recall across different motion conditions, demonstrating the complementary role of semantic reasoning in detecting weak-motion anomalies. The results are included in Section 4.3 (Table 4).

2. Controlled Comparisons and Efficiency (Reviewers UmTj, Htec)

We compared SAVOR with direct VLM inference under matched backbones and further evaluated object-scale performance and inference cost. The results are included in Section 4.4 (Table 6). Additional comparisons with GridVAD using the same VLM are provided in Response to Reviewer Htec (Q3–Q4).

3. Methodological Clarifications and Ablations (Reviewers Htec, Yyn2)

We clarified the differences between SAVOR and GridVAD/TAO, while extending the component ablations across object scales (Section 4.4, Table 5). We also clarified object-level detection reliability and APR's contribution (Section 4.2, Table 2; Section 4.4, Table 9).

4. Additional Clarifications (Reviewers Yyn2, UmTj)

We addressed motion-cue weighting, sampling configurations, and benchmark selection in Response to Reviewer Yyn2 (Q1–Q3, Q5), added SOAL annotation and quality-control details in Appendix A.1, and simplified Figure 2 for improved readability.


Detailed responses are provided under each review. We hope these additional analyses and revisions address the reviewers' concerns. Thank you again for your time and consideration.

Best regards,

Authors

Area_Chair_CeJcOpenReview ↗

AC update

Dear authors,

Thank you for your time and contributions to the discussion.

I understand that you may be concerned if some reviewers have not provided follow-up comments or explicitly acknowledged how their concerns have been addressed in the rebuttal and revised manuscript. Please be assured that the AC will carefully consider all available information, including the original reviews, author rebuttal, revised manuscript, and discussion comments, as part of the final assessment.

The absence of further reviewer comments does not mean that the authors’ responses and revisions have been overlooked. I will carefully examine the issues raised, assess the authors’ clarifications and revisions, and take the discussion into account when forming my recommendation. I will also consult with the reviewers as appropriate to ensure that the relevant issues are fully considered.

Thank you again for your constructive engagement and contributions to the review process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.