- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper investigates the challenge of small-object anomaly localization in complex scenes and proposes SAVOR, a training-free framework that organizes fragmented anomaly evidence through temporal modeling (TCM), global-local spatial event reasoning (SER), and cross-frame mask propagation refinement (APR). The authors also construct SOAL, a small-object anomaly localization benchmark derived from ShanghaiTech and UBnormal. Experiments on standard VAD benchmarks and SOAL demonstrate competitive localization performance, particularly for small-object anomalies. Ablation studies and additional analyses generally support the effectiveness of the proposed components. However, some implementation details and additional experimental analyses are still needed to better justify certain design choices.
Critical strengths
- The paper proposes SAVOR, a training-free video anomaly detection framework that improves small-object anomaly localization through temporal evidence sampling, global-local spatial reasoning, and cross-frame mask propagation refinement. It achieves competitive localization performance on standard VAD benchmarks and substantial improvements on the small-object anomaly localization benchmark.
2.The paper is clearly written, and the proposed methodology is easy to follow. The experimental evaluation is systematic, with ablation studies and targeted analyses supporting the effectiveness of the main components and providing empirical evidence for the paper's key contributions.
Critical weaknesses
-
Trajectory construction is fundamental to the proposed framework and plays an important role in subsequent anomaly localization. Although the overall approach is reasonable, the trajectory-level cues introduced in the Temporal Evidence Accumulation module require further clarification. What specific motion and track-quality metrics are used, and what are their corresponding weights ? How were these metrics and weights determined? Were they manually specified based on empirical experience or determined through other procedures?
-
The temporal sampling strategy used to construct the Global Context Mosaic also affects anomaly localization performance. The current design divides each video into three macro temporal intervals and selects three frames from each interval. However, the paper does not clearly explain how these intervals are determined or why this configuration was chosen. It would also be useful to evaluate different sampling budgets, such as selecting two or four frames per interval, to examine their effects on performance and further justify the design choices.
-
The sampling strategy for constructing the Motion Candidate Strip requires further clarification. Specifically, how are the five temporally distributed observations selected from each candidate trajectory? Additionally, an analysis of how the number of sampled observations affects performance would help justify the current configuration.
-
The paper primarily focuses on improving small-object anomaly localization, and Table 2 reports performance across different object sizes. Meanwhile, the Abstract and Introduction emphasize the complementary roles of TCM, SER, and APR. Therefore, it would be beneficial to extend the component ablation study in Table 3 by reporting results across different object-size ranges. Such an analysis would provide more direct experimental evidence of how each component contributes to small-object anomaly localization and further support the paper's main claims.
Questions for authors
- Evaluation on other spatially annotated benchmarks: SOAL is constructed exclusively from ShanghaiTech and UBnormal, although other VAD benchmarks, such as Street Scene and ComplexVAD, also provide spatial annotations for anomalous objects. Did the authors consider using these datasets to evaluate small-object anomaly localization? If so, why were they not included?
2.Construction and quality control of semantic annotations: The paper introduces a unified event-level semantic annotation layer containing 140 semantic events across 19 anomaly types. Could the authors clarify how these annotations were constructed? Were they manually annotated, derived from existing dataset labels, or obtained through a combination of both? Additionally, what quality-control procedures, such as manual verification, were used to ensure annotation reliability?
5 comments
Q3: Motion Candidate Strip Sampling and Observation-Count Sensitivity.
We select five approximately evenly spaced observations from the primary identity-consistent segment of each candidate trajectory. Additional experiments with 3, 5, and 7 observations show only modest gains beyond five. We therefore retain five to balance temporal evidence and visual input overhead.
(i) Sampling strategy.
We prioritize the identity-consistent primary segment of each trajectory, and uniformly sample five observations from the temporally ordered track samples. If the primary segment contains fewer than five observations, we supplement from the same parent trajectory or repeat available observations as needed. Each observation is cropped with surrounding context and resized to a consistent scale.
(ii) Sensitivity analysis.
We further evaluate 3, 5, and 7 observations per trajectory on ShanghaiTech under otherwise identical settings (Table R4). Increasing from five to seven observations yields only marginal improvements, despite requiring more visual input. This makes five a practical balance between temporal coverage and input efficiency.
Table R4. Sensitivity to the number of observations per trajectory on ShanghaiTech.
| Observations | AUROC | AP | AUPRO | F1 |
|---|---|---|---|---|
| 3 | 75.62 | 45.16 | 49.68 | 57.87 |
| 5 (Ours) | 76.37 | 45.65 | 51.30 | 58.88 |
| 7 | 76.53 | 45.98 | 52.24 | 58.98 |
Q4: Extend Table 3 with component ablation results across different object-size ranges.
As shown in Table R5, we extend the component ablation in Table 3 by reporting Macro Pixel-AP on ShanghaiTech across the five object-size ranges defined in Table 2. These results demonstrate the complementary contributions of TCM, SER, and APR, with full SAVOR achieving the best performance across all five ranges. We will incorporate these scale-wise results into Table 3 of the revised manuscript.
Table R5. Scale-wise Macro Pixel-AP for component ablations on ShanghaiTech test set.
| TCM | SER | APR | ≤0.25% | 0.25–0.5% | 0.5–1% | 1–2% | >2% |
|---|---|---|---|---|---|---|---|
| 0.66 | 2.19 | 9.19 | 18.07 | 32.36 | |||
| ✓ | 1.71 | 4.66 | 11.55 | 20.36 | 34.10 | ||
| ✓ | ✓ | 2.04 | 5.38 | 12.13 | 23.18 | 34.76 | |
| ✓ | ✓ | 5.32 | 10.91 | 19.28 | 27.28 | 37.92 | |
| ✓ | ✓ | ✓ | 6.76 | 11.75 | 24.40 | 30.75 | 39.42 |
Q5: Evaluation on other spatially annotated benchmarks.
We considered several spatially annotated VAD benchmarks during SOAL construction, including Street Scene, ComplexVAD and UCF-Crime.
Since SOAL targets fine-grained and scale-stratified small-object localization, we prioritized datasets with official pixel-level anomaly ground truth to maintain a consistent evaluation protocol.
Street Scene and ComplexVAD provide bounding-box annotations, so including them would require additional mask annotation, verification, and baseline re-evaluation. We also attempted manual pixel-level annotation for UCF-Crime, but low image quality made anomalous-object boundaries difficult to delineate.
For these reasons, we selected ShanghaiTech and UBnormal, covering six real-world scenes and 19 synthetic scenes. SOAL harmonizes their spatial annotations and adds 140 semantic events across 19 anomaly types, making it particularly suitable for evaluating small-object localization across different size ranges and anomaly categories.
Q6: Construction and quality control of semantic annotations.
Our event-level semantic annotations were manually constructed, with reference to the original dataset annotations and relevant prior work. Three annotators independently reviewed and annotated the data, with inconsistent judgments resolved through manual review.
(i) Annotation construction.
- For ShanghaiTech (43 videos), we identified distinct anomaly events and their temporal intervals using the official frame-wise masks and video content. Three annotators independently described each event. We consolidated their descriptions, merged equivalent categories, and aligned the resulting labels with the C-ShT categories from AnyAnomaly where applicable (e.g.,
cyclingandbicycleasbicycle). Cases not covered by C-ShT were manually reviewed. - For UBnormal (47 videos), three annotators independently assigned semantic categories to the official anomaly-object tracks, referencing the 12 categories provided in its test set.
(ii) Quality control and event associations.
After manual verification, we assigned a shared event_id to anomaly trajectories belonging to the same semantic event.
For 160 ShanghaiTech frames where anomalous regions from different categories overlap and cannot be reliably separated, we preserved the original spatial ground truth and excluded these frames from category-wise spatial evaluation. Harmonizing the categories across all 90 videos yielded the unified SOAL annotation layer with 19 anomaly types and 140 semantic events.
(iii) Annotation format.
Each semantic event is represented by its video ID, event ID, category, and temporal intervals:
{
"video_id": "shanghaitech_02_0128",
"event_id": "shanghaitech_02_0128_E001",
"category": "bicycle",
"frame_ranges": [
[start_frame, end_frame]
]
}
We will add the above annotation construction and quality-control procedures to Appendix A of the revised manuscript.
We sincerely appreciate the detailed feedback, which has helped us clarify our design choices and strengthen the supporting analyses.
References
[1] Zhor Chergui and Antonio Jiménez-Martín. On ordinal information-based weighting methods and comparison analyses. Information, 15(9), 2024. ISSN 2078-2489. doi: 10.3390/info15090527.
We thank the reviewer for recognizing the contributions and clarity of SAVOR. We appreciate the detailed and constructive comments on our manuscript. The individual responses to your concerns are presented below.
(i) Trajectory-level cues and weighting strategy.
We organize eight motion and track-quality cues into four groups, prioritizing reliable object-supported motion, followed by local motion contrast, background interference suppression, and auxiliary burst evidence. The resulting group and per-cue weights are summarized in Table R1, where rg, ng, Wg and wℓ denote the priority rank, number of cues, group weight, and per-cue weight, respectively.
Table R1. Trajectory-level cue groups and weighting scheme.
(ii) Weight determination.
Inspired by Rank-Sum ordinal weighting [1], we assign each group a rank score sg=5−rg, yielding (4,3,2,1). Each cue inherits the group score, giving unnormalized group scores ngsg=(12,6,4,1).
We normalize the positive evidence to a unit budget, with Z+=12+6+1=19, while retaining background interference as an explicit negative contribution. The group weights follow: Wg=σgZ+ngsg,Z+=19, where σg=+1 for supporting evidence and σg=−1 for background suppression. In particular, the background-suppression group has WP=−4/19, where the numerator 4=nPsP=2×2 is derived from the same rank-scoring rule. Within each group, individual cues share the same weight wℓ=Wg/ng. All weights are fixed and consistently applied across datasets.
(i) Temporal partitioning and configuration.
Each video is divided into three equal-duration intervals, providing effective coverage of the entire video. Within each interval, we select two separated evidence peaks to avoid relying on a single strong motion event, together with a coverage anchor farther from the peaks to preserve broader temporal context.
Under the same nine-frame budget, we further compare four routing strategies: uniform sampling uses fixed temporal spacing, evidence-focused routing prioritizes high-evidence frames, segmented coverage emphasizes temporal coverage, and TCM combines evidence peaks with coverage anchors. As shown in Table R2 (see also Table 10 of the initial draft), TCM achieves the best balance between informative observations and temporal coverage.
Table R2. Comparison of temporal routing strategies under the same nine-frame budget.
(ii) Sampling Budget Sensitivity.
We additionally evaluate two, three, and four samples per interval under otherwise identical settings (Table R3). Two and three yield comparable performance, while four provides no consistent gains. Hence, SAVOR is relatively insensitive to the sampling budget, and we retain three samples as a balanced default across metrics.
Table R3. Sampling-budget sensitivity on ShanghaiTech.