- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
-- Key Idea:- The authors propose TMVAD, a new tri-modal video anomaly detection (VAD) dataset using video, text, and sound, containing 2000 videos across 4 coarse-grained and 22 fine-grained categories. Additionally, the paper highlights the role of audio in tri-modal Video Anomaly Detection and proposes a novel asymmetrical way to use the audio signal alongside the other modalities.
-- Results:- Experiments are conducted on two datasets, XD-Violence and TMVAD. The results outperform prior methods.
Critical strengths
-
New trimodal dataset: The authors created a new video-audio-text dataset for Video Anomaly Detection, a domain where well-annotated data is very useful. The evaluation results tend to support the quality of the dataset, as the prior methods' results on TMVAD correlate with their performance on the reference dataset for the audio-visual domain, XD-Violence.
-
Well-motivated approach: The authors examine the role of the audio modality in relation to text and video, highlighting the challenges associated with audio's sensitivity to noise, temporal misalignment with the image, and semantic divergence from the video. With this analysis, the authors justify the need for their asymmetrical method in tri-modal anomaly detection.
-
Interesting and reliable method: The paper presents a new look-back strategy triggered by cross-modality discrepancy. This kind of safe strategy is welcome in a VAD model using audio.
-
Generalizability via Plug-and-Play Integration: Beyond their main framework, the authors show in the appendix that both the SIE module and the Look-Back mechanism can be seamlessly integrated into existing baselines, providing consistent performance improvements. This highlights the general applicability and robustness of their approach.
Critical weaknesses
-
Incremental Architectural Novelty: While the integration of an asymmetric pipeline for audio features and a Look-Back mechanism addresses cross-modal noise in VAD, the underlying technical components are not fundamentally novel. Asymmetric modality weighting and iterative refinement loops are widely used techniques in multimodal learning and vision architectures. In the current version of the paper, the contribution lies primarily in the application-specific assembly of existing briques and the creation of the TMVAD dataset, rather than a novel conceptual framework.
-
Limited applicability in real-world surveillance & Over-reliance on the audio modality: In real-world VAD scenarios, such as standard CCTV networks, audio streams are rarely available due to hardware constraints and privacy regulations. The datasets used in this work (XD-Violence and TMVAD) heavily rely on web videos or movie clips containing rich audio tracks. Furthermore, while Table 3 demonstrates that audio improves the proposed model's performance, it also reveals a strong dependency: without audio (Config 1: V+T only, achieving 84.68% AP on XD-Violence), the architecture underperforms compared to existing V+T SOTA baselines reported in Table 1. The authors should discuss the practical limitations of this audio-centric paradigm and clarify how the model handles completely silent or audio-unavailable surveillance settings.
-
Marginal Performance Gains on Coarse-Grained AP: The performance improvement on the XD-Violence benchmark for coarse-grained anomaly detection depicted in Table 1 is relatively narrow. By reporting the results of only one random seed, the authors make it difficult to determine whether this marginal gain represents a true statistical improvement or merely random variance during training. This issue is further compounded by a non-negligible computational cost, as the introduction of the SIE and Look-Back mechanisms increases the model's complexity to 14.03 GFLOPs, more than double that of competing frameworks like CMHKF (5.99 GFLOPs). If the performance gains of these components are neither statistically robust across random seeds nor efficient relative to their computational footprint, the practical value of adopting these modules in future SOTA models becomes questionable and weakens the paper’s claim of novelty.
-
No information on synthetic audio generation for TMVAD: In Appendix B.1, “Tri-modality collection,” the authors mention that for videos lacking original audio in the TMVAD dataset, they generated synthetic audio and synchronized it with the video. The authors do not specify the method or model used for audio synthesis. An interesting analysis could be provided on how synthetic audio enhances or worsens detection compared to natural audio.
-
Minor Formatting and Readability Issues in Tables 3 and 4: The presentation of the ablation studies in Tables 3 and 4 lacks consistency and creates unnecessary friction for the reader. In Table 3, the proposed full model (Config 4) is placed at the bottom, whereas in Table 4 (placed directly adjacent to it), the full model (Index 1) is placed at the top. Aligning the structural order of both tables would make comparisons much clearer. Furthermore, the “Config” and “Index” columns take up redundant space, as the remaining column descriptions are sufficient to identify each baseline in the text. Removing these redundant columns would improve visual clarity for the reader while saving layout space.
Questions for authors
-
Could the authors highlight the specific theoretical or algorithmic innovations of the SIE and Look-Back modules beyond adapting these established principles to tri-modal VAD?
-
Could the authors discuss the practical limitations of this audio-centric paradigm and clarify how the model handles completely silent or audio-unavailable surveillance settings? Also, often in video-audio surveillance settings, the audio part is very noisy and saturated. It is usually not used by security operators because it's not that informative.
-
Could the authors report AP for XD-Violence and TMVAD on their method with 3 or 5 other random seeds to prove that these results are not due to variance?
-
Could the authors provide information on the protocol they used to create synthetic audio streams for the silent videos?
-
Given that synthetic audio is introduced for silent clips in TMVAD, could the authors clarify its influence on the paradigm? Ideally, applying this audio synthesis pipeline to a standard VAD benchmark like UCF-Crime would provide a valuable sanity check. We understand that conducting such experiments on UCF-Crime may not be feasible within the discussion period; in this case, could the authors provide a performance breakdown (both coarse- and fine-grained AP/mAP) separating videos with natural audio from those with synthetic audio in TMVAD? Additionally, it would be insightful to report whether the Look-Back mechanism is triggered at different rates on synthetic versus natural audio clips.
2 comments
Dear Reviewer pM7z,
We truly appreciate your constructive comments helping us improve this paper.
We are sending a gentle reminder that the phase of the author-reviewer discussion is ending, and your problems mentioned in "Weaknesses" and "Questions" have already been addressed properly in both our first response and our revised paper. It would be much appreciated if you could kindly take some time to examine our response and tell us the remaining questions if any.
Thanks for your valuable time. May you have a good day.
Best wishes,
Authors
We clarify the design motivation of our asymmetric paradigm and Look-Back mechanism, demonstrate robustness through multi-seed tests and qualitative demos, and provide a sub-group analysis of natural versus synthetic audio on TMVAD.
Response
[W1 + Q1]:
Thank you for giving us the chance to clarify.
While asymmetric weighting and iterative loops have been explored in broader multimodal learning, existing tri-modal VAD works (e.g., CMHKF) adopt symmetric fusion and one-way feedforward pipelines. We do not apply existing methods, such as uncertainty-based weighting in EAU (CVPR 2024) or recursive cross-attention in RJCMA (CVPRW 2024), because they still allow noisy acoustic feature channels to penetrate the joint latent space. To propose our asymmetric paradigm, we first conducted a systematic statistical analysis on surveillance datasets in main text’s Fig. 3, discovering that video-audio semantic discrepancy is over four times larger than video-text discrepancy, and over 78% of surveillance audio events exhibit temporal latency. Driven by these findings, we designed SIE, which formulates audio as a modulator to preserve visual-textual semantic orientation. Second, rather than relying on generic iterative refinement, we introduced Look-Back as a conditional closed-loop verification process that monitors global semantic disparity and selectively recalibrates biased feature channels only when significant conflict is detected.
Combined with our new TMVAD dataset, we believe our method provides valuable insights for future research.
[W2 + Q2]:
We acknowledge that many standard static CCTV setups may lack audio streams due to privacy or hardware constraints. However, our goal is to explore audio in scenarios where it is available.
As noted by the reviewer, our V+T configuration achieves 84.68% AP on XD-Violence. While it does not surpass the dedicated V+T methods, it remains competitive and outperforms methods like SteerVAD (ICLR2026) and HeadHunt-VAD (AAAI2026). Note that our paradigm is designed not to over-rely on audio. When audio is silent or unavailable, the model degrades to a V+T mode.
To handle noisy or saturated audio (as mentioned by the reviewer), our asymmetric design acts as a safeguard. The SIE module verifies cross-modal alignment and only uses audio as an auxiliary trigger. If the audio is pure noise, it fails the alignment check and is ignored.
To demonstrate this robustness, we have provided demo videos of our method compared to DSANet under two conditions (with audio and without audio) in https://anonymous.4open.science/r/AsymmetricVAD under the folder:
demo-videos/ours_vs_DSANet.[W3 + Q3]:
We evaluated our model over 5 random seeds.
Table 1: Multi-seed performance.
Table 1 demonstrates that our training procedure is robust against random initialization.
Due to page limits, for our response regarding "Marginal Performance" in W3, please refer to our response to [W1] of Reviewer 1.
[W4 + Q4]:
For TMVAD, we use real audio from https://freesound.org/. Specifically, we select the audio corresponding to the actual abnormal event and then manually align it with visual events. Although we can not guarantee that it is the same as what the abnormal sound should be, we believe it can simulate surveillance scenes. We have attached demo videos in the folder:
demo-videos/sample_TMVAD_synthetic.We add the analysis in TMVAD Dataset section of the supplementary.
[W5]:
We have followed your suggestions in the revised manuscript.
[Q5]:
Compared to authentic audio, synthetic audio contains less accurate semantic representations of abnormal events, which slightly limits the joint cross-modal similarity calculation. However, as we discussed in the paper, we believe that leveraging synthetic audio is a practical approach to enhance the model’s robustness in surveillance scenarios, where audio may be absent in certain clips.
Following your suggestion, we provide the performance breakdown in Table 2.
Table 2: Sub-group analysis on TMVAD.
We find that AsymmetricVAD achieves consistent and comparable performance across both natural and synthetic audio. The performance drop in synthetic audio is because authentic audio provides more accurate semantic representations. Regarding Look-Back, the trigger rates on synthetic clips are lower than on natural clips because natural surveillance footage contains more complex background noise.