acceptodds
Withdrawn from ICLR 2027

CAViR: A Synthetic Video-Text Benchmark for Real-World Anomaly Retrieval

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Language-based anomaly clip retrieval searches pre-segmented short clips whose content matches a natural-language description of a normal or anomalous event. However, real-world collection, e.g., the XD-Violence dataset, faces prohibitive cost and privacy constraints, and covers only a fraction of anomalous event types. We introduce CAViR (Cross-modal Anomaly Video Retrieval), a synthetic video-text benchmark comprising 41,315 short video clips (1.36M frames) across 30 normal activity types and 68 anomalous event types. Each clip is paired with 5 captions, yielding 206,575 video-text pairs. To test whether the generated data is learnable and transferable, we further propose the FLAME (Flow-guided Language-video Alignment with Mixture-of-Experts) framework, which integrates Token-Aware Mixture-of-Experts (TAM) and a Cross-modal Flow Predictor (CFP). TAM adaptively routes visual and textual tokens to specialized experts for fine-grained event representation learning, while CFP uses auxiliary optical-flow prediction to align language with short-term visual dynamics. FLAME achieves the highest text-to-video/video-to-text Recall@1 among competitive methods: 71.6%/87.3% on CAViR, 41.0%/39.7% on UCFCrime-AR, and 33.4%/33.8% on XD-Violence, respectively. Extensive experiments verify synthetic-to-real transfer. Without relying on any real anomaly clips, pre-training on CAViR alone improves text-to-video Recall@1 on UCFCrime-AR from 22.4% to 33.8%.

Primary area:
datasets and benchmarks
Keywords:
Anomaly Benchmark, Long-Tail Distribution, Multimodal Learning, Video Text Retrieval

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_1qVwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CAViR, a synthetic video-text benchmark addressing data collection costs, privacy concerns, and limited anomaly diversity. The authors also propose FLAME, combining a Token-Aware Mixture-of-Experts (TAM) module for selective token processing and a Cross-modal Flow Predictor (CFP) for motion representation learning. Experiments report improved retrieval performance on synthetic and real-world datasets, alongside benefits from CAViR pretraining for synthetic-to-real transfer.

Critical strengths

S1. Broad coverage of normal and anomalous events. Unlike benchmarks primarily organized around anomaly categories, CAViR includes 30 normal activity categories. This diversity could support challenging hard negatives and help distinguish anomalies from visually similar normal activities. Dedicated evaluations involving normal events would further substantiate this potential benefit.

S2. Potential benefits of Multi-Style Re-captioning. Multiple descriptions per clip may improve robustness to linguistic variation, support fine-grained alignment through event-, video-, and frame-level descriptions, and reduce dependence on dataset-specific annotation styles. These are useful directions for improving retrieval generalization.

Critical weaknesses

W1. The contribution of FLAME’s components to synthetic-to-real transfer is not isolated. TAM and CFP are motivated by heterogeneous event granularity and motion-language alignment, respectively. However, the main component ablations are conducted on UCFCrime-AR, leaving their individual contributions to synthetic-to-real transfer unclear. Evaluating component variants trained on CAViR and tested on real-world datasets would help distinguish the benefits of the proposed architecture from those of synthetic pretraining.

W2. Potential textual overlap complicates the interpretation of transfer results. CAViR construction uses descriptions from UCFCrime-AR, UCA, and OOPS!, reportedly retaining UCFCrime-AR and UCA descriptions unchanged. Since UCFCrime-AR and OOPS! also serve as transfer evaluation datasets, evaluation queries or close paraphrases may have been used during synthetic training-data construction. If present, this overlap could confound the attribution of gains to generalization to unseen real-world retrieval scenarios. The paper should clarify source-data splits and quantify exact and semantic overlap before drawing strong conclusions about synthetic-to-real transfer.

Questions for authors

Q1. Dataset overlap and transfer validity. Were descriptions from the target evaluation splits excluded during CAViR construction? Please quantify overlap with evaluation queries and report whether transfer gains persist after removing exact duplicates and close paraphrases from synthetic training data.

Q2. Contribution of caption diversity. The advantage of five captions over one may reflect additional training pairs rather than linguistic diversity. Could the authors control the number of video-text pairs and training budget to isolate the benefit of Multi-Style Re-captioning?

Q3. Component contributions to transfer. Could the authors evaluate FLAME with and without TAM and CFP under the same CAViR-to-real transfer protocol? This would clarify how much each module contributes to the reported transfer gains.

Reviewer_mx66OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces CAViR, a synthetic anomaly video-text benchmark, and the FLAME retrieval model. Videos are randomly split within each category into 28,898/4,108/8,309 training/validation/test samples. With separate training on each dataset, FLAME achieves text-to-video R@1 of 71.6%, 41.0%, and 33.4% on CAViR, UCFCrime-AR, and XD-Violence, respectively. Training only on CAViR improves R@1 on all three real-world targets, including an increase from 22.4% to 33.8% on UCFCrime-AR.

Critical strengths
  1. Real-world transfer gains extend across architectures. In Table 4 on page 8, CLIP4Clip, X-CLIP, and FLAME all benefit from CAViR training. FLAME's R@1 on UCFCrime-AR/XD-Violence/OOPS! improves from 22.4/16.5/12.9 to 33.8/20.9/18.3. This is more informative than strong results only on a self-constructed synthetic test set.

  2. The experiments and task boundaries are reasonably well documented. The appendix includes comparisons with VLM2Vec-V2 and analyses of expert count, frame count, caption styles, and reranking costs. The authors acknowledge the single-generator limitation and domain gap, and define the task as retrieval of approximately 2.2-second pre-segmented clips rather than temporal localization in long videos.

Critical weaknesses
  1. The example captions alter core events, calling for direct data-quality auditing.

    In the abuse example in Figure 10 on page 20, one caption describes a woman kicking and hitting an older man, while another describes the woman trying to put socks on him and being kicked away, reversing the actor of the kicking action. The dancing example changes from hazardous dancing to wiping a cabinet door or handling a container. In the drowning example in Figure 11 on page 21, the base description says the boy becomes completely submerged, whereas the holistic video caption describes him joyfully jumping into a puddle. These differences go beyond writing style.

    The reported agreement above 97% concerns text-based category annotation, not the consistency of the complete video event and its five captions. FID, FVD, and CLIPSIM cannot replace event-correctness assessment. Please audit a stratified sample of complete videos for actors, temporal sequence, anomaly attributes, and caption consistency, and report agreement and uncertainty. The displayed captions clearly conflict, but which caption matches the full video and how common such errors are remain unverified. Keyframe captions may legitimately omit dynamic events and should be evaluated separately from conflicting holistic descriptions.

  2. Source separation between synthetic training and real-world testing is unclear.

    Page 3 states that generation uses descriptions from UCA, UCFCrime-AR, and OOPS!. The latter two are also transfer targets, while UCA and UCFCrime-AR share UCF-Crime video sources. The category-wise split after generation on page 7 does not establish whether descriptions associated with target test videos were excluded before generation. The style templates in Appendix B.3 also come from these datasets.

    Please provide the mapping from source video IDs and descriptions to synthetic samples and final splits, explain the exclusion of target test sources and cross-source duplicates, and verify the main transfer results on the source-disjoint data. The use of real-source text is disclosed; the statement about using no real anomaly clips concerns video pixels. The unresolved issue is whether the protocol supports source-independent transfer, not an established finding of leakage.

  3. The dynamic-representation gains are not fully separated from reranking and computational cost.

    In Table 5 on page 15, FLAME without reranking achieves 31.4 R@1, below X-CLIP's 32.5. Top-128 reranking raises R@1 to 41.0 while increasing FLOPs from 3.2G to 419.1G and latency from 1.0 to 61.5 ms. The authors disclose this trade-off. Figure 7b on page 9 also shows an improvement from 33.45 to 41.03 under the same training settings, providing a useful component comparison. The initialization and reranker should nevertheless be specified for each variant, with comparisons at matched cross-modal interaction budgets to identify the added representation modules' contribution.

    The temporal mechanism also needs clearer validation. Under the independent frame encoding and averaging formula on pages 5–6, V=mean⁡(F)+mean⁡(W)V=\operatorname{mean}(F)+\operatorname{mean}(W), the retrieval embedding alone is invariant to frame permutation. Any additional temporal interactions in the actual reranker or CFP should be specified. Please define the frame pairs, resolution, and normalization of the flow targets, and use frame-order and text-removal controls to test the claimed short-term dynamic learning.

Questions for authors
  1. Which source splits were used for generation descriptions and style templates? Were target test videos and associated descriptions excluded before generation, and were cross-source duplicates removed?
  2. How are conflicting event descriptions in Figures 10–12 resolved? Can a video-based human audit separately assess category correctness, event correctness, and consistency across captions?
  3. With the same X-VLM initialization, reranker, and training budget, what do TAM and CFP each contribute? How are positive matches among the five captions defined, and what are the gallery sizes and model-selection rules?
  4. What are the CFP flow frame pairs, output format, and normalization? How do frame permutation and text removal affect the gains? Does inference include temporal interactions absent from the stated formula?
Reviewer_gZuyOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces CAViR, a synthetic video–text benchmark for anomaly clip retrieval, comprising 41,315 short clips (2.2 s) with 206,575 captions covering 30 normal and 68 anomalous event types, generated with Wan2.1 text-to-video and LLM-expanded descriptions. The paper also proposes FLAME, a flow-guided language–video alignment framework with a Token-Aware Mixture-of-Experts (TAM) layer and a Cross-modal Flow Predictor (CFP) that adds auxiliary optical-flow supervision. FLAME reports the best T2V Recall@1 on CAViR (71.6%), UCFCrime-AR (41.0%), and XD-Violence (33.4%), and training on CAViR improves cross-dataset retrieval on UCFCrime-AR, XD-Violence, and OOPS.

Critical strengths
  1. Well-motivated benchmark gap. Existing anomaly video–text benchmarks inherit limited diversity from UCF-Crime/XD-Violence; CAViR expands coverage to 68 anomaly + 30 normal types with 5 stylistically diverse captions per clip.
  2. Privacy and scalability rationale. Synthesizing clips avoids privacy-sensitive footage collection and manual annotation cost, which is a legitimate practical motivation.
  3. Reasonable dataset construction pipeline. LLM-expanded descriptions, manual filtering, Wan2.1 generation with FP8/TeaCache, and multi-style re-captioning are documented, and prompts/templates are provided in the appendix.
  4. Transfer evidence is the strongest part. Training on CAViR consistently improves CLIP4Clip, X-CLIP, and FLAME on real-world targets (Table 4), which is the paper's most convincing claim.
Critical weaknesses
  1. Synthetic-to-real gap is acknowledged but not quantified well. FID/FVD/CLIPSIM (Table 11) are weak evidence of realism; no human study, no downstream VAD evaluation, and no analysis of which anomaly types transfer poorly. The claim that CAViR supports "real-world anomaly retrieval" needs stronger support than retrieval gains on three datasets.
  2. Dataset quality depends heavily on Wan2.1. The paper notes single-generator limitations but does not analyze failure cases of generated clips (e.g., incorrect actions, missing objects, implausible motion). A manual quality audit with inter-annotator agreement would substantially strengthen the benchmark contribution. 3.FLAME's novelty is moderate. TAM is essentially a top-1 Switch Transformer inserted at fixed positions; CFP adds optical-flow MSE as auxiliary supervision. Neither is a fundamentally new idea, and the paper does not compare against a strong video–text embedding baseline (VLM2Vec-V2) under the same fine-tuning protocol in the main tables — it is relegated to the appendix.
Questions for authors
  1. What is the manual quality audit protocol for generated clips? Can you report a human realism/action-correctness rating with inter-annotator agreement, and per-category transfer analysis?
  2. What exactly does CFP predict — flow magnitude, a downsampled field, or a scalar summary? Why is a [CLS]-based MLP sufficient, and what is CFP's ablation contribution in the main table?
  3. Can you provide a clean ablation of TAM, CFP, and multi-style captions in the main paper, not only in the appendix?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.