acceptodds
Withdrawn from ICLR 2027

CAN WE ANTICIPATE VIOLENCE? MULTIMODAL LEARNING FROM PRE-INCIDENT BEHAVIORAL CUES

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Detecting violence after it begins is important from recognizing behavioral cues that appear immediately beforehand. This work studies short-horizon pre-incident risk recognition from multimodal video signals. We construct a binary Normal-versus-Risky setting from temporally annotated XD-Violence clips, using 443 samples with source-level separation across training, validation, and test sets. Each sample consists of a variable-length pre-incident clip, with its duration determined by the observable behavioral context preceding the incident. The incident itself is excluded from all input clips. We evaluate three complementary information sources: facial-region appearance, temporally aligned audio, and body-motion features derived from tracked keypoints. Controlled ablations are performed with Swin-Tiny, ViT-Tiny, and DeiT-Tiny to measure the contribution of each modality under the same split. Results show that combining all modalities is more effective than using any other combination alone. The best configuration, Deit-Tiny with audio, facial appearance, and motion, achieves 91.21% accuracy, 88.96% balanced accuracy, 93.65% F1-score, and 96.38% ROC-AUC on the held-out test set. These results suggest that complementary appearance, acoustic, and kinematic cues provide useful evidence for recognizing elevated pre-incident risk.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Pre-Incident Risk Recognition, Multimodal Learning, Violence Detection, Human Behavior Analysis, Video Understanding, Action Anticipation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_JYfVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates violence anticipation by identifying behavioral cues before violent incidents occur. The authors construct a dataset of 443 pre-incident clips from XD-Violence, categorized into Normal and Risky classes. The proposed framework combines facial appearance, audio, and pose-derived motion features using separate encoders and late feature fusion. Experiments evaluate ViT-Tiny, Swin-Tiny, and DeiT-Tiny backbones with different modality combinations. The best configuration, combining all three modalities with DeiT-Tiny, achieves 91.21% accuracy, 93.65% F1-score, and 96.38% ROC-AUC. The results suggest that multimodal behavioral information can help identify elevated risk before annotated violence onset.

Critical strengths
  • The paper addresses an important and challenging problem by focusing on recognizing behavioral cues before violent incidents rather than detecting violence after onset.
  • The construction of pre-incident clips from XD-Violence provides a useful starting point for studying short-horizon violence anticipation.
  • Combining facial appearance, audio, and body motion provides complementary information and improves the best reported performance over individual modalities.
  • The experiments evaluate multiple visual backbones and modality combinations, helping analyze the contribution of different input signals.
  • The use of source-video-level separation between training, validation, and test sets reduces the risk of information leakage between related clips.
Critical weaknesses
  1. The dataset contains only 443 clips, which limits the statistical reliability of the results and raises concerns about generalization to diverse real-world scenarios.
  2. The proposed architecture mainly combines existing pretrained visual encoders, temporal Transformers, an audio CNN, and late feature fusion, resulting in limited methodological novelty.
  3. The paper does not compare the proposed method directly with existing violence or anomaly anticipation approaches under a matched evaluation protocol.
  4. The annotation process lacks sufficient details about the number of annotators, inter-annotator agreement, and the reliability of the manually assigned risk levels.
  5. The evaluation is limited to pre-segmented clips and does not measure how early violence can be anticipated or the false-alarm rate in continuous videos.
  6. Some modality combinations underperform individual modalities, and the paper does not sufficiently investigate these inconsistencies or the limitations of simple concatenation-based fusion.
  7. The experiments are conducted on a single curated dataset without cross-dataset validation or uncertainty estimates, limiting conclusions about robustness and generalization.
Questions for authors
  1. How were the 443 clips distributed across training, validation, and testing, and what were the class distributions?
  2. How many annotators assigned the original risk labels, and was inter-annotator agreement measured?
  3. How does the proposed approach compare with existing violence or anomaly anticipation methods under the same dataset and evaluation protocol?
  4. How far before the actual incident onset can the model reliably predict elevated risk? Could the authors report performance at different anticipation horizons?
  5. Why do some multimodal combinations perform worse than individual modalities, particularly for ViT-Tiny and Swin-Tiny?
  6. Could the authors report results across multiple random seeds or confidence intervals to establish statistical reliability?
  7. How does the model perform on continuous untrimmed videos, where pre-incident boundaries are unknown and false alarms are important?
  8. How robust is the approach to missing or noisy facial detections, pose-estimation errors, and background audio interference?
Reviewer_GVhLOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies short-horizon pre-incident violence risk recognition using multimodal video signals. The authors construct a dataset of 443 samples from XD-Violence and combine facial appearance, audio, and pose-derived motion features through a late-fusion framework. Experiments with different visual backbones and modality combinations show that the best three-modality configuration achieves 91.21% accuracy and 96.38% ROC-AUC on the held-out test set.

Critical strengths
  1. The paper addresses a relevant and comparatively underexplored problem by shifting the focus from detecting ongoing violence to recognizing potential risk before incident onset.
  2. The authors conduct systematic modality ablations across multiple visual backbones, providing an initial analysis of the complementary contributions of facial appearance, audio, and body motion.
Critical weaknesses
  1. The manuscript does not sufficiently describe the number of annotators, inter-annotator agreement, criteria for determining incident onset, or the overall annotation procedure, raising concerns about the rigor and reproducibility of the dataset annotations.
  2. The study uses only 443 samples from the XD-Violence dataset, with only 91 samples in the test set. The lack of cross-dataset evaluation makes it difficult to establish the generalizability of the proposed method.
  3. The manuscript lacks quantitative comparisons with existing video anomaly anticipation and pre-fight recognition methods, making it difficult to demonstrate the effectiveness and relative advantages of the proposed approach.
  4. The proposed method mainly combines existing feature extraction networks through a simple late-fusion strategy, without a dedicated modeling mechanism for pre-incident risk anticipation, resulting in limited methodological novelty.
  5. Some text in the main framework figure is too small and unclear, hindering the understanding of the proposed architecture and its implementation details.
Questions for authors
  1. Could the authors provide further details on the dataset construction and annotation procedures?
  2. Could the proposed method be evaluated on additional datasets to further assess its generalizability?
  3. Could the authors provide quantitative comparisons with existing video anomaly anticipation and pre-fight recognition methods to validate the effectiveness and advantages of the proposed approach?
  4. Could the authors further clarify the main methodological contributions and compare the proposed method with simpler multimodal fusion baselines?
  5. Could the authors provide a higher-resolution version of the main framework figure with improved text readability?
Reviewer_3T3VOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates whether multimodal cues observed before violent incidents can be used for early recognition of potential violence risks. The authors curate 443 video clips from the XD-Violence dataset and manually annotate the pre-incident clips according to the presence and level of risk. For the classification task, the authors construct a multimodal detection framework that extracts facial appearance, audio, and body motion features using pretrained visual Transformers, an audio CNN, and a pose-based temporal Transformer. These features are then concatenated for binary classification. The experiments compare the performance of facial appearance, audio, motion, and different modality combinations. The best three-modality model achieves 91.21% accuracy on the test set. The research problem has practical significance, and the experimental results are promising. However, the current experiments and dataset construction do not adequately address the problem of violence anticipation.

Critical strengths
  1. A practically meaningful research problem. This paper shifts the focus of violence video analysis from detecting events that have already occurred to recognizing risks before violence begins, with an emphasis on human behavior. This direction has potential practical value.

  2. A relatively clear multimodal experimental framework. The authors integrate three types of information: facial appearance, audio, and body motion. They also compare the performance of different networks and modality combinations. The clear ablation study design helps readers understand the contributions of different information sources.

  3. Comprehensive discussion of related work and consideration of data leakage. The authors provide a relatively comprehensive discussion of the research background and clearly state the contributions of this work. During dataset construction, the authors explicitly exclude annotated violence segments and keep samples from the same source video within the same data partition.

Critical weaknesses
  1. The task formulation does not fully align with the central research question, and the validity of the labels remains unclear. First, the risk levels are manually annotated based on currently observable facial, motion, and audio cues, which are also used as inputs for model classification. Therefore, a high classification accuracy may indicate that the model can reproduce human judgments of current violence risk, but it does not demonstrate whether violence will actually occur in the future. The paper does not clearly explain whether the Risky labels reflect human assessments of potential violence risk or are verified by violent incidents that subsequently occur. Second, the paper states that the annotation criteria for High Risk include physical confrontation, impact sounds, and explosions. These behaviors may already be related to violent incidents themselves. Therefore, whether such cues can be considered evidence for anticipating violence requires further clarification.

  2. Lack of direct evaluation of prediction lead time and early warning capability in continuous scenarios. The central goal of this paper is to recognize violence risk in advance, but how far in advance the model can make predictions is not quantified. The experimental results mainly reflect classification performance on preselected video clips. The paper does not report the time interval between the end of an input clip and the actual onset of a violent incident. This raises the question of whether model performance decreases as the prediction is made further ahead of the incident. In addition, the current classification task relies on preselected video clips, whereas in practical early warning scenarios, the system does not know when violence will occur. This is also an important issue for the central research question of the paper.

  3. Limited methodological novelty and insufficient baseline comparisons. From a methodological perspective, this paper mainly combines existing components, including pretrained visual Transformers, an audio CNN, a pose-based motion feature encoder, and late feature concatenation. The paper does not propose a new modality interaction mechanism or a learning objective specifically designed for event anticipation. Therefore, its contributions mainly lie in the task construction and experimental results. However, the current experiments primarily compare different networks and modality combinations, without demonstrating the advantages of the proposed framework over simple temporal classification methods and existing anticipation methods. In addition, the dataset includes movie footage, which may contain source-specific features unrelated to the target behaviors. Cinematic elements such as camera composition and background music are often designed to create dramatic effects. These features may also correlate with the manually assigned risk labels, causing the model to rely on source-specific cues. The authors could consider analyzing the class distributions across different video sources or including challenging negative samples, such as intense movements without violence or arguments that do not eventually escalate into violent incidents.

  4. Small test set and lack of statistical robustness analysis. The paper uses 443 video clips in total, with only 91 samples in the test set. The best model achieves 91.21% accuracy, corresponding to 83 correctly classified samples, while the best facial-only Swin-Tiny model achieves 89.01%, corresponding to 81 correctly classified samples. Although these two configurations use different visual backbones, the difference is only two correctly classified samples, indicating a relatively small performance gap. Moreover, Table 1 shows that multimodal fusion does not consistently improve performance. This raises the question of whether the complementary benefits of different modalities depend on specific model configurations. In addition, the test set contains 29 Normal samples, of which 5 are incorrectly classified as Risky, resulting in a false-positive rate of approximately 17.24%. This may affect the central objective of the paper and the reliability of early warnings in future practical applications.

Questions for authors
  1. During the annotation process, were the annotators aware of what happened later in the videos? Were the Normal and Risky labels verified based on whether violent incidents actually occurred? How is the onset of a violent incident defined?

  2. What is the distribution of the time intervals between the end of the input clips and the onset of subsequent violent incidents? Could the authors provide model performance at different prediction lead times?

  3. Does the model still show advantages over other classification baselines and existing anticipation methods under the same task setting? Have the authors analyzed the distribution of movie footage and other video sources across different classes, and whether these sources may influence the results?

  4. Could the authors provide results across multiple random seeds, confidence intervals, and paired statistical comparisons? How much would the current false-positive rate affect the model in practical applications?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.