acceptodds
Under review as a conference paper at ICLR 2027

Separate to Deliberate, Fuse to Decide: Perspective-Factorized Latent Reasoning for Video Anomaly Understanding

Abstract

Video Anomaly Understanding (VAU) requires models not only to recognize unusual visual content, but also to understand how scene states change and how events evolve over time. These complementary perspectives provide distinct yet mutually informative cues for interpreting complex anomalies. However, existing LLM-based VAU methods typically encode heterogeneous visual information into a shared representation stream or perform explicit reasoning through a single textual chain of thought, which may prematurely entangle information from complementary perspectives. To address this limitation, we propose BRAID, a perspective-factorized latent reasoning framework based on an observe–deliberate–synthesize paradigm. Specifically, for perspective-aligned observation, BRAID captures object structures, spatial configurations, and state changes in the scene, and models temporal interactions and event evolution. It then performs perspective-specific latent deliberation over these observations, allowing each reasoning stream to develop from the visual evidence captured under its corresponding perspective while avoiding premature cross-perspective interference. Finally, since the relevance of different perspectives varies across contexts, BRAID employs a contextualized latent synthesis mechanism to dynamically assess and integrate perspective-specific information, enabling the model to jointly leverage complementary information from different perspectives for anomaly understanding and response generation. Extensive experiments on multiple widely used benchmarks demonstrate that our method outperforms existing MLLM-based VAU methods in both accuracy and computational efficiency.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Anomaly Understanding, Latent Reasoning, Multi-modal LLM

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_i7x1OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes BRAID, a perspective-factorized latent reasoning framework for video anomaly understanding. It introduces separate visual and thought latent tokens for scene-state and event-dynamics information, restricts cross-perspective interactions through attention masking, and integrates the representations through contextualized latent synthesis. Training combines visual supervision derived from SAM2 and V-JEPA with teacher-generated perspective-specific texts reconstructed through an auxiliary language decoder. The supervisory components are removed at inference. Evaluation covers VAU-Bench, HIVAU-70K, UCF-Crime, and XD-Violence, including question answering, classification, description generation, and anomaly localization. Component ablations and inference latency measurements are also reported.

Critical strengths

1.The paper addresses a meaningful problem with a coherent architectural proposal. Video anomaly understanding requires integrating environmental context with temporally evolving actions and interactions. Separately organizing these cues before contextualized synthesis is a plausible approach. Transferring visual and textual knowledge through training-only supervisory components also offers potential value for deploying a compact anomaly-understanding model.

2.The reported results show promising performance for a small model. BRAID achieves 93.59% QA accuracy and 54.53% multiclass classification accuracy on VAU-Bench. On HIVAU-70K, it achieves the best results in 10 of 12 settings among the listed 2B/3B models. Table 4 also shows improvements from combining the two perspectives and introducing synthesis. These results support the framework’s practical potential, although stronger controls are needed to establish the sources of the gains.

Critical weaknesses

1.The experiments do not sufficiently validate the factorization hypothesis or isolate its contribution. The central motivation is that premature interaction between scene-state and event-dynamics information causes interference. However, Table 4 does not compare separated and shared-stream representations under identical supervision, latent-token budgets, and training conditions. A direct comparison retaining all components while removing cross-perspective masking is absent. The omission of VAU-Eval reasoning scores from the ablations further weakens the connection between the experiments and the claimed reasoning contribution.

2.Reporting inconsistencies and incomplete protocols limit confidence in the evaluation and the claimed accuracy–efficiency advantage. Several Table 1 totals do not reconcile with their displayed VAU-Eval components: BRAID’s components sum to 28.60 rather than 28.43, while VAU-R1’s sum to 22.77 rather than 24.26. Regarding efficiency, Table 6 reports only BRAID’s latency, without matched baseline measurements or a clear timing scope. Table 2 supports strong performance among the listed small models, while BRAID remains below VADER-8B on all 12 metrics. This makes an accuracy–efficiency tradeoff a reasonable contribution to investigate, but the current measurements do not substantiate the paper’s broad claim of superiority in both accuracy and computational efficiency.

Questions for authors

1.Controlled attribution of the gains. Does the Table 4 baseline receive equivalent task training, and which visual and textual supervision signals are retained in each ablation? Under matched data, supervision, training budget, and latent-token count, how does BRAID compare with a shared-stream variant without cross-perspective masking? Please also provide VAU-Eval reasoning results for the relevant variants.

2.Consistency of VAU-Eval reporting. Could the authors reconcile the component scores and totals in Table 1? Besides BRAID and VAU-R1, Qwen2.5VL-7B’s components sum to 29.65 rather than the reported 29.78. Please clarify the aggregation rule and provide consistent results.

3.Evidence for the accuracy–efficiency tradeoff. What exactly does the latency in Table 6 include—visual encoding, prefill, and/or complete answer generation—and what output length is used? Could the authors provide matched measurements against the backbone and representative direct-answer or explicit-CoT baselines? These comparisons would help assess whether BRAID’s smaller deployment model provides a meaningful efficiency advantage and appropriately qualify the superiority claim.

2 comments
AuthorsOpenReview ↗

Response to Reviewer i7x1 – Part 1

We appreciate your valuable feedback.

To Weakness 1: Ablation Study of Shared-Stream Representations and Cross-Perspective Masking

We conduct two additional ablations, BRAID w/o Mask and Unified Latent, to evaluate the effectiveness of perspective separation. In BRAID w/o Mask, we enable cross-perspective attention while keeping all other components, supervision signals, and training settings unchanged. In Unified Latent, we retain the same total number of visual and thought tokens as BRAID but remove the distinction between scene-state and event-dynamics tokens. Specifically, shared visual latents are jointly supervised by SAM2 and V-JEPA features through learnable transformation heads, which accommodate the heterogeneous feature spaces of the two encoders and avoid directly aligning potentially incompatible supervision targets. Meanwhile, unified thought tokens are trained to reconstruct the concatenated scene-state and event-dynamics CoTs generated by the same teacher MLLM. The results are shown in Table A:

Table A: Additional ablation results of BRAID on VAU-Bench, with AUC (%) evaluated on UCF-Crime

SettingParamsQA ↑Acc ↑AUC ↑│CLS ↑KM ↑FLU ↑INF ↑FAC ↑Total (VAU-Eval) ↑
w/o Mask2B91.6352.8584.26│6.114.148.984.864.0828.17
Unified Latent2B90.1249.3783.17│5.412.338.522.693.6222.57
BRAID-2B2B93.5954.5387.89│6.274.189.054.934.1728.60

The results show that Unified Latent underperforms BRAID by 3.47, 5.16 and 4.72 percentage points in QA, Acc and AUC, respectively, highlighting the benefits of perspective-specific latent modeling over unified representations. Moreover, removing the cross-perspective attention mask (w/o Mask) reduces QA, Acc and AUC by 1.96, 1.68 and 3.63 points, respectively, further supporting the effectiveness of restricting premature cross-perspective interactions. The VAU-Eval results of the remaining ablation experiments are provided in Table 7 of the Appendix of the revised manuscript.

To Weakness 2

Clarification of VAU-Eval Score. VAU-Eval employs DeepSeek as an LLM-based evaluator, which independently assesses five component scores and an total score for each sample. Importantly, the total score is generated through a holistic evaluation rather than directly summing the five component scores, which may lead to discrepancies between them. In our original reporting, each component score was averaged across all samples and rounded to two decimal places, while the total score was obtained by averaging the overall scores directly generated by DeepSeek. To ensure consistent reporting, we have now recalculated all VAU-Eval Total scores as the sum of the five averaged component scores. The corrected results are presented in Table 1 of the revised manuscript.

Inference Efficiency. We have additionally measured the inference latency of BRAID, Holmes-VAU, and VAD-R1 with different numbers of input frames:

Table B: Comparison of end-to-end inference latency under different input frame counts (ms). F: frame.

MethodScaleReasoning8F16F32F64F
Qwen3-VL2BDirect Answer129185345618
HolmesVAU2BDirect Answer3635068311143
VAD-R13BExplicit CoT1776249841915657
BRAID2BLatent137194362669
BRAID8BLatent3394158271526

We measure inference latency on the same NVIDIA RTX 4090 GPU using the same input videos, covering the entire process from video loading completion to full response generation, including prompt construction, visual encoding, LLM prefill, and response generation. The results demonstrate BRAID's favorable inference efficiency compared with the evaluated baselines.

Accuracy–Efficiency Tradeoff. We acknowledge that BRAID-2B does not outperform VADER-8B on HIVAU-70K text-generation metrics. However, it achieves the best performance on anomaly-related recognition metrics, including VAU-Bench QA, anomaly classification, and VAU-Eval CLS (Table 1 in the main paper), demonstrating its ability to capture anomaly-relevant visual information. Nevertheless, BRAID-2B outperforms existing 2B VAU models on 10 of the 12 HIVAU-70K metrics (Table 2 in the main paper) and remains competitive with several 7B/8B models, although its smaller backbone may limit open-ended text generation.

We further scale BRAID to Qwen3-VL-8B. As shown in Table 2 of the revised manuscript, BRAID-8B ranks first on 8 of the 12 HIVAU-70K metrics and second on the remaining 4, outperforming VADER-8B and ReactVAU-10B on 10 metrics each. These results demonstrate BRAID's effectiveness across model scales.

AuthorsOpenReview ↗

Response to Reviewer i7x1 – Part 2

To Question 1: Controlled Attribution of the Gains.

Baseline Training Protocol. We clarify that the Qwen3-VL-2B baseline in Tables 1 and 4 is already fine-tuned on task-specific training data. It uses the same task training data as BRAID but without the proposed latent reasoning modules and their auxiliary supervision. Therefore, the improvements cannot simply be attributed to task-specific fine-tuning.

Details Configuration of Ablation Study. A complete description of the removed components and the visual and textual supervision signals retained in each ablation is provided in Section B.1 of the Appendix in the revised manuscript.

Ablation Study of Shared-Stream Variant. As detailed in our response to Weakness 1, we have conducted additional controlled experiments, including w/o Mask and Unified Latent (shared-stream variant), under matched data, supervision, training budgets, and latent-token counts. These experiments, together with the additional VAU-Eval reasoning ablations, provide more direct evidence for the effectiveness of our method.

To Question 2: Clarification of VAU-Eval Score.

As explained in our response to Weakness 2, the discrepancies arise because DeepSeek independently generates the five component scores and an overall score, which may not always equal their sum. We originally followed the official evaluation procedure and averaged these scores separately. To ensure consistency, we have recalculated the Total scores as the sum of the five component scores and corrected the corresponding results in Table 1, including those for BRAID, VAU-R1, and Qwen2.5-VL-7B.

To Question 3: Matched Evaluation of End-to-End Inference Efficiency.

As clarified in our response to Weakness 2, all measurements were conducted on the same hardware, and the scope of latency measurement has been explicitly specified.

As shown in Table I, we uniformly limit the maximum number of visual tokens per video frame to 256 across all methods. All methods use the same prompt for video anomaly classification, except for VAD-R1, which retains its official prompt. With 16 input frames, the average input/output token counts are 1,438/3 for Qwen3-VL-2B, 1,425/2 for HolmesVAU-2B, 2,586/217 for VAD-R1-3B, and 1,482/3 for BRAID-2B/8B.

VAD-R1 has substantially more input and output tokens because it follows a multi-step explicit reasoning process. Specifically, we retain its official prompt (approximately 960 tokens) to preserve its original reasoning procedure, and approximately 210 of its generated tokens correspond to intermediate reasoning steps. In contrast, BRAID performs deliberation in latent space without generating intermediate reasoning text. Compared with its Qwen3-VL-2B backbone, BRAID introduces only 44 additional input tokens with minimal inference overhead, while achieving substantially faster end-to-end inference than VAD-R1. These results highlight BRAID's favorable inference efficiency, with limited overhead over its backbone and substantially lower generation costs than explicit-CoT reasoning.

Reviewer_qTQAOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BRAID, a latent reasoning framework for video anomaly understanding. The main idea is to let the model understand videos from two perspectives, scene states and event dynamics, reason about them separately, and then combine the information to make a final prediction. However, I think the paper needs stronger evidence to show that the proposed separation is truly necessary and that the improvements mainly come from the latent reasoning design rather than the additional supervision. Addressing these concerns would make the paper more convincing.

Critical strengths

Separating scene information from event dynamics makes sense for video anomaly understanding, as both perspectives can provide useful but different information. The authors evaluate the method on several benchmarks and provide ablation studies. It is also impressive that a relatively small 2B model achieves competitive performance against larger models.

Critical weaknesses
  1. This is the key idea of the paper, but I am not fully convinced that separating the two perspectives is better than reasoning about them together. I suggest adding a baseline that uses the same number of latent tokens and the same training setup, but without separating the two reasoning streams. This would help show whether the separation itself really matters.
  2. The authors use different attention masks and supervision signals for the two perspectives. However, does this guarantee that they learn different information? I would like to see more analysis of what these tokens capture and how much the separation contributes beyond the extra visual features.
  3. The method uses a stronger MLLM to generate reasoning supervision. It also uses DINOv2, SAM2, and V-JEPA to provide visual supervision during training. These additional sources of knowledge may contribute significantly to the improvements. It would be helpful to better separate the benefits of these supervision signals from those of the proposed architecture.
  4. Although BRAID performs well on several tasks, it does not achieve the best result on every metric. For example, its VAU-Eval score is lower than that of Qwen2.5-VL-7B in Table 1. I think the authors should discuss why the method improves QA and classification accuracy but does not show the same advantage in reasoning quality. This would give a more balanced view of the method.
Questions for authors

Why does selecting reasoning pairs with the largest semantic difference necessarily lead to better supervision?

2 comments
AuthorsOpenReview ↗

Response to Reviewer qTQA – Part 1

We appreciate your valuable feedback.

To Weakness 1: Effectiveness of Stream Separation and Comparison with a Unified Latent Baseline.

We conduct two additional ablations, BRAID w/o Mask and Unified Latent, to directly evaluate the effectiveness of perspective separation. In BRAID w/o Mask, we enable cross-perspective attention while keeping all other components, supervision signals, and training settings unchanged. In Unified Latent, we maintain the same total number of visual and thought tokens as BRAID but remove the explicit distinction between scene-state and event-dynamics tokens. Specifically, the shared visual latent tokens are jointly supervised by SAM2 and V-JEPA 2 features through learnable transformation heads. These heads accommodate the heterogeneous feature spaces of the two encoders (SAM2 and V-JEPA 2), avoiding direct alignment of the shared representations with potentially incompatible supervision targets. Meanwhile, the unified thought tokens are trained to reconstruct the concatenated scene-state and event-dynamics CoTs generated by the same teacher MLLM. The results are shown in Table A:

Table A: Additional ablation results on VAU-Bench, with AUC (%) on UCF-Crime.

SettingParamsQA ↑Acc ↑AUC ↑│CLS ↑KM ↑FLU ↑INF ↑FAC ↑Total (VAU-Eval) ↑
w/o Mask2B91.6352.8584.26│6.114.148.984.864.0828.17
Unified Latent2B90.1249.3783.17│5.412.338.522.693.6222.57
w/o scene-sup2B92.8753.0286.94│5.934.068.984.853.8027.62
w/o event-sup2B92.4553.2387.10│5.844.058.904.853.7127.35
BRAID-2B2B93.5954.5387.89│6.274.189.054.934.1728.60

The results show that Unified Latent underperforms BRAID by 3.47, 5.16 and 4.72 percentage points in QA, Acc and AUC, respectively, highlighting the benefits of perspective-specific latent modeling over unified representations. Moreover, removing the cross-perspective attention mask (w/o Mask) reduces QA, Acc and AUC by 1.96, 1.68 and 3.63 points, respectively, further supporting the effectiveness of restricting premature cross-perspective interactions.

To Weakness 2: Different Information Captured by Two Perspective Tokens.

Analysis of Perspective-Specific Representations. We provide quantitative and qualitative analyses to examine the learned perspective-specific representations. As shown in Section B.4 and Figure 6(a) of the Appendix, from the initial LLM layer to the intermediate layer 13, the decreasing CKA similarity suggests that scene-state and event-dynamics tokens progressively develop perspective-specific visual representations. Beyond layer 13, the further decrease in CKA indicates continued differentiation, consistent with our design of preserving perspective-specific information during latent synthesis. In Figure 6(b) of the Appendix, visualizations of two Stealing cases show that scene-state tokens mainly focus on objects and their spatial configurations, while event-dynamics tokens emphasize humans and interaction regions, reflecting their complementary roles.

Furthermore, as shown in TableA, BRAID outperforms both the w/o Mask and Unified Latent baselines under matched supervision and latent-token budgets. These results support the representational differences and performance benefits of perspective separation beyond additional visual features.

To Weakness 3: Contributions of Architecture and Additional Supervision.

We would like to clarify that, as described in Section 3.2 of the main paper, DINOv2 is used only for representative frame selection and not for latent supervision.

To disentangle their contributions, we conduct additional controlled ablations. As shown in Table A, removing scene-state supervision (including both scene visual and teacher-generated CoT supervision) leads to drops of 0.72, 1.51, and 0.95 percentage points in QA, Acc, and AUC, respectively, while removing event-dynamics supervision (including both event visual and teacher-generated CoT supervision) leads to decreases of 1.14, 1.30, and 0.79 points. In contrast, under the same supervisory sources and token budget, w/o Mask results in larger decreases of 1.96, 1.68 and 3.63 points, while Unified Latent leads to more substantial drops of 3.47, 5.16 and 4.72 points, respectively. These results indicate that although additional supervision contributes to performance, the proposed architectural design has a more pronounced impact on performance than either perspective's supervision individually, as reflected by the larger performance drops observed in the architectural ablations. This further highlights the importance of perspective factorization in effectively organizing and leveraging heterogeneous visual information.

AuthorsOpenReview ↗

Response to Reviewer qTQA – Part 2

To Weakness 4: Discussion of Small Models' Open-Ended Textual Reasoning Ability.

BRAID primarily aims to enhance visual evidence understanding and anomaly reasoning. At the 2B scale, BRAID outperforms prior methods of comparable size across all metrics and even surpasses several 7B/8B models in classification and QA, demonstrating its effectiveness in capturing anomaly-relevant visual evidence.

However, unlike classification and QA, which typically rely on key visual cues and constrained answer spaces, open-ended reasoning additionally requires strong language generation capabilities, which may also be influenced by model capacity. Notably, BRAID-2B achieves a higher CLS score than Qwen2.5-VL-7B on VAU-Eval, indicating stronger anomaly recognition even in generated textual reasoning about anomalies, while other metrics additionally assess textual fluency, informativeness, and factual consistency.

Furthermore, as shown in Table B, We scale BRAID to the Qwen3-VL-8B backbone using the same training data and training strategy as BRAID-2B, achieving the best performance across all metrics on VAU-Bench. Additionally, Table 2 of the revised manuscript reports the performance of BRAID-8B on HIVAU-70K, where our method also achieves competitive results. These results further demonstrate the scalability and effectiveness of BRAID's perspective-factorized reasoning paradigm across different model scales.

Table B: Performance comparison of BRAID across model scales on VAU-Bench. B-Acc denotes binary normal/abnormal classification accuracy.

MethodParamsQA ↑B-Acc ↑Acc ↑│CLS ↑KM ↑FLU ↑INF ↑FAC ↑Total (VAU-Eval) ↑
Qwen3VL2B81.6769.4941.17│4.032.618.343.613.4021.99
Qwen3VL8B87.5275.2049.38│3.333.817.995.743.2524.12
Qwen2.5VL7B86.1177.6850.85│4.644.378.896.934.8229.65
BRAID-2B (Ours)2B93.5981.7754.53│6.274.189.054.934.1728.60
BRAID-8B (Ours)8B95.0490.6763.73│6.524.789.237.025.2432.79

To Question: Selection of Reasoning Pairs.

Selecting the pair of CoTs with the greatest semantic difference does not guarantee higher-quality supervision. Instead, it serves as a heuristic to encourage complementary supervision across the scene-state and event-dynamics perspectives by reducing redundancy between their descriptions, thereby providing more distinct semantic guidance for the corresponding thought tokens.

To validate this strategy, we further compare three selection methods, as shown in Table C. Although the performance differences are relatively small, Maximum Semantic Difference achieves the best results in both Acc and AUC. These findings suggest that selecting reasoning pairs with the largest semantic difference may be a more effective strategy for perspective-specific supervision.

Table C: Comparison of reasoning-pair selection strategies. Acc (%) is evaluated on VAU-Bench, and AUC (%) on UCF-Crime.

Selection StrategyAcc (%) ↑AUC (%) ↑
Random Pair54.3487.36
Minimum Semantic Difference54.1086.91
Maximum Semantic Difference (BRAID)54.5387.89
Reviewer_2zSBOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that current MLLM-based VAU methods mix different kinds of visual information in one representation stream or one textual CoT, which can entangle complementary cues too early. The authors propose BRAID, which reasons separately from a scene-state view and an event-dynamics view inside the LLM, then fuses the two. Each view gets its own latent tokens, and the two groups are kept apart by an attention mask. During training, the scene tokens are aligned with SAM2 features and the event tokens with V-JEPA 2 features, and the thought tokens are trained to reconstruct scene/event descriptions written by a teacher MLLM. A gated fusion module then combines the two views before the answer is generated. Built on Qwen3-VL-2B, BRAID reports strong results on VAU-Bench, HIVAU-70K, UCF-Crime and XD-Violence, and the authors claim it is both more accurate and more efficient than prior methods.

Critical strengths
  1. The design fits the task and keeps inference simple. The paper models video anomalies from two views, scene states and event dynamics, and gives each its own latent tokens and a suitable teacher: SAM2 for dense scene structure and V-JEPA 2 for motion and interaction. Unlike explicit-CoT methods such as VAD-R1, BRAID does not generate intermediate reasoning text, and all teacher models are removed at inference, so it only adds a small number of latent tokens to a 2B model evaluated on a single RTX 4090. By design, this should make inference cheaper than explicit-CoT approaches.

  2. The method achieves strong results with a small model. With only a 2B backbone, BRAID reports the best QA (93.59%), binary classification (81.77%) and fine-grained classification (54.53%) accuracy on VAU-Bench, ahead of 3B explicit-CoT methods and several 7B/8B models, and is best in 10 of 12 settings among 2B/3B models on HIVAU-70K. It is also competitive on online localization (87.89% AUC on UCF-Crime, 92.51% AUC on XD-Violence), so the gains hold across QA, classification, description and localization. The ablation in Table 4 further shows that using both views consistently outperforms using either view alone on all three metrics, which supports the claim that scene-state and event-dynamics information are complementary.

Critical weaknesses
  1. The central claim that entangling the two perspectives is harmful is not demonstrated. Every row in Table 4 adds or removes token groups or the synthesis module, but none keeps the same tokens and training while allowing cross-perspective attention. Compared with the baseline, BRAID changes many things at once: extra latent tokens, alignment with SAM2 and V-JEPA features, text supervision from a teacher MLLM, three-stage training, and the synthesis module, so the gains cannot be attributed to the separation itself. Moreover, the baseline in the first row of Table 4 has exactly the same numbers as the Qwen3-VL-2B row in Table 1 (81.67 QA, 41.17 Acc), and the paper does not state that it was fine-tuned on VAU-Bench; if it is the off-the-shelf model, part of the gains may simply come from fine-tuning. Similarly, the claim that explicit CoT introduces noisy reasoning is supported only by Figure 1 and the zero-shot thinking modes of general Qwen models, with no failure analysis of trained CoT-based VAU methods such as VAD-R1. I suggest (i) removing the cross-perspective attention mask while keeping everything else identical, and (ii) a baseline fine-tuned on the same data without latent tokens. If performance does not drop in (i), the gains likely come from other parts of the method rather than from the separation.

  2. The paper does not examine how the two views should interact. They meet only once, through a weighted sum at layer 13 (Eq. 9), which can express which view to rely on but not easily a conjunction such as “this action is abnormal only in this scene”. Yet many anomalies are context-dependent: running is normal in a park but suspicious in a bank, so scene and action need to be reasoned about jointly. The rationale is also somewhat inconsistent: the method aims to avoid cross-perspective interference, while Section 3.2 motivates the scene view by using the scene to infer likely events (e.g., theft in malls or banks), which is itself a cross-perspective inference. The authors should clarify this rationale and report results on context-dependent anomalies (e.g., per anomaly category), together with the gate scores.

Questions for authors
  1. The paper criticizes explicit CoT as noisy, yet the thought tokens are trained to reconstruct explicit text generated by a teacher MLLM (Eq. 12). Moreover, the prompts in Figures 4 and 5 exclude anomaly analysis and causal interpretation, so the targets are scene descriptions and event progressions rather than reasoning. What kind of reasoning are the thought tokens expected to perform, and how was the quality of the teacher text checked?
  2. The prompt in Figure 5 asks the teacher to extract the event progression from “a detailed analysis of a video event”. Where does this analysis come from? If it comes from dataset annotations, does it contain ground-truth anomaly information?
  3. Could the authors report, on the same hardware, the end-to-end latency and the number of input and generated tokens for BRAID, the base Qwen3-VL-2B, and explicit-CoT methods such as VAD-R1?
  4. How are frame-level anomaly scores obtained from a model that outputs text (e.g., from the probability of a “Yes” token)?
  5. Which data is BRAID trained on for each benchmark? Are the baseline numbers in Tables 1 and 2 reproduced or taken from prior papers, and which baselines were trained in-domain? In particular, the original Vad-R1 paper and its released checkpoint use a Qwen2.5-VL-7B backbone, so how was the 3B VAD-R1 result obtained (e.g., was it retrained on a 3B backbone, and with which data)?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 2zSB – Part 1

We appreciate your valuable feedback.

To Weakness 1: Additional Controlled Validation of Perspective Separation

(i) Effectiveness of Perspective Separation. We conduct two additional ablations, BRAID w/o Mask and Unified Latent, to directly evaluate the effectiveness of perspective separation. In BRAID w/o Mask, we enable cross-perspective attention while keeping all other components, supervision signals, and training settings unchanged. In Unified Latent, we maintain the same total number of visual and thought tokens as BRAID but remove the explicit distinction between scene-state and event-dynamics tokens. Specifically, the shared visual latent tokens are jointly supervised by SAM2 and V-JEPA features through learnable transformation heads. These heads accommodate the heterogeneous feature spaces of the two encoders (SAM2 and V-JEPA), avoiding direct alignment of the shared representations with potentially incompatible supervision targets. Meanwhile, the unified thought tokens are trained to reconstruct the concatenated scene-state and event-dynamics CoTs generated by the same teacher MLLM. The results are shown in Table A:

Table A: Additional ablation results on VAU-Bench, with AUC (%) on UCF-Crime.

SettingParamsQA ↑Acc ↑AUC ↑│CLS ↑KM ↑FLU ↑INF ↑FAC ↑Total (VAU-Eval) ↑
w/o Mask2B91.6352.8584.26│6.114.148.984.864.0828.17
Unified Latent2B90.1249.3783.17│5.412.338.522.693.6222.57
BRAID-2B2B93.5954.5387.89│6.274.189.054.934.1728.60

The results show that Unified Latent underperforms BRAID by 3.47, 5.16 and 4.72 percentage points in QA, Acc and AUC, respectively, highlighting the benefits of perspective-specific latent modeling over unified representations. Moreover, removing the cross-perspective attention mask (w/o Mask) reduces QA, Acc and AUC by 1.96, 1.68 and 3.63 points, respectively, further supporting the effectiveness of restricting premature cross-perspective interactions.

(ii) Clarification of the Fine-Tuned Baseline. We clarify that the Qwen3-VL-2B baseline reported in Tables 1 and 4 is already fine-tuned on task-specific training data. The baseline is trained using the same VAU-Bench training data. Therefore, the improvements cannot simply be attributed to comparing a fine-tuned BRAID against an unadapted backbone. We will clarify the training protocols and supervision settings in the revised paper.

(iii) The Failure Cases of CoT-Based VAU Methods. We provide four representative failure cases of VAD-R1-3B trained on VAU-Bench in Section B.5 of the revised Appendix. These cases illustrate anomaly category drift, speculative over-reasoning, incorrect causal attribution, and speculation-induced hallucination, highlighting how noisy intermediate reasoning may lead to incorrect anomaly predictions.

AuthorsOpenReview ↗

Response to Reviewer 2zSB – Part 2

To Weakness 2: Clarification of Cross-Perspective Interaction and Context-Dependent Reasoning

Cross-Perspective Interaction. We clarify that cross-perspective interaction is not limited to the gated fusion at mid layer (layer 13 in 2B model). Before this layer, synthesis tokens attend to the video and prompt tokens to establish contextual representations, while remaining isolated from the two perspective-specific streams. After gated fusion, synthesis tokens can attend to both scene-state and event-dynamics tokens throughout the subsequent Transformer layers, enabling iterative joint integration of perspective-specific information and context-dependent reasoning over scene–action relationships. Therefore, the gated fusion serves as the starting point for contextualized joint reasoning, rather than a one-time combination of the two perspectives. The causal attention mask employed in BRAID is illustrated in Table B.

Table B: Attention accessibility before and after latent synthesis.

L ≤ 13VideoPromptSceneEventSyn.│L > 13VideoPromptSceneEventSyn.
Video✓--------│Video✓--------
Prompt✓✓------│Prompt✓✓------
Scene✓✓✓----│Scene✓✓✓----
Event✓✓×✓--│Event✓✓×✓--
Syn.✓✓××✓│Syn.✓✓✓✓✓

Rows denote query tokens and columns denote key/value tokens. ✓ indicates permitted attention, × denotes explicitly restricted attention.

Clarification of the Design Rationale. We clarify that “anomalous events” in Section 3.2 refers to the anomaly categories or instances to be identified, rather than the representations modeled by the event-dynamics stream. Our intention is to emphasize that scene-state information provides contextual cues for anomaly recognition, rather than implying that the scene-state stream directly infers event-dynamics representations.

Furthermore, we emphasize that perspective separation does not imply complete isolation of information between the two perspectives. Instead, our goal is to avoid premature entanglement of heterogeneous visual information during perspective-specific deliberation, allowing each perspective to focus on organizing distinct visual cues. Since the two perspectives capture complementary aspects of the same underlying event, contextualized latent synthesis subsequently integrates their representations to enable joint reasoning and comprehensive anomaly understanding.

Category-Wise Results. We evaluate BRAID-2B on the classification task of VAU-Bench across several representative context-dependent anomaly categories, including Arson, Stealing, Traffic Accident, Fighting, Robbery, and Vandalism, which involve complementary scene-state and event-dynamics cues. We compare BRAID against Qwen3-VL-2B, VAD-R1, Unified Latent, and w/o Mask in Table 8 of Appendix B.3 of the revised manuscript. In particular, BRAID outperforms w/o Mask by 14.29, 4.92, and 4.26 percentage points in classification accuracy on Stealing, Robbery, and Vandalism, respectively. Compared with Unified Latent, the corresponding improvements are 21.43, 16.39, and 4.26 points. The results support the benefits of perspective factorization for anomaly recognition involving complementary visual evidence.

Gate Score Analysis. We compute the average gating weights across samples within each anomaly category. We observe moderate category-dependent variations in gating weights. Arson, Stealing, and Vandalism exhibit nearly balanced weights (0.496/0.504), reflecting the importance of both scene changes and human–object interactions. Traffic Accident assigns greater weight to scene-state (0.593/0.407), emphasizing changes in vehicle states and spatial configurations, while Fighting and Robbery favor event-dynamics (0.386/0.614), highlighting human actions and interactions. These observations suggest that gated fusion adjusts the relative emphasis on complementary visual cues.

AuthorsOpenReview ↗

Response to Reviewer 2zSB – Part 3

To Question 1: Clarification of Thought Latent Token and Teacher Supervision.

Explicit CoT Supervision. We clarify that our concern with explicit CoT is not textual supervision itself, but the reliance on textual intermediates during inference, which may omit fine-grained visual evidence or introduce noise. In BRAID, teacher-generated texts serve only as auxiliary supervision for latent deliberation, without requiring explicit intermediate text generation during inference.

The Reasoning of Thought Token. Perspective-specific deliberation (or thought latent token) is designed to organize and reason over visual evidence. Rather than independently producing complete anomaly judgments, these tokens provide complementary latent representations that are subsequently integrated through contextualized latent synthesis for joint anomaly reasoning. Therefore, we intentionally exclude anomaly analysis and causal interpretation from teacher prompts. Supervising intermediate tokens with explicit anomaly reasoning could shift their focus from learning perspective-specific visual evidence to reproducing teacher-generated anomaly judgments, whereas our goal is to establish complementary visual evidence for subsequent comprehensive anomaly understanding.

Regarding Teacher-Text Quality. We intentionally exclude anomaly judgments and speculative causal interpretations from teacher supervision, as such reasoning may introduce unsupported assumptions or noise. Instead, we constrain the teacher MLLM to describe observable scene characteristics, spatial relationships, human–object interactions, and event progression. These descriptions serve as auxiliary supervision for organizing perspective-specific visual evidence, rather than providing explicit anomaly reasoning or final anomaly judgments. While we do not employ an additional automatic verification mechanism during supervision generation, before model training, we randomly sampled approximately 600 videos and manually inspected their generated descriptions against the corresponding video content, finding no obvious factual inconsistencies in the inspected samples.

To Question 2: Clarification of Teacher MLLM's Prompt.

We apologize for including an incorrect figure in the original submission. The structure of the event CoT prompt in Figure 5 should be consistent with that of the scene CoT prompt in Figure 4. We have corrected this error in the Appendix of the revised manuscript. Specifically, "a detailed analysis of a video event" is not required as input. The teacher MLLM directly takes the video as input and generates the event progression based on its visual content, rather than extracting it from any pre-existing video analysis or dataset annotations. No ground-truth anomaly categories, human-annotated event descriptions, or other anomaly-related annotations are used to construct these supervisory texts.

To Question 3: More Inference Latency Comparison Results.

We measure end-to-end inference latency on VAU-Bench classification using the same videos and NVIDIA RTX 4090 GPU across all methods. Timing starts after video loading and ends with full response generation, including prompt construction, visual encoding, LLM prefill, and decoding. Additionally, we extend BRAID by adopting Qwen3-VL-8B as the base model, train it using the same training data and strategy as BRAID-2B, and evaluate its inference latency. Results are shown in Table C.

Table C: End-to-end inference latency across different input frame counts (ms, lower is better).

MethodScaleReasoning8 Frames16 Frames32 Frames64 Frames
Qwen3-VL2BDirect Answer129185345618
HolmesVAU2BDirect Answer3635068311143
VAD-R13BExplicit CoT1776249841915657
BRAID2BLatent137194362669
BRAID8BLatent3394158271526

We limit visual tokens to 256 per frame for all methods. All use the same video anomaly classification prompt except VAD-R1, which retains its official prompt. With 16 input frames, average input/output token counts are 1,438/3 for Qwen3-VL-2B, 1,425/2 for HolmesVAU-2B, 2,586/217 for VAD-R1-3B, and 1,482/3 for BRAID-2B/8B.

VAD-R1 has substantially more input and output tokens because it follows a multi-step explicit reasoning process. Specifically, we retain its official prompt (approximately 960 tokens) to preserve its original multi-step reasoning procedure and ensure faithful reproduction of the method. Approximately 210 of its generated tokens correspond to intermediate reasoning steps. In contrast, BRAID performs latent reasoning without generating intermediate text, adding only 44 input tokens over its Qwen3-VL-2B backbone with minimal inference overhead. This enables substantially faster inference than VAD-R1 while maintaining efficiency close to its backbone.

AuthorsOpenReview ↗

Response to Reviewer 2zSB – Part 4

To Question 4: Frame-Level Anomaly Score Generation.

Specifically, we sample video frames at 1 FPS using an 8-second sliding window with a stride of 8 seconds, resulting in 8 sampled frames per window. If the final window is shorter than 8 seconds, the number of sampled frames is adjusted accordingly (e.g., 5 frames for a 5-second window). Under the online setting, only frames within the current window are used for inference, without accessing any future frames. For each window, the model predicts whether the input video frames exhibit normal or anomalous behavior. An anomaly score of 0 is assigned when the model predicts "Normal", and 1 when it predicts "Anomalous". Finally, the resulting segment-level anomaly scores are mapped to their corresponding frame intervals to obtain frame-level anomaly scores. For example, for a video recorded at 30 FPS, the anomaly score predicted for the 16–24 s window is assigned to the corresponding frame interval (frames 480–719).

To Question 5: Baseline Training Protocols.

In Table 1, the results for Qwen2.5-VL and InternVL are taken from VAU-R1, while the other VAU methods and Qwen3-VL are reproduced using their official implementations, trained on the same VAU-Bench training data as BRAID, and evaluated under the same protocol.

In Table 2, all results except VAD-R1 and Cue-R1 are taken from the original papers. We reproduce these two methods using their official implementations and the same HIVAU-70K training and evaluation data as BRAID.

For VAD-R1, we use Qwen2.5-VL-3B as the backbone. To ensure faithful reproduction, we first fine-tune it on the VAD-R1 dataset and then further train it on the same data as BRAID.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.