- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes EFG-VLM for offline anomaly understanding in long event streams. The method first identifies candidate intervals and selects temporal anchors within each interval. At these anchors, it combines event-frame features with spatial and time-directed motion graphs built from raw events. The resulting representations are passed to a vision-language model to generate anomaly judgements, temporal boundaries and descriptions.
The main experiments use events simulated with v2e from HIVAU-70K, whose source videos come from UCF-Crime and XD-Violence. The paper reports comparisons with existing methods, component ablations and results across different sequence durations. It also presents three qualitative examples using real event recordings.
Critical strengths
1.The method offers a reasonable way to combine scene semantics with local motion information. Using shared temporal anchors gives the event-frame and graph branches a consistent temporal reference. Restricting graph construction to local windows also addresses a practical difficulty in processing long event streams. This is a promising design for connecting detailed event representations to a language model.
2.The internal comparisons provide evidence that the combined representation is useful. In Table 3, the full model improves accuracy from 28.47 to 48.09 and CIDEr from 0.074 to 0.150 compared with the event-frame-only variant. These gains support further investigation of the approach, although the specific contribution of the proposed graph structure still needs to be isolated.
Critical weaknesses
1.The main evaluation does not adequately test the event-camera advantages used to motivate the paper. The authors emphasise fast interactions and difficult lighting, where ordinary RGB cameras may lose useful information. However, the quantitative experiments rely on events simulated from existing RGB videos. HIVAU-70K was designed for general video anomaly understanding, and the paper does not establish how much of the benchmark requires fine temporal resolution or perception under difficult lighting. Converting RGB recordings into events cannot establish that the method recovers details the original camera failed to capture. The three real-event examples provide preliminary evidence, but they are insufficient to support general claims about these sensor advantages.
2.Temporal localisation is an important output of the method, but it lacks quantitative evaluation. The main tables report classification and text-generation metrics without measuring temporal overlap or boundary error. Precise-looking timestamps in selected examples do not establish accurate localisation. The paper also does not quantify how much the later refinement and language stages improve on the initial candidate intervals. As a result, the claimed ability to locate anomalies in time remains insufficiently tested.
3.The methodological advance over existing work is not yet sufficiently established. Anomaly-focused temporal sampling appears in Holmes-VAU, event-language alignment in EventGPT, and efficient processing of long event sequences in EventFlash. The potential contribution here is the specific combination of shared anchors, local event graphs and proposal-based fusion. However, the ablations mainly remove entire components, without comparing the proposed design against simpler temporal encoders, conventional spatiotemporal graphs or simpler fusion under matched conditions. Removing the motion graph improves Macro-F1 from 16.55 to 18.74 while reducing CIDEr from 0.150 to 0.085. This supports its usefulness for description, but does not isolate the benefit of the proposed time-directed graph structure. The current evidence leaves open whether the gains require the proposed design or could be obtained through a simpler integration of existing components.
Questions for authors
1.Clarification: Which v2e version and conversion settings were used? Given that frame interpolation was disabled, what effective temporal resolution do the generated events have, and how does this relate to the reported 3 ms setting?
2.Experimental evidence: Can the authors provide quantitative results on real event recordings, particularly for fast interactions or difficult lighting? Please specify the evaluation size and sample-selection procedure, and include failures alongside successful cases. For the simulated benchmark, is there evidence that the gains are concentrated in conditions where fine temporal information is useful?
3.Clarification and experimental evidence: Are all candidate intervals used at test time predicted by the model? Can the authors report temporal overlap or boundary-error metrics and compare the initial locator, the boundary-refinement stage and the complete model?
4.Clarification: What backbone, task-specific training and input budget were used for EvQA-FRT and EvQA-ART? Does the event-frame-only ablation retain the same candidate intervals, supervision and training budget as the full model?
5.Experimental evidence: Under matched inputs, training conditions and comparable model capacity, does the proposed graph and fusion design outperform a conventional spatiotemporal graph or a simpler temporal encoder? A controlled comparison of forward-directed and undirected edges would also help establish the contribution of the temporal direction constraint.
I would be willing to raise my score if the additional experiments and explanations adequately address these concerns and substantiate the paper’s claimed contributions.
9 comments
Q2 (W1) — Real-sensor quantitative evidence
Reply:
Thank you for these questions. We address both the real-sensor evaluation and the simulated category-level analysis below:
-
Quantitative & Qualitative Evaluation on Real Event Recordings: revised manuscript Section 4.8, Table 6 reports all nine purposefully selected real synchronized RGB–event sequences (six abrupt physical interactions and three normal controls), comparing EFG-VLM on real events against Holmes-VAU on paired RGB videos. Across the six anomalous sequences, EFG-VLM achieves 44.51–87.03% tIoU (mean 68.50%), whereas frame-based Holmes-VAU does not output localized anomaly intervals. Paired description metrics include both gains and weaker cases: for example, EFG-VLM outperforms Holmes-VAU in BERTScore-F1 on
Jenga(0.8866 vs. 0.8765) andwater-bomb(0.8921 vs. 0.8782), while trailing onmay28_axe_01(0.8431 vs. 0.8614). Figures 8–9 in revised Appendix C also show both accurate grounding and hallucinated descriptions across model variants. -
Category-Wise Gains on Simulated Streams: revised manuscript Table 9 breaks down language gains from the directed motion graph across all 16 activity categories. CIDEr improvements are concentrated in dynamic physical interactions such as Fighting (+0.0396), Explosion (+0.0259), and Car accident (+0.0250), whereas crowd-level or static-scene categories such as Riot (-0.0253) and Vandalism (-0.0371) decline. Stratifying simulated benchmarks directly by motion speed and illumination is a promising direction for future work.
Revision:
Sec. 4.8 (revised manuscript): "We evaluate nine real synchronized RGB–event sequences, including six with manually reviewed anomalous intervals and three without anomalies. EFG-VLM processes real events, whereas the RGB baseline is Holmes-VAU." "Across the six anomalous sequences, EFG-VLM achieves 68.50% mean tIoU (range 44.51–87.03%)."
Q3 (W2) — Temporal localization and GT-free proposals
Reply:
Yes. All candidate intervals are predicted directly from the input event stream at test time without ground-truth interval injection. Stage 1 generates category-agnostic activity proposals, while the downstream language model makes the anomaly judgement and outputs refined start/end bins relative to the top-1 proposal (𝑃 =1) when an anomaly is detected.
On the 38 anomaly-annotated CUVA sequences (revised manuscript Table 2), Locator refine improves category-agnostic proposal overlap over Locator initial from 0.5568 to 0.6249 mean tIoU and from 57.89% to 60.53% R@[email protected] (with 100% proposal emission coverage). When gated by anomaly detection, EFG-VLM (full) achieves 65.79% anomaly-detection coverage, 0.3970 mean tIoU, and 44.74% R@[email protected] (assigning zero tIoU to missed detections across all 38 samples), with a 11.47 s conditional boundary MAE on detected intervals (Section 4.3 and Appendix B.2).
Revision:
Appendix B.2 (revised manuscript): "Evaluation uses the same 38 anomaly-annotated CUVA samples for all methods and stages. Mean tIoU and R@1 use the full evaluation set, assigning zero overlap to missed predictions. Coverage is the proportion of non-empty interval predictions, and boundary MAE is averaged over those predictions."
Q4 (related to W3) — Baseline settings and controls
Reply:
Thank you for asking us to clarify both the external baseline configurations and our internal ablation controls:
-
External Baselines (EvQA-FRT and EvQA-ART in manuscript Table 1): Both baselines were evaluated using the authors’ publicly released inference pipelines, checkpoints, and recommended evaluation settings without task-specific fine-tuning on HIVAU-70K. Specifically, FRT reconstructs dense intensity frames at 24 FPS using the released V2V-E2VID checkpoint, whereas ART performs activity-triggered sparse reconstruction using the released Adaptive-E2VID checkpoint, followed by a Qwen3-VL-Thinking vision-language backbone. Manuscript Table 1 therefore serves as a system-level comparison across distinct pipelines.
-
Internal Event-Frame-Only Ablation (manuscript Table 4): By contrast, the
Event-frame-onlybaseline isolates the contribution of our graph branch within an identical VLM pipeline:-
Shared components: It retains the exact same Stage-1 locator, five shared temporal anchors, rendered event-frame inputs, Qwen-3.5-2B backbone, supervision targets, and Stage-3 LoRA recipe as the full model.
-
Independent optimization: Each retained projection and LoRA adapter is trained independently for event-frame features (with graph-specific parameters and Stage-2 graph pretraining omitted), directly measuring the contribution of event-native graph evidence alongside sparse frames.
-
Revision:
Secs. 4.2 and 4.5 (revised manuscript): "EvQA-FRT/ART use released reconstruction-based front-ends and pretrained VLM checkpoints without HIVAU-specific adaptation"; "Under a shared Stage-1 locator, five anchors, Qwen-3.5-2B backbone, language supervision, and Stage-3 LoRA recipe, removing the graph branch reduces accuracy from 48.09 to 28.47 and CIDEr from 0.150 to 0.074; retained modules are independently optimized for each variant."
Q5 (W3) — Controlled graph-structure comparison
Reply:
Thank you for proposing these controlled experiments. In addition to the component ablations in manuscript Table 4, we have added a controlled architectural comparison (manuscript Table 5) evaluating a single 3D spatiotemporal graph, random partitioning, undirected motion edges, and an MLP–TCN/GRU temporal encoder against our Full Dual design (G0).
To compare all five architectures under identical optimization settings, we trained the four control variants alongside a matched Full Dual reference (G0) using a unified reduced-budget training schedule on the exact same dataset split and test set, varying only the graph/temporal module.
As shown in Table 5, G0 achieves the strongest performance across all metrics (52.71% Accuracy, 16.08% Macro-F1, 0.0753 CIDEr, and 0.2349 METEOR):
-
Compared with the single 3D spatiotemporal graph, separating spatial and motion relations improves Macro-F1 by 4.53 percentage points.
-
Random partitioning causes the largest drop, reducing Accuracy and Macro-F1 by 6.94 and 9.07 points, respectively.
-
Replacing directed motion edges with undirected edges reduces CIDEr by 0.0063 and METEOR by 0.0133, confirming the benefit of temporal directionality.
-
The MLP–TCN/GRU alternative also yields lower recognition accuracy (46.57% vs. 52.71%).
Together, these controlled comparisons validate both our spatial–motion relation decomposition and temporally directed edge formulation.
Revision:
Sec. 4.6 (revised manuscript): "For computational tractability, all five variants are trained from scratch under an identical reduced-budget protocol (Appendix B.2), making G0 a dedicated reference distinct from the full-budget model in Table 4."
Table 5: Controlled graph-structure comparison under a unified reduced-budget protocol (distinct from the full-budget setting in Table 4). Runtime measures average per-sample processing time across core encoding, classification, and description generation.
| Variant | Acc. ↑ | Macro-F1 ↑ | CIDEr ↑ | METEOR ↑ | Encoder Params (M) | Avg. Model Time (s) |
|---|---|---|---|---|---|---|
| G0 Full Dual | 52.71 | 16.08 | 0.0753 | 0.2349 | 0.622 | 6.89 |
| G1 Single 3D | 51.59 | 11.55 | 0.0684 | 0.2294 | Same as G0 | 7.49 |
| G2 Random Partition | 45.77 | 7.01 | 0.0644 | 0.2230 | Same as G0 | 9.42 |
| G3 Undirected Motion | 51.52 | 15.08 | 0.0690 | 0.2216 | Same as G0 | 7.24 |
| G4 MLP + Temporal TCN/GRU | 46.57 | 12.49 | 0.0689 | 0.2310 | Same as G0 | 7.02 |
Q1 (W1) — v2e version and timing resolution
Reply:
Thank you for catching this ambiguity and for your constructive questions. We clarify the exact conversion pipelines, effective temporal resolution, and code release below:
v2e Version & Conversion Settings (HIVAU-70K): All HIVAU-70K source RGB videos were converted using
SensorsINI/v2e(Hu et al., 2021) (commitcac6384445, Oct. 2025) under a fixed configuration (Appendix B.1, manuscript Table 8): DVS346 preset (346 ×260), nominaltimestamp_resolution=0.003 s,auto_timestamp_resolutiondisabled, frame interpolation disabled (--disable_slomo), duration exposure mode (0.005 s), positive/negative contrast thresholds of 0.15/0.15, threshold noise std. of 0.03, refractory period of 0.5 ms, cutoff frequency of 10 Hz, and default seed 0. Because untrimmed videos in UCF-Crime and XD-Violence can exceed the 32-bit unsigned microsecond limit (232−1μs≈4,294.97 s or 71.58 min), we patched thev2eHDF5 writer to store 64-bit integer timestamps. We will release all model code, patched conversion scripts, and exact configuration files in our public repository.Effective Temporal Resolution vs. Nominal 3 ms Setting: When slow-motion frame interpolation is disabled (
--disable_slomo),v2eignores the nominal 0.003 s step and synthesizes events by interpolating log-intensity changes directly between consecutive source RGB frames. Accordingly, the effective temporal observation interval is bounded by the source RGB frame rate (e.g., ∼ 33.3 ms for 30-FPS videos) rather than providing independent 3 ms physical measurements. We have updated manuscript Table 8 and Appendix B.1 to explicitly mark the 3 ms parameter as inactive.High-Temporal-Rate Protocol (Supplementary CUVA Evaluation): Separately from HIVAU-70K, our CUVA localization evaluation upsamples source videos to 240 FPS (∼ 4.17 ms per synthetic frame) via Practical-RIFE 4.25 lite and simulates events using a modified ESIM pipeline (
uzh-rpg/rpg_vid2e) with 64-bit nanosecond structured NPY output, fixed seed 12345, and matched DVS346 resolution (346×260), contrast thresholds (0.15/0.15), noise (0.03), and refractory period (0.5 ms), as detailed side-by-side in manuscript Table 8.Revision:
Appendix B.1 (revised manuscript & Table 8): "Because untrimmed surveillance videos in UCF-Crime and XD-Violence can exceed the 32-bit unsigned microsecond limit (232−1μs≈4,294.97s or 71.58min), we patch the
v2eHDF5 writer to store 64-bit integer timestamps, preventing timestamp wrap-around in long streams." "With frame interpolation disabled inv2e, the nominal 0.003 s step is inactive and effective temporal sampling follows the source video frame interval."