acceptodds
Under review as a conference paper at ICLR 2027

Sparse Frames, Causal Graphs: Long Event-Stream Anomaly Reasoning with Vision-Language Models

Abstract

RGB sensing can lose critical anomaly evidence under fast motion or adverse illumination, while event cameras preserve high-dynamic-range, microsecond-scale changes. Most event-language methods nevertheless aggregate events into frame-like inputs, obscuring the temporal detail that makes event sensing valuable. We propose EFG-VLM, an offline hybrid event-frame/event-graph framework for long-stream open-world anomaly understanding. A category-agnostic locator establishes shared anchors where sparse event frames provide scene semantics and bounded raw-event graphs preserve complementary spatial structure and causal motion. Hierarchical fusion combines landmark, proposal, and video context and exposes structured evidence to a pretrained multimodal language model for anomaly judgement, proposal-conditioned temporal localization, and description. Because localization is category-agnostic and final interpretation is language-generated rather than closed-set classification, the same interface can describe interactions outside benchmark categories. On large-scale v2e-simulated event streams derived from HIVAU-70K, EFG-VLM substantially improves event-source recognition and language description while remaining robust across clip durations. Native RGB–event cases further illustrate transfer to high-dynamic-range interactions beyond the benchmark categories.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Event-based Vision, Video Anomaly Understanding, Vision-Language Models, Spatiotemporal Graph Neural Networks, Multimodal Representation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_FrnDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes EFG-VLM for offline anomaly understanding in long event streams. The method first identifies candidate intervals and selects temporal anchors within each interval. At these anchors, it combines event-frame features with spatial and time-directed motion graphs built from raw events. The resulting representations are passed to a vision-language model to generate anomaly judgements, temporal boundaries and descriptions.

The main experiments use events simulated with v2e from HIVAU-70K, whose source videos come from UCF-Crime and XD-Violence. The paper reports comparisons with existing methods, component ablations and results across different sequence durations. It also presents three qualitative examples using real event recordings.

Critical strengths

1.The method offers a reasonable way to combine scene semantics with local motion information. Using shared temporal anchors gives the event-frame and graph branches a consistent temporal reference. Restricting graph construction to local windows also addresses a practical difficulty in processing long event streams. This is a promising design for connecting detailed event representations to a language model.

2.The internal comparisons provide evidence that the combined representation is useful. In Table 3, the full model improves accuracy from 28.47 to 48.09 and CIDEr from 0.074 to 0.150 compared with the event-frame-only variant. These gains support further investigation of the approach, although the specific contribution of the proposed graph structure still needs to be isolated.

Critical weaknesses

1.The main evaluation does not adequately test the event-camera advantages used to motivate the paper. The authors emphasise fast interactions and difficult lighting, where ordinary RGB cameras may lose useful information. However, the quantitative experiments rely on events simulated from existing RGB videos. HIVAU-70K was designed for general video anomaly understanding, and the paper does not establish how much of the benchmark requires fine temporal resolution or perception under difficult lighting. Converting RGB recordings into events cannot establish that the method recovers details the original camera failed to capture. The three real-event examples provide preliminary evidence, but they are insufficient to support general claims about these sensor advantages.

2.Temporal localisation is an important output of the method, but it lacks quantitative evaluation. The main tables report classification and text-generation metrics without measuring temporal overlap or boundary error. Precise-looking timestamps in selected examples do not establish accurate localisation. The paper also does not quantify how much the later refinement and language stages improve on the initial candidate intervals. As a result, the claimed ability to locate anomalies in time remains insufficiently tested.

3.The methodological advance over existing work is not yet sufficiently established. Anomaly-focused temporal sampling appears in Holmes-VAU, event-language alignment in EventGPT, and efficient processing of long event sequences in EventFlash. The potential contribution here is the specific combination of shared anchors, local event graphs and proposal-based fusion. However, the ablations mainly remove entire components, without comparing the proposed design against simpler temporal encoders, conventional spatiotemporal graphs or simpler fusion under matched conditions. Removing the motion graph improves Macro-F1 from 16.55 to 18.74 while reducing CIDEr from 0.150 to 0.085. This supports its usefulness for description, but does not isolate the benefit of the proposed time-directed graph structure. The current evidence leaves open whether the gains require the proposed design or could be obtained through a simpler integration of existing components.

Questions for authors

1.Clarification: Which v2e version and conversion settings were used? Given that frame interpolation was disabled, what effective temporal resolution do the generated events have, and how does this relate to the reported 3 ms setting?

2.Experimental evidence: Can the authors provide quantitative results on real event recordings, particularly for fast interactions or difficult lighting? Please specify the evaluation size and sample-selection procedure, and include failures alongside successful cases. For the simulated benchmark, is there evidence that the gains are concentrated in conditions where fine temporal information is useful?

3.Clarification and experimental evidence: Are all candidate intervals used at test time predicted by the model? Can the authors report temporal overlap or boundary-error metrics and compare the initial locator, the boundary-refinement stage and the complete model?

4.Clarification: What backbone, task-specific training and input budget were used for EvQA-FRT and EvQA-ART? Does the event-frame-only ablation retain the same candidate intervals, supervision and training budget as the full model?

5.Experimental evidence: Under matched inputs, training conditions and comparable model capacity, does the proposed graph and fusion design outperform a conventional spatiotemporal graph or a simpler temporal encoder? A controlled comparison of forward-directed and undirected edges would also help establish the contribution of the temporal direction constraint.

I would be willing to raise my score if the additional experiments and explanations adequately address these concerns and substantiate the paper’s claimed contributions.

9 comments

Q1 (W1) — v2e version and timing resolution

Reply:

Thank you for catching this ambiguity and for your constructive questions. We clarify the exact conversion pipelines, effective temporal resolution, and code release below:

  1. v2e Version & Conversion Settings (HIVAU-70K): All HIVAU-70K source RGB videos were converted using SensorsINI/v2e (Hu et al., 2021) (commit cac6384445, Oct. 2025) under a fixed configuration (Appendix B.1, manuscript Table 8): DVS346 preset (346 ×260), nominal timestamp_resolution =0.003 s, auto_timestamp_resolution disabled, frame interpolation disabled (--disable_slomo), duration exposure mode (0.005 s), positive/negative contrast thresholds of 0.15/0.15, threshold noise std. of 0.03, refractory period of 0.5 ms, cutoff frequency of 10 Hz, and default seed 0. Because untrimmed videos in UCF-Crime and XD-Violence can exceed the 32-bit unsigned microsecond limit (232−1μs≈4,294.972^{32}-1\mu\text{s}\approx 4{,}294.97 s or 71.5871.58 min), we patched the v2eHDF5 writer to store 64-bit integer timestamps. We will release all model code, patched conversion scripts, and exact configuration files in our public repository.

  2. Effective Temporal Resolution vs. Nominal 3 ms Setting: When slow-motion frame interpolation is disabled (--disable_slomo), v2e ignores the nominal 0.003 s step and synthesizes events by interpolating log-intensity changes directly between consecutive source RGB frames. Accordingly, the effective temporal observation interval is bounded by the source RGB frame rate (e.g., ∼\sim 33.3 ms for 30-FPS videos) rather than providing independent 3 ms physical measurements. We have updated manuscript Table 8 and Appendix B.1 to explicitly mark the 3 ms parameter as inactive.

  3. High-Temporal-Rate Protocol (Supplementary CUVA Evaluation): Separately from HIVAU-70K, our CUVA localization evaluation upsamples source videos to 240 FPS (∼\sim 4.17 ms per synthetic frame) via Practical-RIFE 4.25 lite and simulates events using a modified ESIM pipeline (uzh-rpg/rpg_vid2e) with 64-bit nanosecond structured NPY output, fixed seed 1234512345, and matched DVS346 resolution (346×260346\times 260), contrast thresholds (0.15/0.150.15/0.15), noise (0.030.03), and refractory period (0.50.5 ms), as detailed side-by-side in manuscript Table 8.

Revision:

Appendix B.1 (revised manuscript & Table 8): "Because untrimmed surveillance videos in UCF-Crime and XD-Violence can exceed the 32-bit unsigned microsecond limit (232−1μs≈4,294.97s2^{32}-1\mu\text{s}\approx 4{,}294.97\text{s} or 71.58min71.58\text{min}), we patch the v2e HDF5 writer to store 64-bit integer timestamps, preventing timestamp wrap-around in long streams." "With frame interpolation disabled in v2e, the nominal 0.003 s step is inactive and effective temporal sampling follows the source video frame interval."

Q2 (W1) — Real-sensor quantitative evidence

Reply:

Thank you for these questions. We address both the real-sensor evaluation and the simulated category-level analysis below:

  • Quantitative & Qualitative Evaluation on Real Event Recordings: revised manuscript Section 4.8, Table 6 reports all nine purposefully selected real synchronized RGB–event sequences (six abrupt physical interactions and three normal controls), comparing EFG-VLM on real events against Holmes-VAU on paired RGB videos. Across the six anomalous sequences, EFG-VLM achieves 44.51–87.03% tIoU (mean 68.50%), whereas frame-based Holmes-VAU does not output localized anomaly intervals. Paired description metrics include both gains and weaker cases: for example, EFG-VLM outperforms Holmes-VAU in BERTScore-F1 on Jenga (0.8866 vs. 0.8765) and water-bomb (0.8921 vs. 0.8782), while trailing on may28_axe_01 (0.8431 vs. 0.8614). Figures 8–9 in revised Appendix C also show both accurate grounding and hallucinated descriptions across model variants.

  • Category-Wise Gains on Simulated Streams: revised manuscript Table 9 breaks down language gains from the directed motion graph across all 16 activity categories. CIDEr improvements are concentrated in dynamic physical interactions such as Fighting (+0.0396), Explosion (+0.0259), and Car accident (+0.0250), whereas crowd-level or static-scene categories such as Riot (-0.0253) and Vandalism (-0.0371) decline. Stratifying simulated benchmarks directly by motion speed and illumination is a promising direction for future work.

Revision:

Sec. 4.8 (revised manuscript): "We evaluate nine real synchronized RGB–event sequences, including six with manually reviewed anomalous intervals and three without anomalies. EFG-VLM processes real events, whereas the RGB baseline is Holmes-VAU." "Across the six anomalous sequences, EFG-VLM achieves 68.50% mean tIoU (range 44.51–87.03%)."

Q3 (W2) — Temporal localization and GT-free proposals

Reply:

Yes. All candidate intervals are predicted directly from the input event stream at test time without ground-truth interval injection. Stage 1 generates category-agnostic activity proposals, while the downstream language model makes the anomaly judgement and outputs refined start/end bins relative to the top-1 proposal (𝑃 =1) when an anomaly is detected.

On the 38 anomaly-annotated CUVA sequences (revised manuscript Table 2), Locator refine improves category-agnostic proposal overlap over Locator initial from 0.5568 to 0.6249 mean tIoU and from 57.89% to 60.53% R@[email protected] (with 100% proposal emission coverage). When gated by anomaly detection, EFG-VLM (full) achieves 65.79% anomaly-detection coverage, 0.3970 mean tIoU, and 44.74% R@[email protected] (assigning zero tIoU to missed detections across all 38 samples), with a 11.47 s conditional boundary MAE on detected intervals (Section 4.3 and Appendix B.2).

Revision:

Appendix B.2 (revised manuscript): "Evaluation uses the same 38 anomaly-annotated CUVA samples for all methods and stages. Mean tIoU and R@1 use the full evaluation set, assigning zero overlap to missed predictions. Coverage is the proportion of non-empty interval predictions, and boundary MAE is averaged over those predictions."

Q4 (related to W3) — Baseline settings and controls

Reply:

Thank you for asking us to clarify both the external baseline configurations and our internal ablation controls:

  1. External Baselines (EvQA-FRT and EvQA-ART in manuscript Table 1): Both baselines were evaluated using the authors’ publicly released inference pipelines, checkpoints, and recommended evaluation settings without task-specific fine-tuning on HIVAU-70K. Specifically, FRT reconstructs dense intensity frames at 24 FPS using the released V2V-E2VID checkpoint, whereas ART performs activity-triggered sparse reconstruction using the released Adaptive-E2VID checkpoint, followed by a Qwen3-VL-Thinking vision-language backbone. Manuscript Table 1 therefore serves as a system-level comparison across distinct pipelines.

  2. Internal Event-Frame-Only Ablation (manuscript Table 4): By contrast, the Event-frame-onlybaseline isolates the contribution of our graph branch within an identical VLM pipeline:

    • Shared components: It retains the exact same Stage-1 locator, five shared temporal anchors, rendered event-frame inputs, Qwen-3.5-2B backbone, supervision targets, and Stage-3 LoRA recipe as the full model.

    • Independent optimization: Each retained projection and LoRA adapter is trained independently for event-frame features (with graph-specific parameters and Stage-2 graph pretraining omitted), directly measuring the contribution of event-native graph evidence alongside sparse frames.

Revision:

Secs. 4.2 and 4.5 (revised manuscript): "EvQA-FRT/ART use released reconstruction-based front-ends and pretrained VLM checkpoints without HIVAU-specific adaptation"; "Under a shared Stage-1 locator, five anchors, Qwen-3.5-2B backbone, language supervision, and Stage-3 LoRA recipe, removing the graph branch reduces accuracy from 48.09 to 28.47 and CIDEr from 0.150 to 0.074; retained modules are independently optimized for each variant."

Q5 (W3) — Controlled graph-structure comparison

Reply:

Thank you for proposing these controlled experiments. In addition to the component ablations in manuscript Table 4, we have added a controlled architectural comparison (manuscript Table 5) evaluating a single 3D spatiotemporal graph, random partitioning, undirected motion edges, and an MLP–TCN/GRU temporal encoder against our Full Dual design (G0).

To compare all five architectures under identical optimization settings, we trained the four control variants alongside a matched Full Dual reference (G0) using a unified reduced-budget training schedule on the exact same dataset split and test set, varying only the graph/temporal module.

As shown in Table 5, G0 achieves the strongest performance across all metrics (52.71% Accuracy, 16.08% Macro-F1, 0.0753 CIDEr, and 0.2349 METEOR):

  • Compared with the single 3D spatiotemporal graph, separating spatial and motion relations improves Macro-F1 by 4.53 percentage points.

  • Random partitioning causes the largest drop, reducing Accuracy and Macro-F1 by 6.94 and 9.07 points, respectively.

  • Replacing directed motion edges with undirected edges reduces CIDEr by 0.0063 and METEOR by 0.0133, confirming the benefit of temporal directionality.

  • The MLP–TCN/GRU alternative also yields lower recognition accuracy (46.57% vs. 52.71%).

Together, these controlled comparisons validate both our spatial–motion relation decomposition and temporally directed edge formulation.

Revision:

Sec. 4.6 (revised manuscript): "For computational tractability, all five variants are trained from scratch under an identical reduced-budget protocol (Appendix B.2), making G0 a dedicated reference distinct from the full-budget model in Table 4."

Table 5: Controlled graph-structure comparison under a unified reduced-budget protocol (distinct from the full-budget setting in Table 4). Runtime measures average per-sample processing time across core encoding, classification, and description generation.

VariantAcc. ↑Macro-F1 ↑CIDEr ↑METEOR ↑Encoder Params (M)Avg. Model Time (s)
G0 Full Dual52.7116.080.07530.23490.6226.89
G1 Single 3D51.5911.550.06840.2294Same as G07.49
G2 Random Partition45.777.010.06440.2230Same as G09.42
G3 Undirected Motion51.5215.080.06900.2216Same as G07.24
G4 MLP + Temporal TCN/GRU46.5712.490.06890.2310Same as G07.02
Reviewer_sZdHOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EFG-VLM, a framework for long-stream anomaly understanding that combines sparse event frames for semantic context with spatial–motion graphs for fine-grained temporal dynamics. By aligning both representations through shared temporal anchors and integrating hierarchical context into a pretrained multimodal language model, the method supports anomaly recognition, temporal localization, and language description. Experiments on simulated event streams and native RGB–event examples demonstrate improvements over event-based baselines, particularly in identifying and describing short-lived anomalous interactions.

Critical strengths
  1. The paper proposes a well-motivated framework that combines sparse event frames for semantic understanding with spatial–motion graphs for capturing fine-grained temporal dynamics. The shared temporal anchors and hierarchical fusion design effectively address the limitations of frame-based event representations in long-stream anomaly understanding.

  2. The proposed method achieves notable improvements in anomaly recognition and language description over event-source baselines. The ablation studies demonstrate the complementary contributions of spatial and causal motion graphs, while qualitative results illustrate the method's ability to identify and localize short-duration abnormal interactions.

Critical weaknesses

1、The comparison with event-based baselines is limited. Table 1 includes only two event-source baselines, making it difficult to fully assess the proposed method against existing event-based understanding approaches. The authors should consider including additional relevant baselines or providing a stronger justification for the limited comparison.

2、Although Table 3 quantifies the contributions of individual components, the paper does not provide qualitative results for different ablation variants. Visual comparisons of the full model and its ablated versions would help clarify how each component affects anomaly recognition, temporal localization, and language description.

3、 Although the paper outlines a three-stage training pipeline and identifies the main trainable components and objectives, important implementation details remain insufficiently specified. In particular, the loss formulations and weighting strategies for the first two stages, as well as key optimization hyperparameters and the construction of language-supervision targets, should be described more explicitly to facilitate reproducibility.

Questions for authors

see Weaknesses

5 comments

W1 — Limited event-based comparisons

Reply:

Thank you for raising this point. Evaluating on the full HIVAU-70K benchmark in Table 1 requires a model to simultaneously satisfy three task requirements: (i) strictly event-only input at inference, (ii) support for multi-second to minute-long event streams, and (iii) an instruction-following language interface capable of both anomaly recognition and free-form description.

As summarized in Section 4.1 and Appendix B.1, existing event-based methods differ in supported temporal horizons, input modalities, or task interfaces, which prevents direct full-benchmark comparison under this protocol:

  1. Short-Window Event-MLLMs (e.g., EventGPT (Liu et al., 2025)): EventGPT is designed for short (∼100ms\sim100\mathrm{ms}) event segments rather than multi-second or minute-long streams. To provide a direct comparison within its supported temporal regime, we evaluate EventGPT in the <1s<1\mathrm{s} duration bin in manuscript Table 3.

  2. RGB–Event Fusion MLLMs (e.g., LLaFEA (Zhou and Lee, 2025), EventSTU (Xu et al., 2025)): These frameworks require synchronized RGB frames alongside event streams during inference, whereas our benchmark evaluates strictly event-only inference.

  3. Non-Generative Event Vision & SNN VAD Methods (e.g., AEGNN (Schaefer et al., 2022), UCF-Crime-DVS (Qian et al., 2025)): Classical event graph networks and spiking VAD models output closed-set classification scores without a language interface, and therefore cannot perform instruction-based reasoning or generate anomaly descriptions.

  4. Concurrent Event-MLLMs (e.g., EventFlash (Liu et al., 2026a), GEP (Cao et al., 2026)): Complete public inference pipelines and checkpoints compatible with long-stream anomaly evaluation were not available at the time of our experiments.

Consequently, EvQA-FRT and EvQA-ART are currently the only publicly runnable external Event-MLLM baselines that support long event streams and open-ended anomaly description for full-benchmark evaluation in Table 1. Together with EventGPT on short clips (manuscript Table 3) and our backbone-aligned Event-frame-only baseline (manuscript Table 4), these comparisons cover the available event-based baselines across both short and long temporal regimes.

Revision:

Sec. 4.1 (revised manuscript): "EventGPT is reported only for the shortest-duration group because its supported event segment is approximately 100 ms, while LaGoVAD (Liu et al., 2026b) performs classification only and therefore has no language metrics. Accordingly, Table 1 evaluates long-stream EvQA-FRT/ART, Table 3 includes short-window EventGPT, and Table 4 isolates graph contributions under a shared backbone."

W2 — Qualitative component ablations

Reply:

Thank you for this suggestion. We have added qualitative comparisons covering the full model and all component ablations in Figures 1–2 (also included in Appendix C). In these panels, bold black text highlights accurate salient details, while red text marks unsupported or hallucinated content; RGB frames are shown solely for visual reference, whereas all EFG-VLM variants take event streams as input.

  • Scene and anchor grounding. Removing event graphs (Event-frame-only) or adaptive landmarks (w/o change-aware anchors) leads to generic street descriptions or unrelated hallucinations (such as a car dashboard), while removing the spatial graph weakens scene grounding.

  • Spatial and motion evidence. In the protest sequence (Figure 1), the full model captures demonstrators, flags, and smoke; the motion-graph ablation also retains protest cues, consistent with our category-wise analysis. In the separate road-smoke sequence (Figure 2), the full model appropriately identifies a possible accident or mechanical failure from the smoke plume, whereas removing the directed motion graph hallucinates an unsupported two-car collision, illustrating how motion evidence constrains interaction descriptions.

Figure 1: Qualitative component comparison on a protest sequence. Bold black text highlights salient accurate details; red marks unsupported content. (Also in revised manuscript Appendix C, Figure 8.)

Figure 2: Qualitative component comparison on a road-smoke sequence, distinct from the two-car accident example in the main paper. (Also in revised manuscript Appendix C, Figure 9.)

Revision:

Appendix C (revised figure captions): "The full model captures demonstrators, flags, and smoke; some ablations lose scene grounding or introduce unrelated details." "The full model describes visible smoke and a possible accident or mechanical failure; the motion-graph ablation introduces an unsupported two-car collision."

W3 — Training reproducibility

Reply:

Thank you for requesting more detailed training formulations. We have expanded Appendix A.4 to specify the loss functions, weighting coefficients, supervision-target construction, and optimization hyperparameters for the first two stages:

  • Stage 1 (Category-Agnostic Activity Localization): Trained with an eight-term objective combining eventness, start/end boundary, center, dense offset, observation-utility quality, boundary refinement, and pairwise ranking losses (Eq. 28), optimized via AdamW (lr =5 ×10−5, 2,000 steps, global batch size 16).

  • Stage 2 (Joint Graph–Context Learning): Optimizes the graph encoder and context module with seven losses: boundary refinement, binary anomaly prior, multi-horizon future event prediction (Δ∈{1,2,4,8}), motion reconstruction, semantic alignment, proposal validity (tIoU ≥0.2), and description readiness (tIoU ≥0.5) (Eq. 31), trained for 1,200 steps with AdamW (lr =10−4 for new modules and 5 ×10−6 for the graph encoder).

  • Language-Supervision Targets: Constructed from the frozen text backbone of Qwen3.5-2B using final-layer masked mean pooling and 𝐿2 normalization (2048-D embeddings), aligning video-, event-, and clip-level descriptions with their corresponding visual representations.

Revision:

Appendix A.4 (revised manuscript): "Stage 2 trains the graph encoder and the scene–motion context module jointly from step 0, while the Stage-1 locator and the adapted Qwen visual encoder remain frozen." "Description readiness requires a stricter match, tIoU ≥0.5." "Stage 2 runs for 1,200 optimizer steps with AdamW."

AuthorsOpenReview ↗

Follow-up on Our Rebuttal and Manuscript Revisions

Dear Reviewer,

We sincerely appreciate your encouraging review and constructive suggestions. In our rebuttal and revised manuscript, we have worked to address the points you raised: we clarified the scope and fairness of our event-based baseline comparisons, incorporated new component-ablation visualizations to strengthen the qualitative analysis, and documented the complete training objectives, loss formulations, and optimization settings to facilitate reproducibility.

Whenever your schedule permits, we would be grateful to hear whether these additions and clarifications have helped address your concerns, or if there are any remaining questions we can further address during the discussion phase.

Thank you again for your valuable time and feedback.

Reviewer_D6qbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes EFG-VLM, an event-based vision-language framework for long-stream anomaly understanding. The main motivation is that RGB videos may lose important information under rapid motion or difficult illumination, while event cameras preserve fine temporal changes. Existing event-language methods commonly aggregate asynchronous events into frames, which improves compatibility with vision-language models but sacrifices part of the temporal structure. EFG-VLM therefore separates semantic understanding from fine motion modeling: sparse event frames provide scene and object semantics, while raw-event graphs preserve short-term spatial structure and temporally directed motion relationships.

The framework first builds a coarse 4-Hz timeline and uses a category-agnostic temporal locator to generate candidate intervals. Each proposal is represented using five shared temporal anchors: start, center, end, and two change-aware internal landmarks. Around these anchors, the system renders event frames for semantic encoding and separately constructs bounded raw-event graphs.

The graph branch models two different types of relationships. Spatial edges connect nearby events that occur almost simultaneously, with the goal of preserving contours and local structure. Motion edges connect earlier events to later events within bounded spatial and temporal neighborhoods. These two relations are processed separately and then fused before additional temporal aggregation. The event-frame and graph representations are then combined at the same temporal anchors, contextualized using proposal-level and long-range video information, and converted into structured tokens for a Qwen language model. The language model outputs anomaly judgement, description, and proposal-conditioned temporal localization.

For quantitative evaluation, RGB videos from HIVAU-70K are converted one-to-one into simulated event streams using v2e. The authors separately report results for the XD-Violence and UCF-Crime source subsets. EFG-VLM substantially improves accuracy and language metrics over the evaluated event-source baselines. For example, compared with EvQA-FRT, accuracy increases from 28.89% to 50.38% on the XD subset and from 16.91% to 35.61% on UCF-Crime, while CIDEr increases from 0.005 to 0.150. However, Macro-F1 is only slightly better on XD and slightly worse on UCF-Crime.

The paper also evaluates performance across different clip-duration groups, provides component ablations, and shows qualitative examples on both simulated and native event-camera sequences. The ablations indicate that event-native graphs provide substantial information beyond event frames alone, while spatial and motion graphs appear to serve different roles: spatial structure contributes more strongly to recognition, whereas temporally directed motion appears particularly useful for language description.

Critical strengths

The strongest aspect of the paper is the representation design. The authors correctly identify a tension in event-based multimodal reasoning: converting everything to event frames makes the data compatible with pretrained visual encoders but discards some of the temporal structure that makes event sensing attractive in the first place. Conversely, processing the complete raw event stream with a graph would be computationally very expensive. Restricting raw-event graph reasoning to proposal-guided local windows while maintaining a coarse global timeline is therefore a sensible compromise.

The separation between spatial structure and temporally directed motion is also well motivated. Rather than defining a single generic neighborhood in space-time, the method uses one graph to represent near-simultaneous geometry and another to represent earlier-to-later event evolution. This is a reasonable inductive bias because contours/contact structure and motion history are genuinely different forms of evidence.

The use of shared temporal anchors across the event-frame and graph branches is another good design decision. It avoids the difficult problem of aligning independently extracted semantic and motion evidence after the fact. The adaptive landmarks are also supported empirically: replacing them with uniformly spaced five-point anchors causes a large decrease in CIDEr, from 0.150 to 0.074.

The hierarchical interface to the language model is stronger than simply giving Qwen one pooled visual token. The model receives global context, proposal summaries, ordered landmark tokens, temporal coordinates, and explicit proposal priors. This makes the reasoning interface relatively transparent and preserves more of the temporal structure recovered by the perception stages.

The ablation study is informative. Removing event-native graph evidence and using event frames alone decreases accuracy from 48.09% to 28.47% and CIDEr from 0.150 to 0.074, providing reasonably strong evidence that raw-event structure adds useful information.

I also appreciate that the paper does not overinterpret RGB-versus-event comparisons. The authors explicitly state that RGB and event systems are evaluated under different sensing modalities and that their scores should not be interpreted as controlled modality comparisons. This is a careful and appropriate qualification.

Critical weaknesses

The most important weakness is that the paper's main quantitative benchmark is built from simulated events generated from RGB video. Every HIVAU-70K RGB video is converted one-to-one with v2e, using a fixed configuration and without frame interpolation. This is a serious limitation relative to the paper's motivation. If the original RGB recording already missed a microsecond-scale collision or lost information because of saturation, motion blur, or exposure, v2e cannot reconstruct physical evidence that was never recorded. Therefore, the simulated benchmark can evaluate whether the proposed representation is effective for event-like data, but it cannot strongly validate the claimed sensing advantage of event cameras under fast motion or difficult illumination. The native event experiments partially address this issue, but they are qualitative and very small.

A second major weakness is the lack of a convincing quantitative temporal localization evaluation. Precise localization is repeatedly presented as a central capability of the model, and Figure 1 explicitly motivates the framework around answering not only “what happened?” but also “exactly when?”. Yet the primary quantitative tables report recognition and language-generation metrics, not temporal IoU, average precision at temporal IoU thresholds, boundary error, center error, or similar localization metrics. The timestamped examples are useful but cannot establish localization accuracy across the benchmark.

Third, the term “causal graph” is potentially misleading. The motion graph is causal only in the limited sense that edges point from earlier events to later events. It does not perform causal discovery, causal intervention, counterfactual reasoning, or establish causal relationships in the usual causal-inference sense. I would prefer terminology such as “temporally directed motion graph” unless the authors can justify the stronger causal interpretation.

Fourth, the ablation results show that the motion graph has a somewhat ambiguous contribution. Removing the causal motion graph actually increases recognition accuracy from 48.09% to 49.80% and Macro-F1 from 16.55% to 18.74%, although language metrics become substantially worse. This is interesting and potentially valuable, but it means the motion graph is not clearly improving anomaly recognition itself. The paper should therefore be more precise: the spatial graph appears more important for recognition, while directed motion is mainly useful for richer semantic explanation.

Fifth, baseline fairness needs stronger analysis. EFG-VLM uses a Qwen 3.5-2B language backbone with LoRA and a substantial perception pipeline, while the event-source baselines may use different backbone sizes, representations, training data, and language models. The large gains over EvQA are encouraging, but without matched-capacity baselines it is difficult to determine how much improvement comes from event graphs versus a stronger language model and more elaborate architecture.

Sixth, the long-stream robustness evidence is limited. The duration bins contain different clips rather than different-length versions of the same underlying sequences, and the authors correctly acknowledge that the results therefore show performance across regimes rather than a causal effect of increasing duration. Because content distribution may vary substantially across bins, the reported increase in Macro-F1 with clip length does not by itself establish long-context robustness.

Seventh, the paper would benefit from much clearer efficiency and scalability analysis. A graph chunk can retain up to 12,000 nodes, the graph operates at 120-Hz chunks, and multiple graph neighborhoods are constructed around five anchors per proposal. This could be computationally heavy. Runtime, GPU memory, number of graph nodes/edges processed per proposal, number of proposals per video, and scaling with stream length should be reported.

Finally, the Stage-1 locator appears to be trained using activity boundaries from annotated training data. This is compatible with the stated task, but the paper's “open-world” wording could leave readers with an overly strong impression of zero-shot localization. The distinction between open-vocabulary semantic interpretation and supervised proposal/localization learning should be made more explicit.

Questions for authors
  1. Can you provide quantitative temporal localization results? Since localization is one of the major claimed outputs of EFG-VLM, please report standard metrics such as temporal IoU, mean boundary error, or temporal AP at multiple IoU thresholds. It would also be useful to separate Stage-1 proposal quality from the final Qwen-refined localization.

  2. How can the v2e benchmark validate the claimed event-camera advantage under fast motion and HDR conditions? Since v2e receives conventional RGB videos as input, information already lost to RGB exposure, motion blur, or frame rate cannot be recovered. Could the authors provide quantitative evaluation on a native event-camera anomaly/action dataset, rather than only selected qualitative native sequences?

  3. Would the authors consider renaming “causal motion graph”? The current graph enforces earlier-to-later message passing, but this is temporal directionality rather than causal inference. What specifically justifies the word “causal” beyond temporal ordering?

  4. Why does removing the motion graph improve recognition accuracy and Macro-F1 while substantially hurting language metrics? Does this indicate that the motion graph mainly improves event explanation rather than anomaly recognition? Could the authors provide per-class or per-anomaly-type results showing where motion topology is actually helpful?

  5. Can the authors provide a matched event-frame baseline using the same Qwen backbone, training data, proposal locator, and language interface but without event graphs? The current event-frame-only ablation is helpful, but more detail is needed to confirm that the large gains over published EventQA-style baselines come from the proposed event representation rather than backbone or training differences.

  6. Can you report runtime and memory requirements? In particular, how many proposals are processed per video, how many event nodes and edges are typically constructed, what is the end-to-end inference time for a 10-second and 30-second stream, and what GPU memory is required?

  7. Can long-duration robustness be evaluated more directly? For example, could the same underlying long clips be progressively truncated or padded with normal context so that performance can be measured as context length changes while the anomaly remains fixed? This would avoid the confounding effect of using different clips in different duration bins.

  8. How much supervision does the temporal locator use? Are Stage-1 activity boundaries directly derived from ground-truth anomaly intervals? If so, please clarify the distinction between supervised proposal localization and open-world language interpretation.

  9. How many native event-camera samples were evaluated in total? The paper currently shows a few strong qualitative examples. A small but systematic native-event benchmark, even without perfect anomaly labels, would substantially strengthen the sensing-related claims.

7 comments

Q1 (W2) — Quantitative temporal localization

Reply:

Thank you for this suggestion. To evaluate temporal grounding quantitatively, we construct an evaluation benchmark of 38 anomaly-annotated sequences from the CUVA dataset using the high-temporal-rate RIFE+ESIM event conversion documented in Appendix B.2. All methods and model stages are evaluated on the exact same 38 sequences against annotated anomaly spans, with all test-time intervals predicted directly from the event stream without ground-truth input (Table 1; manuscript Table 2):

  • Activity proposals: Locator initial to Locator refine. Both stages emit category-agnostic activity intervals for every sample (100% proposal emission coverage). Refinement raises mean tIoU from 0.5568 to 0.6249 and R@[email protected] from 57.89% to 60.53%, confirming improved proposal alignment with annotated anomaly spans.

  • Anomaly-gated final output: EFG-VLM (full). The full model outputs a localized interval only when it detects an anomaly (65.79% anomaly-detection coverage). Missed detections receive zero tIoU across all 38 samples, so its mean tIoU of 0.3970 and R@[email protected] of 44.74% jointly reflect anomaly detection and localization, with a 11.47 s conditional boundary MAE on detected cases. Because we retain the top-1 proposal (𝑃 =1), we report top-1 recall (R@1) and mean tIoU.

Given the substantial computational overhead of 240-FPS RIFE interpolation, ESIM simulation, and full raw-event graph construction across all baselines, we start with this controlled 38-sequence evaluation set and are expanding this pipeline to a larger split for the final version (Section 4.3 and Appendix B.2).

Revision:

Sec. 4.3 (revised manuscript): "Note that locator stages emit unconditional activity proposals for all samples (100% coverage), whereas EFG-VLM (full) outputs timestamps selectively for detected anomalies (65.79% coverage). Because missed detections count as zero overlap in full-set metrics, unconditional tIoU is 0.3970 (corresponding to $\sim$0.603 conditional tIoU on detected cases), while LLM-conditioned refinement sharply reduces boundary MAE from 30.35 s to 11.47 s."

Table 1: Stage-wise temporal localization on CUVA. [email protected]/0.5/0.7 denote top-1 recall at the indicated tIoU thresholds; recall and coverage are percentages, and MAE is in seconds.

Method[email protected] ↑[email protected] ↑[email protected] ↑tIoU ↑CoverageMAE ↓
RGB-video baselines
VadCLIP60.5357.8955.260.570078.9521.71
MoniTor23.6810.5310.530.171365.7933.84
LaGoVAD23.6821.0521.050.230447.3632.74
Holmes-VAU (ATS)26.3218.4210.530.191347.3637.69
Event-stream localization stages
Locator initial60.5357.8955.260.5568100.0032.47
Locator refine68.4260.5357.890.6249100.0030.35
EFG-VLM (full)50.0044.7436.840.397065.7911.47

Note: R@τ and mean tIoU are evaluated over all 38 samples; MAE is conditioned on non-empty predictions. Coverage reflects unconditional activity proposals for locator stages and anomaly-gated intervals for full models.

Q3 (W3) — Temporally directed graph terminology

Reply:

Thank you for this constructive suggestion. We originally adopted the term "causal" because motion edges propagate strictly from earlier to later events (0 <𝑡𝑗 −𝑡𝑖 ≤𝜏𝑚0 <𝑡_𝑗 −𝑡_𝑖 ≤𝜏_𝑚), aiming to preserve the temporal precedence of physical interactions (e.g., an impact preceding a collapse) as directional cues for the downstream VLM rather than supervising the graph with formal causal-inference objectives.

We agree that "causal graph" carries a specific formal meaning in machine learning (causal discovery, intervention, or counterfactual reasoning) and could be misleading here. Following your recommendation, we have renamed this module to Temporally Directed Motion Graph (and directed motion graph/edges for brevity) across the title, main text, Figure 2, manuscript Table 4, and Appendix A.2.

Revision:

Sec. 3.2 (revised manuscript): "Temporally directed motion edges instead connect earlier events to later targets under bounded temporal and spatial gaps"; "The resulting ordered chunk sequence is processed by a causal multi-scale temporal encoder and a short GRU whose hidden state is reset for every bounded window, so no recurrent state leaks across proposals or videos."

Q2 (W1) — Simulated and real event evidence

Reply:

Thank you for highlighting this distinction between hardware sensing advantages and event-stream representation learning.

We agree that v2e simulation cannot recover HDR or high-frequency motion details already lost in source RGB recordings. While these established sensor properties motivate adopting event cameras, EFG-VLM addresses a complementary algorithmic challenge: how to preserve and exploit the spatial and temporal structure of event streams for open-world anomaly understanding. By combining sparse event-frame semantics with event-native spatial and temporally directed motion graphs, our framework enables an LLM to interpret abnormal activities through structural, dynamic, and contextual evidence without relying on a predefined anomaly-classification head.

The HIVAU-v2e benchmark provides a large-scale, consistently annotated testbed for evaluating this representation and reasoning challenge. In manuscript Table 4, incorporating the event-native graph branch improves recognition accuracy from 28.47% to 48.09% and CIDEr from 0.074 to 0.150 over the Event-frame-only variant, confirming that spatial and directed motion graphs contribute critical dynamic evidence beyond sparse event frames on identical simulated inputs.

To examine real-sensor applicability, we also report a paired evaluation on nine real synchronized RGB–event sequences: six contain manually reviewed anomalous intervals and three are normal controls (revised manuscript Section 4.8, Table 6; rebuttal Table 4 under Q9). Using native event streams, EFG-VLM achieves 68.50% mean tIoU (44.51–87.03%) across the six anomalous sequences, whereas the RGB comparator (Holmes-VAU) does not output localized anomaly intervals for these cases.

Revision:

Sec. 4.8 (revised manuscript): "To assess transferability to real event sensors, we conduct a paired feasibility study on nine synchronized RGB–event sequences from three public datasets (six containing manually verified abrupt physical disruptions and three depicting normal activities). We compare EFG-VLM (real event input) against Holmes-VAU (paired RGB input)..."

Q4 (W4) — Contribution of the motion graph

Reply:

Thank you for this observation. In manuscript Table 4, removing the directed motion graph raises accuracy from 48.09% to 49.80% and Macro-F1 from 16.55% to 18.74%, while reducing CIDEr from 0.150 to 0.085 and ROUGE-L from 0.239 to 0.186. This supports complementary roles: spatial structure supplies stable category cues, while temporally directed motion primarily enriches descriptions of how interactions unfold.

Table 2 (manuscript Table 9) reports language gains across all 16 activity categories. CIDEr improves for Fighting (+0.0396), Explosion (+0.0259), and Car accident (+0.0250), with sample-weighted gains of +0.0381 overall and +0.0100 across the 15 abnormal categories. Riot (-0.0253) and Vandalism (-0.0371) instead decline, showing category-dependent effects.

The qualitative comparisons in Appendix C also illustrate description fidelity: in the road-smoke example, the full model describes the smoke and a possible accident or mechanical failure, whereas the motion-graph ablation invents a two-car collision.

Revision:

Sec. 4.5 (revised manuscript): "The ablation therefore suggests that recognition relies more strongly on structural evidence, while fine-grained description benefits more from directed motion evolution."

Table 2: Category-wise language gains from the directed motion graph (Δ = Full − w/o Motion). Means are weighted by category sample counts.

CategoryΔ BLEU-4Δ CIDErΔ METEORΔ ROUGE-L
Normal+0.0263+0.0904+0.0528+0.0570
Fighting+0.0425+0.0396+0.0483+0.0681
Riot+0.0359−0.0253+0.0387+0.0526
Car accident+0.0266+0.0250+0.0240+0.0360
Shooting+0.0166−0.0027+0.0184+0.0256
Explosion+0.0185+0.0259+0.0355+0.0589
Shoplifting+0.0085+0.0233+0.0075−0.0022
Burglary0.0000−0.0202+0.0143+0.0062
RoadAccidents+0.0033+0.0060+0.0345+0.0154
Abuse+0.0045−0.0070+0.0062+0.0018
Arson+0.0042+0.0065+0.0162+0.0058
Arrest−0.0128+0.0157−0.0012−0.0017
Robbery−0.0023−0.0085−0.0079−0.0041
Assault+0.0217−0.0021+0.0090+0.0081
Vandalism0.0000−0.0371+0.0077+0.0083
Stealing0.0000−0.0267+0.0085−0.0073
Weighted Mean (all 16)+0.0215+0.0381+0.0345+0.0401
Weighted Mean (15 abnormal)+0.0189+0.0100+0.0247+0.0310

Q5 (W5) — Frame baseline

Reply:

Thank you for the opportunity to clarify how representation gains are isolated from backbone and training differences.

First, in manuscript Table 4, the Event-frame-only baseline is a strictly aligned internal ablation: it retains the exact same pretrained Qwen-3.5-2B backbone, HIVAU-70K splits, Stage-1 locator, five temporal anchors, language supervision targets, and Stage-3 LoRA recipe as the full model. Each variant independently optimizes its retained modules (omitting graph parameters and Stage-2 graph pretraining when the graph branch is removed). Under this aligned interface, adding spatial and directed motion graphs raises accuracy from 28.47% to 48.09% and CIDEr from 0.074 to 0.150.

Second, by contrast, EvQA-FRT and EvQA-ART in manuscript Table 1 are cross-system referencesevaluated using the authors’ released reconstruction front-ends and pretrained checkpoints. Because their visual front-ends and VLM configurations differ from ours, we rely on manuscript Table 4 to isolate the specific contribution of our event graphs (clarified in Sections 4.2 and 4.5).

Revision:

Sec. 4.5 (revised manuscript): "Under a shared Stage-1 locator, five anchors, Qwen-3.5-2B backbone, language supervision, and Stage-3 LoRA recipe, removing the graph branch reduces accuracy from 48.09 to 28.47 and CIDEr from 0.150 to 0.074; retained modules are independently optimized for each variant."

Q6 (W7) — Efficiency

Reply:

Table 3 (manuscript Table 10) profiles inference on an NVIDIA RTX 4090 at batch size 1, using three videos per duration and three runs per video:

  • Proposal budget: one highest-ranked proposal (𝑃 =1) and five shared anchors per video.

  • Graph scale: 749,342 nodes / 20,125,117 edges for 10-s inputs and 1,127,481 nodes / 30,698,410 edges for 30-s inputs.

  • Latency and memory: for 10-s / 30-s inputs, mean times are 0.71 / 2.45 s for the locator, 12.01 / 16.82 s for graph encoding, and 9.58 / 9.86 s for context plus Qwen. End-to-end latency is 30.22 / 55.13 s, with 6.45 GB peak VRAM including resident weights.

End-to-end timing includes raw-event I/O, preprocessing, and three 64-token task responses in a shared GPU/NAS environment, excluding weight loading. The table gives precise latencies, standard deviations, and numerical precision settings.

Revision:

Sec. 4.5 (revised manuscript): "On an RTX 4090, end-to-end processing takes 30.219 s for 10-s inputs and 55.132 s for 30-s inputs, with 6.45 GB peak VRAM (𝑃 =1)."

Table 3: Measured inference latency and resource usage of EFG-VLM for 10-s and 30-s event streams. Latencies are reported as mean ± standard deviation across trials.

Method / stage10-s latency (s)30-s latency (s)Peak VRAM (GB)Nodes / edges (M; 10 / 30 s)Params (M)
Stage 1 Locator0.707 ± 0.2612.446 ± 0.7524.54–0.662
Graph Encoder12.008 ± 9.21716.820 ± 1.1996.450.749/20.125 ; 1.127/30.6980.622
Context + Qwen9.576 ± 1.2489.858 ± 1.3754.61–2237.499
Full EFG-VLM30.219 ± 21.38155.132 ± 33.0906.450.749/20.125 ; 1.127/30.6982238.783

Note: Measured on an NVIDIA RTX 4090 (batch size 1, P = 1 proposal with 5 anchors, 9 trials across 3 videos per duration) using FP32 graphs and BF16 Qwen. Full EFG-VLM latency additionally includes raw-event disk I/O and preprocessing overhead (~7.9 s for 10-s and ~26.0 s for 30-s streams over shared storage, which also accounts for the higher variance), whereas weight loading is excluded.

Q7 (W6) — Duration robustness

Reply:

We agree with you that binning evaluation clips by duration does not isolate the causal effect of progressively extending background context around a fixed anomaly.

To clarify the construction of MiniDataset in manuscript Table 3: evaluation clips in HIVAU-70K are segmented at original annotated activity/event boundaries and grouped into five bins (<1s to >20s) by the resulting clip duration. Because different bins contain distinct scenes and event instances rather than truncated or padded versions of the same sequence, the variation in Macro-F1 across bins characterizes performance across naturally occurring short- to long-duration activity regimes rather than a controlled context-extension effect.

The primary goal of manuscript Table 3 is to verify that EFG-VLM maintains stable recognition and coherent description across a wide span of input durations, while a controlled context-padding protocol remains a valuable direction for future evaluation. We have updated the Table 3 caption, Section 4.4, and Appendix B.1 accordingly.

Revision:

Sec. 4.4 (revised manuscript): "Because MiniDataset bins contain distinct activity instances, these results compare duration regimes rather than the effect of extending context around a fixed event."

Comments on the paper

3 comments
AuthorsOpenReview ↗

Appreciation for Reviewers' Positive Feedback and Manuscript Revisions

We sincerely thank the reviewers for their thoughtful evaluations, constructive comments, and recognition of our work. We are particularly grateful for their positive assessments of the motivation, novel representation design, complementary spatial–motion modeling, shared temporal anchors, hierarchical language grounding, and experimental improvements of EFG-VLM. We summarize these encouraging comments below and sincerely appreciate the valuable suggestions. We have carefully considered all concerns and address them individually in the following responses.

1. Well-Motivated Hybrid Representation and Efficient Long-Stream Processing

D6qb (S1): "The strongest aspect of the paper is the representation design." "Restricting raw-event graph reasoning to proposal-guided local windows while maintaining a coarse global timeline is therefore a sensible compromise."

sZdH (S1): "The paper proposes a well-motivated framework that combines sparse event frames for semantic understanding with spatial–motion graphs for capturing fine-grained temporal dynamics."

FrnD (S1): "The method offers a reasonable way to combine scene semantics with local motion information." "Restricting graph construction to local windows also addresses a practical difficulty in processing long event streams."

2. Complementary Spatial Structure and Temporally Directed Motion Modeling

D6qb (S2): "The separation between spatial structure and temporally directed motion is also well motivated." "This is a reasonable inductive bias because contours/contact structure and motion history are genuinely different forms of evidence."

sZdH (S2): "The ablation studies demonstrate the complementary contributions of spatial and causal motion graphs."

3. Shared Temporal Anchors and Adaptive Landmark Selection

D6qb (S3): "The use of shared temporal anchors across the event-frame and graph branches is another good design decision." "The adaptive landmarks are also supported empirically: replacing them with uniformly spaced five-point anchors causes a large decrease in CIDEr, from 0.150 to 0.074."

sZdH (S1): "The shared temporal anchors and hierarchical fusion design effectively address the limitations of frame-based event representations in long-stream anomaly understanding."

FrnD (S1): "Using shared temporal anchors gives the event-frame and graph branches a consistent temporal reference."

4. Hierarchical Fusion and Structured Language Grounding

D6qb (S4): "The hierarchical interface to the language model is stronger than simply giving Qwen one pooled visual token." "This makes the reasoning interface relatively transparent and preserves more of the temporal structure recovered by the perception stages."

sZdH (S1): "The shared temporal anchors and hierarchical fusion design effectively address the limitations of frame-based event representations in long-stream anomaly understanding."

FrnD (S1): "This is a promising design for connecting detailed event representations to a language model."

5. Strong Experimental Improvements and Informative Ablation Studies

D6qb (S5): "The ablation study is informative." "Removing event-native graph evidence and using event frames alone decreases accuracy from 48.09% to 28.47% and CIDEr from 0.150 to 0.074, providing reasonably strong evidence that raw-event structure adds useful information."

sZdH (S2): "The proposed method achieves notable improvements in anomaly recognition and language description over event-source baselines." "Qualitative results illustrate the method’s ability to identify and localize short-duration abnormal interactions."

FrnD (S2): "The internal comparisons provide evidence that the combined representation is useful." "These gains support further investigation of the approach."

6. Careful and Fair Interpretation of Cross-Modality Comparisons

D6qb (S6): "I also appreciate that the paper does not overinterpret RGB-versus-event comparisons." "This is a careful and appropriate qualification."

Responses to Reviewer Comments and Manuscript Revisions

We sincerely thank you again for their constructive feedback and valuable suggestions. We have carefully addressed each concern in the individual responses below and revised the manuscript accordingly. For the complete responses, including additional experimental results, detailed tables, qualitative visualizations, and implementation details, please refer to the revised manuscript PDF provided in the Supplementary Material, where all corresponding updates are documented.

AuthorsOpenReview ↗

Consolidated Summary of Additional Experiments and Revisions

Dear Area Chair and Reviewers,

We sincerely thank all reviewers for their constructive feedback. As the discussion period approaches its conclusion, we would like to summarize the key additional experiments and clarifications in our revised manuscript and supplementary material.

1. Graph-Structure and Motion Analyses (Secs. 4.5–4.6; App. B.2; Tables 5, 9)

We added controlled G0–G4 architectural comparisons, demonstrating the benefits of spatial–motion decomposition and temporal directionality (e.g., +4.53 Macro-F1 points over a conventional 3D spatiotemporal graph and +6.94 percentage points in accuracy over random partitioning). Category-wise analyses further demonstrate gains in several rapid physical-interaction categories. We also clarified the terminology temporally directed motion graph to avoid implying formal causal inference.

2. Quantitative and Qualitative Temporal Localization (Sec. 4.3; Table 2; App. C.2)

On 38 CUVA test sequences, Stage-1 refinement improves mean tIoU from 0.5568 to 0.6249. On the matched 25 detected sequences, Locator refine achieves 0.7279 tIoU / 12.59 s MAE, versus 0.6035 tIoU / 11.47 s MAE for the final VLM. We explicitly acknowledge this trade-off and added six qualitative timeline case studies (Table 11; Figs. 10–15) illustrating the distinction between broader activity coverage and anomaly-focused temporal grounding.

3. Real Event-Camera Evaluation and Simulation Details (Sec. 4.8; App. B.1; Tables 6, 8)

We added a paired evaluation on nine real-sensor RGB–event sequences, achieving 68.50% mean tIoU across six anomalous cases, framed as a small-scale feasibility study. We also documented the v2e, RIFE-240 + ESIM, and effective temporal-resolution settings for reproducibility.

4. Baseline Fairness and Component Ablations (Secs. 4.1–4.5; Table 4; App. C.1)

We clarified the comparison scope of EventGPT and EvQA. Our backbone-controlled event-frame-only ablation demonstrates the contribution of event-native graphs (28.47% → 48.09% accuracy; 0.074 → 0.150 CIDEr). Additional qualitative ablations (Figs. 8–9) illustrate the effects of individual graph components on scene grounding.

5. Training Reproducibility and Efficiency (Apps. A.4, B.2; Table 10)

We expanded the documentation of training objectives, loss formulations, supervision, and optimization settings, clarified GT-free inference, and reported stage-wise latency, graph sizes, and peak VRAM (6.45 GB on an RTX 4090).

We sincerely appreciate the reviewers' and Area Chair's time and guidance. We would be grateful for any remaining questions or concerns that we could clarify before the discussion period closes.

Thank you again for your valuable consideration.

Area_Chair_Z27tOpenReview ↗

Please join the discussion and reply to the authors if you still have concerns

Dear reviewers,

Thank you for your contribution to the ICLR review.

The Paper Discussion phase is coming to an end very soon. Please could you take a look at the authors' reply, if you have not yet, and let the authors know if you still have any questions or concerns.

Thanks, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.