acceptodds
Withdrawn from ICLR 2027

TFE-GraphRAG: Temporal-Frequency Evidence Retrieval for Efficient Event-Based MLLMs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Event-based multimodal large language models (MLLMs) enable robust perception in high-speed and low-light scenarios. However, current event-based MLLMs often convert event streams into frame-like inputs, fixing the event evidence representation before reasoning, while independent token selection can omit evidence connections needed to compose an answer. We propose TFE-GraphRAG, a temporal-frequency GraphRAG framework with two major modules. Offline Wavelet-Guided Event Memory Construction (WEMC) applies CWT directly to raw events, uses patch-local frequency-group energy flux to perform CWT Adaptive Binning (CAB), and builds a patch–time–frequency Event Evidence Graph whose frequency-specific nodes each carry a semantic address key, one signed coefficient block pointer, and conservative typed edges. Online Query-Conditioned Evidence Retrieval (QCER) scores those lightweight addresses, expands candidate seeds into a compact connected component, and loads only the referenced signed CWT coefficient blocks for the MLLM. This division bounds online visual context while preserving relationships among the selected regions. On EventMind, TFE-GraphRAG achieves higher reported scores than EventFlash on all four metrics at both 3B and 7B scales. At 7B, it also improves EventChat-Sub GDC, while EventFlash remains higher on EventChat-Sub FGQA. In the controlled 3B comparison, TFE-GraphRAG provides the online throughput of the Dense Event Baseline while improving all four EventMind metrics.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Event-based vision, multimodal large language models, GraphRAG, continuous wavelet transform.

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_JVaMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a Time-Frequency-Graph Retrieval-Augmented Generation (TFE-GraphRAG) framework designed for event-centric Multimodal Large Language Models (MLLMs). The framework comprises an offline "Wavelet Event Memory Construction" (WEMC) module, which employs "Continuous Wavelet Transform-based Adaptive Slicing" (CAS) to generate frequency-specific frames and organize them into event evidence graphs featuring temporal, frequency, and spatial edges; and an online "Query-Conditioned Evidence Retrieval" (QCER) module, which retrieves evidence for the MLLM's inference process by selecting query-relevant root nodes and performing bounded multi-hop expansion. Experiments conducted on the EventMind and EventChat-Sub datasets using the Qwen2.5 backbone model (in two sizes) demonstrate that this method enhances both task performance and online processing throughput.

Critical strengths
  1. This framework combines frequency-aware adaptive event representation with graph-based retrieval, aiming to address the issues of temporal information loss and spatiotemporal context fragmentation.

  2. TFE-GraphRAG improves task performance over EventFlash on EventMind and EventChat-Sub and increases online throughput over the Dense Event Baseline.

Critical weaknesses
  1. Evaluations focused on the backbone model offer limited evidence regarding generalizability. TFE-GraphRAG was evaluated using only the 3B and 7B versions of Qwen2.5; while this demonstrates consistency across model scales, it fails to verify performance across different model families or multimodal architectures. Incorporating an additional compatible backbone model for evaluation would help determine whether the performance gains achieved by this method extend to architectures beyond the specific model family tested.

  2. Evaluations based on Large Language Models (LLMs) lack human verification. Although GDC and FGQA scores were generated by GPT-5.6 Luna, no independent human assessment was reported to calibrate these judgments. A representative subset should undergo human evaluation via blind testing, with clear reporting on the sampling process, sample size, annotator qualifications, scoring criteria, disagreement resolution mechanisms, inter-annotator agreement, and the level of alignment between human and LLM evaluations.

Questions for authors

Table 4 does not clearly demonstrate the benefit of adaptive frequency selection. The adaptive method can retrieve from all three frequency groups, while each fixed policy uses only one. A fixed strategy that also accesses all three groups might perform equally well. This baseline is needed to show whether query adaptation itself provides additional gains.

Reviewer_jNcgOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces TFE-GraphRAG, a temporal-frequency retrieval framework for event-based multimodal large language models. The method combines Continuous Wavelet Transform (CWT)-based adaptive event slicing with a structured evidence graph containing temporal, frequency, and spatial similarity connections. A query-conditioned retrieval mechanism then selects relevant graph nodes and constructs compact visual evidence for downstream MLLM reasoning.

Critical strengths
  • The paper investigates efficient event-stream understanding, an important problem given the high temporal resolution and asynchronous nature of event-camera data.
  • Combining multi-resolution event representations with structured evidence retrieval is an interesting direction for handling long or complex event streams.
Critical weaknesses
  • The Introduction identifies two major challenges, temporal information loss and spatiotemporal context fragmentation, but does not sufficiently establish their individual effects on existing event-based MLLMs. In particular, the connection between fixed event grouping and the need for graph-based retrieval is not clearly motivated. The paper moves from adaptive event representations to video RAG without demonstrating why conventional retrieval mechanisms are fundamentally insufficient for the proposed setting.
  • The proposed framework largely combines established techniques from wavelet-based event representations, adaptive event slicing, and graph-based retrieval. Prior approaches such as Adaptive Spatial-Temporal Window (CVPR 2026) and Wavelet-Based Feature Representation Framework for Event Stream Recognition (ICME 2025) already investigate adaptive or frequency-aware event representations. Similarly, Vgent (NeurIPS 2025) and Graph-to-Frame RAG (CVPR 2026) explore graph-structured retrieval for video reasoning.
  • The evidence graph is presented as preserving temporal, spatial, and frequency relationships. However, spatial edges are defined through CLIP embedding similarity rather than explicit spatial correspondence or geometric relationships.
  • The experimental evaluation is restricted to Qwen2.5-3B and Qwen2.5-7B. Both backbones belong to the same model family, and no experiments investigate larger or architecturally different MLLMs.
  • The method emphasizes efficiency and reports improved token-generation throughput. However, the full framework requires 40.2 seconds of offline preprocessing and 6.5 seconds of online processing per second of input event stream. These costs are substantial and are not adequately reflected by generated tokens per second. The paper does not sufficiently analyze end-to-end query latency, preprocessing amortization, memory footprint, or graph construction scalability.
  • The evaluation covers only two event-stream benchmarks, and the paper provides limited evidence regarding robustness under different motion speeds, lighting conditions, event densities, and sensor noise. The improvements on EventChat-Sub are relatively modest for some metrics. Additionally, Section 4.1 (page 6) refers to an unresolved "Appendix??" for dataset descriptions and split protocols, while the provided manuscript lacks sufficient details about the training or initialization of the event-language projector.
Questions for authors
  • Can the authors provide direct experimental evidence demonstrating that fixed event grouping causes temporal information loss and that independent retrieval causes spatiotemporal fragmentation?
  • Since CLIP similarity does not directly encode physical spatial correspondence and adaptive intervals may be temporally misaligned, could the authors evaluate edge correctness, temporal overlap, and retrieval relevance against ground-truth event relationships?
  • Could the authors evaluate larger and more diverse MLLMs, including alternative model families and stronger visual reasoning backbones?
Reviewer_jRLrOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TFE-GraphRAG, an event-stream interface that separates offline evidence construction from online question-conditioned retrieval. CWT Adaptive Slicing uses wavelet-energy changes to determine patch and frequency-specific temporal intervals and constructs frequency-specific frames. These frames form an evidence graph with temporal, cross-frequency, and visual-similarity edges. Experiments on EventMind and EventChat-Sub with 3B and 7B Qwen2.5 backbones report improvements over EventFlash and a dense event baseline, together with higher reported online generation throughput.

Critical strengths
  1. The offline or online decomposition is well motivated for answering multiple questions about the same event stream. Frequency-specific CAS frames provide an explicit intermediate representation, while bounded graph expansion offers a concrete mechanism for selecting related evidence rather than independently ranking every frame.
  2. The method reports improvements over EventFlash on the exact-match tasks at both model scales and the graph ablations also provide encouraging evidence that connected retrieval can outperform independent retrieval under the stated common memory and retrieval budget.
Critical weaknesses
  1. CAS independently determines temporal boundaries for each patch–frequency pair, then assembles blocks with the same ordinal index into a frame. However, the manuscript does not specify how variable-duration coefficient blocks, scale dimensions, and polarity channels are converted into the tensor consumed by CLIP. The implementation section also does not establish how the event encoder and two-layer projector are trained or initialized beyond the CLIP initialization; keeping the Qwen2.5 LLM frozen does not resolve the provenance of the multimodal alignment

  2. There is also an unresolved temporal-correspondence issue: Eq. 8 connects the same interval index across frequency groups, although those groups have independently selected boundaries. Equal indices therefore need not represent overlapping physical intervals. Patchwise temporal supports are stored, but their use in constructing these correspondences or informing the MLLM is not specified.

  3. Section 4.1 states that GDC and FGQA are evaluated using GPT-5.6 Luna. However, the EventFlash entries in Table 1 match those in the publicly available EventFlash paper, whose Section 5.1 specifies GPT-4o as the judge. It is unclear whether the baseline answers were rescored using the same judge and prompt as the proposed method.

  4. The online throughput improvement is useful, but Table 3 additionally reports 40.2 seconds of offline processing per second of input. An end-to-end and amortized comparison with the same-device dense baseline is needed to establish the workload under which the method offers a favorable accuracy–cost tradeoff. The offline cost is disclosed; the concern is the scope of the demonstrated efficiency advantage, not its omission.

Questions for authors
  1. Were the GDC/FGQA outputs of the Table 1 baselines, particularly EventFlash, evaluated using exactly the same judge version, scoring prompt, and test examples as TFE-GraphRAG?
  2. What is the exact tensor construction from the variable-duration CWT blocks to a CAS frame accepted by CLIP, including treatment of time, scale, and polarity dimensions?
  3. When frequency groups and spatial patches have different temporal boundaries, how should equal-index cross-frequency edges be interpreted?
  4. Please provide comments on weaknesses if possible.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.