- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes TRACE, a training-free framework for streaming long-video understanding built around an SAO-centric memory mechanism. Unlike visual-feature-centric memory approaches, TRACE primarily organizes historical observations through text-only Subject-Action-Object (SAO) triples and entity-centric temporal trajectories. Based on these textual semantic representations, Semantic Surprise-Guided Event Filtering (SSEF) selectively retains key-frame or full-segment visual features, while an Intention-Driven Resource-Aware Router (IRR) retrieves historical evidence under resource constraints.
Critical strengths
The paper aims to address an important challenge in streaming long-video understanding: preserving historical evidence and temporal relationships under resource constraints. The proposed SAO-centric memory provides an intuitive and interpretable way to organize historical events. Its training-free design may be appealing in settings with limited training resources.
Critical weaknesses
W1. Fundamental limitations of SAO-centric memory for general video understanding. My primary concern is whether representing historical observations primarily through text-only, coarse-grained SAO triples (e.g., "man2, look at, man1") is sufficiently expressive for general video understanding. Such representations may omit substantial fine-grained perceptual information, including object attributes, scene text, spatial relationships, and subtle state changes. Although TRACE selectively retains visual features, these decisions rely on SAO-based semantic similarity and surprise. Semantic redundancy does not necessarily imply visual redundancy, and discarded visual evidence cannot be recovered for future queries. Even retained visual features may be difficult to retrieve when relevant attributes are absent from the semantic index.
Moreover, it remains unclear how such a restricted representation supports complex cross-temporal state reasoning, entity identity consistency, and evolving interactions among multiple objects. These are not exceptional cases but common requirements across practical streaming scenarios and established video understanding benchmarks. Consequently, the proposed memory appears better suited to a relatively narrow subset of coarse-grained event retrieval tasks than to the broad video understanding setting claimed in the paper. The current experiments do not adequately establish its general applicability to either offline long-video understanding or streaming video understanding.
W2. Unquantified memory costs and missing modality-specific ablations. Although TRACE claims to achieve resource-aware memory retention by selectively preserving visual features alongside SAO-based textual memory, the actual memory footprint and relative contributions of textual and visual evidence remain unclear. SSEF determines merge/key/full retention using fixed semantic thresholds, yet the proportions of retained key-frame and full-segment features, their storage costs, and memory growth with video duration are not reported. More importantly, the cumulative ablations cannot establish whether the improvements primarily originate from the SAO-based textual memory, the retained visual features, or their combination.
Questions for authors
My questions arise mainly from the methodology. It would be helpful for my understanding of this work if the authors could clarify or address them.
Q1. How would TRACE answer common visual understanding questions such as "What color was the cup previously on the table?" or "What was written on the black car that passed earlier?" Such questions require fine-grained visual evidence that may not be represented in SAO triples. How can SSEF ensure that this evidence is retained, and how can IRR retrieve it if the relevant attributes are absent from the semantic index? Would the proposed memory mechanism fail in such cases?
Q2. How does EETC preserve evolving object states, spatial relationships, and complex interactions across distant timestamps? How does it distinguish visually similar entities or repeated but temporally separate events, particularly as the number of entities increases?
Q3. How necessary and effective is SAO-centric memory? What advantages does the proposed SAO-centric memory offer over the richer text-structured event memory (e.g., OASIS, cvpr2026)? Could the authors compare SAO-only memory, SAO memory with retained visual features, and representative alternative memory structures under matched storage budgets? Such experiments are essential to isolate the contributions of textual and visual evidence and justify the restricted SAO representation.
Q4. What are the end-to-end computational and latency costs of SAO extraction, semantic prediction, and entity-trajectory updates? For example, how much wall-clock time is required to construct the proposed memory for a video longer than three minutes? Does the reported processing throughput include these operations, and how do the costs scale with video duration?