- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents a training-free framework for long-form egocentric video question answering. The proposed Dynamic Hierarchical Memory (DH-Mem) organizes temporally grounded substeps into higher-level procedural steps based on semantic similarity, allowing non-contiguous video segments to be grouped together. The Evidence Construction Skills (EC-Skills) component employs predefined procedures to adapt evidence retrieval, visual processing, tool use, and answer verification to different question types. Experiments on HD-EPIC, EgoSchema, and QAEgo4D demonstrate consistent improvements over frozen MLLM baselines.
Critical strengths
- One interesting insight of this paper is that some procedural steps consist of temporally non-contiguous segments that can be semantically grouped together.
- The paper is clearly presented, and the figures and tables effectively convey the core ideas.
Critical weaknesses
- I think the contributions are somewhat limited. The paper presents the overall pipeline as one contribution, while counting its two components as separate contributions. Moreover, although the paper considers temporally non-contiguous action segments, hierarchical representations of procedural steps are not a new concept, as seen in prior work such as the Ego4D Goal-Step dataset.
- The EC-Skills in this paper rely on predefined mappings from question types to inference procedures, rather than capabilities learned through RL or embedded in the model harness. This raises concerns about their generalizability, particularly to unseen question types or more open-ended scenarios.
- Some of the benchmarks used in the experiments seem somewhat outdated, particularly EgoSchema and QAEgo4D. The authors could extend the evaluation to more recent benchmarks, such as EgoLifeQA and MM-Lifelong. Additionally, several baselines in Table 1 are relatively old, including VideoLLaMA and LLaVA-Video, both from 2024. The specific version of Gemini used in the comparison is also not clearly specified.
- The necessity of incorporating specialized models such as SAM3 and Depth Anything 3 is not sufficiently justified. These tools introduce additional computational overhead and installation requirements, which may limit the practicality of the proposed framework. It would be helpful to evaluate whether the performance gains justify these additional costs.
Questions for authors
- How much of the performance improvement can be specifically attributed to grouping temporally non-contiguous segments into the same procedural step compared with continuous merging?