- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This manuscript proposes a benchmark called EgoR1-Bench1K and builds a baseline on this dataset.
Critical strengths
- The structure of this manuscript is easy to follow.
Critical weaknesses
- The differences and necessity of the introduced benchmark, EgoR1-Bench1K, are not adequately described. The paper does not clarify how EgoR1-Bench1K compares with other related datasets, nor does it explain what specifically distinguishes this benchmark from existing ones. A clearer comparison would help justify the need for a new benchmark.
- The effectiveness of the proposed Ego-R1 method is not sufficiently demonstrated. The comparisons are conducted only on the proposed benchmark. It is recommended that the authors compare Ego-R1 with other methods on existing datasets (e.g., Ego4D and Ego-Exo4D), and further include comparisons between Ego-R1 and other anticipation methods. Such experiments would provide stronger evidence for the method's effectiveness.
Questions for authors
See Weaknesses.
2 comments
Thanks for the clarification. However, W1 remains qualitative, i.e., no direct comparison with existing benchmarks is given. W2 is unresolved: all results are still on EgoR1-Bench1K only, with no Ego4D/Ego-Exo4D experiments and no comparison with specialist anticipation methods (Mirage/LVR are not such methods). Promised future comparisons do not provide current evidence.
Benchmark Contributions and Effectiveness of EgoR1
1. Differences and contributions of EgoR1-Bench1K.
EgoR1-Bench1K is designed to evaluate future interaction forecasting in MLLMs, with several key differences from existing benchmarks.
Compared with existing benchmarks, EgoPlan-Bench focuses on MCQ-based next-step prediction, Ego4D STA evaluates future active-object localization and interaction prediction through structured outputs, and Ego-Exo4D emphasizes skilled activities across first- and third-person perspectives. EgoR1-Bench1K provides a unified MLLM-compatible setting that combines open-vocabulary future interaction selection, challenging distractors, and target-object grounding.
The benchmark also uses source-video-level data splits and removes contact-frame leakage, near-duplicate observations, and ambiguous samples to support reliable evaluation. We will make these distinctions more explicit in the benchmark comparison.
2. Effectiveness and contributions of EgoR1.
EgoR1 introduces progressive interaction-state grounding, which differs from generic latent visual reasoning by explicitly modeling the transition from a predicted future interaction state to its target object.
Specifically, Future Interaction State Distillation uses future contact frames and paired interaction text to construct interaction-aware latent supervision. Object-Centric Latent Reasoning then conditions on the predicted state and uses region alignment and contrastive grounding to distinguish the target object from other plausible candidates.
The experimental results support this design:
EgoR1 improves over LVR by 5.85 percentage points in accuracy and 0.0260 mIoU. The component ablations further show that combining future interaction state distillation and object-centric reasoning achieves 0.8222 Acc. and 0.4023 mIoU before GRPO. GRPO then improves these results to 0.8485 Acc. and 0.4219 mIoU.
These results highlight the complementary roles of state-level prediction and object-level grounding, showing that the gains come from task-aligned latent supervision and structured reasoning rather than simply introducing additional latent tokens.
3. Relationship to specialist anticipation methods.
EgoR1 and specialist anticipation methods address related prediction problems through different modeling approaches. Traditional methods typically use dedicated prediction heads for verbs, nouns, bounding boxes, or temporal offsets. EgoR1 instead performs future interaction prediction and object grounding through a unified generative MLLM, using interaction-centric latent representations as intermediate reasoning states.
Our goal is not to replace specialist anticipation architectures, but to investigate how general-purpose MLLMs can acquire interaction forecasting capabilities through structured latent reasoning. We will further clarify this distinction and include comparisons with representative anticipation methods under compatible evaluation settings.
EgoR1-Bench1K provides a challenging evaluation setting for joint semantic and spatial forecasting, while EgoR1 demonstrates how interaction-centric latent reasoning can improve both capabilities within a unified MLLM framework.