- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents HoliWorld, an integrated framework for automatically constructing structured 3D/4D annotations from egocentric videos. By combining static scene reconstruction, dynamic object tracking, and interaction understanding, the authors construct a dataset for spatial-temporal reasoning and demonstrate its effectiveness through downstream VLM training.
Critical strengths
- The paper presents a comprehensive and practically valuable framework for automatically constructing structured spatial-temporal annotations, integrating static geometry, dynamic object states, and interaction semantics.
- The experiments are extensive and solid, demonstrating substantial improvements in annotation quality across multiple benchmarks and validating the effectiveness of the generated data for downstream spatial-temporal reasoning.
Critical weaknesses
- Limited novelty. The overall data construction pipeline can be viewed as an incremental extension of Holi-Spatial, primarily incorporating existing dynamic tracking techniques and temporal supervision. The methodological novelty beyond integrating existing components is relatively limited.
- Limited validation of world modeling capabilities. Despite the paper's emphasis on holistic world intelligence, the downstream evaluation primarily focuses on VLM-based spatial-temporal reasoning. The potential value of the constructed data for world modeling, particularly future state prediction or dynamics understanding, remains unexplored.
Questions for authors
Could the authors provide additional experiments demonstrating the utility of HoliWorld-Data for world modeling tasks, such as future state prediction or action-conditioned dynamics prediction? Such evaluations would help substantiate the broader claim of holistic world intelligence beyond VLM-based spatial-temporal reasoning.
2 comments
Thank you for the clarifications. I acknowledge the practical value of the proposed framework and the solid experimental results. However, my main concern regarding the limited methodological novelty relative to Holi-Spatial remains.
For now, I will maintain my current rating and revisit it during the Reviewer–AC discussion period.
Response to Reviewer zPdi
We sincerely thank the reviewer for recognizing HoliWorld’s practical value and the strength of our experimental evaluation. Below, we address the comments on methodological novelty and the potential utility of HoliWorld-Data for world modeling.
W1: Creativity Compared to Holi-Spatial. For static perception, HoliWorld incorporates updated perception components and, more importantly, a more effective VLM-guided verification and object consolidation mechanism, aiming to improve annotation reliability in complex egocentric environments.
More substantially, HoliWorld introduces dynamic 4D perception, extending beyond static scene reconstruction to recover time-varying object states, persistent identities, 3D trajectories, and interaction semantics. These extensions together enable the construction of unified spatial-temporal world records and corresponding QA supervision, which are beyond the scope of the original static-focused Holi-Spatial framework.
Q1 / W2: Scope of Current Evaluation Focused on Spatial Intelligence. Thank you for this valuable suggestion. We agree that future state prediction and action-conditioned dynamics modeling represent promising applications of HoliWorld-Data.
Our current evaluation focuses primarily on spatial-temporal reasoning, as the central goal of this work is to establish a scalable pipeline for grounding spatial-temporal understanding in real egocentric experiences and constructing high-quality, structured 3D/4D supervision. Rather than directly developing a predictive world model, we focus on building the reliable world representations and large-scale supervision that could serve as a foundation for broader world intelligence tasks.
Nevertheless, HoliWorld-Data contains rich spatial-temporal and interaction information that could be further exploited for learning object dynamics and future state prediction. We appreciate the reviewer's suggestion and are particularly interested in exploring this promising extension in future work.