- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces MM-Lifelong, a benchmark for multimodal understanding over a persistent subject’s history at Day, Week, and Month scales. It distinguishes recorded duration from physical temporal span, and includes 181.1 hours of video and 1,289 clue-grounded questions; the Month subset contains 105.6 hours of livestream footage spanning 51 days. The paper evaluates a broad range of MLLMs and video agents using both answer quality and temporal grounding, finding that current systems remain far from human performance.
Critical strengths
A valuable new evaluation setting
The distinction between recorded duration and physical temporal span is useful. The Month subset extends prior day-scale longitudinal benchmarks such as EgoLife and TeleEgo to a substantially longer, sparse history, where recurring events and changing contexts can matter.
Strong benchmark construction and evaluation
The paper provides manually annotated clue intervals and evaluates answer quality separately from temporal grounding. The comparison covers both end-to-end MLLMs and agentic systems, making the empirical picture more informative than answer accuracy alone.
Critical weaknesses
The central claim about unobserved intervals is not directly evaluated
The paper motivates the Lifelong Horizon by arguing that models must reason about events occurring during unobserved periods. However, the QA pairs are grounded in recorded clue intervals. The authors explicitly acknowledge in Appendix D(iii) that the benchmark contains no questions or annotations directly probing the content of an unobserved interval. It therefore evaluates connecting observed events across time, rather than the stronger gap-inference ability used to motivate the setting.
The distinct value of the Month scale is not established
Extending prior day-scale longitudinal benchmarks such as EgoLife and TeleEgo to the Month scale is a main contribution. However, temporal scale is fully coupled with domain and subject: Day uses gameplay, Week uses egocentric daily life, and Month uses livestreams. The paper intentionally introduces this content variation, but provides no matched comparison that separates temporal horizon from domain difficulty. Indeed, the paper notes that most MLLMs perform worse on the shorter Day subset than on Week. The results establish several challenging long-history settings, but do not show what unique value the Month horizon adds beyond broader data coverage.
Questions for authors
-
Could the authors add a small set of questions that directly probe reasoning about the content of unobserved intervals? This would provide direct evidence for the central capability used to motivate the Lifelong Horizon.
-
Since the Month-scale horizon is a central contribution beyond EgoLife and TeleEgo, could the authors provide an analysis of what substantive challenges or failure modes become more prominent at this scale? This would help establish the added value of Month-scale evaluation beyond expanding the benchmark’s coverage.