- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces MRBench, which aligns recap-derived narrative queries with movie intervals, and EdgeMemory, a framework with persistent Plan, Event, and Observation nodes and progressively updated relations. The four-stage procedure constructs candidate groundings, diagnoses uncertain matches, hypothesizes relevant events, and verifies local visual evidence. The evaluated core contains 50 movies and 22,778 localized queries.
Critical strengths
The recap-to-movie alignment pipeline offers a practical route to a sizable narrative-query localization benchmark. The appendix distinguishes the 141-pair candidate pool from the 50-movie evaluation core, excludes non-localizable queries, and reports 95.54% correctness on 2,603 audited visual matches (Appendix B.1). This validation supports the usefulness of the benchmark resource.
EdgeMemory provides meaningful localization and efficiency improvements in several settings. With GPT-5.4, Acc@30 rises from 27.10% to 33.79%, and Acc@300 from 41.97% to 60.48%. The query-type breakdown shows larger gains for narrative/context-dependent queries. The five-movie Qwen audit includes construction costs and reports 66.81 versus 288.27 minutes per movie for the direct protocol (Table 6), supporting the practical relevance of structured memory.
Critical weaknesses
The evaluation does not sufficiently separate narrative evidence integration from chronological alignment. The controller exposes early incomplete Plans in recap order and uses neighboring matches as temporal anchors (Appendix B.7), making an order/progress baseline consequential. Localization and Beat/Sequence/Act membership are not direct measures of narrative understanding, as Appendix B.6 acknowledges. Since 88.39% of target intervals are shorter than 20 seconds, coarse tolerances also need careful interpretation. These points limit the central claim about long-range reasoning, rather than the usefulness of the benchmark itself.
The evidence supports benefits that depend on query type and backbone, with insufficient characterization of variation across movies. For Gemini, Act alignment falls from 42.45% to 41.12%, and local-query Acc@30 falls from 37.73% to 19.00%, despite narrative-query gains (Tables 1 and 4). Movie-level uncertainty and clarification of interval scoring are important for assessing whether the reported aggregate improvements reliably reflect better grounding.
Questions for authors
How does the method compare with a simple chronological alignment baseline using recap progress and established anchors, or a text-only prior? A limited control would help establish how much of the gain depends on visual narrative evidence.
For interval-localizable queries, how are valid alternative intervals represented in scoring, given that Equation 2 compares with one interval center? Can predictions inside a valid but broad interval be incorrectly penalized by a small tolerance?
Can the main claims be reconciled with the Gemini regressions, and can movie-level uncertainty be reported for the core localization improvements? Please clarify whether query averaging gives longer recaps more weight.
Response To Reviewer vNjP
We thank the reviewer for the constructive feedback. We added order-only baselines, audited center-based scoring, and evaluated movie-level robustness. We respond below.
(W1, Q1) On Temporal-Order Priors and Narrative Reasoning
Query order is part of MRBench's input, not privileged supervision for EdgeMemory: the Direct MLLM baseline can also access the ordered recap queries during window scanning. EdgeMemory uses query order and established temporal anchors to narrow candidate searches and maintain temporal consistency, not as a substitute for narrative reasoning.
To quantify this prior, we added two baselines using neither video frames nor query semantics. Order-only (linear) maps relative recap position linearly to movie time. Order-only (learned monotone) learns a non-decreasing order-to-time mapping under Leave-One-Movie-Out, maximizing training Acc@30 on the other 49 movies and applying it to the held-out movie.
The learned baseline achieves only 4.23% Acc@30, versus 27.10% for GPT-5.4 and 33.79% for EdgeMemory; even its Acc@300 is only 33.05%. Thus, recap progression alone cannot account for accurate grounding. Semantic correspondence to movie evidence remains necessary; EdgeMemory further integrates that evidence through event-level representations and cross-event links. We will include these controls and clarify baseline access to ordering information.
(W1, Q2) On Wide Ground-Truth Intervals and Center-Based Scoring
We acknowledge that center-based scoring (Eq. 2) could penalize valid predictions within wide ground-truth intervals. However, for interval [a,b], if b−a≤2δ, any prediction inside it necessarily passes the center-distance criterion. Thus, such false negatives are possible only for intervals longer than 2δ.
As reported in Appendix B.3, 88.39%, 99.10%, and 99.91% of intervals are shorter than 20, 60, and 120 seconds, respectively. Consequently, at most 11.61%, 0.90%, and 0.09% of queries could be affected at Acc@10/30/60. For the primary Acc@30 metric, the theoretical upper bound is 0.90%. In an additional audit of GPT-5.4, 559 predictions fell inside their annotated intervals; none was incorrectly penalized by center-based scoring (0% observed among these predictions).
MRBench assigns one human-verified continuous interval per query. During construction, we retained recap descriptions with reliable event/segment correspondence and excluded highly ambiguous or ungroundable queries. We will clarify these assumptions and limitations. While the concern is theoretically valid, our interval statistics and audit indicate limited impact on the reported results.
(W2, Q3) On Gemini Performance Degradation, Backbone Dependence, and Movie-Level Robustness
We agree that EdgeMemory does not improve every model, query type, or metric. With Gemini 3 Pro, Local Acc@30 falls from 37.73% to 19.00%, and Act alignment from 42.45% to 41.12%. Additional structure and multi-stage constraints can disrupt straightforward local matching. However, Narrative/Context-dependent Acc@30 rises from 21.60% to 33.61% (+12.01 points), lifting overall Acc@30 from 24.91% to 30.61% (+5.70 points). We will qualify our claims: benefits are backbone- and query-type-dependent, strongest for context-dependent grounding.
Because query-level micro averages weight movies by query count, we additionally computed movie-level macro accuracy, averaging per-movie scores equally across all 50 movies:
EdgeMemory improves at every threshold; macro Acc@30 increases by 6.22 points. In paired per-movie comparisons, it achieves strictly higher Acc@30 on 37/50 movies (74%), showing that gains are widespread rather than driven by a few outliers, though not universal. We will add these analyses and discuss the observed limitations in the revision.
We will incorporate these clarifications and further improve the corresponding descriptions in the final revision, with all revised text highlighted in blue for ease of reference.