acceptodds
Withdrawn from ICLR 2027

Long-Horizon Movie Reasoning: A Scalable Benchmark and Structured Memory Framework

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Long-form video understanding requires models to reason consistently over extended temporal contexts, yet existing benchmarks provide limited evaluation of long-range memory and high-level narrative reasoning. We introduce a semi-automated pipeline for constructing long-form video reasoning benchmarks from full-length movies and their recap narrations, reducing annotation cost while preserving reliable temporal and semantic alignment. Based on this pipeline, we construct Movie Reason Benchmark (MRBench), which evaluates holistic reasoning over long videos with narrative semantics at multiple levels of abstraction. We further introduce complementary metrics for temporal localization under multiple tolerance thresholds and semantic alignment across narrative granularities. To address the challenges exposed by MRBench, we propose EdgeMemory, an edge-centric graph-structured memory framework that decomposes complex reasoning tasks, organizes multi-level semantic representations, and progressively refines their relations through a dedicated reasoning pipeline. Extensive experiments show that EdgeMemory consistently improves long-form video reasoning across diverse models. Further analysis reveals distinct capability boundaries and failure modes across model scales, highlighting persistent challenges in long-range evidence integration and abstract narrative reasoning.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Long-Form Video Understanding, Long-Horizon Reasoning, Structured Memory

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vNjPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces MRBench, which aligns recap-derived narrative queries with movie intervals, and EdgeMemory, a framework with persistent Plan, Event, and Observation nodes and progressively updated relations. The four-stage procedure constructs candidate groundings, diagnoses uncertain matches, hypothesizes relevant events, and verifies local visual evidence. The evaluated core contains 50 movies and 22,778 localized queries.

Critical strengths

The recap-to-movie alignment pipeline offers a practical route to a sizable narrative-query localization benchmark. The appendix distinguishes the 141-pair candidate pool from the 50-movie evaluation core, excludes non-localizable queries, and reports 95.54% correctness on 2,603 audited visual matches (Appendix B.1). This validation supports the usefulness of the benchmark resource.

EdgeMemory provides meaningful localization and efficiency improvements in several settings. With GPT-5.4, Acc@30 rises from 27.10% to 33.79%, and Acc@300 from 41.97% to 60.48%. The query-type breakdown shows larger gains for narrative/context-dependent queries. The five-movie Qwen audit includes construction costs and reports 66.81 versus 288.27 minutes per movie for the direct protocol (Table 6), supporting the practical relevance of structured memory.

Critical weaknesses

The evaluation does not sufficiently separate narrative evidence integration from chronological alignment. The controller exposes early incomplete Plans in recap order and uses neighboring matches as temporal anchors (Appendix B.7), making an order/progress baseline consequential. Localization and Beat/Sequence/Act membership are not direct measures of narrative understanding, as Appendix B.6 acknowledges. Since 88.39% of target intervals are shorter than 20 seconds, coarse tolerances also need careful interpretation. These points limit the central claim about long-range reasoning, rather than the usefulness of the benchmark itself.

The evidence supports benefits that depend on query type and backbone, with insufficient characterization of variation across movies. For Gemini, Act alignment falls from 42.45% to 41.12%, and local-query Acc@30 falls from 37.73% to 19.00%, despite narrative-query gains (Tables 1 and 4). Movie-level uncertainty and clarification of interval scoring are important for assessing whether the reported aggregate improvements reliably reflect better grounding.

Questions for authors

How does the method compare with a simple chronological alignment baseline using recap progress and established anchors, or a text-only prior? A limited control would help establish how much of the gain depends on visual narrative evidence.

For interval-localizable queries, how are valid alternative intervals represented in scoring, given that Equation 2 compares with one interval center? Can predictions inside a valid but broad interval be incorrectly penalized by a small tolerance?

Can the main claims be reconciled with the Gemini regressions, and can movie-level uncertainty be reported for the core localization improvements? Please clarify whether query averaging gives longer recaps more weight.

1 comment
AuthorsOpenReview ↗

Response To Reviewer vNjP

We thank the reviewer for the constructive feedback. We added order-only baselines, audited center-based scoring, and evaluated movie-level robustness. We respond below.

(W1, Q1) On Temporal-Order Priors and Narrative Reasoning

Query order is part of MRBench's input, not privileged supervision for EdgeMemory: the Direct MLLM baseline can also access the ordered recap queries during window scanning. EdgeMemory uses query order and established temporal anchors to narrow candidate searches and maintain temporal consistency, not as a substitute for narrative reasoning.

To quantify this prior, we added two baselines using neither video frames nor query semantics. Order-only (linear) maps relative recap position linearly to movie time. Order-only (learned monotone) learns a non-decreasing order-to-time mapping under Leave-One-Movie-Out, maximizing training Acc@30 on the other 49 movies and applying it to the held-out movie.

MethodAcc@10Acc@30Acc@60Acc@120Acc@180Acc@300
Order-only (linear)1.123.436.7912.5717.3626.88
Order-only (learned monotone)1.464.237.5614.0820.5133.05

The learned baseline achieves only 4.23% Acc@30, versus 27.10% for GPT-5.4 and 33.79% for EdgeMemory; even its Acc@300 is only 33.05%. Thus, recap progression alone cannot account for accurate grounding. Semantic correspondence to movie evidence remains necessary; EdgeMemory further integrates that evidence through event-level representations and cross-event links. We will include these controls and clarify baseline access to ordering information.

(W1, Q2) On Wide Ground-Truth Intervals and Center-Based Scoring

We acknowledge that center-based scoring (Eq. 2) could penalize valid predictions within wide ground-truth intervals. However, for interval [a,b][a,b], if b−a≤2δb-a\leq 2\delta, any prediction inside it necessarily passes the center-distance criterion. Thus, such false negatives are possible only for intervals longer than 2δ2\delta.

As reported in Appendix B.3, 88.39%, 99.10%, and 99.91% of intervals are shorter than 20, 60, and 120 seconds, respectively. Consequently, at most 11.61%, 0.90%, and 0.09% of queries could be affected at Acc@10/30/60. For the primary Acc@30 metric, the theoretical upper bound is 0.90%. In an additional audit of GPT-5.4, 559 predictions fell inside their annotated intervals; none was incorrectly penalized by center-based scoring (0% observed among these predictions).

MRBench assigns one human-verified continuous interval per query. During construction, we retained recap descriptions with reliable event/segment correspondence and excluded highly ambiguous or ungroundable queries. We will clarify these assumptions and limitations. While the concern is theoretically valid, our interval statistics and audit indicate limited impact on the reported results.

(W2, Q3) On Gemini Performance Degradation, Backbone Dependence, and Movie-Level Robustness

We agree that EdgeMemory does not improve every model, query type, or metric. With Gemini 3 Pro, Local Acc@30 falls from 37.73% to 19.00%, and Act alignment from 42.45% to 41.12%. Additional structure and multi-stage constraints can disrupt straightforward local matching. However, Narrative/Context-dependent Acc@30 rises from 21.60% to 33.61% (+12.01 points), lifting overall Acc@30 from 24.91% to 30.61% (+5.70 points). We will qualify our claims: benefits are backbone- and query-type-dependent, strongest for context-dependent grounding.

Because query-level micro averages weight movies by query count, we additionally computed movie-level macro accuracy, averaging per-movie scores equally across all 50 movies:

MethodAcc@10Acc@30Acc@60Acc@120Acc@180Acc@300
Vanilla (GPT-5.4)16.2426.9633.3338.0740.0642.24
EdgeMemory (GPT-5.4)18.7433.1843.3352.0255.7759.99
Improvement (points)+2.50+6.22+10.00+13.95+15.71+17.75

EdgeMemory improves at every threshold; macro Acc@30 increases by 6.22 points. In paired per-movie comparisons, it achieves strictly higher Acc@30 on 37/50 movies (74%), showing that gains are widespread rather than driven by a few outliers, though not universal. We will add these analyses and discuss the observed limitations in the revision.

We will incorporate these clarifications and further improve the corresponding descriptions in the final revision, with all revised text highlighted in blue for ease of reference.

Reviewer_u8MxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes MRBench, a benchmark for localizing movie-recap narration inside full-length movies, and EdgeMemory, a graph-structured memory method for this task. Recap narration is transcribed with Whisper, and each narration segment is grounded in the source movie by DINOv3 sequence matching between the recap clip and the movie frames. From a pool of 141 movie–recap pairs, 50 movies with 22,778 localizable queries form the evaluation set. Metrics are Acc@δ (predicted time within δ of the center of the ground-truth interval, δ from 10 to 300 s) and Beat/Sequence/Act alignment against a narrative hierarchy generated by GPT-5.4. EdgeMemory splits the recap into Plans and the movie into Events and Observations, and updates the edges between them in four stages (Construct, Diagnose, Hypothesis, Verify). It is tested with GPT-5.4, Gemini 3 Pro and Qwen3-VL-8B against the base models, WorldMM and One-CLIP RAG (Table 1), with stage and node ablations (Tables 2–3), a local/narrative query split (Table 4), a cost comparison (Table 6) and a transfer test on EgoMemReason (Table 7).

Critical strengths
  1. A practical way to get dense temporal labels at movie scale. Recap videos give both the narration and footage cut from the movie, so labels come from visual matching instead of people watching whole movies. The audit in B.1 finds 2,487 of 2,603 checked matches correct (95.54%, Wilson CI 94.68–96.27%; I recomputed this interval and it is correct). The task is far from solved: GPT-5.4 alone gets 16.43% at 10 s and 41.97% at 300 s (Table 1). This is the main reason the benchmark could be useful. The release plan is also careful (annotations, metadata, code and source identifiers, no video).

  2. Large gains with GPT-5.4, and lower cost with Qwen3-VL-8B. With GPT-5.4, EdgeMemory improves Acc@300 from 41.97 to 60.48 and Act alignment from 45.46 to 67.51 (Table 1). In the five-movie cost audit with Qwen3-VL-8B (Table 6), it uses 41.0 multimodal calls and 66.81 min per movie, against 256.6 calls and 288.27 min for direct inference and 828.34 min for WorldMM. B.7 describes the controller rules in useful detail (but no prompts are given, although the reproducibility statement says they are in the appendix). If the concerns below are resolved, these results would support the method.

Critical weaknesses

W1. It is not clear that MRBench measures narrative reasoning, which is the central claim. (a) Burned-in subtitles, and what the ground truth means. 7 of the 14 frames in Fig. 7 carry Chinese/English dialogue subtitles. In the Temporal Reasoning Error row, the query is "It must have slipped down from your neck.", the predicted frame shows the subtitle "Must've slipped right off your neck.", and GPT's reason is "The caption of this frame corresponds directly to the line." This prediction is counted as an error (the model "selects a similar event from the wrong time segment"), although the subtitle of the predicted frame says the same thing. So (i) dialogue text is visible to the models, and GPT's rationale suggests it uses it, which the design wants to avoid ("rather than surface-level dialogue matching", A.5), and (ii) the ground truth is the source of the recap's footage, which may differ from where the narrated line is spoken in the movie. (b) The task includes "cross-query dependency" (Sec. 3.1), and EdgeMemory uses query order in several places: the active pool keeps "the earliest incomplete Plans in narrative order", Diagnose checks whether a match is "compatible with the ordering of neighboring Plans" (A.4.2), Hypothesis ranks Events partly by "temporal proximity to existing anchors", and Verify enforces "within-Plan temporal order" (B.7). Using order is legitimate, but if recaps mostly follow the plot order (the paper does not report this), order alone may explain much of the score. There is no order-only baseline, so we cannot tell how much of the gain is narrative reasoning. (c) The Beat/Sequence/Act units are generated by GPT-5.4 (B.6). They are fixed before model evaluation and shared by all methods, but GPT-5.4 is also the best backbone, so its own segmentation may favour it. B.6 itself says this metric "should not be interpreted as a direct measure of narrative understanding". Still, Sec. 5.3 and the conclusion use the Sequence/Act gains as evidence of better high-level reasoning. Chance levels are not given. (d) Sec. 3.2 says the candidates are manually verified and corrected (the Fig. 2 caption: "manually verified"), but A.5 says "we performed sampled manual audits", and B.1 reports a match audit of 2,603 pairs, about 11% of the 22,778 queries. All segments were categorized for localizability (B.1), but it is not clear whether the other matched intervals were checked.

W2. The results do not support the "consistent" claims, and the setup is not described well enough to check them.

  • Sec. 5.2 says EdgeMemory "consistently achieves the best results", and Sec. 5.3 and the conclusion repeat that it consistently improves. In Table 1, EdgeMemory (Gemini3 Pro) is below Gemini3 Pro on Act (41.12 vs 42.45), and EdgeMemory (Qwen3-VL-8B) is below WorldMM (Qwen3-VL-8B) at 10 s (4.27 vs 5.92). In Table 4, EdgeMemory (Gemini 3 Pro) is below the Gemini base on local queries at every threshold (e.g., 11.66 vs 22.92 at 10 s), and it is the only row whose local accuracy is below its narrative accuracy at all six thresholds. B.2 admits that EdgeMemory "does not uniformly outperform all alternatives on local queries", but the main text does not.
  • The protocol for the base GPT-5.4 and Gemini rows and for WorldMM (GPT-5.4) is missing: frame rate, number of frames and chunking (A.2 says at most 50 frames per call, while the Table 1 footnote marks only Qwen as needing chunking), how WorldMM, a QA agent, outputs a timestamp, and whether these baselines get the queries in order. B.7 gives these settings for EdgeMemory, and B.4 gives costs for the Qwen methods only, in the five-movie audit. There are no repeated runs or variance, and EdgeMemory runs at temperature 0.2.
  • The ablations (Tables 2–3) use "a subset of MRBench" with no backbone given. The full model's Acc@30 of 38.19 does not match any row of Table 1, and Err@30 and MatchRate are not defined in the text (Sec. 5.4 only names "accuracy, error rate, and coverage").
  • Several numbers do not add up, and the Table 7 baseline rows are identical to published EgoMemReason results that are not cited (Questions 4–7).
Questions for authors

Answers to 1–7 could change my score; 8–9 are clarifications.

  1. How many of the 50 movies have burned-in subtitles, and were they masked before the frames were given to the models? Please report results with the subtitle area masked (or on subtitle-free movies). Please also give the language of the recap narrations (English, Chinese, and Japanese movies are mentioned in A.5) and say whether queries were translated.
  2. What is the rank correlation between recap order and ground-truth movie order? Please add an order-only baseline (e.g., map each query's recap time linearly to the movie timeline, or a monotonic alignment) under Acc@δ and Beat/Sequence/Act, and give the chance level of each level.
  3. For the base GPT-5.4 and Gemini rows and WorldMM (GPT-5.4) in Table 1: frame rate, total frames, number of calls, whether all queries are given at once and in order, and how WorldMM and One-CLIP RAG produce a timestamp. Please also give call and frame totals for the 50-movie runs (Table 6 covers five movies with Qwen3-VL-8B only).
  4. Tables 2–3: which backbone and how many movies? Please define Err@30 and MatchRate. If Err@30 = 1 − Acc@30/MatchRate, five of the six rows fit within 0.01 (reading the "–" MatchRate of "+ Hypothesis" as 100%, since with Hypothesis but no Verify each unresolved Plan takes the midpoint of a hypothesized Event; p. 8 and B.7). The "w/o Diagnose" row does not: it gives 61.49, not 62.50, so one of its three values seems wrong (MatchRate 90.75 or Acc@30 33.14 would fit).
  5. Table 4: Table 1 equals 0.2053 × Local + 0.7947 × Narrative in every row and threshold (to within 0.005), so the local split has about 4.7k queries and one query changes Local accuracy by about 0.021 points. Yet several Local columns rise in steps of exactly 0.01, e.g., WorldMM (Qwen3-VL-8B) 16.46/16.47/16.48/16.49 at 30–180 s and Base (Qwen3-VL-8B) 23.74/23.75/23.76. In fact, no local-split size between 4,000 and 5,500 gives all Local values of Table 4 as a per-query average (Eq. (2)). How were these numbers computed? Per-query predictions would settle this.
  6. Table 7: the Qwen3-VL-8B and WorldMM rows are identical to Table 1 of EgoMemReason (Wang et al., arXiv:2605.09874), which is not cited. Were these baselines re-run? Which backbone and frame budget does EdgeMemory use there? In that table Qwen-3-VL-32B reaches 36.8 and Gemini-3-Flash 39.6, which puts EdgeMemory's 36.9 in context.
  7. Sec. 3.3 gives 44 to 260 minutes and 471.60 queries per movie for the verified movies, but 471.60 = 66,496/141, the candidate-pool mean (Fig. 2b, A.5). The 50 evaluated movies have 24,862/50 = 497.24 segments and 22,778/50 = 455.56 queries per movie, and the Fig. 5 bins span 86.4–233.3 min (if Fig. 5 covers all 50 movies). Please give the statistics of the evaluated set. Also, in B.1, 15,971 + 6,907 + 2,084 = 24,962, not 24,862.
  8. Sec. 5.5 says Act and Sequence accuracy declines as movie duration increases while Beat stays stable, and that fine-grained localization remains comparatively stable. In Fig. 5a, from the shortest to the longest bin, Act is 69/71/64/66/67 (no monotone decline), Sequence 53/57/46/51/44 and Beat 21/20/16/18/14, so Beat loses 33% of its value, more than Sequence (17%). In Fig. 5b, <10 s goes from 23 to 14 (−39%) and <300 s from 64 to 52 (−19%). So these statements hold only partly in absolute points and not in relative terms. Which method is shown, and how many movies are in each bin?
  9. How does MRBench differ from MAD (Soldan et al., CVPR 2022), which grounds over 384,000 sentences in over 1,200 hours of movies; the Temporal Narration Grounding task of Movie101 (Yue et al., ACL 2023); and M-SyMoN (Sun et al., EMNLP 2024 Findings), which uses movie summary videos with manual annotation of fine-grained video-text correspondences? None of them is discussed; Sec. 2.1 describes long-video benchmarks with question answering as the primary evaluation paradigm.
3 comments
AuthorsOpenReview ↗

Response To Reviewer u8Mx(1/3)

We thank the reviewer for the thorough assessment and constructive feedback. We address each concern below, supported by additional experiments and clarifications.

(W1a, Q1) On Burned-in Subtitles, Ground-Truth Semantics, and Language Settings

Retaining dialogue subtitles embedded in movie frames was an intentional setting, not inadvertent data leakage. Movie understanding involves dialogue and narrative context in addition to visual content; subtitles provide linguistic information analogous to the dialogue heard by human viewers. Avoiding surface-level matching (Appendix A.5) means that grounding should not reduce to matching isolated frame descriptions with query text; it does not imply excluding dialogue evidence.

We nevertheless acknowledge that visible subtitles may provide direct lexical cues. We therefore added an evaluation using subtitle-free video sources, rather than artificially masking image regions:

MethodAcc@10Acc@30Acc@60Acc@120Acc@180Acc@300BeatSequenceAct
GPT-5.416.2326.9433.3138.0440.0342.2114.5736.9345.93
EdgeMemory (GPT-5.4)18.7533.1943.3452.0255.7759.9917.6450.8168.62

Relative to the original setting, GPT-5.4's Acc@30 changes from 27.10% to 26.94%, and EdgeMemory's from 33.79% to 33.19%. Without subtitles, EdgeMemory still achieves 50.81% Sequence and 68.62% Act alignment, retaining its advantage over the backbone. Thus, the observed gains do not primarily depend on visible dialogue subtitles or direct text matching.

MRBench includes English, Chinese, and Japanese movies. We will explicitly report the recap-narration language distribution.

(W1b, Q2) On Temporal-Order Priors and Narrative Reasoning

We agree that temporal ordering introduces a prior, but query order is part of the MRBench input, not privileged supervision unique to EdgeMemory. Recap queries follow their original narrative order, and Direct MLLM baselines can also access the ordered list during window scanning. EdgeMemory uses this order and established temporal anchors to narrow candidate search and preserve consistency over long movies with hundreds of queries; we do not equate ordering itself with narrative reasoning. We will clarify baseline-specific access to ordering information.

To measure how much performance order alone explains, we added two baselines using neither video frames nor query semantics. Order-only (linear) maps each query's relative recap position linearly onto the movie timeline. Order-only (learned monotone) learns a non-decreasing mapping from relative query position to relative movie time under Leave-One-Movie-Out evaluation: it optimizes training Acc@30 using the other 49 movies and then predicts timestamps for the held-out movie.

MethodAcc@10Acc@30Acc@60Acc@120Acc@180Acc@300
Order-only (linear)1.123.436.7912.5717.3626.88
Order-only (learned monotone)1.464.237.5614.0820.5133.05

Although learned monotone mapping captures typical cross-movie narrative progression, it achieves only 4.23% Acc@30, versus 27.10% for GPT-5.4 and 33.79% for EdgeMemory; even its Acc@300 is only 33.05%. These results show that relative recap progress and monotonic ordering cannot account for the grounding performance. Accurate localization requires semantic correspondence to concrete movie evidence. EdgeMemory additionally integrates context through event-level representations and cross-event relationships.

AuthorsOpenReview ↗

Response To Reviewer u8Mx(2/3)

(W1c, W1d) On Narrative Hierarchies and Human Verification of Ground Truth

The Beat–Sequence–Act hierarchy is not arbitrary: it is inspired by Story: Substance, Structure, Style, and the Principles of Screenwriting and adapted for long-video semantic grounding. We have added this reference. As explained in Appendix B.6, GPT-5.4 generates the initial hierarchy offline, but annotations are subsequently assessed through automated structural checks, human audits, and repeated-generation stability analysis. These checks examine temporal-boundary plausibility, semantic coherence within narrative units, and cross-level containment. We will elaborate on the validation protocol.

Crucially, Beat–Sequence–Act annotations are not MRBench's primary ground truth. They support supplementary structural alignment analysis; the primary timestamp-based Acc@δ metrics do not use model-generated hierarchical boundaries. Thus, the main temporal-grounding results are not directly determined by narrative segmentation. We agree that membership within a narrative unit should not be equated with narrative understanding.

We also apologize for ambiguity concerning human verification. The 2,603 matched pairs in Appendix B.1 constitute a separate sampled audit of automatic DINOv3 visual matching, not the total number of benchmark samples verified by humans. In the full validation workflow, reviewers assess recap sentences alongside candidate movie segments for semantic and temporal correspondence and flag cases requiring correction. All 50 evaluation movies underwent this process, with at least two reviewers per movie. We will clearly distinguish the DINOv3 audit from this broader verification procedure.

(W2, Q3) On the Scope of Performance Improvements and Baseline Protocols

We appreciate the reviewer's observation that “consistently achieves the best results” overstates our findings. We intended to describe EdgeMemory's aggregate advantages, not claim superiority on every backbone, query type, or metric. We have revised the wording and will explicitly acknowledge non-uniform performance.

Indeed, Appendix B.2 (Table 4) reveals notable query-type dependence. On Gemini 3 Pro, EdgeMemory improves Narrative/Context-dependent Acc@30 from 21.60% to 33.61%, whereas Local Acc@30 falls from 37.73% to 19.00%. EdgeMemory targets long-range dependencies and abstract narrative grounding; its event-level structure and multi-stage constraints may be unnecessary or even disruptive for queries already solvable through direct local matching. Thus, the evidence supports backbone- and query-type-dependent improvements, with the clearest gains for context-dependent queries.

We also agree that the baseline protocols require clearer documentation. The revision will standardize and expand the reproduction and evaluation details for GPT-5.4, Gemini 3 Pro, Qwen3-VL-8B, WorldMM, and One-CLIP RAG, including the relevant implementation and evaluation settings.

AuthorsOpenReview ↗

Response To Reviewer u8Mx(3/3)

(Q4, Q5, Q7) On Ablation Results, Metric Definitions, and Dataset Statistics

 We thank the reviewer for identifying numerical inconsistencies. Our audit of experimental records and evaluation scripts revealed issues in dataset scope and result aggregation, which have now been corrected. Tables 2–3 use GPT-5.4 as the ablation backbone. Their earlier statistics inadvertently included five additional movies, making them inconsistent with the main benchmark; we recomputed all ablations on the same 50-movie evaluation set.  We will also define the metrics explicitly. Let NN be the total number of queries, MM the number producing a valid match, and CC the correctly localized queries among them: 

Acc@30=CN,MatchRate=MN,Err@30=M−CM.\mathrm{Acc@30}=\frac{C}{N},\qquad \mathrm{MatchRate}=\frac{M}{N},\qquad \mathrm{Err@30}=\frac{M-C}{M}.

 Thus, Acc@30 measures correctness across all queries, whereas Err@30 measures the error rate conditional on a valid match. The former w/o Diagnose discrepancy also resulted from the inconsistent evaluation scope and aggregation and has been corrected. For Table 4, a confusion between strict (<) and non-strict (≤) inequality operators in the evaluation loop caused deviations of approximately 0–0.03 percentage points; we corrected the comparison conditions and recomputed the results from per-query integer counts.  Finally, Sec. 3.3 and Appendix B.1 mixed candidate-pool and final-benchmark statistics and contained some incorrect records. The average of 471.60 queries/movie refers to the initial 141-movie pool. The final benchmark comprises 22,778 valid queries across 50 movies (455.56/movie). We apologize for these reporting errors and have standardized the dataset version, evaluation protocol, and reported totals. 

(Q6) On the Source and Comparison Scope of the EgoMemReason Baselines

 The Qwen3-VL-8B and WorldMM results in Table 7 were taken from the original EgoMemReason paper (Wang et al., 2026), not independently rerun. We have added the missing citation. This experiment examines transfer beyond movie temporal grounding, emphasizing a common Qwen3-VL-8B backbone: EdgeMemory obtains 36.9% Overall, versus 29.6% for Qwen3-VL-8B and 30.6% for WorldMM. We will clarify the provenance and comparison scope of these results. 

(Q8) On the Relationship Between Video Length and Grounding Performance

 We agree that our description of Figure 5 was overly categorical. The claim that Sequence/Act performance progressively declines with movie length while Beat remains stable did not adequately account for non-monotonic fluctuations or differences across metrics. We have revised the discussion: some metrics show downward trends with increasing duration, but these are not strictly monotonic and vary by narrative granularity and tolerance threshold. We will identify the evaluated model and include the movie count in each duration bin to ensure that interpretations remain proportional to the evidence. 

(Q9) On the Distinctions from MAD, Movie101, and M-SyMoN

 We agree that the related-work comparison should be expanded. MAD grounds relatively localized movie audio descriptions; Movie101 addresses character-aware narration generation and temporal narration grounding; and M-SyMoN investigates fine-grained video–text alignment in multilingual movie summaries. These are important related settings, but MRBench grounds recap narration of a movie's overall storyline back to the original full-length movie timeline. Such queries may summarize character motivations, causal relationships, and plot developments across multiple scenes. Accordingly, grounding requires recovering concrete visual events from abstract, context-dependent narrative descriptions over long time spans, rather than only matching local scene descriptions. We will acknowledge these benchmarks more fully and clarify MRBench's distinct evaluation focus.

We will incorporate these clarifications and further improve the corresponding descriptions in the final revision, with all revised text highlighted in blue for ease of reference.

Reviewer_YWohOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work focuses on the problem of long-form video understanding---specifically on aspects of temporal modeling and narrative reasoning over full-length movies. To this end, this paper create a benchmark (MRBench) to test temporal localization in full-length movies. Using a semi-automated pipeline (involving ASR model, visual feature matching) with human-in-the-loop, authors construct ~23k queries across three narrative levels they define (Beat, Sequence, and Act). Furthermore, authors propose an agentic memory framework for the temporal localization task and show that the framework outperforms naive direct inference and other existing agentic frameworks.

Critical strengths
  • The benchmark addresses a critical gap in long-video evaluation by moving beyond needle-in-haystack retrieval to temporal localization. Also, the semi-automated pipeline substantially reduces benchmark annotation costs.
  • The principles behind the proposed agentic framework---the different stages and how those stages restrict expensive multimodal LLM calls---are well-motivated and explained.
Critical weaknesses
  1. Narrative levels defined in this work (Beat--Sequence--Act) are ungrounded and no formal definitions or citations from literature are provided to justify how these boundaries are determined. Also, from Appendix B.6 states that the structural hierarchy is generated offline using a proprietary model, without any human annotation or formal validation.
  2. Instead of conducting a thorough diagnostic evaluation of existing MLLM capabilities, e.g., testing modality reliance or visual retrieval limits, this work couples the benchmark with a custom agentic framework. This conflates the diagnosis of base model limitations with the performance of a specific multi-stage agent.
Questions for authors

Questions:

  • Despite the optimizations, the latency of the proposed framework is still prohibitively high (~67 minutes per movie)? How does this compare to naive MLLM inference?
1 comment
AuthorsOpenReview ↗

Response To Reviewer YWoh

Response We thank the reviewer for the careful assessment and constructive comments. We address each concern below.

(W1) Definition and Reliability of the Beat–Sequence–Act Hierarchy

We agree that the definitions and validation protocol require clearer exposition. Our hierarchy is not arbitrary: it is inspired by the narrative framework in Story: Substance, Structure, Style, and the Principles of Screenwriting and simplified for long-video semantic grounding. We have added the corresponding reference in the revised manuscript.

We also clarify that these labels were not generated by a proprietary model without validation. As stated in Appendix B.6, GPT-5.4 produced the initial hierarchy offline, followed by structural consistency checks, human audits, and repeated-generation stability analysis. Human audits examined the plausibility of temporal boundaries, semantic coherence within units, and containment across levels. We will further detail this protocol.

Importantly, Beat–Sequence–Act labels are not the primary ground truth of MRBench. The principal metric, Acc@δ, uses independently annotated timestamps, while Beat/Sequence/Act alignment serves only as a supplementary structural diagnostic. Hence, potential subjectivity in narrative boundaries does not affect the primary temporal-grounding scores or the conclusions drawn from them.

(W2) Separating Benchmark Diagnosis from the EdgeMemory Method

MRBench is constructed and evaluated independently of EdgeMemory, which is an additional method motivated by the challenges identified by the benchmark. We do not infer base-model limitations solely from our method's improvements.

The paper already provides independent diagnostic evaluations. In Table 1, we evaluate GPT-5.4, Gemini 3 Pro, Qwen3-VL-8B, WorldMM, and One-CLIP RAG across temporal grounding and narrative granularities. In Appendix B.2 (Table 4), we distinguish local from narrative/context-dependent queries. Notably, GPT-5.4's Acc@30 drops from 49.98% to 21.19%, highlighting the difficulty of integrating cross-event context. Furthermore, Sec. 5.5 and Appendix A.3 provide human case analyses identifying Perception, Understanding, Reasoning, and Sampling errors, separating failures in visual perception, semantic interpretation, abstract narrative reasoning, and temporal grounding. Finally, Sec. 5.5 (Figs. 4–5) examines temporal-progress awareness and the relationship between movie duration and grounding performance across narrative granularities.

These analyses are independent of EdgeMemory's performance gains and demonstrate MRBench's value in diagnosing existing MLLM capabilities. Building on these findings, EdgeMemory explores structured memory and explicit relational modeling to address cross-event evidence integration and abstract narrative grounding. Thus, MRBench contributes an independent evaluation benchmark and diagnostic insights, while EdgeMemory investigates a possible solution to the identified limitations.

(Q1) End-to-End Latency

We agree that absolute computational cost is important. Unlike single-query short-video QA, MRBench requires grounding hundreds of narrative queries within a feature-length movie. The reported 66.81 minutes represents the end-to-end processing time for an entire movie and all its queries, rather than per-query latency.

Appendix B.4 (Table 6) already reports complete efficiency comparisons, including node construction and subsequent reasoning. Under the same Qwen3-VL-8B backbone, execution environment, and five-movie efficiency-audit subset, the average processing times are 288.27 min/movie for Qwen3-VL Direct, 828.34 min/movie for WorldMM, 132.00 min/movie for One-CLIP RAG, and 66.81 min/movie for EdgeMemory. Accordingly, EdgeMemory achieves 4.31×, 12.40×, and 1.98× speedups over Qwen3-VL Direct, WorldMM, and One-CLIP RAG, respectively.

These efficiency gains primarily arise from reusing structured memory and narrowing candidate search through event-level reasoning, thereby reducing redundant multimodal inference. As reported in Table 6, EdgeMemory requires only 41.0 multimodal calls and 981.6 cumulative input frames on average, substantially fewer than the compared methods.

We acknowledge that 66.81 minutes remains a substantial absolute computational cost and do not position EdgeMemory as a real-time system. Nevertheless, given the need to process full-length movies and hundreds of queries, these results demonstrate a clear relative end-to-end efficiency advantage over the evaluated baselines.

We will incorporate these clarifications and further improve the corresponding descriptions in the final revision, with all revised text highlighted in blue for ease of reference.

Comments on the paper

1 comment
Area_Chair_9PfeOpenReview ↗

Friendly reminder from AC

Dear Reviewers,

Thank you for your reviews. The initial ratings are 2 / 2 / 2. The authors posted their responses, and none of the reviewers have replied yet. Please let us know whether the responses resolve your concerns, which issues remain, and whether your assessment has changed.

Thank you for your efforts.

Best regards, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.