acceptodds
Under review as a conference paper at ICLR 2027

NEXUS: A Causal-Aware Framework for Multimodal Long-Form Movie Understanding

Abstract

Understanding long-form videos, especially movies, requires jointly integrating visual, audio, and dialogue signals while persistently tracking character dynamics and event causality. Agentic models are increasingly used to tackle this challenge. However, these approaches either rely on subtitles as a stand-in for video, leaving other modalities underutilized, or reason directly over raw frames at a computational cost that scales sharply with video length and query complexity. Moreover, the field is largely evaluated through multiple-choice (MC) benchmarks, which do not always demand the depth of reasoning that free-form responses require. Under this lens, we introduce NEXUS, a framework for long-form video understanding built around three core components. First, we propose NEXUS-MF, an end-to-end multimodal understanding pipeline that processes long videos into a structured context representation capturing character dynamics and long-term causal dependencies. Second, we present NEXUS-OEBench, a large-scale, human-annotated, open-ended QA benchmark comprising 1,701 question–answer pairs spanning 150+ hours of video content, with questions demanding deep causal and multimodal reasoning. Third, we introduce NEXUS-QA, a video QA agent that causally reasons over NEXUS-MF's structured representation, actively seeking and verifying evidence to construct complete causal chains before forming a final response. NEXUS outperforms baselines on QA benchmarks with 3.4× fewer tokens and 3.3× lower costs, while maintaining stable computational overhead as video length increases. Additionally, NEXUS-MF's multimodal representation generalizes well on the context-retrieval task.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
causal-aware, multimodal understanding, video question answering

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_h1a7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents NEXUS, a framework for long-form movie understanding with three parts. NEXUS-MF processes a video once into a persistent multimodal understanding package containing character-attributed captions from speaker diarization plus agent-based assignment, context-aware chunk summaries, and relation and event graphs. NEXUS-OEBench is a human-annotated open-ended benchmark of 1,701 questions over 23 context sources from 14 titles, reaching 28.5 hours per source and scored by keypoint recall from a panel of LLM judges. NEXUS-QA is an agent that answers over the package with retrieval tools, and NEXUS plus a structural causal model adds a causal-role instruction that labels evidence as cause, mediator, or confounder.

Critical strengths
  • The persistent representation is a sensible answer to the cost problem of per-query video agents, and the paper demonstrates reuse across QA, public benchmarks, and visual retrieval rather than claiming it.
  • NEXUS-OEBench is a substantial resource: fully human-written questions and keypoint answers over genuinely long narrative video, with documented categories, difficulty levels, and genre composition.
Critical weaknesses
  • No annotation reliability statistics are reported for the benchmark's questions and keypoints, and no contamination analysis addresses that all 14 titles are famous works likely present in the pretraining data of the evaluated models, which the web-search result itself makes plausible.
  • The primary metric rewards enumeration and cannot penalize unsupported content. In the human comparison, NEXUS beats human viewers by about 20 points on two of three titles, which more plausibly reflects metric inflation from exhaustive listing than superhuman understanding, and the paper does not confront this reading.
Questions for authors
  • How can third parties obtain the videos needed to use the benchmark?
  • Can you measure the causal-role classifier against human labels, and run an ablation that replaces the causal instruction with a generic keep-searching instruction of similar length? If the gain over plain NEXUS survives that control, the causal framing is supported; if not, the title overclaims.
1 comment
AuthorsOpenReview ↗

Response to Reviewer h1a7

We thank the reviewer for the thoughtful and constructive feedback, and we respond to each point below.

  1. No annotation reliability statistics are reported for the benchmark's questions and keypoints, and no contamination analysis addresses that all 14 titles are famous works likely present in the pretraining data of the evaluated models, which the web-search result itself makes plausible.

Contamination. To measure the effect of prior knowledge directly, we ran a closed-book control in which the model receives only the title and the question, without the video or web access.

SettingModelRecall (%)
Closed-book (title and question only)Claude Opus-57.0
Closed-book (title and question only)Gemini 3.7 Flash9.9
Web search (title and question only)Claude Opus-528.6
Web search (title and question only)Gemini37.2
Web search (title and question only)ChatGPT41.1
Include VideoVideoSeek54.1
Include VideoNexus+SCM65.3

Overall, prior knowledge and online discussion recover only a small part of the answers, and both settings remain far below the methods that process the video, which indicates that the gain of Nexus reflects the method itself. In addition, many questions require frame-level or speaker-level detail that is available only in the video, such as the example in Figure 6.


  1. The primary metric rewards enumeration and cannot penalize unsupported content. In the human comparison, NEXUS beats human viewers by about 20 points on two of three titles, which more plausibly reflects metric inflation from exhaustive listing than superhuman understanding, and the paper does not confront this reading.

Enumeration. We agree that keypoint recall focuses on coverage of the reference, and that scoring content beyond the annotated keypoints remains an open direction, as we acknowledge in Limitations (Appendix B). To complement recall, the new Table 10 reports answer length. Nexus's answers are shorter than the baseline's, with 38.3 words on average for Nexus+SCM and 44.5 words for Nexus, compared with 50.8 words for VideoSeek. Higher recall with fewer words indicates that the gain comes from locating the right evidence rather than from exhaustive listing.

Human comparison. Our intent with the human comparison is to provide a reference point. Human viewers were able to re-watch the video while answering and were scored with the same protocol. The human evaluators reported that understanding and localizing the context from the given questions came naturally to them, while recalling long and multiple contexts and finding the exact moment needed for the answer was more demanding.

  • For more context, we intentionally created three diverse titles at different difficulty levels. Title 1 is the most challenging, with dense events and many twists, Title 2 sits in the middle, with a regular pace and a few major twists, and Title 3 is the easiest, with a straightforward flow.
  • The conclusions for Titles 1 and 2 roughly match our intuition.
  • However, human attention is easily distracted by different factors. For example, although Title 3 is the easiest one, humans surprisingly scored the lowest on it. This suggests that even with simple titles, it's hard for humans to catch certain details without rewatching or intentionally keeping things in mind, which shows our approach checks details carefully regardless of title difficulty.

  1. How can third parties obtain the videos needed to use the benchmark?

We appreciate the reviewer's interest in using the benchmark. We are happy to share more details on how to use this benchmark upon acceptance.


  1. Can you measure the causal-role classifier against human labels, and run an ablation that replaces the causal instruction with a generic keep-searching instruction of similar length? If the gain over plain NEXUS survives that control, the causal framing is supported; if not, the title overclaims.

Keep-searching control. Nexus+SCM is plain Nexus with the causal instruction added on top, and every other part of the agent is identical. Plain Nexus already continues to search until the thinking model judges the gathered evidence to be sufficient, which is the behavior a generic keep-searching instruction would encourage. The comparison between Nexus and Nexus+SCM in our experiments therefore already serves as this control, where the only difference between the two is the causal instruction itself. In addition, Nexus+SCM reaches its answers with fewer tokens and in less time than plain Nexus on Nexus-OEBench (Table 2), which suggests that the instruction guides the agent toward the right evidence rather than toward a longer search.

We would appreciate it if the reviewer could share more detail on the intended setup, in case we have misunderstood the question, and we would be glad to follow up.

Reviewer_Z2V7OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a benchmark for long-video open-ended question answering. It comprises three processes: (1) graph creation, (2) benchmark generation, and (3) agentic experiments. For graph creation, the video is preprocessed once (speaker diarization, character attribution of closed captions, character profiles, and multi-level chunk summaries) into a relation graph and an event graph that are reused at inference. The dataset consists of only 14 movies/TV shows (re-cut into 23 sources of different lengths, 150+ hours in total) and a total of 1,701 questions (~74 questions per source). The authors state that the questions and 1–6 free-form ground-truth keypoints per question were written manually by annotators familiar with the titles. Each question is then tagged with one or more of 7 question types. The benchmark is evaluated by checking whether the keypoints appear in the generated answer, using a panel of LLM judges. The authors then propose their own model, which is essentially an agentic system with 3 tools (search_script, get_script, get_event_range) that operate on the chunked summaries of the video's script, and 2 tools (skim, focus) that operate on the video itself. They also propose a variant (NEXUS+SCM) that adds a causal-reasoning instruction to the agent's prompt. They compare their approach against two other models and show that it achieves the highest recall of keypoints in the generated answers, with ~3.4x fewer tokens and ~3.3x lower cost per question. They additionally report results on two public multiple-choice benchmarks (LVBench, MINERVA-Ego) and a visual frame-retrieval task.

Critical strengths
  • Most long-video QA benchmarks are multiple-choice. The paper motivates why open-ended answers better test understanding, and it provides human-written questions over long contexts.
Critical weaknesses
  • Dataset scale and diversity. The benchmark covers only 14 movies/TV shows (1,701 questions). Although the 23 sources span 150+ hours, the questions are heavily concentrated: a single fantasy title contributes 551 questions (~32%), and several genres are represented by a single title with 12–51 questions (Table 7). This limits how well the results generalize across genres and narrative styles.

  • Annotation process is under-specified. The paper states only that the questions were written by "a group of annotators who have good knowledge of these titles." It is unclear:

    • how many annotators there were and what their background was;
    • what guidelines they followed;
    • how keypoints were selected, and how many keypoints a typical question has;
    • whether any question was validated by a second annotator (no inter-annotator agreement is reported);
    • how difficulty levels were calibrated across annotators. Without guidelines or real examples, it is difficult to assess the quality of the dataset.
  • Missing dataset statistics and examples. The paper does not report basic statistics such as question length, number of keypoints per question, keypoint length, or the expected answer length (in tokens). The examples shown in the main paper are either invented (Appendices E, F, H) or AI-generated (Figure 1, per the AI Use Statement). Only a few real questions appear (Figure 6, Appendix O). Since the dataset is internal and will be released only upon acceptance, reviewers cannot inspect it.

  • Validity of the evaluation. It is unclear whether keypoint recall measures the deep causal and multimodal reasoning the benchmark claims to target. The real examples shown (Figure 6, Appendix O) ask for a character's name, a list of names, or a yes/no answer with a name; these are short factual lookups rather than causal reasoning. In addition, recall does not penalize incorrect or hallucinated content (acknowledged in Appendix B), so longer answers that list more candidates may be rewarded. No agreement between the LLM judges and human raters is reported.

  • Web-search baseline. The success of this baseline depends heavily on how much online discussion exists for each title, which the paper does not report. If the titles are obscure or internal (one genre is labeled "Unknown"), web search fails by construction, so the comparison says little about the need for multimodal understanding. The very low token count for Gemini (746 tokens/question) also suggests the model may not have searched at all.

  • Benchmark–method co-design. The same authors designed both the benchmark and the method, and the question categories (e.g., who-said-what, character dynamics) closely mirror what NEXUS-MF is built to extract (speaker attribution, relation graphs). The paper does not state whether annotators had access to any NEXUS-MF outputs, or whether design choices were tuned on benchmark titles (e.g., the 4-title ablation subset). Clarifying this would help rule out an evaluation biased toward the proposed method.

  • Missing related work. Several long-video and movie QA benchmarks are not discussed or compared in Table 1, e.g., Movie Recaps. Some are cited in the introduction (MovieQA, TVQA, CinePile) but left out of the related-work comparison.

  • Missing Ablation. A big point the authors brought up was the spread out nature of the key moments, yet this was never mentioned in the experiments or the results.

Questions for authors
  • Why is Figure 6 a screenshot of a computer screen? The benchmark is described as covering movies and TV shows, so it is unclear whether this frame comes from a scene in one of the titles (e.g., a character's laptop) or from another source. The caption also says the line is "spoken on screen," but the frame appears to show the text as a chat notification. Could the authors clarify where this frame comes from and how the line is delivered?

  • The paper contains very few real examples from the benchmark (Figure 6 and the two examples in Appendix O); the other examples are invented or AI-generated. Could the authors provide a representative set of real questions, keypoints, and model answers, ideally covering each of the seven question categories?

  • How exactly were the questions and keypoints created? Please describe the annotation process, the guidelines given to annotators, and the source material the annotators used (e.g., the raw video, subtitles, or scripts).

  • Which movies and TV shows are included? The paper reports genre labels (Table 7) but does not name the titles, and one title is labeled "Unknown." What is this title, and why is its genre unknown? Since the dataset is described as internal, what are the licensing terms, and will the videos themselves be released, or only the questions?

  • What source material did annotators use when writing questions and keypoints (e.g., the raw video, subtitles, scripts, or any model-generated summaries)? Were the annotators given access to any NEXUS-MF outputs? If keypoints were derived from text that overlaps with NEXUS-MF's representation, the evaluation may favor the proposed method.

  • What happens if the key moments are not unique. Meaning, there are two scenes where the vase is broken, how are these captured and detected?

4 comments
AuthorsOpenReview ↗

Response to Reviewer Z2V7 (1/2)

We thank the reviewer for the thoughtful and constructive feedback, and we respond to each point below.

  1. Why is Figure 6 a screenshot of a computer screen? Could the authors clarify where this frame comes from and how the line is delivered?

Figure 6 shows a full, uncropped frame of a movie in Nexus-OEBench. In this scene, the film cuts to a close-up of a character's laptop screen, and the laptop display fills the entire frame. We have added a film-strip border to the figure in the revision to make this clear. The line "Thanks, it was keeping me up all night" appears only as on-screen text, outside both the spoken dialogue and the closed captions. This makes the example a clear case for visual search, where a transcript-only search would miss the line.


  1. The paper contains very few real examples from the benchmark. Could the authors provide a representative set of real questions, keypoints, and model answers, ideally covering each of the seven question categories?

We have added Table 8 (Appendix H) with seven real questions and their keypoints, one from each genre, which together cover all seven question categories.


  1. How exactly were the questions and keypoints created? Please describe the annotation process, the guidelines given to annotators, and the source material the annotators used.
  • Annotation process. The questions and keypoints were written 6 annotators with professional experience in movie-related annotation, all of whom are highly familiar with the titles.
  • Source material. The questions and keypoints were written independently of the model.
    • Included. The following information are provided to the annotators: the video with its audio and subtitles, and our question-writing instructions, which primarily emphasize the format, context isolations.
    • Kept separate. We did't provide the following information to the annotators: The movie script, the model design, the Nexus's ideas, and all Nexus-MF and model outputs. The questions were kept as originally written.
  • Guidelines. The instructions asked for open-ended, challenging questions across seven categories (Appendix H.2).
  • Keypoints. Each question has 1–6 keypoints, with 1.52 on average, and 35.6% of questions have more than one (new Table 9). The keypoints cover exactly the facts and information needed to answer the question, complete and concise.
  • Validation. We validate the questions and keypoints through an independent review, in which question is cross-checked by a second reviewer who is also highly familiar with the title.
  • Difficulty. We (the authors) first define the seven question categories and challenges (Appendix H.2). The people who wrote and cross-checked each question then rate its difficulty on two shared criteria, namely whether it spans multiple categories, and whether the evidence is caught in one viewing or requires multiple viewings.

  1. What happens if the key moments are not unique. Meaning, there are two scenes where the vase is broken, how are these captured and detected?

We handle this scenario in both the benchmark and the method.

Benchmark. Each question is instructed and cross-checked so ensure that it's written with enough context to isolate the intended moment. For similar key moments, we instructed the annotators to add more surrounding context to help uniquely locate the specific moments, such as mentioning locations, characters involved in the target scenes,...

Method. Nexus-MF stores the two scenes as separate entries, each with its own time window and timestamped events. Nexus-QA's script search returns every matching chunk, and the agent selects the right one from the context in the question, or collects all occurrences when the question asks for them, as in the Appendix O example with six different wrong names for Alice.

AuthorsOpenReview ↗

Response to Reviewer Z2V7 (2/2)

  1. Which movies and TV shows are included? One title is labeled "Unknown." What are the licensing terms, and will the videos themselves be released, or only the questions? Dataset scale and diversity. The benchmark covers only 14 movies/TV shows, and the questions are heavily concentrated.

Titles and release. The full list of titles will be made public upon acceptance. We selected well-known movies and TV shows that have rich discussion on the internet in text, frames, and video.

"Unknown" genre. That title is a horror title. We have corrected its label, and Table 7 now lists seven genres.

Scale and concentration. Nexus-OEBench favors depth per title, with long videos and dense questions.

  • Diversity. We chose the 14 titles based on the expertise of the people who wrote the questions and on genre diversity. They span movies and TV shows in seven genres (Table 7), with questions in seven categories.

  • Scale. Compared with the other long-form benchmarks in Table 1, our 14 titles cover 150+ hours of video, with a mean length of 6.80 hours and the largest set of questions (1,701), all open-ended.

  • Repeated questions. The concentration comes from the multi-version titles, where we intentionally repeat the same questions at several context lengths, and each repeat counts as a question. We intentionally repeat questions across context lengths to measure the ability of models dealing with extreme long context. The repeats fall into one of two cases.

    • The keypoints change. "Who said 'I'll handle it'?" has the answer "Character A" after 5 episodes, and "Character A, Character B, and Character C" by episode 20.
    • The keypoint stays same but model context is become more challenging and noise. For example, "When did Character A first meet Character B?” always points to one unique scene. However, later episodes covering different events may reference that first meeting in flashbacks or conversations, and a robust model needs to pick out such detail from noisier context inputs.
Reviewer_Z2V7OpenReview ↗

Response to authors

The authors did not address any of the main weaknesses I had pointed out but just the side questions. For that reason, I still believe my review is correct and will not be changing my score. The dataset is too small without explanation of the creation process, no human study, no inter-annotator scores. The paper is too ambiguous and not unique enough for me to grant a change in the score.

AuthorsOpenReview ↗

Following up on Reviewer Z2V7

We thank the reviewer for the follow-up.

The weaknesses and the questions in the review LARGELY OVERLAP, and we organized our response around the questions to keep it concise. We map each weakness to our response below.

  • Dataset scale and diversity. We have covered this in Point 5, with details of how the 14 titles were chosen, how the benchmark compares with the long-form benchmarks in Table 1 (150+ hours, 1,701 questions), and how the intentionally repeated questions explain the concentration.
  • Annotation process and inter-annotator validation. We have covered this in Point 3, with details of the six annotators and their professional experience, the guidelines, and the cross-check of each question by a second reviewer.
  • Dataset statistics and examples. We have covered this in Points 2 and 3, with details of the new Table 8 (real questions and keypoints across all seven categories) and the new Table 9 (keypoint statistics).
  • Validity of the evaluation and human study. We have covered this in Point 2, with details of real examples on character dynamics, psychology, and narrative intent. The paper also reports a human evaluation on three titles (Appendix I), and the new Table 10 shows that Nexus reaches higher recall with shorter answers.
  • Benchmark–method co-design. We have covered this in Point 3, with details of the source material given to the annotators, which was independent of the model design and of all Nexus-MF and Nexus-QA outputs.
  • Web-search baseline. We have covered this in Point 5, with details of the titles, which are well-known and have rich online discussion, which makes the comparison with internet-search baselines fair.

We also address the remaining points here.

  • Dataset size. On the comment in the follow-up that the dataset is small, Table 1 shows that Nexus-OEBench has the largest set of questions among the compared long-form benchmarks (1,701, against 1,549 for LVBench, 1,515 for MINERVA, and 500 for EgoLifeQA). Its videos are also much longer, with a mean length of 6.80 hours (against 1.1 hours for LVBench and 0.2 hours for MINERVA) and 150+ hours in total.
  • Related work. Table 1 focuses on long-form benchmarks at the hour scale, which is the setting of Nexus-OEBench. MovieQA, TVQA, and CinePile are built on short clips of a few minutes or less, which is why we cite them in the introduction. Appendix D discusses further benchmarks, including NExT-QA, MovieChat-1K, MLVU, ActivityNet-QA, FriendsQA, and Neptune. We thank the reviewer for the pointer to Movie Recaps, and we will add it, together with MovieQA, TVQA, and CinePile, to Table 1 and Appendix D in the revision.
  • Spread-out key moments. We address this in both the benchmark and the agent.
    • Benchmark. The question categories of character dynamics (45.9% of questions), character psychology (23.1%), and behavioral pattern (7.5%) mostly cover long-term dependency, where the answer requires linking events from multiple parts of the movie (Figure 2 and Appendix H.2; categories are multi-label).
    • Agent. Nexus-MF records each chunk of the video as a separate entry, and Nexus-QA searches across all of these records to gather evidence from distant scenes.
    • Example. The Appendix O example illustrates both aspects. The question asks for the wrong names a character uses for Alice, which appear at six different points of the title, and the agent collects all six occurrences.

We would be glad to expand on any of these points.

Reviewer_2uTsOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

NEXUS constructs a persistent representation of long videos using character-attributed captions, multi-granularity summaries, and relation/event graphs, then answers questions through retrieval. It introduces NEXUS-OEBench with 1,701 open-ended questions and a causal-role prompting variant, NEXUS+SCM. Experiments report improved keypoint recall and reduced amortized costs, supported by component ablations, public-benchmark evaluations, and visual-context retrieval.

Critical strengths
  1. Useful representation with substantial empirical support. Character-attributed captions and persistent relation/event information improve access to long-range narrative evidence. Character-attribution and graph ablations support their contribution to the overall performance gains.

  2. Validation beyond the proposed benchmark. Matched-subset evaluations on LVBench, experiments on MINERVA-Ego, an alternative thinking backbone, and visual-context retrieval provide complementary evidence that the system’s benefits extend beyond its own open-ended evaluation.

Critical weaknesses
  1. Keypoint recall does not establish overall answer correctness. The evaluation rewards reference coverage but does not penalize incorrect statements outside the annotated keypoints. Appendix I scores human answers with the same protocol rather than independently auditing model-answer precision. Overlapping questions across context versions also require clear aggregation and uncertainty estimates.

  2. The SCM framing exceeds the implementation and evidence. Appendix G implements causal-role classification through prompting rather than an operational structural causal model. Its treatment of confounders and narrative intent needs greater precision. Incremental gains are mixed across settings, so the results support a potentially useful causal-inspired search heuristic more clearly than established causal identification.

Questions for authors
  1. Can you provide a small blinded audit of model answers for incorrect or unsupported statements, alongside answer length? Please clarify how keypoints, questions, and overlapping context versions are weighted.

  2. Can the SCM component be described more precisely as causal-inspired prompting, with paired effects by question type? How are candidate causes, confounders, and narrative-intent evidence distinguished?

  3. How are the preprocessing costs in Figure 3 amortized into Table 2, and how many queries are assumed? Can existing predictions on questions shared across context lengths isolate the effect of longer context?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 2uTs (1/2)

We thank the reviewer for the thoughtful and constructive feedback, and we respond to each point below.

  1. The evaluation does not penalize incorrect statements outside the annotated keypoints. Overlapping questions across context versions require clear aggregation and uncertainty estimates.

Precision. We agree that keypoint recall focuses on coverage of the reference. We acknowledge this in Limitations (Appendix B), and we see a metric that also measures precision as a valuable direction for future work.

Answer length. We have added the answer length of each system in the new Table 10. Nexus reaches higher recall with shorter answers than VideoSeek (38.3–44.5 words against 50.8 words), which indicates that the gain comes from locating the right evidence rather than from longer answers.

Overlapping questions across context versions. As the context grows, a model must isolate the target information from more noise. We intentionally repeat questions across context lengths to measure this, and the new Figure 12 (Appendix Q) shows that shared-question recall closely matches recall on all questions.


  1. The SCM framing exceeds the implementation and evidence.

Implementation. Nexus+SCM uses the structural causal model as the inspiration for a prompting strategy (Appendix G) that guides evidence search, which matches the reviewer's reading of a causal-inspired search heuristic.

Evidence. Relative to plain Nexus, Nexus+SCM is 1.2× faster with fewer tokens and improves recall by 0.7 points on Nexus-OEBench (Table 2). It also improves recall by 3.3 points with the Kimi K3 backbone (Table 12) and accuracy by 3.6 points on MINERVA-Ego, while staying comparable on LVBench with 83.6% against 83.8% (Table 4).


  1. Can you provide a small blinded audit of model answers, alongside answer length? How are keypoints, questions, and overlapping context versions weighted?

Unsupported statements. The questions repeated across context versions already provide a check on unsupported statements. Consider the question "What is the name of Zach's dad?" as an example.

  • First version. The name is yet to be revealed in this context, and the correct answer is "not mentioned".
  • Second version onward. The name is revealed in the added episodes, and the correct answer becomes "Tom".
  • Scoring. A model that states a name in the first version makes an unsupported statement and receives zero recall for that question, while a model that stays grounded in the given context is rewarded.

Weighting. Keypoints are weighted equally within a question, questions are averaged within each context source, and the overall recall in Table 2 is the unweighted mean over the 23 context sources, where each context version contributes equally.


  1. Can the SCM component be described as causal-inspired prompting, with paired effects by question type? How are causes, confounders, and narrative-intent evidence distinguished?

The SCM prompt (Appendix G) asks the thinking model to assign a causal role to each piece of retrieved evidence before using it.

  • Direct cause. A stated action or reason that immediately precedes and explains the outcome.
  • Mediator. Evidence between an earlier trigger and the outcome, which leads the agent to search for that trigger.
  • Confounder. Evidence that only co-occurs with the outcome, which the agent sets aside.
  • Narrative-intent evidence. A planted detail or callback, accepted as a cause once an action or statement in the story connects it to the outcome.
AuthorsOpenReview ↗

Response to Reviewer 2uTs (2/2)

  1. How are the preprocessing costs in Figure 3 amortized into Table 2, and how many queries are assumed? Can predictions on shared questions isolate the effect of longer context?

Amortization. Nexus-MF processes each video once, and Table 2 amortizes this one-time cost over an estimated 200 queries per video.

Overlapping questions across context versions. As the context grows, a model must isolate the target information from more noise. We intentionally repeat questions across context lengths to measure this, and the new Figure 12 (Appendix Q) shows that shared-question recall closely matches recall on all questions.

A repeated question falls into one of two cases:

  • The keypoints change. For example, "Who said 'I'll handle it'?" has the answer "Character A" after the first 5 episodes, and becomes "Character A, Character B, and Character C" by episode 20, once the other two characters have repeated the line.

  • The keypoints stay the same, while the event is mentioned more times, making the context surrounding one event become more challenging. For example, "When did Character A first meet Character B?” always points to one unique scene. However, later episodes covering different events may reference that first meeting in flashbacks or conversations, and a robust model needs to pick out such detail from noisier context inputs.

Following the reviewer's suggestion, the new Figure 12 (Appendix Q) reports recall on the 62 questions shared across versions of the 5-version title. The results agree closely with Figure 5, which confirms that the trend reflects the longer context itself, with Nexus remaining more stable than VideoSeek.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.