- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents NEXUS, a framework for long-form movie understanding with three parts. NEXUS-MF processes a video once into a persistent multimodal understanding package containing character-attributed captions from speaker diarization plus agent-based assignment, context-aware chunk summaries, and relation and event graphs. NEXUS-OEBench is a human-annotated open-ended benchmark of 1,701 questions over 23 context sources from 14 titles, reaching 28.5 hours per source and scored by keypoint recall from a panel of LLM judges. NEXUS-QA is an agent that answers over the package with retrieval tools, and NEXUS plus a structural causal model adds a causal-role instruction that labels evidence as cause, mediator, or confounder.
Critical strengths
- The persistent representation is a sensible answer to the cost problem of per-query video agents, and the paper demonstrates reuse across QA, public benchmarks, and visual retrieval rather than claiming it.
- NEXUS-OEBench is a substantial resource: fully human-written questions and keypoint answers over genuinely long narrative video, with documented categories, difficulty levels, and genre composition.
Critical weaknesses
- No annotation reliability statistics are reported for the benchmark's questions and keypoints, and no contamination analysis addresses that all 14 titles are famous works likely present in the pretraining data of the evaluated models, which the web-search result itself makes plausible.
- The primary metric rewards enumeration and cannot penalize unsupported content. In the human comparison, NEXUS beats human viewers by about 20 points on two of three titles, which more plausibly reflects metric inflation from exhaustive listing than superhuman understanding, and the paper does not confront this reading.
Questions for authors
- How can third parties obtain the videos needed to use the benchmark?
- Can you measure the causal-role classifier against human labels, and run an ablation that replaces the causal instruction with a generic keep-searching instruction of similar length? If the gain over plain NEXUS survives that control, the causal framing is supported; if not, the title overclaims.
Response to Reviewer h1a7
We thank the reviewer for the thoughtful and constructive feedback, and we respond to each point below.
Contamination. To measure the effect of prior knowledge directly, we ran a closed-book control in which the model receives only the title and the question, without the video or web access.
Overall, prior knowledge and online discussion recover only a small part of the answers, and both settings remain far below the methods that process the video, which indicates that the gain of Nexus reflects the method itself. In addition, many questions require frame-level or speaker-level detail that is available only in the video, such as the example in Figure 6.
Enumeration. We agree that keypoint recall focuses on coverage of the reference, and that scoring content beyond the annotated keypoints remains an open direction, as we acknowledge in Limitations (Appendix B). To complement recall, the new Table 10 reports answer length. Nexus's answers are shorter than the baseline's, with 38.3 words on average for Nexus+SCM and 44.5 words for Nexus, compared with 50.8 words for VideoSeek. Higher recall with fewer words indicates that the gain comes from locating the right evidence rather than from exhaustive listing.
Human comparison. Our intent with the human comparison is to provide a reference point. Human viewers were able to re-watch the video while answering and were scored with the same protocol. The human evaluators reported that understanding and localizing the context from the given questions came naturally to them, while recalling long and multiple contexts and finding the exact moment needed for the answer was more demanding.
We appreciate the reviewer's interest in using the benchmark. We are happy to share more details on how to use this benchmark upon acceptance.
Keep-searching control. Nexus+SCM is plain Nexus with the causal instruction added on top, and every other part of the agent is identical. Plain Nexus already continues to search until the thinking model judges the gathered evidence to be sufficient, which is the behavior a generic keep-searching instruction would encourage. The comparison between Nexus and Nexus+SCM in our experiments therefore already serves as this control, where the only difference between the two is the causal instruction itself. In addition, Nexus+SCM reaches its answers with fewer tokens and in less time than plain Nexus on Nexus-OEBench (Table 2), which suggests that the instruction guides the agent toward the right evidence rather than toward a longer search.
We would appreciate it if the reviewer could share more detail on the intended setup, in case we have misunderstood the question, and we would be glad to follow up.