acceptodds
Under review as a conference paper at ICLR 2027

StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding

Abstract

Deploying autonomous multimodal agents in continuous, real-world environments requires them to ingest unbounded audio-visual streams and maintain hour-scale memory. However, current evaluations predominantly rely on brief clips and multiple-choice formats. This design allows minimal baselines that process only the last four frames to match or surpass complex streaming models, while answer options also expose language shortcuts. We introduce StreamArena, a benchmark for hour-scale, interactive streaming video understanding. StreamArena contains 243 full-length videos averaging 88.8 minutes and 3,646 rigorously annotated, open-ended question-answer pairs that evaluate real-time perception, historical retrospection, proactive interaction, and multimodal tool utilization. Evaluation across diverse systems exposes a tension between continuous interaction and long-horizon multimodal comprehension. Methods that retain only recent frames cannot recover distant events, methods that convert past observations into text lose visual evidence, and methods that repeatedly compress visual memory struggle to preserve fine-grained details over time. We address this tension with StreamMind, a two-tier architecture that assigns latency-critical interaction and proactive monitoring to independently scheduled frontend workers, while backend workers asynchronously construct persistent multimodal memory and perform historical recall and external search. StreamMind outperforms streaming baselines across four capabilities and reduces query-to-answer latency by reusing persistent state.

Primary area:
foundation or frontier models, including LLMs
Keywords:
streaming video understanding, multimodal agents, long-horizon memory, proactive interaction, tool use, evaluation benchmarks

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_deueOpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents StreamArena, an evaluation benchmark for continuous, interactive, long-duration video understanding. It addresses the limitation of existing video understanding evaluations, which rely on short videos and multiple-choice questions and thus fail to measure real streaming comprehension capabilities.

In addition, the authors propose StreamMind, a two-layer multi-agent architecture. It achieves coordination between low-latency responses and long-term multimodal memory via front-end real-time interaction and active monitoring, as well as back-end asynchronous memory construction and retrieval. Experiments demonstrate that StreamMind outperforms the evaluated streaming baselines across all four capabilities.

Critical strengths
  1. The dataset has advantages in scale and annotation quality. The benchmark consists of 243 videos with an average duration of 88.8 minutes and 3,646 open-ended question-answering tasks spanning seven distinct scenarios.

The authors adopt multi-stage manual cross-validation and independent review, and attach precise timestamp annotations for questions and supporting evidence, which helps enhance evaluation reliability and causal consistency.

  1. The experimental results show a favorable performance-latency trade-off. StreamMind outperforms the evaluated streaming baselines on all four tasks.

In comparisons using the same base model, the system reduces average response latency by 66.2% while retaining approximately 89.7% of the overall accuracy. This illustrates the potential of continuous memory construction and state reuse for improving efficiency.

Critical weaknesses
  1. StreamMind adopts Qwen3.5-397B-A17B as its base model, while its primary streaming baselines AURA, MiniCPM-o-4.5, StreamForest and ThinkStream employ Qwen3-VL-8B-Instruct (8B), Qwen3-8B (8B), Qwen2-7B (7B), and Qwen2.5-VL-3B (3B), respectively.

These base models exhibit substantial discrepancies in scale and capability. Therefore, the performance advantage of StreamMind across the four tasks cannot be fully attributed to the proposed multi-agent architecture.

  1. The paper lacks independent ablation studies on key components including the Memory Writer, hierarchical event memory, entity relation graph, and keyframe retrieval. As such, the actual contribution of each module to the final performance gains cannot be clearly identified.

I recommend the authors supplement streaming architecture comparisons under the same backbone and module ablation experiments to strengthen the persuasiveness of the experimental conclusions.

Questions for authors

Refer to Critical Weaknesses.

  1. For weakness 1, how abour performance of our work with baseline's backbone LLM? Or performance of baselines with our backbone LLM?

  2. For weakness 2, I recommend the authors supplement module ablation experiments to strengthen the persuasiveness of the experimental conclusion.

qualitative explanations are provided in Section 3.2 on Page 8.

  • History recall: The authors argue that continuously maintained multimodal memory, especially the joint retrieval of structured events and keyframes, is the major contributor to improved long-duration understanding.
  • Tool use: The authors attribute the 228.1% relative performance gain to the collaboration among Router, Recall and Search Workers.
  • Proactive interaction: The authors identify independently running Monitor Workers as the key to performance gains in proactive interaction.
  • Response latency: Section 3.3 attributes latency reduction to persistent state reuse, which continuously builds memory before questions arrive and avoids reprocessing historical videos for every query.

However, these are all the authors’ interpretations of the overall system experimental results, rather than conclusions validated via module-level experiments. I recommend the authors supplement quantitative and rigorous module ablation experiments.

5 comments
AuthorsOpenReview ↗

Response to Weakness 1 (1/2)

Response to W1. Streaming methods with the same 397B backbone

Thank you for recognizing the value of StreamArena and for raising the question of how much backbone scale contributes to the results. In Table 4, we use each baseline’s released checkpoint and inference procedure. For AURA and VST, those checkpoints incorporate method-specific fine-tuning. Simply replacing them with a larger model would discard what the original models learned during that training. We therefore evaluated the released systems first, while recognizing that their scores reflect both the backbone and the method.

To compare the methods at the same model scale, we have adapted the inference procedures of AURA, VST, StreamForest, and ThinkStream to the same Qwen3.5-397B-A17B checkpoint used by StreamMind, without additional fine-tuning. In these 397B runs, AURA uses at most 30 recent frames, while VST uses at most 8,192 visual tokens per segment, following their respective original settings. The comparison covers all 243 StreamArena videos (3,646 questions), with the same scoring rules for every method.

MethodRTPHR avg.HR L1HR L2HR L3HR L4ToolPro. avg.Pro. L1Pro. L2Pro. L3
AURA35.424.529.827.025.413.8-8.915.46.42.2
VST36.930.231.535.533.016.6-----
StreamForest †25.518.424.317.618.413.8-----
ThinkStream †12.910.315.510.68.16.64.53.75.83.11.1
StreamMind44.534.931.546.734.617.156.111.616.69.57.5

Accuracy is reported in percent. HR L1–L4 group questions by the time from supporting evidence to the question (≤5 min, 5–15 min, 15–30 min, and >30 min). Pro. L1–L3 group questions by the time from the monitoring instruction to the target event (≤30 s, 30 s–4 min, and >4 min).

A dash (-) denotes a capability unsupported by the method’s evaluation interface.

With the backbone held fixed, StreamMind reaches 34.9% HR accuracy, compared with 30.2% for VST, the strongest baseline on this task.

† With the 397B backbone, HR accuracy improves by 4.0 percentage points for StreamForest and 2.8 for ThinkStream. These gains may be modest because we used the checkpoint without training it to use StreamForest’s merged visual tokens or the reasoning tokens that ThinkStream keeps in its KV cache after evicting older visual tokens.

We have added the comparison using the shared 397B backbone to Appendix C.9 (Table 13).

AuthorsOpenReview ↗

Response to Weakness 2 (2/2)

Response to Q2. Worker roles and memory ablations

Thank you for highlighting the value of StreamArena and for asking how its components contribute. The Recall Worker retrieves past evidence for historical questions, the Search Worker looks up external information for tool questions, and the Monitor Worker issues an alert when a requested event occurs. Removing any of these workers would disable the corresponding operation. For example, without the Monitor Worker, StreamMind could never issue an alert and would score zero on proactive interaction.

We therefore keep all workers and the Qwen3.5-397B-A17B backbone fixed and vary one memory component at a time across all 877 historical retrospection (HR) questions. The variant without the event hierarchy keeps micro events and removes the macro and super levels. We report accuracy (%) overall and by the time between the supporting evidence and the question.

ConfigurationHR avg.L1 (≤5 min)L2 (5–15 min)L3 (15–30 min)L4 (>30 min)
Full StreamMind34.931.546.734.617.1
No event hierarchy30.130.441.827.011.6
No entity graph34.130.945.833.516.6
No historical key frames27.426.536.426.512.7

Historical key frames contribute most to overall HR accuracy. Removing them lowers accuracy from 34.9% to 27.4%, a drop of 7.5 percentage points. Without these frames, Recall cannot revisit visual details omitted from the event descriptions. Removing the event hierarchy reduces overall accuracy by 4.8 points. Among the three ablations, this change causes the largest accuracy drop on L4, from 17.1% to 11.6%. This suggests that grouping related events helps Recall locate evidence more than 30 minutes after it appears. Removing the entity graph has the smallest overall effect (0.8 points). Its links support more specific queries, such as tracing a character’s relationships across scenes.

We have added the memory ablation results for all 877 HR questions, including the breakdown by evidence-to-question gap, to Appendix C.8 (Table 12).

Reviewer_deueOpenReview ↗

Reponse to Rebuttal

While I appreciate the authors' additional experiments, I believe Weakness 2 is only partially addressed. The new ablations provide useful evidence regarding the contributions of hierarchical event memory, the entity relation graph, and historical keyframes to HR performance. However, the Memory Writer itself remains unevaluated, and the contributions of the overall multi-agent coordination mechanism are still unclear. In particular, the current experiments do not sufficiently validate the authors' explanations for the improvements in tool utilization, proactive interaction, and response latency.

The authors also provide insufficient explanations and justifications for the individual components and design choices. Your rebuttal also seems to merely list numerical results without providing sufficient analysis of the underlying mechanisms or explaining why these improvements occur.

I also note that another reviewer, TpSn, raised essentially the same concern: "StreamMind employs multiple specialized workers and integrates hierarchical event memory, an entity relation graph, and keyframe storage. However, the paper lacks sufficient module-level ablation studies to clarify the individual contributions of these components." This suggests that the concern is not merely a minor experimental detail, but a fundamental limitation in the current evaluation.

Overall, StreamMind presents an interesting engineering effort, but its architecture appears highly handcrafted, with multiple specialized workers and heuristically designed memory components. The interactions among these components are insufficiently analyzed, making the overall design difficult to interpret and its individual contributions difficult to establish.

There have already been numerous similar works on Video Agents. Without a clear explanation of the novelty and rationale behind their design choices, I am left with the impression that this work is merely an amalgamation of existing ideas.

Although the system demonstrates promising empirical results, I do not think the current evidence sufficiently supports the proposed architectural contributions at the level expected for publication at ICLR. Consequently, I am lowering my rating from 3 (Weak Accept) to 2 (Weak Reject).

Reviewer_TpSnOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces StreamArena, a benchmark for continuous, interactive, and long-horizon streaming video understanding. The benchmark comprises 243 full-length videos with an average duration of 88.8 minutes and 3,646 manually annotated and validated open-ended question-answer pairs, covering four key capabilities: real-time multimodal perception, historical retrospection, proactive interaction, and multimodal tool utilization.

Critical strengths

The paper introduces a practically relevant benchmark for long-horizon interactive streaming video understanding. The paper provides systematic experimental analyses and presents an effective system architecture.

Critical weaknesses

Substantial differences in experimental settings across methods limit the ability to directly assess the advantages of the proposed StreamMind architecture.

Questions for authors
  1. StreamMind employs Qwen3.5-397B-A17B as its backbone, whereas several streaming baselines use substantially smaller models. Could the authors conduct further comparisons with existing streaming architectures under matched backbone models, input frame budgets, and inference resources to better establish the independent advantages of StreamMind?

  2. StreamMind employs multiple specialized workers and integrates hierarchical event memory, an entity relation graph, and keyframe storage. However, the paper lacks sufficient module-level ablation studies to clarify the individual contributions of these components.

  3. Could the authors provide a more comprehensive analysis of the total end-to-end computational cost, including continuous video processing, GPU utilization, memory storage growth, and the inference overhead of background workers? Furthermore, can the system continuously process hour-long videos at their actual playback rate without accumulating processing delays?

  4. StreamArena employs Gemini 3.1 Pro as an LLM judge to assess open-ended responses using binary correctness decisions. Could the authors provide further analysis of the agreement between the LLM judge and human evaluators, as well as the consistency of evaluation results across different judge models, to establish the reliability of the evaluation protocol?

4 comments
AuthorsOpenReview ↗

Response to Weakness1 & Question 1 (1/4)

Response to W1/Q1. Streaming methods with the same 397B backbone

Thank you for recognizing the value of StreamArena and for raising the question of how much backbone scale contributes to the results. In Table 4, we use each baseline’s released checkpoint and inference procedure. For AURA and VST, those checkpoints incorporate method-specific fine-tuning. Simply replacing them with a larger model would discard what the original models learned during that training. We therefore evaluated the released systems first, while recognizing that their scores reflect both the backbone and the method.

To compare the methods at the same model scale, we have adapted the inference procedures of AURA, VST, StreamForest, and ThinkStream to the same Qwen3.5-397B-A17B checkpoint used by StreamMind, without additional fine-tuning. In these 397B runs, AURA uses at most 30 recent frames, while VST uses at most 8,192 visual tokens per segment, following their respective original settings. The comparison covers all 243 StreamArena videos (3,646 questions), with the same scoring rules for every method.

MethodRTPHR avg.HR L1HR L2HR L3HR L4ToolPro. avg.Pro. L1Pro. L2Pro. L3
AURA35.424.529.827.025.413.8-8.915.46.42.2
VST36.930.231.535.533.016.6-----
StreamForest †25.518.424.317.618.413.8-----
ThinkStream †12.910.315.510.68.16.64.53.75.83.11.1
StreamMind44.534.931.546.734.617.156.111.616.69.57.5

Accuracy is reported in percent. HR L1–L4 group questions by the time from supporting evidence to the question (≤5 min, 5–15 min, 15–30 min, and >30 min). Pro. L1–L3 group questions by the time from the monitoring instruction to the target event (≤30 s, 30 s–4 min, and >4 min).

A dash (-) denotes a capability unsupported by the method’s evaluation interface.

With the backbone held fixed, StreamMind reaches 34.9% HR accuracy, compared with 30.2% for VST, the strongest baseline on this task.

† With the 397B backbone, HR accuracy improves by 4.0 percentage points for StreamForest and 2.8 for ThinkStream. These gains may be modest because we used the checkpoint without training it to use StreamForest’s merged visual tokens or the reasoning tokens that ThinkStream keeps in its KV cache after evicting older visual tokens.

We have added the comparison using the shared 397B backbone to Appendix C.9 (Table 13).

AuthorsOpenReview ↗

Response to Question 2 (2/4)

Response to Q2. Worker roles and memory ablations

Thank you for highlighting the value of StreamArena and for asking how its components contribute. The Recall Worker retrieves past evidence for historical questions, the Search Worker looks up external information for tool questions, and the Monitor Worker issues an alert when a requested event occurs. Removing any of these workers would disable the corresponding operation. For example, without the Monitor Worker, StreamMind could never issue an alert and would score zero on proactive interaction.

We therefore keep all workers and the Qwen3.5-397B-A17B backbone fixed and vary one memory component at a time across all 877 historical retrospection (HR) questions. The variant without the event hierarchy keeps micro events and removes the macro and super levels. We report accuracy (%) overall and by the time between the supporting evidence and the question.

ConfigurationHR avg.L1 (≤5 min)L2 (5–15 min)L3 (15–30 min)L4 (>30 min)
Full StreamMind34.931.546.734.617.1
No event hierarchy30.130.441.827.011.6
No entity graph34.130.945.833.516.6
No historical key frames27.426.536.426.512.7

Historical key frames contribute most to overall HR accuracy. Removing them lowers accuracy from 34.9% to 27.4%, a drop of 7.5 percentage points. Without these frames, Recall cannot revisit visual details omitted from the event descriptions. Removing the event hierarchy reduces overall accuracy by 4.8 points. Among the three ablations, this change causes the largest accuracy drop on L4, from 17.1% to 11.6%. This suggests that grouping related events helps Recall locate evidence more than 30 minutes after it appears. Removing the entity graph has the smallest overall effect (0.8 points). Its links support more specific queries, such as tracing a character’s relationships across scenes.

We have added the memory ablation results for all 877 HR questions, including the breakdown by evidence-to-question gap, to Appendix C.8 (Table 12).

AuthorsOpenReview ↗

Response to Question 3 (3/4)

Response to Q3. Real-time playback and resource use

Thank you for highlighting the practical value of StreamArena. Yes, StreamMind continues to receive video at normal playback speed while it answers questions.

A live video understanding system must monitor the stream for as long as it plays to respond when an event occurs. A one-hour stream therefore requires one hour of online operation, even when the user asks no questions. During that time, StreamMind continues to read frames, check active monitoring instructions, and update memory.

We estimate the serving cost for Qwen3.5-397B-A17B on eight H800 GPUs at four concurrent streams. At this load, four one-hour streams share eight GPU-hours, or approximately two allocated H800 GPU-hours per video-hour.

The table summarizes all 243 video sessions in the Qwen-based StreamMind run.

MeasureFull run
Model calls for memory updates90,436 total. Of the 304 calls per video-hour, 242 are for memory writing, 35 for event aggregation, and 27 for entity and event review.
Tokens for those calls4.90M input and 0.303M output per video-hour
Search rounds2,523 total (8.48 per video-hour)
Recall rounds1,584 total (5.33 per video-hour)
Mean final memory per video28.73 MiB (23.28 MiB for 683.3 keyframes + 5.45 MiB for the database)

We have added the GPU cost estimate to Appendix C.6, where Table 11 reports model calls, token counts, Search and Recall rounds, and final memory use.

AuthorsOpenReview ↗

Response to Question 4 (4/4)

Response to Q4. Agreement with human evaluators and alternative judges

Thank you for recognizing the practical value of StreamArena and for asking how reliably we evaluate open-ended responses. We checked how often Gemini 3.1 Pro, our original judge, agrees with human evaluators and whether its judgments hold when we use a different model.

We sampled 100 StreamArena questions. For each question, three PhD-level evaluators independently graded the same model response as correct or incorrect according to whether it contained the factual core of the reference answer. We also graded those responses using Gemini 3.1 Pro, Qwen3.5-397B-A17B, and GPT-6.1 Sol.

The table reports how often Gemini 3.1 Pro and each evaluator gave the same verdict on the 100 responses. Agreement was at least 96% on this sample.

Evaluator compared with Gemini 3.1 ProAgreement
Human evaluator 199%
Human evaluator 299%
Human evaluator 398%
Qwen3.5-397B-A17B96%
GPT-6.1 Sol99%

Two features of the evaluation help explain the high agreement on this sample. Each question has a single verifiable factual answer, even when several phrasings are acceptable. The judge receives the question, the reference answer, and only the final answer from the evaluated model. It checks whether that final answer conveys the required facts and accepts equivalent wording. This gives all judges the same evidence and a well-defined decision to make.

Reviewer_oZaeOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces StreamArena, a benchmark for evaluating continuous, interactive video understanding over hour-scale audiovisual streams. It contains 243 videos averaging 88.8 minutes and 3,646 manually annotated, open-ended question-answer pairs covering four capabilities: real-time multimodal perception, historical retrospection, proactive interaction, and multimodal tool use. Queries and supporting evidence are timestamped to support causal evaluation and assess long-range dependencies. The authors also propose StreamMind, a multi-worker framework that separates user interaction and proactive monitoring from asynchronous memory construction, historical recall, and external search. Its persistent memory combines hierarchical event descriptions, entity relations, and key frames, enabling the system to retrieve historical evidence and coordinate subsequent tool calls. Experiments compare offline MLLMs and several classes of streaming systems, with StreamMind achieving the highest accuracy among the evaluated streaming baselines across all four capabilities. A comparison using the same backbone further shows substantially lower query-to-answer latency, accompanied by a reduction in pooled accuracy on reactive tasks.

Critical strengths
  1. A valuable benchmark for long-horizon streaming interaction. StreamArena provides 3,646 manually annotated questions across 243 hour-long videos, covering real-time perception, historical recall, tool use, and proactive response. Its temporally grounded annotations and causal evaluation protocol provide a useful testbed for assessing sustained interaction beyond short, isolated video questions.

  2. A unified framework with improved responsiveness. StreamMind coordinates continuous memory construction, historical retrieval, external search, and event monitoring through specialized workers. Using the same backbone, it reduces average reactive response latency from 81.4 s to 27.5 s while retaining most of the offline accuracy (48.6% versus 54.2%), demonstrating a practical trade-off between responsiveness and answer quality.

Critical weaknesses
  1. Insufficient evidence for the framework’s design choices. StreamMind uses a substantially larger backbone than most streaming baselines, making it difficult to separate the gains from model capacity and architectural design.
  2. Insufficient validation of annotation reliability. Although the paper describes multiple rounds of human validation, it does not report inter-annotator agreement for answers or temporal annotations. The reported 73% retention rate is also not accompanied by clear criteria for revising versus discarding drafts or resolving annotator disagreements. Reporting agreement statistics and clarifying these procedures would strengthen confidence in the benchmark’s reference annotations, particularly for proactive tasks whose scores depend on precise event timing.
Questions for authors

See weakness, please.

2 comments
AuthorsOpenReview ↗

Response to Weakness 1 (1/2)

Response to W1. Streaming methods with the same 397B backbone

Thank you for recognizing the value of StreamArena and for raising the question of how much backbone scale contributes to the results. In Table 4, we use each baseline’s released checkpoint and inference procedure. For AURA and VST, those checkpoints incorporate method-specific fine-tuning. Simply replacing them with a larger model would discard what the original models learned during that training. We therefore evaluated the released systems first, while recognizing that their scores reflect both the backbone and the method.

To compare the methods at the same model scale, we have adapted the inference procedures of AURA, VST, StreamForest, and ThinkStream to the same Qwen3.5-397B-A17B checkpoint used by StreamMind, without additional fine-tuning. In these 397B runs, AURA uses at most 30 recent frames, while VST uses at most 8,192 visual tokens per segment, following their respective original settings. The comparison covers all 243 StreamArena videos (3,646 questions), with the same scoring rules for every method.

MethodRTPHR avg.HR L1HR L2HR L3HR L4ToolPro. avg.Pro. L1Pro. L2Pro. L3
AURA35.424.529.827.025.413.8-8.915.46.42.2
VST36.930.231.535.533.016.6-----
StreamForest †25.518.424.317.618.413.8-----
ThinkStream †12.910.315.510.68.16.64.53.75.83.11.1
StreamMind44.534.931.546.734.617.156.111.616.69.57.5

Accuracy is reported in percent. HR L1–L4 group questions by the time from supporting evidence to the question (≤5 min, 5–15 min, 15–30 min, and >30 min). Pro. L1–L3 group questions by the time from the monitoring instruction to the target event (≤30 s, 30 s–4 min, and >4 min).

A dash (-) denotes a capability unsupported by the method’s evaluation interface.

With the backbone held fixed, StreamMind reaches 34.9% HR accuracy, compared with 30.2% for VST, the strongest baseline on this task.

† With the 397B backbone, HR accuracy improves by 4.0 percentage points for StreamForest and 2.8 for ThinkStream. These gains may be modest because we used the checkpoint without training it to use StreamForest’s merged visual tokens or the reasoning tokens that ThinkStream keeps in its KV cache after evicting older visual tokens.

We have added the comparison using the shared 397B backbone to Appendix C.9 (Table 13).

AuthorsOpenReview ↗

Response to Weakness 2 (2/2)

Response to W2. Annotation reliability and retention

Thank you for highlighting the value of StreamArena for sustained interaction and StreamMind’s balance between response time and answer quality. Below, we clarify how disagreements between annotators are resolved and how the final audit determines which questions to retain.

Two annotators first solve each question independently, before seeing the drafter’s answer or explanation. For proactive questions, each marks the time at which the condition in the monitoring instruction is satisfied. They then compare their solutions with each other and with the draft. If they disagree, they revisit the video together and, for tool-use questions, compare the sources supporting their answers. They keep the draft unchanged if the evidence confirms it, or revise the wording, answer, or timestamp if they can resolve the discrepancy. If they still disagree or cannot verify the answer or event time, they flag the question and pass their proposed answers and supporting evidence to the final reviewer.

A third reviewer audits every item. For each revised or flagged question, the reviewer first solves it independently before seeing the earlier answers or reasoning. The reviewer then checks whether the revised answer can be reproduced or, for a flagged question, whether the evidence resolves the disagreement. We discard any question whose answer or, for proactive questions, target event time remains uncertain. Each retained proactive question has one reference timestamp.

Approximately 73% of the initial drafts were retained through this process. We have added the revision and retention criteria to Section 2.1.1 (lines 169–175) and a description of the adjudication procedure to Appendix B.1 (lines 690–703).

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response to the Follow-up Question on Backbone and Judge Selection

Thank you for the follow-up question. Qwen3.5-397B-A17B serves as StreamMind’s backbone, and Gemini 3.1 Pro serves as the judge for all benchmark results reported in the paper. Using a separate judge avoids having StreamMind’s backbone grade its own answers.

In the rebuttal, we compared Gemini’s verdicts with those of Qwen and GPT-6.1 Sol to give readers additional points of reference. We also assessed how closely Gemini’s verdicts aligned with independent human judgments. On the same 100 responses, Gemini agreed with the three human evaluators in 99%, 99%, and 98% of cases, respectively. This agreement supports our choice of Gemini as an independent judge and provides evidence for the reliability of the scores we report.

AuthorsOpenReview ↗

Summary of Rebuttal Responses and Discussion Status

Summary of Rebuttal Responses and Discussion Status

Dear ACs, SACs, and PCs,

Thank you for your time and effort. We summarize the additional evidence addressing the reviewers’ concerns and the discussion status below.

ReviewerInitial ratingInitial concernsOur rebuttal and additional evidenceReviewer’s further feedbackOur further reply and current status
oZae2Backbone fairness; annotation reliability and retention criteria.Compared four baselines with StreamMind using the same 397B backbone across 243 videos. StreamMind achieves 34.9% HR, versus 30.2% for the strongest baseline (Table 13). Clarified verification by two independent annotators, adjudication by a third reviewer, and the criteria underlying 73% retention.No follow-up received.We consider the concerns addressed. No further questions or objections received.
TpSn2Backbone fairness; component ablations; resource costs; judge reliability.Added shared-backbone comparisons (Table 13), memory ablations on 877 HR questions (Table 12), and resource profiling: 304 background calls per video-hour and 28.73 MiB mean final memory per video (Table 11). On 100 responses, judge agreement with three human evaluators is 99%, 99%, and 98%; agreement with alternative judges is 96% and 99%.Raised rating from 2 to 3. Asked whether Qwen serves as both backbone and judge.Clarified: Qwen is the backbone; Gemini 3.1 Pro is the benchmark judge. Alternative judges were used only for the agreement study. Follow-up answered; no further questions received.
deue3Backbone fairness; component contributions and mechanisms.Added shared-backbone comparisons and controlled memory ablations (Tables 12–13). Removing keyframes, event hierarchy, and entity graph reduces HR by 7.5, 4.8, and 0.8 percentage points, respectively. Removing hierarchy reduces long-horizon L4 accuracy from 17.1% to 11.6%. Explained the mechanisms behind these results.Acknowledged useful ablation evidence, but requested further justification of the Writer, coordination, and novelty.Explained the Writer’s role and expanded mechanism analysis. Clarified novelty: 3,646 manually authored tasks, videos averaging 88.8 minutes, and joint evaluation of four capabilities, supported by coordinated memory, monitoring, and retrieval. We consider both follow-up concerns addressed; no further response received.

Best regards,
All Authors of Submission #10590

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.