acceptodds
Under review as a conference paper at ICLR 2027

OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction

15 views

Abstract

Streaming video LLMs must retain evidence before its relevance to future tasks is known and respond when sufficient evidence becomes available. The challenge is to form reusable factual memory without compromising real-time perception. We introduce OneStreamer, which jointly learns query-independent evidence recording and task response through a shared proactive generation process. Its Proactive Hierarchical Caption Memory (PHCM) produces time-grounded local-detail captions and summaries of completed events. Streaming caption targets supervise the interpretation of observed video prefixes during training. At inference, model-generated records complement a recent visual window, providing reusable factual context without revisiting historical visual features. Proactive State Transition Learning (PSTL) reduces the dominance of repeated waiting states by preserving supervision at all output anchors and selecting representative state-change and state-persistence tokens. We further develop a streaming data synthesis pipeline that aligns output content and timing with available evidence. Combining the resulting streaming captions and QA with cleaned open-source data yields OneStreamer-1M, a broad-coverage streaming video interaction dataset with over one million records spanning diverse tasks. Our 4B model achieves the best results among the compared methods across all eight evaluated streaming video understanding benchmarks. Ablations show that retaining generated captions improves historical QA without degrading real-time perception. PSTL also outperforms dense state supervision while supervising only 27.5% of annotated state tokens. Together, these results support proactive generation as a shared learning interface connecting perception, memory formation, and timely response in streaming video interaction.

Primary area:
foundation or frontier models, including LLMs
Keywords:
MLLM, Streaming Video Understanding, Interaction Model, Online Video

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.60 over 5 reviews, above 99% of the others.
Reviewer_fTrgOpenReview ↗
Rating
4
Confidence
4(+1)
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes OneStreamer, a streaming video LLM built on Qwen3-VL-4B. Using a set of control tokens, it unifies three functions within a single causal generation process: recording evidence, waiting, and responding. The paper makes three main contributions: 1.PHCM: The model proactively generates time-stamped local-detail captions and event-level summaries, which serve as textual memory. At inference, this memory is concatenated with a Recent-N FIFO visual window, so historical visual tokens no longer need to be retained. 2.PSTL: A selective supervision scheme for the state-token loss. All output anchors are kept, while the remaining transition groups are subsampled using a quota defined by the largest number of transitions into an anchor. This alleviates the class imbalance caused by the dominance of silence tokens. 3.OneStreamer-1M: A streaming data synthesis pipeline consisting of a caption branch and a QA branch with response-time calibration. The synthesized data is combined with cleaned open-source data.

Critical strengths
  1. Strong experiments and valuable data. The empirical results are very strong. The proposed data pipeline and the OneStreamer-1M dataset would be a substantial contribution to the community.
  2. Solid ablations. The ablations cover both data and method. The control-token ablation in particular offers useful insights into how proactive response is learned.
  3. Effective PHCM design. PHCM is naturally integrated into training, which avoids the train–inference gap of prior memory methods. Ablations show that it reduces computation and extends the accessible temporal range, while maintaining real-time performance comparable to the recent-window-only baseline.
  4. Interesting state-transition formulation. Modeling response control as state transitions offers a simple and effective new approach for the streaming community.
Critical weaknesses
  1. Repetitive caption outputs. Since captions are constructed segment by segment, each segment gets its own independent description. This may lead to repetitive captions when an action spans multiple segments. Unlike humans, the model may not learn to say that an action is still ongoing, or simply stay silent, and may instead describe the same action again in every segment.
  2. Limited evaluation of long-term memory. The paper does not evaluate on long streams such as hour-long videos, and there is no mechanism to handle the growing caption memory. It is also unclear how accurate the generated captions are, and how errors in memory affect later answers.
  3. Lack of memory correction. Early segments may sometimes lead to a wrong interpretation, for example when an action or scene is ambiguous at first and only becomes clear later. However, the model has no ability to correct earlier caption records based on later evidence, so such errors remain in memory and may mislead subsequent answers.
Questions for authors
  1. How are the KV cache and positional encoding handled when evicted frames are replaced by captions, and does training simulate this eviction to avoid a train–inference mismatch?
  2. Since PHCM is only applied on two benchmarks, does enabling it on the other tasks or in real-world use cause any degradation, for example in proactive response timing or latency?
7 comments

We thank the reviewer for the careful evaluation of our work. We address each concern below.

W1: Repetitive caption outputs. Since captions are constructed segment by segment, each segment gets its own independent description. This may lead to repetitive captions when an action spans multiple segments. Unlike humans, the model may not learn to say that an action is still ongoing, or simply stay silent, and may instead describe the same action again in every segment.

We thank the reviewer for this comment and clarify how captions are constructed. PHCM does not require a caption for every input segment. The frame- or clip-level captions used to construct </Observe> targets are associated with annotated temporal intervals, and each target is released only after its supporting interval ends. Caption release times therefore follow the annotated intervals rather than a fixed schedule. At inference, the model conditions its writing decisions on the current visual input and retained textual history, and can generate </Silence> without writing to memory. Access to preceding records allows writing decisions to take earlier observations into account. The model is thus not required to describe each segment independently. In the example reported in Table 19, it generates 26 local-detail records over a 360-second stream, corresponding to approximately one record per 14 seconds on average. This example illustrates that memory writing can be sparse rather than occurring at every update.

W2: Limited evaluation of long-term memory. The paper does not evaluate on long streams such as hour-long videos, and there is no mechanism to handle the growing caption memory. It is also unclear how accurate the generated captions are, and how errors in memory affect later answers.

We thank the reviewer for raising these concerns about long-term memory evaluation and reliability. We have conducted additional duration-stratified experiments and clarify below how caption memory is constructed, its current capacity limitations, and its utility for subsequent QA.

Scalability of long-video memory: We agree that hour-long video streams are important for evaluating long-term memory. Given the limited availability of standardized hour-long streaming benchmarks, we conducted additional experiments on OVOBench-Backward to examine PHCM across the video durations available in this benchmark. We divided the benchmark into three duration groups and measured memory length, GPU memory consumption, inference latency, and QA accuracy on a single NVIDIA H200 GPU. We compared PHCM with a Full Video baseline that retains the complete observed video history sampled at 4 FPS. The results are shown below:

Video LengthAvg. DurationMemory Tokens (PHCM)GPU Memory (PHCM)Latency (PHCM)Full Video Acc.PHCM Acc.Δ
Short (≤90s)37.5s2959.076 GiB0.118s80.581.1+0.6
Medium (90–300s)182.4s1,3199.374 GiB0.155s71.676.1+4.5
Long (>300s)430.3s4,48610.059 GiB0.260s65.473.7+8.3

Across the three duration groups, PHCM's memory length and inference latency increase with video duration, while GPU memory consumption rises from 9.076 GiB to 10.059 GiB. PHCM outperforms the Full Video baseline in all three groups, with the accuracy gap increasing from 0.6 to 8.3 percentage points. These results support the utility of compact textual records for retaining historical information within the evaluated duration range.

Regarding memory capacity, PHCM generates an average of approximately 558.1 memory tokens per minute of video on OVOBench-Backward. The longest video in this benchmark is 19 minutes and 58 seconds and produces 8,920 tokens of textual memory. Extrapolating this average rate to the base model's 256K context window gives a token-budget estimate of approximately 7.6 hours. Our current evaluations do not reach the context limit, and we do not employ an explicit memory eviction policy. A possible extension for longer streams would be to selectively evict earlier local-detail records (</Observe>) while retaining their corresponding event-level summaries (</Summary>) as the context approaches its capacity limit. This could reduce context consumption while retaining coarser descriptions of historical events.

Caption reliability and its impact on subsequent QA: During data construction, we use Gemini and Seed to generate timestamp-grounded captions at multiple temporal granularities. These training annotations are verified against the source videos for factual accuracy and temporal consistency. Captions containing unsupported content, incorrect temporal boundaries, or inconsistencies across annotation granularities are filtered out. At inference, PHCM-generated captions serve as explicit historical context for subsequent QA. Downstream QA therefore provides an end-to-end measure of their utility. In the duration-stratified experiments above, PHCM achieves an accuracy of 73.7 in the Long group, outperforming Full Video by 8.3 percentage points. This supports the usefulness of generated records as historical context over the evaluated video durations.

We will include these duration-stratified experiments and the accompanying discussion in the revised manuscript.

W3: Lack of memory correction. Early segments may sometimes lead to a wrong interpretation, for example when an action or scene is ambiguous at first and only becomes clear later. However, the model has no ability to correct earlier caption records based on later evidence, so such errors remain in memory and may mislead subsequent answers.

We thank the reviewer for pointing out this limitation. PHCM currently uses append-only memory: newly generated timestamped captions are appended to the existing caption history without modifying or deleting previous records. When early observations are ambiguous or incorrectly described, subsequent evidence does not automatically trigger revisions to those records.

Later records can provide additional evidence that may help resolve earlier ambiguities during QA. For example, if an action becomes clearer in later frames, the model can append observations describing that evidence. OneStreamer then answers questions using the accumulated local descriptions, summaries of completed events, and recent visual evidence. Any resolution of conflicting records therefore depends on the model's ability to integrate evidence across time during answer generation. Explicit memory revision remains important directions for future work.

We will clarify the scope of PHCM's append-only memory in the revised manuscript and explicitly discuss the lack of memory revision as limitations of the current approach.

Reviewer_EhLEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

OneStream is a streaming video LLM that connects real-time perception, memory-based QA, and proactive QA through a shared format by learning when and what to record or answer, using 4 types state tokens: <Observe>, <Summary>, <Silence>, and <Response>. The authors construct OneStream-1M, which annotates new caption and QA dataset from multiple video domains with existing streaming datasets. Their method includes two main components:

  1. Proactive Hierarchical Caption Memory (PHCM): learns when to write local observations and event summaries in text, retaining these caption records with a fixed sliding window of recent visual inputs
  2. Proactive State Transition Learning (PSTL): selects state tokens for loss to reduce the effect of dominant silence tokens while preserving supervision for memory-writing and response states.

Compared with existing recent streaming video LLMs, OneStream has strong performance across 8 streaming video LLM benchmarks. Ablations on PHCM compared with vision-only memory and the experiment on state loss design, and training data shows effectiveness of their method.

Critical strengths
  1. A unified design for recording and responding: OneStreamer trains one model with state tokens to write caption records and generate responses. It's aligned training data includes perception, memory QA, and proactive interaction.

  2. Broad evaluation with useful ablations: The paper compares recent streaming video LLMs across 8 benchmarks. Ablations of their designs, caption retention, PSTL, and data curation, provide evidence for their method's effectiveness.

Critical weaknesses

The benefit of learning when to write memory remains unclear relative to the closest prior methods. Think While Watching[1] uses also the same model to write a caption for each video segment and answer later questions from those notes. VST[2] also stores pre-query text at specified visual boundaries. OneStreamer learns when to record, and it showed that reatining these records helps, but do not compare this with periodic writing and with a record budget.


References

[1] Lu Wang et al., Think While Watching: Online Streaming Segment-Level Memory for Multi-Turn Video Reasoning in Multimodal Large Language Models, arXiv:2603.11896, 2026.

[2] Yiran Guan et al., Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously, arXiv:2603.12262, 2026.

Questions for authors
  • Regarding ODVBench result in Table 1, the public VideoChat3 paper reports 72.3, which would change the ranking against OneStreamer's 71.3 under the same protocol. What is the difference in the evaluation setting?
4 comments

We are very grateful for your constructive comments, which have been instrumental in improving our paper. The following is a point-by-point response to the reviewer's comments.

W1: The benefit of learning when to write memory remains unclear relative to the closest prior methods. Think While Watching[1] uses also the same model to write a caption for each video segment and answer later questions from those notes. VST[2] also stores pre-query text at specified visual boundaries. OneStreamer learns when to record, and it showed that reatining these records helps, but do not compare this with periodic writing and with a record budget.

We thank the reviewer for raising this important concern. We agree that Think While Watching and VST have already explored generating and retaining pre-query textual information during video streaming. Therefore, the contribution of OneStreamer is not simply to introduce streaming textual memory, but to further investigate whether a model can autonomously decide when to write memory and at what granularity based on the visual evidence observed so far.

Specifically, Think While Watching generates segment-level memory for individual video segments, while VST accumulates historical semantic information through streaming thinking. In contrast, our PHCM explicitly formulates memory writing as a model-predicted control state: </Observe> records local facts from the currently observed content, while </Summary> summarizes completed events or segments. Rather than writing records at predefined temporal intervals, the model autonomously determines whether and when to generate each type of record based on the ongoing visual evidence. This allows memory writing to adapt to variations in scene changes and event density across different videos.

To directly examine whether periodic writing can achieve comparable performance under the same record budget, we conduct an additional ablation using Periodic Writing. This baseline adapts Think While Watching’s one-note-per-segment writing scheme to our model. For each question’s observed video interval, we set K to the number of valid cached PHCM records, counting both Observe and Summary records. The Periodic Writing divides this interval into K equal-duration segments and writes one note after each segment. Each note is newly generated from the current segment (sampled at 4 fps with a 128-frame cap) and the previous notes. All other settings including the model and inference configuration remain unchanged. The results are as follows:

VariantOVOBench (Overall)StreamingBench (Real-Time)
Periodic Writing70.886.5
PHCM (Ours)72.186.9

Under the same record budget, PHCM achieves gains of 1.3 and 0.4 percentage points on OVOBench and StreamingBench, respectively. These results suggest that allowing the model to autonomously decide when to write to memory based on ongoing observations further improves memory utilization while maintaining real-time perception performance.

Furthermore, existing ablations in our paper provide complementary evidence for the PHCM design. Table 2 shows that retaining Caption Memory under the same Recent-16 visual window improves OVOBench Backward ASI from 63.5 to 71.6, demonstrating that historical records provide useful evidence for subsequent questions. Table 18 further demonstrates the complementary roles of the two recording granularities: using only Observe, only Summary, and both achieves OVOBench Overall scores of 71.7, 71.2, and 72.1, respectively, with the complete hierarchical memory achieving the best result.

In summary, our experiments further demonstrate that the benefit of PHCM goes beyond continuously retaining pre-query textual information. By allowing the model to autonomously control memory-writing times and combining local factual observations with event-level summaries, PHCM constructs reusable historical memory that adapts to the content and temporal structure of different videos. We will further clarify the methodological distinctions and contributions of OneStreamer relative to Think While Watching and VST in the revised manuscript.

Q1: Regarding ODVBench result in Table 1, the public VideoChat3 paper reports 72.3, which would change the ranking against OneStreamer's 71.3 under the same protocol. What is the difference in the evaluation setting?

Thank you for carefully checking these results. Since the VideoChat3 paper does not provide a detailed breakdown of results by benchmark subtask, we re-evaluated its official checkpoint in our local environment. The dagger (†) indicating re-evaluated results was omitted from the original manuscript. We apologize for this oversight and will correct it in the revised version.

We carefully checked our evaluation configuration and confirmed that the context length, frame sampling rate, and maximum number of frames match the official VideoChat3 evaluation settings. We suspect that the discrepancies may be related to differences in hardware or software environments. The VideoChat3 paper does not report the GPU model used for evaluation, and its public repository does not specify exact versions of dependencies such as PyTorch, making it difficult to reproduce the original evaluation environment exactly.

The table below compares the results reported in the VideoChat3 paper with our re-evaluation. Even when using VideoChat3’s reported results, OneStreamer scores slightly lower on ODVBench (71.3 vs. 72.3) but consistently outperforms VideoChat3 on the other five benchmarks listed. These differences therefore do not alter our main conclusion regarding OneStreamer’s overall performance.

BenchmarkOVOBenchStreamingBenchOVBenchODVBenchProactiveVideoQAOVO-TimingAvg.
VideoChat3 (reported)57.883.060.072.337.635.557.7
VideoChat3 (re-evaluated)58.581.962.570.837.633.657.5
Difference (re-evaluated − reported)+0.7−1.1+2.5−1.50.0−1.9−0.2
OneStreamer72.186.966.871.348.741.664.6

Thank you for the additional experiments and for clarifying the VideoChat3 results.

Provided comparison helped to address my concern about the benefit of learning when to write memory. It would be helpful if you could specify the visual context and record-generation settings for the periodic baseline in the revised paper.

I will keep my rating unchanged.

Dear Reviewer EhLE,

Thank you for your follow-up comments and for acknowledging that our additional experiments and clarifications regarding the VideoChat3 results have addressed your concerns. We agree that the revised manuscript should explicitly specify the visual context and record-generation settings of the periodic-writing baseline. We will incorporate these details into the revised manuscript to ensure that the comparison protocol is clearly described and reproducible.

Reviewer_ufKWOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper addresses streaming video understanding by jointly modeling real-time perception, reusable memory, and proactive response. The authors argue that retaining historical visual tokens increases context overhead, while sliding windows discard evidence needed for future queries. To address this, the paper proposes OneStreamer, a unified streaming framework built on Qwen3-VL-4B-Instruct that (i) introduces Proactive Hierarchical Caption Memory (PHCM) to retain time-aligned local captions and event-level summaries alongside recent visual frames, (ii) develops Proactive State Transition Learning (PSTL) to preserve output-triggering states while selectively supervising state transitions and persistence, and (iii) constructs a streaming data synthesis pipeline that aligns captions and response timing with available visual evidence. The resulting OneStreamer-1M dataset combines newly constructed and curated open-source data, totaling approximately 1.16 million records. Experiments across eight benchmarks report the highest aggregate scores among compared methods, while ablations show improved historical QA through caption memory and stronger proactive response performance while supervising only 27.5% of annotated state tokens.

Critical strengths
  1. A useful formulation with supporting memory ablations. Fig. 2 and Fig. 3 make the paper's motivation concrete: a streaming model must preserve evidence before queries arrive, maintain current visual understanding, and decide when to respond. Although these ideas are individually familiar, organizing memory writing and response control within one causal generation process is a useful formulation. Tab. 2 shows that retaining generated captions improves historical action-sequence understanding from 63.5 to 71.6 while maintaining comparable real-time perception (80.9 vs. 81.4). Tab. 17 further shows that dedicated caption supervision improves the usefulness of generated memory (+2.0 vs. −0.1). These controlled comparisons provide meaningful evidence for the memory component beyond cross-system benchmark results.

  2. Substantial data construction and practical evaluation. Sec. 4 and Appendix B describe approximately 1.16 million records combining newly constructed examples with curated existing data. The pipeline includes evidence verification and response-time calibration, providing a concrete approach to streaming supervision. In Tab. 20, the caption-quality and timing-calibration stage improves OVO-Timing from 33.0 to 42.0 F1 without adding training samples. The resulting 4B model also substantially outperforms its Qwen3-VL-4B baseline (+12.2 F1 on OVO-Timing). Tab. 19 further reports an average online update time of 0.636 s under a 1 s update interval, supporting practical feasibility in the tested setting.

Critical weaknesses
  1. The paper's central claim of unifying perception, memory, and proactive response is not directly validated under joint operation. Appendix D describes caption-memory inference for OVOBench and StreamingBench but does not establish its use in proactive-response evaluations. Fig. 3 depicts simultaneous memory writing and response decisions, while Tab. 5 defines task-specific control states without explaining how competing decisions are handled. Furthermore, the necessity of the specific mechanisms remains unclear. In Tab. 18, adding event summaries to local captions improves OVOBench by only 0.4 points and provides no gain on StreamingBench. Although Tab. 3 shows improvements over focal loss, standard class-balanced sampling and weighted CE baselines are missing. The paper demonstrates several useful capabilities but does not sufficiently establish the benefits of their unification or the necessity of the proposed designs.

  2. The claim of best results across all eight benchmarks requires qualification, and the overall gains are not cleanly attributable to the proposed memory mechanism. In Tab. 8, HLD increases from 41.9 to 99.5, accounting for approximately 6.4 of the 13.3-point OVOBench improvement over Qwen3-VL. Excluding this abstention-oriented subtask gives approximately 68.5, but whether OneStreamer retains its lead is unclear without corresponding results for competing models. On OmniMMI, the reported 36.6 uses ASR for four tasks, whereas Tab. 20 reports 30.6 without ASR, below MOSS-VL-Realtime's reported 32.7; baseline input parity remains unclear. Moreover, the model without proactive caption supervision already reaches 70.0 on OVOBench (Tab. 16), compared with 58.8 for the original Qwen3-VL, while enabling caption memory adds only 2.0 points (Tab. 17). Finally, Full and FIFO differ in frame sampling rate (Tab. 16), and a token-matched visual-memory baseline is missing, weakening conclusions about the advantages of textual memory over visual history.

Questions for authors
  1. Fig. 3 presents memory writing and proactive response as part of one generation process. Appendix A.2 suggests that proactive QA uses the same history, while Appendix D only explicitly describes caption-memory inference for OVOBench and StreamingBench. Could the authors clarify whether memory writing was enabled for the proactive benchmarks and, if not, evaluate them with memory writing running concurrently, reporting response accuracy, timing, and latency against the same model without memory writing? How are competing memory-writing and response decisions handled under the task-specific control states in Tab. 5?

  2. In Tab. 8, HLD reaches 99.5 and accounts for approximately 6.4 of the 13.3-point OVOBench improvement over Qwen3-VL. Could the authors report OVOBench Overall excluding HLD for all compared models where possible, the HLD score of a trivial always-abstain policy, and how often OneStreamer chooses an abstention option when one is added to answerable Backward Tracing questions? Additionally, did all OmniMMI baselines receive ASR transcripts? Without ASR, OneStreamer scores 30.6 (Tab. 20), below the 32.7 reported for MOSS-VL-Realtime, so a matched-input comparison is needed.

  3. In Tab. 16, the model without proactive caption training already achieves 70.0 on OVOBench, while enabling caption memory in the full model adds 2.0 points (Tab. 17). Could the authors compare against (a) a training-budget-matched variant in which the 60K caption examples are replaced by an equal number of non-caption streaming examples, (b) a token-matched visual-memory baseline (Recent-16 plus approximately 1.3K tokens of downsampled historical frames), and (c) FIFO answering with the same 128-frame, 4 fps context used to construct memory (Appendix D)? These comparisons would separate the contributions of caption supervision and textual memory from extra training data, memory representation, and denser visual observations.

  4. Tab. 3 shows that the proposed state supervision achieves 41.6 F1 on OVO-Timing, but the random sparse baseline reaches only 1.4 despite using the same 27.5% supervision ratio. Could the authors add class-balanced random undersampling at the same ratio and inverse-frequency weighted cross-entropy? This would clarify whether the improvement comes from transition-specific selection or simply correcting the imbalance between waiting and response states.

10 comments

We would like to extend our heartfelt gratitude to the reviewer for your insightful comments and valuable feedback. Your questions have significantly contributed to the improvement of our work, and we appreciate the opportunity to address them.

W1 (Part 1): The paper’s central claim of unifying perception, memory, and proactive response is not directly validated under joint operation. The paper demonstrates several useful capabilities but does not sufficiently establish the benefits of their unification or the necessity of the proposed designs.

We thank the reviewer for highlighting the need to clarify the benefits of our unified framework. OneStreamer uses proactive response as a shared interface through which a single model learns both factual recording and task response by predicting control tokens and their associated text. A key connection is that streaming caption proactive generation provides supervision for interpreting observed video during training and produces reusable factual memory at inference. Perception learning and memory formation are therefore connected through the same proactive response process, while state supervision supports learning when to produce outputs. Our ablations provide evidence for the following three roles:

  • Proactive caption supervision supports perception learning. Caption targets require the model to generate local descriptions and event summaries grounded in visual evidence observed so far. Table 16 evaluates all settings without caption memory, examining the contribution of caption supervision independently of inference-time memory use. Adding this supervision improves performance across all six reported visual-understanding settings. For example, StreamingBench improves from 85.8 to 86.3 under FIFO evaluation. These results support the value of streaming caption targets as supervision for video understanding.

  • Generated factual records provide reusable historical memory. At inference, descriptions generated by the same model are retained as context for subsequent QA after the relevant frames leave the recent visual window. Table 17 further connects caption supervision with memory use. Under the same fact-recording prompt and inference settings, retaining generated records improves OVOBench by 2.0 points for the full model, whereas the corresponding change is −0.1 points for the variant trained without caption data. These results indicate that dedicated caption supervision helps the model benefit more from the records it generates.

  • Selective state supervision supports proactive response learning. Factual recording and task response share a generation format in which control tokens initiate textual outputs. PSTL uses task-specific output anchors to select state supervision, preserving all output-initiating tokens while selecting examples of both state changes and state persistence. Table 3 shows that PSTL outperforms the evaluated dense, random sparse, and transition-only supervision strategies while supervising only 27.5% of annotated state tokens. For example, it achieves 41.6 on OVO-Timing, compared with 26.5 for frequency-balanced focal loss, supporting the value of this supervision strategy for proactive response learning.

These results support the practical value of a shared generative interface connecting perception supervision, memory formation, and response learning. In particular, the complementary roles of proactive caption generation during training and inference provide direct evidence for the connection between perception and memory. Our unification claim concerns these shared modeling and information-reuse mechanisms.

W1 (Part 2): Furthermore, the necessity of the specific mechanisms remains unclear. In Tab. 18, adding event summaries to local captions improves OVOBench by only 0.4 points and provides no gain on StreamingBench.

We thank the reviewer for highlighting the need to clarify the incremental contribution of event summaries. Summary records are designed to capture the overall semantics of completed events or segments, complementing local factual observations. Observe records describe specific objects, actions, and state changes, while Summary records organize observed content into coherent event-level descriptions that provide explicit context for subsequent tasks. For questions requiring connections across multiple observations or events, this representation allows the model to draw on an integrated view of each segment without having to reconstruct all event-level structure from individual local records.

Table 18 also provides evidence for the utility of event summaries as memory. Retaining only Summary records improves OVOBench from 70.1 to 71.2 and StreamingBench from 86.3 to 86.6 relative to using no caption memory. These gains of 1.1 and 0.3 points indicate that event summaries retain useful information for subsequent understanding even when local observation records are not retained. Combining Summary with Observe further raises OVOBench to 72.1. These results suggest that the two granularities provide complementary information and that combining them can improve performance.

In addition, Summary records offer a potential option for memory retention under limited context budgets. When the text budget cannot accommodate the complete history, one could prioritize event summaries and progressively discard local observations already covered by those summaries, preserving the overall semantics of past events at a coarser granularity. Such a policy could offer a flexible trade-off between local detail and temporal coverage, potentially retaining useful information over longer periods within a fixed context budget.

W1 (Part 3): Appendix D describes caption-memory inference for OVOBench and StreamingBench but does not establish its use in proactive-response evaluations. Fig. 3 depicts simultaneous memory writing and response decisions, while Tab. 5 defines task-specific control states without explaining how competing decisions are handled.

Q1: Fig. 3 presents memory writing and proactive response as part of one generation process. Appendix A.2 suggests that proactive QA uses the same history, while Appendix D only explicitly describes caption-memory inference for OVOBench and StreamingBench. Could the authors clarify whether memory writing was enabled for the proactive benchmarks and, if not, evaluate them with memory writing running concurrently, reporting response accuracy, timing, and latency against the same model without memory writing? How are competing memory-writing and response decisions handled under the task-specific control states in Tab. 5?

We thank the reviewer for raising this important question about the joint operation of memory writing and proactive response.

Evaluation setting: In our original evaluation, memory writing is disabled for proactive-response benchmarks. This choice follows the nature of these tasks: questions or instructions are typically provided before the target event occurs, and the evidence needed for a response primarily comes from subsequent video observations. Historical caption memory is expected to provide limited direct benefit for these tasks, while additional memory records may introduce context overhead and affect response decisions. We adopt the same FIFO evaluation setting as existing streaming proactive-response methods to assess the model's ability to respond based on incoming visual evidence.

Joint-operation evaluation: To directly examine the effect of enabling memory writing during proactive response, we additionally evaluate the same OneStreamer checkpoint with PHCM memory writing enabled.

MethodProactiveVQAOmniMMIOVO-TimingViSpeak
OneStreamer (without memory writing)48.736.641.62.87
OneStreamer (with PHCM)48.136.438.02.88
Δ-0.6-0.2-3.6+0.01

Enabling memory writing leads to largely stable performance on ProactiveVQA, OmniMMI, and ViSpeak. OVO-Timing shows a larger decrease of 3.6 points, indicating that timing-sensitive response decisions can be more affected by the additional context introduced through memory writing. Overall, these results demonstrate that memory writing and proactive response can operate within the same streaming interaction, although their joint operation introduces task-dependent trade-offs.

Latency analysis: We further measure the inference latency using 130 duration-stratified samples drawn from the four proactive-response benchmarks. Each sample is evaluated three times.

SettingMemory Writing (s)Response Decision (s)
FIFO (without memory writing)—0.2513
PHCM (with memory writing)0.34120.3942

Under sequential execution, PHCM incurs a cumulative latency of 0.7354 s, compared with 0.2513 s for FIFO. However, the two stages do not necessarily need to execute sequentially on the critical path of user response. Since the recent visual window already retains newly observed frames, we can prioritize response generation using the recent visual window and existing memory, while deferring memory writing to idle periods. The deferred writing can be completed before its source frames leave the recent visual window, preserving the evidence needed for subsequent memory construction. This response-first scheduling strategy offers a practical way to reduce user-perceived response latency without sacrificing historical evidence retention.

Handling task-specific control states: Memory writing and proactive response do not compete for the same control-state decision. Instead, we implement their joint operation as two sequential inference stages. First, the model processes the current video context in the memory-writing stage and generates </Observe>, </Summary>, or </Silence> to update PHCM. Second, the updated caption history, together with the recent visual window and user query, is provided to the same model for the response-decision stage, which predicts </Silence>, </Standby>, or </Response>. Thus, the two stages use their respective task-specific control-state. This inference procedure also explains why the original training examples use task-specific state supervision: memory-writing and response tasks are supervised independently, rather than through jointly annotated trajectories.

We apologize that the original presentation did not clearly distinguish task-specific evaluation from joint inference. We will clarify the evaluation settings in Appendix D, describe the two-stage joint-inference procedure in Appendix A.2, and include the joint-operation results and latency measurements in the appendix.

Reviewer_MLgtOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents OneStreamer, which combines hierarchical caption memory (PHCM), selective state-token supervision (PSTL), and the OneStreamer-1M training dataset for streaming video perception, memory, and proactive response. Built on Qwen3-VL-4B, the model achieves the highest aggregate scores among the compared methods across eight evaluations.

Critical strengths
  1. The paper addresses an important practical problem: retaining evidence for future questions while maintaining current perception and responding at appropriate times. Combining factual records with a recent visual window is an intuitive design.
  2. The evaluation is broad and includes useful component ablations. Table 2 shows that retaining generated records improves historical QA, particularly ASI from 63.5 to 71.6. Table 3 supports PSTL’s effectiveness, improving OVO-Timing F1 from 26.5 with focal loss to 41.6.
Critical weaknesses
  1. The methodological advance over closely related work remains insufficiently established. ThinkStream also combines recent visual context with generated textual history, and the reported experiments do not directly demonstrate the additional benefit of PHCM’s query-independent factual records over this approach.
  2. The PSTL ablation supports the effectiveness of the overall strategy but does not isolate the source of improvement. PSTL preserves all output anchors, whereas Random Sparse is not described as doing so. The gains may therefore partly reflect preservation of critical output supervision rather than transition-based grouping itself.

Reference: Liu et al. Thinking in Streaming Video. arXiv:2603.12938, 2026.

Questions for authors
  1. Does the Random Sparse baseline in Table 3 preserve all output anchors? If not, does PSTL still outperform an anchor-preserving random baseline at the same supervision budget?
  2. Have PHCM and PSTL been evaluated on other backbones or model sizes? Are the gains consistent across these configurations?
  3. Does textual memory have a fixed capacity or an eviction policy? What is the longest video tested, and how do accuracy, latency, and GPU memory usage change over its duration? How is history handled when it reaches the context limit?
8 comments

We sincerely thank the reviewer for the constructive comments and valuable feedback. We appreciate the opportunity to clarify our work and address the concerns raised. Below, we provide a point-by-point response to each comment.

W1: The methodological advance over closely related work remains insufficiently established. ThinkStream also combines recent visual context with generated textual history, and the reported experiments do not directly demonstrate the additional benefit of PHCM’s query-independent factual records over this approach.

We thank the reviewer for raising this concern. We would like to clarify how PHCM differs from ThinkStream in terms of memory content, writing mechanism, and training objectives, and to provide experimental evidence for its additional benefits.

First, the key difference lies in what information the textual history preserves. ThinkStream's RCSM retains intermediate reasoning processes and uses reinforcement learning to optimize their generation toward final answer correctness. Its memory is therefore primarily oriented toward solving the question. In contrast, PHCM decouples memory formation from specific queries by explicitly supervising the model to record observed local facts and summaries of completed events through </Observe> and </Summary>. Both types of records are temporally grounded in their source intervals. This enables the model to proactively accumulate reusable factual information before future questions are known.

Second, the two approaches differ in their memory-writing mechanisms. According to ThinkStream's public implementation, RCSM generates memory periodically based on predefined video chunks, whereas PHCM allows the model to decide when and whether to generate a local record or an event summary based on the currently observed content. For example, when processing a 600-second video with max_chunks=60, the RCSM in ThinkStream divides the video into 10-second chunks. Such fixed-interval writing makes it difficult to adapt the recording frequency to the actual density of scene changes. To examine the benefit of adaptive writing, we conduct an additional ablation in which PHCM's model-controlled writing times are replaced with fixed chunk-based writing (the implementation of RCSM in ThinkStream), while keeping other settings unchanged. The results are as follows:

VariantOVOBench (Overall)StreamingBench (Real-Time)
Fixed-Chunk Writing70.886.5
PHCM (Ours)72.186.9

PHCM improves over fixed-chunk writing by 1.3 and 0.4 percentage points on OVOBench and StreamingBench, respectively. These results suggest that allowing the model to determine when to record information based on ongoing observations further benefits historical understanding while maintaining real-time perception.

Furthermore, OneStreamer achieves higher scores in this system-level comparison. The particularly large improvement on OVOBench Backward highlights its advantage in tasks requiring access to historical evidence. These results further demonstrate that organizing historical information into explicit, reusable factual records improves the utilization of distant visual evidence while maintaining strong real-time perception.

MethodOVOBench (Backward)OVOBench (Real-Time)StreamingBench (Real-Time)
ThinkStream52.367.075.0
OneStreamer77.981.486.9

Finally, three existing ablations in our paper provide complementary evidence for the PHCM design.

  • Table 2 examines the effectiveness of retaining factual records by comparing FIFO and PHCM using the same checkpoint and Recent-16 visual window. Retaining Caption Memory improves OVOBench Backward ASI from 63.5 to 71.6 and EPM from 62.0 to 62.6, demonstrating that historical factual records provide useful evidence for subsequent questions.
  • Table 17 further establishes the importance of dedicated caption supervision. Under the same recording prompt and inference settings, retaining generated descriptions from a model trained without proactive caption supervision changes OVOBench Overall by −0.1 points, whereas the full model gains 2.0 points. This indicates that inference-time prompting alone is insufficient to establish effective factual memory.
  • Table 18 examines the complementary roles of the two recording granularities: using only Observe, only Summary, and both achieves OVOBench Overall scores of 71.7, 71.2, and 72.1, respectively, with the full hierarchy achieving the best result.

In summary, PHCM goes beyond simply combining textual history with recent visual context. Through query-independent factual recording, content-adaptive memory writing, and hierarchical supervision that integrates local observations with event-level summaries, it enables the model to proactively construct reusable historical memory without knowing future queries in advance.

W2: The PSTL ablation supports the effectiveness of the overall strategy but does not isolate the source of improvement. PSTL preserves all output anchors, whereas Random Sparse is not described as doing so. The gains may therefore partly reflect preservation of critical output supervision rather than transition-based grouping itself.

Q1: Does the Random Sparse baseline in Table 3 preserve all output anchors? If not, does PSTL still outperform an anchor-preserving random baseline at the same supervision budget?

We thank the reviewer for identifying this important confounding factor. Random Sparse in Table 3 does not preserve supervision at all output anchors. It uniformly samples state-token positions at a 27.5% supervision ratio, leaving some output-anchor positions unsupervised. We agree that this comparison alone does not isolate the contribution of transition-based grouping.

To address this concern, we evaluated an anchor-preserving random baseline "Random (matched state counts)" that additionally controls for the target-state distribution. For each training sequence, let kYk_Y denote the number of positions with target state YY supervised by PSTL. The baseline preserves supervision at every output anchor. For each target state YY, we randomly sample kYk_Y positions irrespective of the preceding control state XX. It therefore matches PSTL's exact supervised target-state counts and total supervision budget within every sequence, yielding the same overall supervision ratio of 27.5%.

State selection strategyAll output anchors supervisedState supervision ratioProactiveVQA (Avg)OmniMMI (Avg)OVO-Timing (Avg F1)
Random Sparse (Table 3)No27.5%26.627.41.4
Random (matched state counts)Yes27.5%37.733.823.4
PSTL (Ours)Yes27.5%48.736.641.6

The matched random baseline improves over Random Sparse on all three benchmarks. However, PSTL still outperforms this baseline by 11.0 points on ProactiveVQA, 2.8 points on OmniMMI, and 18.2 points on OVO-Timing, despite identical output-anchor supervision, target-state counts, and supervision budgets.

The remaining difference lies in how supervision is allocated among transitions entering the same target state. Uniform sampling within YY allocates supervision in proportion to the frequencies of incoming transitions. In expectation, frequent self-transitions can dominate the selected positions. PSTL instead caps supervision separately for each X→YX→Y group, retaining coverage of less frequent transitions alongside state persistence.

These results provide evidence that transition-based grouping contributes beyond output-anchor preservation and target-state balancing. We will include this baseline in the revised Table 3 and clarify both sampling protocols.

Q2: Have PHCM and PSTL been evaluated on other backbones or model sizes? Are the gains consistent across these configurations?

We thank the reviewer for raising this important question. To evaluate the transferability of PHCM and PSTL across different model backbones, we reproduce the ablation experiments in Tables 2 and 3 using Qwen3.5-4B. We keep the training data, training configurations, and ablation settings unchanged, modifying only the model backbone and its initialization checkpoint to assess the effectiveness of both methods under a different backbone.

(1) Effectiveness of PHCM across backbones. We re-evaluate the Full, FIFO, and PHCM variants using Qwen3.5-4B. The results are reported below.

VariantOVOBench (Overall)StreamingBench
Full70.080.7
FIFO70.586.8
PHCM (Ours)72.787.1

PHCM consistently outperforms both Full and FIFO on the two benchmarks. In particular, compared with FIFO, which uses the same Recent-N visual window, PHCM achieves improvements of 2.2 and 0.3 percentage points on OVOBench and StreamingBench, respectively. These results suggest that the benefits of combining recent visual context with historical caption memory are not specific to the Qwen3-VL backbone.

(2) Effectiveness of PSTL across backbones. We further compare CE, Focal, and PSTL using Qwen3.5-4B. The results are as follows.

VariantProactiveVQAOmniMMIOVO-Timing
CE29.333.315.0
Focal38.532.927.3
PSTL (Ours)48.434.842.8

PSTL achieves the best results on all three benchmarks. Compared with CE, it improves performance by 19.1, 1.5, and 27.8 percentage points, respectively. Compared with Focal, the improvements are 9.9, 1.9, and 15.5 percentage points, respectively. These results further demonstrate that PSTL effectively improves proactive response learning through selective supervision of state transitions and persistence, even with a different model backbone.

Overall, despite the change from Qwen3-VL to Qwen3.5, both PHCM and PSTL maintain their performance advantages, demonstrating their transferability across different model backbones. Regarding scalability to larger models, since OneStreamer-1M contains over one million training records, conducting the corresponding experiments with larger backbones entails substantial computational costs and training time. We therefore leave this investigation to future work, where we plan to further evaluate the effectiveness of PHCM and PSTL across different model scales.

Reviewer_1P1EOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents OneStreamer, a streaming video language model that jointly learns query-independent evidence recording and task response through a shared proactive generation process. Its Proactive Hierarchical Caption Memory (PHCM) combines a recent visual window with time-aligned local captions and event summaries to support historical question answering. Proactive State Transition Learning (PSTL) addresses imbalanced control-state supervision by preserving all output anchors and selectively supervising state transitions and persistence. To support training, the authors construct OneStreamer-1M by combining synthesized streaming captions and QA with curated existing datasets. The data synthesis framework focuses on caption quality and response timing. Built through supervised fine-tuning of Qwen3-VL-4B-Instruct, OneStreamer achieves the highest aggregate scores among the compared methods on eight streaming benchmarks. Ablations show that caption memory improves historical QA while maintaining real-time perception, and that PSTL improves proactive-response performance over dense and randomly subsampled state supervision.

Critical strengths
  1. A valuable dataset for training streaming video models. OneStreamer-1M provides over one million training records covering perception, memory, and proactive interaction. It combines new caption and QA annotations generated by Gemini3.1Pro and Seed2.1Pro with curated existing data. Beyond this broad coverage, the authors focus on caption quality and response timing. The data ablation demonstrates the benefit of these efforts.

  2. An effective training recipe for proactive streaming video understanding. PSTL addresses the imbalance between frequent waiting states and sparse response states through selective state supervision. It achieves 41.6 F1 on OVO-Timing, compared with 26.5 for frequency-balanced focal loss and 15.3 for transition-only supervision, demonstrating the effectiveness of the training strategy in learning when to respond.

Critical weaknesses

Unclear attribution of the overall performance gains. The paper introduces both a new training dataset and a training and inference framework, but it remains unclear how much of the total improvement comes from the data and how much comes from the method. Although the component ablations support PHCM and PSTL, they do not fully assess the contribution of annotations generated by Gemini3.1Pro and Seed2.1Pro. A comparison using a weaker annotator on a subset would help clarify the role of annotation quality in the overall gains.

Questions for authors

See weakness, please.

1 comment

We would like to express our sincere gratitude for your review. Your suggestions will greatly help us improve our paper.

W1: Unclear attribution of the overall performance gains. The paper introduces both a new training dataset and a training and inference framework, but it remains unclear how much of the total improvement comes from the data and how much comes from the method. Although the component ablations support PHCM and PSTL, they do not fully assess the contribution of annotations generated by Gemini3.1Pro and Seed2.1Pro. A comparison using a weaker annotator on a subset would help clarify the role of annotation quality in the overall gains.

We thank the reviewer for raising this important concern. We address this concern from two aspects: (1) an additional annotator-substitution experiment to directly examine the effect of annotation quality, and (2) existing controlled ablations that separately assess the contributions of PHCM, PSTL, and our data construction pipeline.

(1) Effect of annotation quality. To further investigate the role of annotation quality, we select the 60k Proactive Caption Memory records originally annotated by Gemini3.1Pro and Seed2.1Pro and regenerate them using a comparatively weaker open-source model Qwen3.5-397B-A17B-FP8 with exactly the same prompts. We then replace the original 60K records entirely with the Qwen-generated annotations and retrain the model while keeping all other training data, model initialization, and training configurations unchanged. The results are reported below.

Annotator for the subsetOVOBenchStreamingBenchProactiveVQAOmniMMIAvg.
Without this subset69.986.149.236.460.4
Qwen3.5-397B-A17B72.187.546.337.460.8
Gemini3.1Pro & Seed2.1Pro (Ours)72.186.948.736.661.1

The results show that both annotation schemes provide overall benefits: compared with removing this data subset, the Qwen-annotated and original annotations improve the average score across the four benchmarks by 0.4 and 0.7 points, respectively. Their benefits vary across tasks: Qwen-generated annotations perform better on StreamingBench and OmniMMI, whereas Gemini3.1Pro and Seed2.1Pro achieve better results on ProactiveVQA and a slightly higher overall average. These results indicate that annotator choice can affect task-specific performance, but the overall gains do not depend only on a particular closed-source annotator. A comparatively weaker open-source annotator can also provide effective training supervision.

(2) Contributions of the method and data construction. Our existing controlled ablations further disentangle their contributions:

  • PHCM (Table 2): Using the same checkpoint and Recent-16 visual window, we vary only whether generated captions are retained as memory. Compared with FIFO, PHCM improves OVOBench-Backward ASI from 63.5 to 71.6 (+8.1), OVOBench Overall from 70.1 to 72.1 (+2.0), and StreamingBench Real-Time from 86.3 to 86.9. Compared with Full, it improves ASI by 4.0 points and OVOBench Real-Time by 10.5 points (81.4 vs. 70.9). These comparisons isolate the contribution of inference-time memory.

  • PSTL (Table 3): With identical training data, input sequences, and text supervision, we vary only state-token supervision. Compared with dense CE, PSTL improves ProactiveVQA, OmniMMI, and OVO-Timing by 22.6, 5.8, and 40.1 points, respectively. It also outperforms frequency-balanced focal loss by 1.7, 3.8, and 15.1 points, respectively. Random Sparse performs substantially worse under the same 27.5% supervision budget, showing that sparsity alone does not explain the gains.

  • Data construction (Table 20): Progressively adding OneStreamer-Core components under the same training configuration increases the five-benchmark average from 49.8 to 56.0. Notably, CQTC adds no training samples but improves existing annotations through content verification and response-time calibration, raising OVO-Timing from 33.0 to 42.0 (+9.0) and the average from 53.4 to 55.6 (+2.2). This highlights the importance of annotation quality and temporal alignment beyond data scale.

Overall, these results show that performance gains arise from complementary contributions of data quality, training strategies, and inference mechanisms. We will include the annotator-substitution experiment and clarify these contributions in the revised manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overall Summary of Rebuttal and Revisions

We sincerely thank all reviewers for their constructive feedback during the review and discussion stages, and we thank the Area Chair for their time and dedication in ensuring a fair and rigorous evaluation process.

We are pleased to have received encouraging feedback in the review stage, with initial ratings of 4 (1P1E), 2 (MLgt), 3 (ufKW), 3 (EhLE), and 4 (fTrg). Reviewers noted that OneStreamer-1M is a valuable dataset for training streaming video models (1P1E, fTrg) and a substantial contribution to the community (fTrg), that PSTL is an effective training recipe for proactive streaming video understanding (1P1E) and an interesting state-transition formulation that offers a simple and effective new approach (fTrg), that the paper addresses an important practical problem with an intuitive design (MLgt), that organizing memory writing and response control within one causal generation process is a useful formulation (ufKW), and that the evaluation is broad and supported by useful and solid ablations (MLgt, ufKW, EhLE, fTrg).

During the discussion stage, we provided detailed, point-by-point responses to every reviewer's concerns. Reviewers MLgt, ufKW, EhLE, and fTrg engaged in the discussion and gave positive follow-up remarks. Reviewer MLgt raised their score from 2 to 3 (weak acceptance), Reviewer ufKW raised their score from 3 to 4 (clear acceptance), Reviewer EhLE maintained their positive score of 3 (weak acceptance), and Reviewer fTrg kept a score of 4 (clear acceptance) and raised their confidence from 3 to 4. The updated ratings are 4, 3, 4, 3, and 4, with all five reviewers now recommending acceptance:

Reviewer MLgt: "Thank you for the additional experiments. The matched random baseline addresses my main concern about PSTL, and the additional backbone and video-duration analyses strengthen the empirical evidence. These results resolve my major concerns, and I am raising my rating to Weak Accept."

Reviewer ufKW: "Thank you for the detailed response and the additional experiments. They are clear and convincing, and they fully address my concerns. I am raising my score to 4."

Reviewer EhLE: "Provided comparison helped to address my concern about the benefit of learning when to write memory."

Reviewer fTrg: "I thank the authors for their detailed response, which has fully resolved my concerns. OneStreamer is a solid and valuable contribution, and I believe both the proposed training paradigm and the curated data will have a significant positive impact on the community. Accordingly, I will increase my confidence score."

Summary of new experiments. During the discussion stage, we conducted the following experiments and incorporated them into the revised manuscript. Details are provided in our individual responses.

  1. Model-controlled vs. periodic memory writing under the same record budget (MLgt, EhLE)
  2. Class-balanced random sampling and inverse-frequency weighted CE baselines for PSTL (MLgt, ufKW)
  3. Controls for training data volume, memory token budget, and visual density (ufKW)
  4. Duration-stratified analysis of accuracy, latency, and GPU memory (MLgt, fTrg)
  5. Joint memory writing and proactive response, with latency analysis (ufKW, fTrg)
  6. Transfer of PHCM and PSTL to the Qwen3.5-4B backbone (MLgt)
  7. Annotator substitution with a weaker open-source model (1P1E)
  8. OVOBench results excluding HLD for all compared models (ufKW)
  9. Matched-input OmniMMI comparison with and without ASR (ufKW)

Clarifications. We also clarified the differences from ThinkStream, Think While Watching, and VST (MLgt, EhLE), the two-stage joint inference procedure (ufKW, fTrg), KV-cache and positional-encoding handling (fTrg), and the append-only memory limitation (fTrg). Following Reviewer ufKW's suggestion, we adopted VideoChat3's reported ODVBench score and qualified the claim of best results across all eight benchmarks (ufKW, EhLE).

We will continue to refine the main text and appendix based on all reviewer suggestions, and we are confident that the final version will meet the reviewers' expectations. We are grateful to the reviewers and the Area Chair for a constructive review process that has substantially improved the quality of our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.