acceptodds
Under review as a conference paper at ICLR 2027

ProV-Mem: Proactive Memory for Streaming Video Question Answering

Abstract

Proactive streaming video question answering requires a model to continuously monitor an incoming stream and autonomously decide when to speak, what to say, and whether to emit a response. Existing streaming systems face a fundamental difficulty: retaining long visual histories is costly, whereas discarding them can lead to missed events and repeated responses. We propose **ProV-Mem**, a training-free proactive memory framework built around a frozen VLM backbone. ProV-Mem decomposes the proactive decision into three lightweight memory tiers: a temporal-trigger memory controls when to query the backbone and recovers from prolonged silence, an instructional memory guides what to say through a fixed prompting prior for concise and visually grounded generation, and an episodic memory determines whether to emit by filtering near-duplicate responses. Under an idealized conditional-sufficiency assumption, we derive a stagewise approximation bound relating the decomposed process to a full-state joint policy. We further introduce Cost-Aware Area Under the Curve (**CAUC**), which extends PAUC by explicitly discounting redundant, overly frequent, and unnecessarily long emissions. Experiments on the four scenarios of ProactiveVideoQA show that ProV-Mem achieves the best CAUC across all scenarios and the best PAUC on WEB, EGO, and TV, while maintaining low per-response inference latency.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Proactive Video QA, LLM memory

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aXx5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ProV-Mem, a training-free wrapper around a frozen Qwen2.5-VL-7B for proactive streaming VQA. It queries the backbone at a fixed stride (K=3 frames) with a silence-recovery trigger, prompts for a concise grounded description of the current window, and suppresses near-duplicate outputs via prefix-LCS similarity to the last emission. A union-bound lemma relates this three-stage factorization to a joint policy, and CAUC extends PAUC with redundancy, frequency, and length penalties. On ProactiveVideoQA, it reports the best CAUC on all four tasks, the best PAUC on WEB/EGO/TV, and 1.3 s per-response latency.

Critical strengths
  1. The proposed method is training-free on a frozen backbone, and its prompts and per-scenario settings are disclosed. It is strong on EGO against published numbers. Its gains over the same backbone’s published chunked baseline are large on EGO and TV. Broad sweeps (η, K, τ_sup, κ, ω) and a 3B/7B/32B scaling study are included.
  2. PAUC does not charge for redundant, overly frequent, or verbose emissions. Reporting a cost-aware score and Rep% alongside PAUC is a step in the right direction.
Critical weaknesses
  1. The baselines do not support the SOTA claims.
    • The re-run baselines are much weaker than their published results. The same GPT-4.1 judge is used, yet most offline open-source baselines score far below their numbers in the ProactiveVideoQA paper.
      • Qwen2.5-VL-7B is ProV-Mem's own backbone. Its PAUC falls from 52.7/46.5/36.5/29.3 (published) to 35.82/39.27/29.46/23.25 (this paper).
      • IXC-2.5 on EGO falls from 50.3 to 13.19. That is below the 25 a model scores by never responding.
      • The online models, by contrast, roughly match their published values.
      • The "Baseline" in Fig. 3 is never defined and matches no row of Table 1.
    • The strongest systems are missing. GPT-4.1 and GPT-4.1-mini are the best published systems, and both are omitted.
      • GPT-4.1-mini's published PAUC on TV (59.4) and VAD (47.7) exceeds ProV-Mem's (57.43, 32.10).
      • On WEB, ProV-Mem leads the published LLaVA-OV result (55.0) by under 3 points.
    • The scales do not match. MMDuet2 is a 3B model, but it is compared against a 7B ProV-Mem. At matched 3B scale (Fig. 3), ProV-Mem leads on CAUC only on EGO.
  2. The reported numbers are inconsistent, and some contradict the paper's conclusions.
    • VAD CAUC. Under the default setting it is 31.68 in Tables 1–2 but ≈28.6 in Figs. 3–5. At ≈28.6, ProV-Mem would rank below four proprietary baselines. So the claim of "best CAUC across all scenarios" rests on the inconsistent number.
    • EGO PAUC without de-duplication. It is reported as 71.78 (Table 2), 70.64 (App. C.5) and 66.5 (Fig. 4). Sec. 4.5 builds its argument on the lowest value, yet Table 2 shows that removing M_ep improves PAUC on all four tasks.
    • Silence recovery. Turning it off (κ=∞) beats the default κ=4 on both PAUC and CAUC, yet κ=4 is still presented as optimal.
    • Cooldown. Rep% doubles, from 16% to 32%, when the cooldown τ_sup grows from 30 s to 60 s. A longer cooldown can only suppress more, so Rep% should not rise.
    • Scaling. EGO PAUC drops from 7B to 32B (70.9 → ≈65), which contradicts the claimed monotonic scaling.
  3. The theory and the memory design are overstated.
    • Lemma 1 is weaker than presented.
      • It is a union bound that holds for any two policies, so it does not justify this particular decomposition.
      • Its coupling ignores how the system's own outputs change later states. This removes compounding error by construction.
      • The ε terms are never estimated.
      • Assumption 1 is never used in the proof.
      • Sec. 4.5's discussion of η and ρ also contradicts App. B.
    • The "three-tier memory" is thin. It amounts to a frame counter, a fixed prompt and a 200-character prefix of the last response. The generator sees only the current 3-frame window and, apparently, not the question. This sits at odds with the introduction's criticism of methods that discard visual history.
Questions for authors
  1. Baselines.
    • Why are most re-run offline baselines far below the published ProactiveVideoQA numbers for the same models and judge (up to ~38 PAUC points)?
    • Why were GPT-4.1 and GPT-4.1-mini dropped?
    • How was Time measured for each baseline?
  2. Inputs and tuning.
    • Is the question q passed to the backbone at all?
    • If not, how does a minimal baseline score: the same backbone captioning every K frames with a generic prompt and no gating?
    • How were K, η, τ_sup and κ chosen without a held-out split?
  3. Which numbers are correct?
    • For the default VAD CAUC, is it 31.68 or ≈28.6?
    • For EGO without de-duplication, is it 71.78, 70.64 or 66.5?
    • Why is Rep% non-monotonic in τ_sup?
2 comments
AuthorsOpenReview ↗

Response to Reviewer aXx5

We would like to express our sincere gratitude to you for your review and detailed suggestions.

W1: Concerns regarding baseline comparability and SOTA claims

A1: Thank you for these specific comparisons. We clarify the baseline comparisons by distinguishing independently reproduced results, published benchmark results, and capacity-matched evaluations. These comparisons involve different inference protocols and serve different purposes.

1. Reproduction gaps and the definition of the Figure 3 baseline.
Although both studies use GPT-4.1 as the PAUC evaluator, the same evaluator does not guarantee identical inference protocols or generated response traces. In particular, offline models require adaptation to proactive inference, and differences in chunk lengths, sampling rates, and answerability-checking procedures can affect the resulting responses. Our results are based on independent baseline reproductions rather than the original benchmark's stored response traces. We will clarify these protocol differences and provide more detailed reproduction configurations.

To better understand the observed gaps, we audited the EGO outputs of Qwen2.5-VL-7B and IXC-2.5-7B. Qwen produces 436 empty or "No Reply" outputs among 1,575 outputs (27.68%), with a mean evaluator score of 1.0774/2. IXC, in contrast, produces only four empty outputs, but its responses receive a mean evaluator score of only 0.1396/2. These observations suggest distinct failure patterns: Qwen exhibits frequent non-responses, whereas IXC's low performance is primarily associated with the quality of its generated responses.

We also clarify that the baseline in Figure 3 refers to the corresponding Qwen2.5-VL backbone without the ProV-Mem controller (i.e., without the three memory tiers), at each model scale. It serves as a backbone-only reference for evaluating the contribution of ProV-Mem and is distinct from the chunked-inference Qwen2.5-VL-7B baseline in Table 1.

2. Omission of stronger published baselines.
To make the comparison more complete, we will explicitly report the published results of GPT-4.1 and GPT-4.1-mini at ω=0.5\omega=0.5 in ProactiveVideoQA. Their PAUC scores are 51.7/58.8/56.8/46.2 and 47.8/65.8/59.4/47.7 on WEB/EGO/TV/VAD, respectively. In particular, GPT-4.1-mini outperforms ProV-Mem on TV and VAD. But these published scores were obtained under the original benchmark's offline inference protocol, whereas ProV-Mem operates under a causal streaming protocol. Although both use PAUC, their inference paradigms and response-generation schedules differ. We therefore report these results as published cross-protocol references rather than strictly controlled comparisons against our independently reproduced baselines. Similarly, LLaVA-OV achieves a published WEB PAUC of 55.0, which is close to ProV-Mem's 57.90. Under our reproduction setup, LLaVA-OV-7B obtains 34.31.

3. Model-scale mismatch between MMDuet2 and ProV-Mem.
The comparison in Table 1 involves MMDuet2-3B and ProV-Mem-7B, and therefore does not constitute a capacity-matched evaluation. To address this concern, we provide an additional comparison between MMDuet2-3B and ProV-Mem-3B. As shown in the table below, ProV-Mem-3B achieves PAUC improvements of 0.18, 35.70, and 8.43 points on WEB, EGO, and TV, respectively, while scoring 0.60 points lower on VAD. Its four-scenario average is 50.18, compared with 39.25 for MMDuet2-3B, yielding an overall improvement of 10.93 points.

Table 1. Capacity-matched comparison at the 3B scale (PAUC).

SystemWEBEGOTVVAD4-scenario average
Backbone-only (3B)42.6923.4936.851.2726.08
MMDuet2-3B54.8331.5841.6928.8939.25
ProV-Mem-3B55.0167.2850.1228.2950.18
Difference vs. MMDuet2-3B+0.18+35.70+8.43−0.60+10.93

We further distinguish the PAUC and CAUC results. At the matched 3B scale, ProV-Mem leads in CAUC only on EGO, and we do not claim otherwise. Thus, the matched-scale results demonstrate consistent PAUC improvements in three scenarios, while its CAUC advantage remains scenario-dependent.

The two systems also differ in their training requirements: MMDuet2 relies on reinforcement-learning training for proactive interaction, whereas ProV-Mem operates with a frozen backbone without additional training.

AuthorsOpenReview ↗

Looking forward to your feedback aXx5

Dear Reviewer aXx5,

We appreciate your thoughtful comments and suggestions for our manuscript.

As the discussion deadline is approaching, we kindly invite you to check our rebuttal. Please feel free to raise any additional questions, and we will respond in a timely manner.

If our responses have addressed your concerns, we would appreciate your consideration in updating your score. Thank you for your time. We look forward to your feedback.

Best Regards,
Authors of Paper 15303

Reviewer_8YEAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses challenges in proactive streaming video question answering, such as controlling response timing and reducing redundancy, by proposing a training-free memory framework called ProV-Mem. It employs three memory modules—Temporal-Trigger Memory, Instructional Memory, and Episodic Memory—to control model invocation, response generation, and emission filtering, respectively. Additionally, the paper introduces CAUC, a cost-aware metric that penalizes redundant responses. Experiments on four scenarios of ProactiveVideoQA, including comparisons with multiple baselines and ablation studies, demonstrate the effectiveness of the proposed framework and its components.

Critical strengths
  1. The paper addresses an important challenge in proactive streaming video QA by jointly considering response timing, content generation, and redundancy suppression under real-time constraints.

  2. ProV-Mem introduces a training-free framework using a frozen VLM and lightweight memory mechanisms, avoiding additional model training while offering interpretable controls and low reported response latency.

  3. The method is well presented and easy to read.

Critical weaknesses
  1. The novelty remains incremental. The proposed three-tier memory framework largely combines heuristic triggering, fixed prompt engineering, and lexical deduplication. Its advantages over a straightforward combination of existing techniques remain insufficiently demonstrated.

  2. CAUC penalizes redundancy, response frequency, and length—behaviors directly targeted by ProV-Mem. Without independent human validation and stronger sensitivity analyses, its validity and potential bias toward the proposed method remain unclear.

  3. Although four scenarios are evaluated, they all belong to ProactiveVideoQA and involve scenario-specific configurations. Evaluation on more independent benchmarks is needed to substantiate claims of generalization and broad applicability.

Questions for authors
  1. Can the authors demonstrate that the proposed three-tier memory framework offers substantive benefits beyond a straightforward combination of heuristic triggering, prompt engineering, and lexical deduplication, under matched experimental conditions?

  2. What independent evidence establishes the validity and fairness of CAUC, given that its penalties directly align with the behaviors explicitly optimized by ProV-Mem? In particular, can the authors validate the metric against human judgments and demonstrate its robustness to alternative penalty weights?

  3. Can the authors evaluate ProV-Mem on more independent benchmarks or unseen domains under a causal streaming protocol, preferably without benchmark-specific prompt or hyperparameter tuning, to substantiate its generalization claims?

2 comments

We sincerely appreciate you for your careful reading, constructive comments, and valuable suggestions.

W1 & Q1:The novelty remains incremental.

A1: Thank you for raising this concern. We would like to clarify that ProV-Mem is not a simple aggregation of three existing memory components. Instead, it organizes decisions at different stages into a unified proactive response control framework through cross-stage state dependencies and feedback.

As described in Section 3.3, the three stages are coupled through state transitions rather than operating independently. When a candidate is accepted, the system updates episodic memory with the emitted response and resets the silence counter to zero. In contrast, when a regular trigger opportunity produces no emission, the silence counter is incremented, eventually activating the silence-recovery mechanism once the threshold is reached. Consequently, the emission decision directly affects the temporal state and subsequent triggering behavior, establishing a feedback loop between episodic filtering and temporal control. This cross-tier state coupling distinguishes ProV-Mem from a straightforward aggregation of independent heuristic components.

Furthermore, the ablation results in Table 2 demonstrate the distinct contributions of the three components. Removing Temporal Memory reduces WEB PAUC from 57.90% to 37.51%, a decrease of 20.39 percentage points, highlighting its role in controlling response triggering. Removing Instructional Memory leads to PAUC drops of 12.84 and 11.94 percentage points on WEB and EGO, respectively, demonstrating its contribution to candidate response generation. Episodic Memory exhibits a different effect: its removal increases PAUC across all four scenarios, while decreasing CAUC across all four. For example, on EGO, PAUC increases from 70.87% to 71.78%, whereas CAUC decreases from 38.20% to 37.80%. This opposite movement reflects the module's role in regulating the trade-off between response coverage and redundancy, rather than simply duplicating the functions of the other components.

Therefore, the central contribution of ProV-Mem lies not in stacking three independent memory operations, but in establishing a coordinated, state-coupled mechanism through which response decisions influence subsequent interactions.

AuthorsOpenReview ↗

Looking forward to your feedback

Dear Reviewer 8YEA,

We are truly grateful for your careful evaluation and valuable feedback on our work.

The discussion window will expire soon. We would appreciate it if you could take a moment to examine our point-by-point responses. Should you have any remaining questions or require further clarification on our revisions, please let us know at your earliest convenience so we can respond promptly.

If you find our revisions and responses resolve the issues you raised, we would be honored if you could reconsider your recommendation. Thank you for your time and efforts. We are looking forward to hearing from you.

Best Regards,
Authors of Paper 15303

Reviewer_jVCVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ProV-Mem is a training-free framework for proactive streaming VQA. It uses three memory-based components - a trigger memory with silence counter, an instructional memory with for context specific generation and an episodic memory with redundancy avoidance feature using longest common-subsequence (LCS).

Critical strengths
  1. The proposed memory-based framework for Proactive streaming VQA is an elegant training-free solution.

  2. Strong empirical gains on WEB and EGO scenarios.

  3. The PAUC critical analysis is valid, and the proposed CAUC covers the identified problems.

  4. The paper reported results on offline closed-source and open-source, as well as streaming VLMs.

  5. Backbone scaling and scenario-specific analysis are interesting.

Critical weaknesses
  1. The proposed solution is the aggregation of three existing memory components, instead of a novel memory design.
  2. Hyperparameter setting is a challenge in the proposed method: There is a requirement to set hyperparameters for different scenarios in the dataset itself.
  3. There is not enough discussion on configuring penalty weights in CAUC
  4. There is no explanation of how the Time(s) column in Table 1 gets its values, and why ProV-Mem gets 1.30: is it per query? Per video for other VLMs?
  5. Hyperparameters are empirically set, and results are reported on the entire dataset: is it possible to divide the dataset into validation/test and get the hyperparameters (like thresholds, etc.) set using the validation set and use the best-performing parameters to check performance on the train set?
  6. Table 1 shows limited gains on EGO, VAD, and WEB with the CAUC metric; VAD (no performance gain) with PAUC metric.
  7. Lexical LCS-based check for redundancy in episodic memory will not detect semantic collisions when text is paraphrased. A comparison with BERTScore will be a good addition.

Minor weakness: There are issues with sentence framing, for example, on Line 204 “at at”. Authors might like to recheck the paper for sentence framing and grammatical errors.

Questions for authors
  1. Can authors provide sensitivity-specific analysis for CAUC penalty weights?

  2. Can authors include a collison analysis for all the scenarios in the dataset and provide failure cases of ProV-Mem?

  3. Do all VLM baselines in Table 1 have similar frame sampling rate and token count budget?

5 comments
AuthorsOpenReview ↗

Response to Reviewer jVCV

W4: Concern on the timing unit and calculation

A4: Thank you for raising this point. The original Time column used different measurement units across systems, making direct comparisons potentially misleading. We clarify the meaning of the reported 1.30 s, explain the measurement scopes used for the other systems, and provide two complementary comparisons using normalized measurements.

1. The 1.30 s represents ProV-Mem's average latency per emitted response.
The reported 1.30 s is the macro-averaged wall-clock latency per emitted response, measured from entering a trigger window to producing its reply, including backbone prefill and token decoding. It is averaged across four scenarios: (1.00 + 1.33 + 1.93 + 0.94)/4 = 1.30 s (WEB, EGO, TV, and VAD). Measurements were conducted on A100-80GB GPUs using four near-average-length samples per scenario. Since ProactiveVideoQA contains one benchmark query per video but ProV-Mem can emit multiple proactive responses during processing, this value represents the latency of generating one response, not the total video-processing time or benchmark query-answering latency.

2. The original measurements used inconsistent units across systems.
The reviewer correctly identified an inconsistency in the original table: different systems were reported using different measurement scopes.

  • Offline open-weight VLMs: Their latency was measured in seconds per video. For example, LLaVA-OV-7B requires 348.08 s to process one EGO video, and its reported 94.87 s is the unweighted average across the four scenarios.
  • MMDuet and its variant: Their latency was measured in seconds of computation per second of video. For example, MMDuet requires 10.09 s of computation per second of video on WEB, while its reported 3.03 s is the average across scenarios.
  • ProV-Mem: Its latency was measured in seconds per emitted response.

These quantities characterize different aspects of computational cost and cannot be directly compared in a single column without explicitly stating their respective units. We also identified an inconsistency in the original ProV-Mem-3B row: its WEB and TV entries were reported per second of video, whereas its EGO and VAD entries were reported per emitted response. Consequently, its 0.765 s macro-average does not represent a consistent physical quantity and should not have been included as a comparable latency figure.

3. Recomputing latency using a common measurement unit.
To enable a fairer comparison, we normalize the available stored run records in two complementary ways, without performing new inference. Table 3 reports the average computation time per emitted response, calculated as the total processing time per video divided by the number of responses generated for that video. Table 4 reports the computation time per second of video, together with the average number of responses emitted per video.
Table 3. Latency per emitted response (seconds).

SystemWEBEGOTVVADMacro
ProV-Mem-7B1.001.331.930.941.30
Qwen2.5-VL-7B0.505.022.761.202.37
InternVL-2.5-8B0.593.833.946.043.60
IXC-2.5-7B4.757.0810.189.547.89
LongVA-7B3.74109.98110.40143.4291.89
LLaVA-OV-7B1.69276.0973.5320.3592.92

Under the normalized metric in Table 3, ProV-Mem achieves the lowest macro-averaged computation time per emitted response (1.30 s), compared with 2.37 s for Qwen2.5-VL-7B and 92.92 s for LLaVA-OV-7B. This corresponds to approximately 1.8× and 71.5× lower computation time per response, respectively. While ProV-Mem is not the fastest in every individual scenario, it achieves the lowest overall macro-average among the systems listed.

Table 4. Computation time per second of video and average number of emitted responses per video.

SystemWEBEGOTVVADMacroResponses per video
ProV-Mem-7B0.13540.03420.01140.00090.045510.91
Qwen2.5-VL-7B0.06060.01660.00900.00260.02222.66
InternVL-2.5-8B0.06440.01640.01080.00730.02472.77
IXC-2.5-7B0.50560.03190.03210.01880.14712.95
LongVA-7B0.05000.01340.00580.00290.01800.16
LLaVA-OV-7B0.18440.24170.05650.01710.12491.35

4. Interpreting the two measurements
The two tables capture different efficiency dimensions: Table 3 measures amortized computational cost per emitted response, while Table 4 measures computational cost per second of input video. Although ProV-Mem incurs higher stream-level computation than Qwen2.5-VL-7B, InternVL-2.5-8B, and LongVA-7B, it also generates substantially more responses (10.91 per video versus 0.16–2.95), resulting in a lower average cost per response. Thus, ProV-Mem improves response-level efficiency but does not necessarily minimize total computation over a given video duration. These two metrics should therefore be interpreted separately.

AuthorsOpenReview ↗

Response to Reviewer jVCV

W5 Concenrs on alidation-based hyperparameter selection.

A5: Thank you for raising this point. To examine the robustness of the reported configurations, we conducted 500 repeated video-level validation/test splits, selecting configurations by validation PAUC and comparing their performance against the originally reported settings on the same held-out test videos.

1. Video-level validation/test protocol.
We performed 500 repeated random 70/15/15 splits per scenario, using videos as the partitioning unit. The 70% partition was reserved, while the remaining videos were divided equally into validation and test sets. All questions and turns from the same video remained in one subset to prevent information leakage. All candidate records were matched by question identity, with none excluded. PAUC was averaged across valid turns per video and then equally across videos for both selection and evaluation; these subset-level results do not replace the official full-dataset scores in Table 1. The predefined candidates were τsup=30\tau_{\mathrm{sup}}=30 s versus ∞\infty for EGO, κ=4\kappa=4 versus ∞\infty for TV, and event cooldown on versus off for VAD. WEB had only one configuration and was included only to characterize split-to-split variation. For each split, we selected the highest-validation-PAUC candidate and compared its test performance with the originally reported configuration on the same held-out videos, using the corresponding stored emission trajectories.

2. Validation-based selection does not yield consistent test-set improvements.
Table 5 reports the validation selection frequency and corresponding test PAUC across 500 splits. The mean test difference is calculated as the paired PAUC difference between the validation-selected and originally reported configurations on the same test subset, averaged across splits.
Table 5. Validation-based configuration selection over 500 repeated video-level splits.

ScenarioCandidate configurationsReported configuration selected by validationSelected test PAUCFixed reported test PAUCMean test difference
WEBOne configuration100%59.7759.770.00
EGOτsup=30\tau_{\mathrm{sup}}=30 s vs. ∞\infty70.0%70.6770.81−0.15
TVκ=4\kappa=4 vs. ∞\infty79.2%56.7957.11−0.33
VADEvent cooldown on vs. off47.6%32.6932.38+0.30

Validation selected the originally reported configuration in 70.0% of EGO splits and 79.2% of TV splits. In VAD, the selection frequency was closer to an even split. On the held-out test subsets, the mean differences were −0.15, −0.33, and +0.30 PAUC points for EGO, TV, and VAD, respectively. These results show that validation-based selection did not yield a consistent improvement across scenarios. The absolute mean differences were all below 0.4 PAUC points, indicating comparable test performance between the validation-selected and originally reported configurations within the candidate set examined.

To further characterize the variability of the held-out results, Table 6 reports the mean, standard deviation, and 2.5th and 97.5th percentiles of test PAUC across the repeated splits.
Table 6. Test PAUC variation across the 500 repeated splits.

ScenarioMean test PAUCSplit-to-split SD2.5th Percentile97.5th Percentile
WEB59.771.9556.0263.46
EGO70.672.3066.0174.92
TV56.793.0350.7462.28
VAD32.694.3925.2542.16

The percentile values characterize the variation in test performance across the repeated partitions; they should not be interpreted as confidence intervals for the mean or as significance tests of the paired performance differences. VAD exhibits greater split-to-split variation, consistent with its smaller number of videos and correspondingly smaller test subsets. Since the repeated splits overlap, the 500 repetitions should also not be interpreted as 500 independent experiments.

3. Rationale for the originally reported configurations
The original configurations reflect scenario-specific temporal and event characteristics: EGO's 30-second suppression cooldown aligns with the median recurrence gap of approximately 28.7 seconds (Appendix C.5); TV's κ=4\kappa=4 yields a 12-second silence-recovery horizon under 1 fps sampling and a trigger cadence of K=3K=3; and VAD's 8-second event cooldown reduces repeated emissions for the same event (Appendix C.2). The additional validation/test analysis assesses selection among predefined alternatives but does not establish global hyperparameter optimality or imply that the original settings were selected through this procedure. Rather, it shows that the original configurations are frequently selected by validation in EGO and TV and achieve comparable held-out test performance to the validation-selected alternatives. We therefore retain the original settings, supported by both their empirical performance and scenario-specific rationale.

AuthorsOpenReview ↗

Response to Reviewer jVCV

W6: Limited performance gains on certain scenarios.

A6: Thank the reviewer for your suggestions. ProV-Mem's performance advantages vary across scenarios and evaluation metrics. In particular, its gains over competing systems are limited or negative on certain scenario–metric combinations, especially VAD. We therefore distinguish two aspects of the results: its performance relative to the strongest competing systems evaluated in Table 1, and its improvement over the underlying frozen Qwen2.5-VL-7B baseline. These comparisons address different questions and should not be conflated.

1. Comparison with the strongest competing systems.
Table 1 shows that ProV-Mem achieves the highest CAUC across all four scenarios, although the margins vary substantially. We summarize the margins over the strongest competing systems below.
Table 6. ProV-Mem versus the strongest competing system on each scenario.

ScenarioProV-Mem PAUCBest competing PAUCΔ PAUCProV-Mem CAUCBest competing CAUCΔ CAUC
WEB57.9054.83 (MMDuet2)+3.0740.5940.29 (MMDuet2)+0.30
EGO70.8750.28 (GLM-4.5V)+20.5938.2038.16 (GLM-4.5V)+0.04
TV57.4341.69 (MMDuet2)+15.7440.2234.89 (MMDuet2)+5.33
VAD32.1036.92 (Gemini-2.5-Pro)−4.8231.6830.96 (GLM-4.5V)+0.72

The comparison reveals substantial PAUC improvements on EGO and TV, but only marginal CAUC advantages on WEB and EGO (+0.30 and +0.04 points, respectively). On VAD, ProV-Mem achieves a slightly higher CAUC (+0.72 points), but falls short of the best PAUC by 4.82 points. These results indicate that the advantage of ProV-Mem is scenario- and metric-dependent rather than uniform across all comparisons.

2. VAD remains a limitation under PAUC.
The relatively weaker PAUC performance on VAD can be attributed to the distinct characteristics of this scenario, where sparse but critical hazard events require accurate event recognition and timely localization. ProV-Mem primarily regulates when to query and emit responses through temporal triggering, prompt-based guidance, and redundancy filtering. Although its VAD-specific two-stage scouting mechanism and event cooldown help control emission frequency, these mechanisms do not directly improve the backbone's underlying ability to recognize anomalous events. This limitation is also reflected in the sensitivity analysis, where VAD exhibits limited sensitivity to the lexical similarity threshold, suggesting that its response behavior is more strongly governed by temporal constraints than by lexical redundancy filtering. Thus, while ProV-Mem improves CAUC over competing methods, further gains in VAD PAUC may require stronger event-level recognition and localization capabilities.

3.Improvements over the underlying frozen backbone.
Although the margins over the strongest competing systems are limited in some scenarios, ProV-Mem consistently improves upon its underlying frozen Qwen2.5-VL-7B backbone. The CAUC comparisons are as follows.
Table 7. Improvement over the underlying Qwen2.5-VL-7B backbone.

ScenarioQwen PAUCProV-Mem PAUCΔ PAUCQwen CAUCProV-Mem CAUCΔ CAUC
WEB35.8257.90+22.0825.6540.59+14.94
EGO39.2770.87+31.6031.3638.20+6.84
TV29.4657.43+27.9723.8340.22+16.39
VAD23.2532.10+8.8519.1031.68+12.58

The consistent improvements over the frozen backbone demonstrate the contribution of the proactive memory framework across all four scenarios, including VAD. These gains, however, should be distinguished from the margins over the strongest competing systems, which vary considerably across scenarios.

Overall, the results support two distinct conclusions: ProV-Mem consistently improves upon its underlying frozen backbone and achieves the highest CAUC among the evaluated systems in all four scenarios. However, its advantages over competing systems are not uniform, and VAD remains a limitation in terms of PAUC. We have clarified these distinctions to provide a more balanced interpretation of the experimental results.

Comments on the paper

1 comment

Dear Reviewers,

The authors have provided their rebuttal, but no follow-up comments have been posted yet. As the author-reviewer discussion period is ending soon, please review and acknowledge the rebuttal, and engage in further discussion if any concerns remain unresolved.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.