acceptodds
Under review as a conference paper at ICLR 2027

Freeze the Student, Train the Researcher: Learning the Value of Experiments for Streaming Video Understanding

Abstract

Streaming video systems are typically engineered against a fixed benchmark, yet deployment brings unfamiliar scenes, event statistics, and query types, where the tuning loop must start over. What survives such shifts is the research process itself: hypothesizing about failures, running diagnostic experiments, and revising the design. We propose StreamRSI, which keeps the deployed 7B video answerer frozen and instead trains a 27B research policy to conduct this process, learning the value of experiments rather than any single system. The central obstacle is credit: an unsuccessful experiment can rule out a misleading explanation and improve subsequent search, while an immediately successful candidate can teach nothing transferable. StreamRSI therefore assigns controlled evidence credit. Paired continuations restore the same candidate state and differ only in access to a diagnostic result, isolating that result's contribution to later research; a third continuation spends the diagnostic budget on direct search, pricing the experiment's opportunity cost. These comparisons supervise when to investigate, which probe to run, and when to reopen a diagnosis. The discovered inference-time program lifts the frozen answerer to 65.50 OVO-Bench Overall and 80.63 StreamingBench real-time accuracy. Under a matched 20,000 GPU-s budget, full StreamRSI reaches the target quality in all eight seeds and outperforms outcome-only and single-credit training. The resulting research policy generalizes to novel domains, accelerating search on unseen benchmarks even when stripped of its historical archive.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
streaming video understanding, recursive self-improvement

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_4kiFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes StreamRSI: Freezing the Qwen2.5-VL-7B video question-answering model and training a 27B-parameter research agent to improve the program that runs during inference of the question-answering model. The main contribution was the design of A controlled evidence-credit protocol for diagnostic experiments: restoring the candidate program and archive before the same experiment, comparing with subsequent research branches (A and B ') that could not see the diagnostic results, and comparing with branches (A and C) that used the saved diagnostic budget for direct search. The above estimates are supervised through preference optimization for research decisions. The paper reports an Overall score of 65.50 for OVO-Bench and an accuracy rate of 80.63% for StreamingBench real-time video understanding. In the ablation experiment with eight random seeds, each seed lasting 20,000 GPU seconds, the final quality of the full objective was higher than that of the scheme that only used outcome-only supervision and a single credit term. Other experiments analyzed the differences between pre-experiment state restoration, archive-free transfer, frame selection, and evidence retention and reading.

Critical strengths
  1. The paper focuses on how the experiment contributes to subsequent improvement, not just whether the experiment improves the score in the moment. During the study, an experiment that does not improve accuracy may also rule out false explanations and provide directions for the next improvement. This paper tries to incorporate this kind of information value into the training objective, which has research significance.
  2. The paper sets up multiple controls to test whether performance improvement comes from the learning of the research agent. The authors compared the follow-up study process with and without seeing the diagnosis, and compared different training methods with the same budget and eight paired random seeds. In addition, the authors cleaned out the existing programs and archives to conduct transfer experiments, trying to test whether the research agent can find effective solutions on new tasks. These designs provide positive evidence for the effectiveness of the method.
Critical weaknesses
  1. The paper defines the information value of the experiment and the net value after considering the cost respectively, and shows the performance advantages of using the two together through ablation experiments. However, existing analyses mainly stop at final score comparison and have not yet fully illustrated which research decisions are improved by the two signals respectively and how they complement each other. Thus, the experiments support that the combination approach works, but the explanation for why it works is still more limited.
  2. Table 6 shows the advantage of the trained research agent over the untrained model on the new task, but there is no research agent trained only on the final results as a comparison. Therefore, the existing results support the transfer ability of the trained research agent, but are not sufficient to prove that the credit mechanism proposed in this paper can bring better transfer effects than normal training.
Questions for authors
  1. When an experiment can provide useful information, but its cost results in a final performance lower than that of a direct search, how does the complete method determine whether to conduct this experiment? Please explain how the two types of credit signals jointly influence the decision in this situation.
  2. Can you give a case study of an experiment that didn't immediately improve accuracy, but helped the research agent eliminate incorrect explanations and get better results later? This case will help to understand the role of the core idea of the paper in the actual research process.
  3. Under the same transfer conditions as in Table 6, how does the proposed method perform compared to the research agent trained only on the final results? This contrast helps to determine whether the migration gain comes from the credit mechanism proposed in this paper.
2 comments

Thank you for the review. Below we explain how the two credit signals enter the researcher's decisions, give a case of an informative negative result, and add the outcome-only researcher to the transfer comparison of Table 6.

W1 / Q1. Which research decisions does each signal improve, how do they complement each other, and what does the full method do when an experiment is informative but still loses to direct search?

The two signals supervise different decisions, because each produces its own preference pairs:

  • Net credit compares a diagnostic with a direct edit at the same state, preferring the diagnostic if Vnet=E[QA−QC]>0V_{\mathrm{net}}=\mathbb E[Q_A-Q_C]>0 and the direct edit if Vnet<0V_{\mathrm{net}}<0. It trains when to investigate.
  • Evidence credit compares two diagnostics at the same state by Vevidence=E[QA−QB′]V_{\mathrm{evidence}}=\mathbb E[Q_A-Q_{B'}]. It trains which probe to run.

Each decision state is forked into five branches. For each of two candidate diagnostics D0D_0 and D1D_1 there is a branch AA, which runs the diagnostic, restores the program and archive, and continues with the result visible, and a branch B′B', which does the same with the result hidden. A fifth branch CC skips the diagnostic and searches directly from the same state. Net pairs compare AiA_i with CC, and the evidence pair compares D0D_0 with D1D_1 by QAi−QBi′Q_{A_i}-Q_{B'_i}. The two kinds of pairs are combined only in the loss, L=13(Lnet+Levidence+Ledit)\mathcal L=\tfrac13(\mathcal L_{\mathrm{net}}+\mathcal L_{\mathrm{evidence}}+\mathcal L_{\mathrm{edit}}), where Ledit\mathcal L_{\mathrm{edit}} is the outcome preference over program edits. A pair is kept only if the 95% bootstrap CI of the difference excludes zero (8 paired continuation seeds, 5,000 resamples over seeds and video clusters). The training pool has 144 net pairs and 80 evidence pairs.

With only one of the two signals, one of these decisions gets no supervision. Evidence credit alone teaches which probe is the most informative, but not that a probe can cost more than its information is worth. Net credit alone teaches when to probe, but gives no basis for choosing between probes. The full objective supervises both, and Table 4 shows the result: it is 1.20 [0.89, 1.51] above net credit and 2.02 [1.71, 2.34] above evidence credit.

If an experiment is informative but loses to direct search (Vevidence>0V_{\mathrm{evidence}}>0, Vnet<0V_{\mathrm{net}}<0), the net pair at that state prefers the direct edit. The experiment's evidence value may still rank it above a less informative probe, but the net pair trains the researcher not to run it from this state. Case B in Table R1 is an example.

Table R1. Two diagnostics from the source search. The immediate change is on the support panel; QAQ_A, QB′Q_{B'}, QCQ_C are on the credit panel.

Case ACase B
Current programrecent-44 recent + 4 uniform historical frames
ExplanationsH1: too few recent frames; H2: key historical event missingH1: historical event located wrongly; H2: prompt poorly organized
Probe (cost)recent-4 → recent-8, layout and prompt fixed (120 GPU-s)several historical-window localization runs, frame count and layout fixed (800 GPU-s)
Immediate change73.15 → 70.37 (−2.78)78.70 → 79.63 (+0.93)
ResultMore recent frames do not fix the errors; earlier event evidence is needed. H1 ruled outKey historical interval located; replacing those frames fixes the errors under the same prompt. H1 supported
QAQ_A / QB′Q_{B'} / QCQ_C81.49 / 78.59 / 79.9479.85 / 78.66 / 81.20
VevidenceV_{\mathrm{evidence}} / VnetV_{\mathrm{net}}+2.90 / +1.55+1.19 / −1.35
Full objective prefersthe probe over a direct editthe direct edit over the probe

In Case B the probe is informative (Vevidence=+1.19V_{\mathrm{evidence}}=+1.19): branch A uses the localized interval to improve historical sampling. The probe, however, costs 800 GPU-s. Branch C spends that budget validating more candidates and finishes higher, so Vnet=−1.35V_{\mathrm{net}}=-1.35.

Q2. Can you give a case of an experiment that did not immediately improve accuracy but helped the research agent rule out a wrong explanation and reach a better result later?

Case A in Table R1 is one. The current program was recent-4, and the researcher had two explanations for its errors: too few recent frames (H1) or a missing historical event (H2). It ran a controlled probe that changed recent-4 to recent-8 with the layout and prompt fixed, at a cost of 120 GPU-s, and recorded beforehand what each explanation predicts. The support score fell by 2.78 points. The extra recent frames did not fix the main errors, and the failing queries needed evidence from earlier events, which ruled out H1.

In branch A the researcher saw this result, reopened the diagnosis, and switched to query-relevant historical retrieval. Its final program uses 4 recent and 4 relevant historical frames and scores QA=81.49Q_A=81.49 on the credit panel. The branch that ran the same probe without seeing the result reaches 78.59, and direct search with the saved budget reaches 79.94. The probe therefore has Vevidence=+2.90V_{\mathrm{evidence}}=+2.90 and Vnet=+1.55V_{\mathrm{net}}=+1.55 and became a preferred action. Outcome-only training would have seen only the 2.78-point drop and labeled the probe negative.

W2 / Q3. Under the transfer conditions of Table 6, how does the method compare with a researcher trained only on final outcomes?

We ran the outcome-only researcher of Table 3 under the Table 6 protocol: weights transferred, candidate code, diagnosis archives and task memory cleared, three seeds per target.

Table R2. Weights-only transfer, mean over three seeds.

Transferred weightsT1T2T3
None (base search)78.8377.8776.84
Outcome-only researcher79.1878.2077.10
StreamRSI full80.3279.0877.57
Full − Outcome-only+1.14+0.88+0.47
Paired 95% CI[+0.64, +1.64][+0.63, +1.13][+0.10, +0.84]

Outcome-only training gives a small transfer gain over the base search (+0.35, +0.33 and +0.26). StreamRSI is above the outcome-only researcher on every target, and every paired interval excludes zero. Relative to the base search, 77% of the weight-transfer gain on T1, 73% on T2 and 64% on T3 goes beyond what outcome-only training provides, so most of the transfer gain in Table 6 comes from the credit signals.

Reviewer_ihz1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes StreamRSI, which trains a 27B researcher to improve how a frozen 7B model answers video questions. The researcher modifies frame selection, input layout, and prompts. To assess whether an experiment was worth running, the method compares subsequent searches with its result, without its result, and without running the experiment. I find the topic interesting, but the paper leaves too many basic details unexplained. Addressing these issues would require a substantial revision beyond what is feasible during rebuttal. I recommend rejection.

Critical strengths
  • Applying automated research to streaming video understanding is interesting and potentially useful. The difficulty of assigning credit to diagnostic experiments is well motivated.

  • The experiments include extensive ablations, separate data for different stages of evaluation, and account for computational costs. The reported improvements are substantial.

Critical weaknesses
  • The method is difficult to follow. For example, Figure 1 is cluttered, and neither the learning algorithm nor the researcher’s execution framework is adequately explained. A complete example showing how the researcher proposes an experiment, interprets its result, and improves the program would help considerably.

  • Important details needed to reproduce the method are missing. For example, how evidence value and net value are combined, how the reference model is updated, and when researcher updates occur.

Questions for authors
  • Could the authors provide the full training objective, explain how action-preference pairs are constructed, and specify the 27B model and its training configuration?

  • Which dataset is used to compute QQ in Table 3? Is it also used to train the researcher?

3 comments

Thank you for the review. We give a worked example of one research step, the full training objective and configuration of the researcher, and the panels behind Q in Table 3, together with a re-evaluation of Table 3 on questions not used in researcher training.

W1. The method is hard to follow. Figure 1 is cluttered, and a complete example of how the researcher proposes an experiment, interprets its result, and improves the program would help.

We will redraw Figure 1 as three separate panels (research state and actions, the A/B′/C fork, preference construction) and add an algorithm box. The worked example below comes from the source search and will be added to the paper.

  1. The current program is the recent-4 view. The researcher has two explanations for its errors: recent frames are sampled too sparsely (H1), or the key historical event is missing from the input (H2).
  2. Instead of another direct edit, it issues a Controlled probe that changes recent-4 to recent-8, keeps the layout and prompt fixed, and checks which error types remain. Before running it, the researcher records the controlled factor, what each explanation predicts, and the cost (120 GPU-s).
  3. The support score drops from 73.15 to 70.37 (−2.78). The extra recent frames do not fix the main errors, and the failing queries need evidence from earlier events, so H1 is ruled out.
  4. In branch A the researcher sees this result, reopens the diagnosis, and switches to query-relevant historical retrieval. The final program uses 4 recent and 4 relevant historical frames.
  5. From the restored pre-probe state, the credit panel gives QA=81.49Q_A=81.49, QB′=78.59Q_{B'}=78.59 and QC=79.94Q_C=79.94, so Vevidence=+2.90V_{\mathrm{evidence}}=+2.90 and Vnet=+1.55V_{\mathrm{net}}=+1.55. Since Vnet>0V_{\mathrm{net}}>0, the pair (probe ≻ direct edit) is added for this state. A researcher trained on outcomes alone would have counted the probe as a failure, because its immediate score fell.

One training round of the researcher runs as follows:

  1. Run research with the current checkpoint πϕ\pi_\phi fixed.
  2. At a decision state, snapshot the candidate, archive, checkpoint, evidence and random state, and run five branches from the snapshot (A0A_0, B0′B'_0, A1A_1, B1′B'_1, CC; described under W2), each with 8 paired continuation seeds.
  3. Each branch picks its final program on the selection panel, and the evaluator scores that program on the credit panel to give QAQ_A, QB′Q_{B'}, QCQ_C.
  4. Build preference pairs at the shared state and commit them.
  5. Train πϕ\pi_\phi for one epoch on the preference pool; the gate panel then accepts or rejects the update.

W2 / Q1. What is the full training objective, how are evidence value and net value combined, how are action-preference pairs constructed, how is the reference model updated, when do updates occur, and which 27B model is used with what configuration?

The researcher is trained with three DPO losses of the form in Eq. 5, weighted equally: L=13(Lnet+Levidence+Ledit),\mathcal L=\tfrac13\big(\mathcal L_{\mathrm{net}}+\mathcal L_{\mathrm{evidence}}+\mathcal L_{\mathrm{edit}}\big), that is, Lresearch=12(Lnet+Levidence)\mathcal L_{\mathrm{research}}=\tfrac12(\mathcal L_{\mathrm{net}}+\mathcal L_{\mathrm{evidence}}) mixed 2∶1 with Ledit\mathcal L_{\mathrm{edit}}.

Evidence value and net value are not merged into one score. Each produces its own pairs, and the pairs are combined only in the loss:

  • Lnet\mathcal L_{\mathrm{net}} compares a diagnostic with a direct edit at the same state, preferring the diagnostic if Vnet>0V_{\mathrm{net}}>0 and the direct edit if Vnet<0V_{\mathrm{net}}<0. It trains when to investigate.
  • Levidence\mathcal L_{\mathrm{evidence}} compares two diagnostics at the same state by VevidenceV_{\mathrm{evidence}}. It trains which probe to run.
  • Ledit\mathcal L_{\mathrm{edit}} compares program edits by their outcome.

Each decision state is forked into five branches. For each of two candidate diagnostics D0D_0 and D1D_1 there is a branch AA, which runs the diagnostic, restores the program and archive, and continues with the result visible, and a branch B′B', which does the same with the result hidden. A fifth branch CC skips the diagnostic and searches directly from the same state. Net pairs compare AiA_i with CC, and the evidence pair compares D0D_0 with D1D_1 by QAi−QBi′Q_{A_i}-Q_{B'_i}. A pair is kept only if its ordering is resolved: each branch runs 8 paired continuation seeds, we draw 5,000 bootstrap resamples over seeds and video clusters, and the 95% CI of the difference must exclude zero. Invalid executions and inconclusive measurements remain in the context but never receive positive credit.

πref\pi_{\mathrm{ref}} is the initial researcher checkpoint and stays fixed for all of training. The researcher parameters are frozen while the branches of a state run, and they are updated only after those branches finish, their credit is committed, and the pairs are built.

Table R1. Researcher training configuration.

ItemSetting
Frozen answererQwen2.5-VL-7B-Instruct
ResearcherQwen3.8-27B
Fine-tuningLoRA, r = 8, α = 16, on the text-attention q_proj and v_proj; all other parameters frozen
DPOβ = 0.1, learning rate 1e−4, one epoch over the preference pool per round, effective batch of 8 pairs
Reference policyInitial researcher checkpoint, fixed
Uncertainty rule8 paired continuation seeds; 5,000-sample seed/video-cluster bootstrap; 95% CI excludes 0
Branches128 states × 8 seeds × 5 branches (A0A_0, B0′B'_0, A1A_1, B1′B'_1, CC) = 5,120
Preference pairs992: 768 outcome, 144 net, 80 evidence
Fork compute1,024,000 GPU-s, counted as offline data-collection cost, including diagnostics and all continuations
Search budget20,000 GPU-s per seed, counted separately

Q2. Which dataset is used to compute Q in Table 3, and is it also used to train the researcher?

Q in Table 3 is measured on the source credit panel: each search picks its final program on the selection panel, and that program is scored once on the credit panel. The four panels share no video (395 videos, 836 questions in all). Support, selection and gate come from the nine OVO-Bench real-time and backward-tracing tasks. The credit panel (200 videos, 512 questions) holds 312 OVO-Bench questions from the real-time, backward-tracing and forward categories and 200 StreamingBench RTVU questions. Q weights the two benchmarks equally, whatever the 312∶200 split: Q(P)=12[AccRT(P)+AccBT(P)+AccFwd(P)3+AccSB(P)],Q(P)=\frac12\left[\frac{\mathrm{Acc}_{RT}(P)+\mathrm{Acc}_{BT}(P)+\mathrm{Acc}_{Fwd}(P)}{3}+\mathrm{Acc}_{SB}(P)\right], where each OVO category accuracy is the unweighted mean of its task accuracies, StreamingBench accuracy is question-weighted, and all scores are on a 0–100 scale.

Table R2. Development panels.

PanelBenchmarkVideosQuestionsRole in researcher training
SupportOVO (RT, BT)64108Indirect: diagnostic feedback enters the research state
SelectionOVO (RT, BT)74108Indirect: selects candidate programs
CreditOVO (RT, BT, Fwd) 312 + SB 200200512Direct: provides the preference supervision; Q in Table 3
GateOVO (RT, BT)57108No gradient; accepts or rejects updates

The credit panel does train the researcher, since its outcomes give QAQ_A, QB′Q_{B'} and QCQ_C. Candidate programs never see its labels and program selection never uses it, so no program in Table 3 is selected on the panel it is scored on. We will state this in Section 3.4 and in the caption of Table 3.

We also re-evaluated Table 3 on data that played no part in researcher training. The 56 frozen final programs of Table 3 (7 arms × 8 seeds) were rescored on every OVO-Bench and StreamingBench question whose video lies outside all four panels: 1,280 OVO-Bench and 2,000 StreamingBench questions. These videos were not used to train the researcher or to select programs, and Qheld-outQ_{\text{held-out}} uses the same formula as Q.

Table R3. Table 3 arms on held-out questions. Means over eight seeds; 95% t intervals (df = 7).

ResearcherQ on credit panel (Table 3)OVO-Bench OverallStreamingBenchQheld-outQ_{\text{held-out}} (95% CI)
StreamRSI full82.7065.7380.0672.89 [72.15, 73.63]
Net credit81.5064.6778.7871.72 [71.05, 72.39]
Evidence credit80.6863.7277.9570.84 [70.22, 71.46]
Outcome-only79.6062.5676.7469.65 [69.08, 70.23]
Frozen researcher + tools78.8061.8475.9468.89 [68.32, 69.46]
Direct search78.4061.4375.4068.42 [67.86, 68.98]
Ordinary reflection77.9060.7174.6967.70 [67.20, 68.20]

Table R4. Paired contrasts over the same eight seeds.

ContrastCredit panel (Table 4)Held-outHeld-out 95% CI
Full − Net credit+1.20+1.17[+1.03, +1.31]
Full − Evidence credit+2.02+2.05[+1.83, +2.28]
Full − Outcome-only+3.10+3.24[+2.90, +3.57]
Full − Direct search+4.30+4.48[+4.09, +4.86]

The seven arms keep their Table 3 order, no contrast moves by more than 0.18 points, and every paired interval excludes zero. The held-out scores come from a different question set and are not on the same scale as Q on the credit panel, so the Q=80.0Q=80.0 target does not apply to them.

The manuscript needs further revision to help readers clearly understand the main idea and contributions. I therefore maintain my original score.

Comments on the paper

4 comments

Thank you for the detailed review. The main additions are benchmark scores with all panel videos removed, a re-evaluation of Table 3 on questions outside every panel, the definition of Q with the full researcher configuration, and a readout test on natural video. We also withdraw the extrapolated costs.

W1. The benchmark results look tuned on the test benchmarks and are not comparable to the baselines.

We agree with all three points and will narrow what Tables 1 and 2 are used to claim.

The four development panels share no video (395 videos, 836 questions in all). Support, selection and gate come from the nine OVO-Bench real-time and backward-tracing tasks. The credit panel (200 videos, 512 questions) holds 312 OVO-Bench questions from the real-time, backward-tracing and forward categories and 200 StreamingBench RTVU questions. All panels are therefore part of the full OVO-Bench and StreamingBench evaluations. Table R1 rescores the programs after removing every question on a panel video.

Table R1. Benchmark scores with all panel videos removed.

BenchmarkQuestionsOursSimpleStream-4fΔ
OVO-Bench Overall, full set (Table 1)3,03565.5057.32+8.18
OVO-Bench Overall, panel-free1,28064.9056.80+8.10
StreamingBench RTVU, full set (Table 2)2,50080.6476.90+3.74
StreamingBench RTVU, panel-free2,00079.8576.35+3.50

Removing the panel videos lowers our OVO-Bench score by 0.60 and SimpleStream-4f by 0.52, so the margin moves by 0.08 points. On StreamingBench, 2,000 of the 2,500 questions remain (1,597 and 1,527 correct), and the margin goes from 3.74 to 3.50 points.

The OVO-Bench and StreamingBench programs are different programs: the StreamingBench program uses 8 frames for Counting, and the harness uses task-conditioned routing. The baselines pay no search cost, whereas each of our programs comes from a 20,000 GPU-s search with a 27B researcher. Tables 1 and 2 therefore report what a searched inference-time program reaches with a frozen 7B answerer, and they are not evidence of a better streaming method. We will remove the comparison language around HERMES and SimpleStream, add a search-cost column, and mark our rows as searched per benchmark with task-conditioned routing. The paper's central claim rests on Table 3, where every arm has the same answerer, program space and 20,000 GPU-s budget, and the revision will put it first.

W2. The main research-policy result (Table 3) is hard to verify, and part of it rests on modeling rather than measurement.

Q is the quality of a search's final program, which is selected on the selection panel and scored on the credit panel (200 videos, 512 questions). The credit panel shares no video with the support, selection and gate panels. Q weights OVO-Bench Overall and StreamingBench RTVU accuracy on the credit panel equally, whatever the 312∶200 split: Q(P)=12[AccRT(P)+AccBT(P)+AccFwd(P)3+AccSB(P)],Q(P)=\frac12\left[\frac{\mathrm{Acc}_{RT}(P)+\mathrm{Acc}_{BT}(P)+\mathrm{Acc}_{Fwd}(P)}{3}+\mathrm{Acc}_{SB}(P)\right], where each OVO category accuracy is the unweighted mean of its task accuracies, StreamingBench accuracy is question-weighted, and all scores are on a 0–100 scale. The initial program has Q0=70.1Q_0=70.1, and the target is Q=80.0Q=80.0. Confidence intervals in Tables 3 and 4 are t intervals over the eight research seeds (df = 7), paired by seed for the contrasts.

Because the credit panel also supplies the researcher's training signal, we re-evaluated Table 3 on held-out data. The frozen final programs of the seven Table 3 arms and of the two closest fork-ablation variants, naive fork and unstructured MC credit, were rescored (72 programs: 9 variants × 8 seeds) on every OVO-Bench and StreamingBench question whose video lies outside all four panels: 1,280 OVO-Bench and 2,000 StreamingBench questions. These videos were not used to train the researcher or to select programs, and Qheld-outQ_{\text{held-out}} uses the same formula as Q.

Table R2. Researcher variants on held-out questions. Means over eight seeds; 95% t intervals (df = 7).

ResearcherQ on credit panelOVO-Bench OverallStreamingBenchQheld-outQ_{\text{held-out}} (95% CI)
StreamRSI full82.7065.7380.0672.89 [72.15, 73.63]
Naive fork81.6064.8879.1071.99 [71.33, 72.65]
Net credit81.5064.6778.7871.72 [71.05, 72.39]
Unstructured MC credit81.3064.4178.5671.48 [70.82, 72.15]
Evidence credit80.6863.7277.9570.84 [70.22, 71.46]
Outcome-only79.6062.5676.7469.65 [69.08, 70.23]
Frozen researcher + tools78.8061.8475.9468.89 [68.32, 69.46]
Direct search78.4061.4375.4068.42 [67.86, 68.98]
Ordinary reflection77.9060.7174.6967.70 [67.20, 68.20]

Table R3. Paired contrasts over the same eight seeds.

ContrastCredit panelCredit 95% CIHeld-outHeld-out 95% CI
Full − Naive fork+1.10[+0.93, +1.27]+0.90[+0.73, +1.07]
Full − Net credit+1.20[+0.89, +1.51]+1.17[+1.03, +1.31]
Full − Unstructured MC credit+1.40[+1.21, +1.59]+1.41[+1.21, +1.61]
Full − Evidence credit+2.02[+1.71, +2.34]+2.05[+1.83, +2.28]
Full − Outcome-only+3.10[+2.84, +3.36]+3.24[+2.90, +3.57]
Full − Direct search+4.30[+3.92, +4.68]+4.48[+4.09, +4.86]

All nine variants keep their credit-panel order, no contrast moves by more than 0.20 points, and every paired interval excludes zero. The held-out scores come from a different question set and are not on the same scale as Q on the credit panel, so the Q=80.0Q=80.0 target does not apply to them. The programs in Tables 1, 2 and R1 come from separate per-benchmark searches and are not among these 72, which is why their held-out scores differ from the full-researcher row (64.90 vs. 65.73 on OVO-Bench).

Table R4. Researcher training configuration.

ItemSetting
Frozen answererQwen2.5-VL-7B-Instruct
ResearcherQwen3.8-27B
Fine-tuningLoRA, r = 8, α = 16, on the text-attention q_proj and v_proj; all other parameters frozen
DPOβ = 0.1, learning rate 1e−4, one epoch over the preference pool per round, effective batch of 8 pairs; reference policy fixed to the initial checkpoint
Objective13(Lnet+Levidence+Ledit)\tfrac13(\mathcal L_{\mathrm{net}}+\mathcal L_{\mathrm{evidence}}+\mathcal L_{\mathrm{edit}})
Uncertainty rule8 paired continuation seeds; 5,000-sample seed/video-cluster bootstrap; 95% CI excludes 0
Branches128 states × 8 seeds × 5 branches = 5,120; per state, AA and B′B' for each of two candidate diagnostics, plus one direct-search branch CC
Preference pairs992: 768 outcome, 144 net, 80 evidence
Fork compute1,024,000 GPU-s

The forks cost 1,024,000 GPU-s, covering diagnostic execution and all continuations. This is a one-time offline cost of collecting training data for the researcher, separate from the 20,000 GPU-s per-seed search budget in Table 3, which compares searches run by researchers that are already trained. The revision will report both numbers in Section 4.1.

We withdraw the extrapolated costs of 28k, 34k and 41k GPU-s in Figure 3b and Table 8, along with the curve-based AUC, which relies on the same assumed asymptote for these variants. Variants that do not reach Q=80.0Q=80.0 are reported as right-censored at 20,000 GPU-s, and all variants are compared by measured final quality.

Table R5. All researcher variants, eight seeds, 20,000 GPU-s each.

VariantFinal Q (95% CI)Seeds at Q ≥ 80First target cost (GPU-s)
StreamRSI full82.70 [82.28, 83.12]8/89,400
Naive fork81.60 [81.06, 82.14]8/810,400
Net credit81.50 [80.95, 82.05]8/810,800
Unstructured MC credit81.30 [80.74, 81.86]8/811,200
Evidence credit80.68 [80.10, 81.25]8/811,600
Outcome-only79.60 [79.05, 80.15]0/8> 20,000
Mask only79.29 [78.71, 79.87]0/8> 20,000
Evidence permutation78.90 [78.29, 79.51]0/8> 20,000
Frozen researcher + tools78.80 [78.21, 79.39]0/8> 20,000
Noise evidence78.55 [77.91, 79.19]0/8> 20,000
Direct search78.40 [77.73, 79.07]0/8> 20,000
Ordinary reflection77.90 [77.17, 78.63]0/8> 20,000

The conclusions of Section 4.3.2 hold without the extrapolation: mask only is below outcome-only in final quality, and evidence permutation and noise evidence fall to the level of direct search.

W3. The contribution doesn't fit the framing, and the supporting evidence is thin.

We agree that the paper is about credit assignment for a research agent, and that the evidence comes from one answerer, one program space, and streaming and long-video benchmarks. We have no results in another research domain. The revision will state the scope in these terms and remove wording that suggests broader generality.

Deletion and substitution interventions [Meng et al., 2026; Hu et al., 2026] remove or replace one step of a trajectory. Removing an experiment removes both its result and its effect on the artifact, which confounds the two; the naive-fork and mask-only rows of Table R5 measure this. The naive fork is the closest variant to full StreamRSI, and the gap holds on held-out questions: +0.90 [+0.73, +1.07] (Table R3). Restoring the pre-experiment state accounts for 1.10 of the 3.10-point gain over outcome-only training on the credit panel and 0.90 of 3.24 on held-out questions; the rest comes from the forked comparisons themselves. Our B′ arm runs the same experiment, restores the artifact, charges the same budget, and withholds only the result. The C arm compares against spending that budget on direct search, a cost the step-level interventions do not account for. Value of information is the quantity we estimate. Classical formulations compute it from a model of outcomes; we estimate it from continuations of the actual researcher and use it as preference supervision.

For the readout analysis, we ran the same test on natural OVO-Bench videos with the same frozen answerer.

Table R6. Readout with and without a relevant focus frame. Paired differences with 95% bootstrap CIs.

ComparisonPairsControlFocus armΔ (pp) [95% CI]
Synthetic (Table 7): one focus frame vs. long KV only2416.6775.00+58.33 [+41.3, +75.3]
Natural: relevant vs. irrelevant focus frame5451.85 (28/54)62.96 (34/54)+11.11 [+3.70, +20.37]
Natural: relevant focus frame vs. original cache5664.29 (36/56)64.29 (36/56)0.00 [−8.93, +8.93]
Natural: KV relevant vs. irrelevant focus633.33 (2/6)50.00 (3/6)+16.67 [0.00, +50.00]

On natural video, a relevant focus frame beats an irrelevant one by 11.11 points, and the interval excludes zero. Added on top of the original cache, it gives no gain on these 56 pairs, so the large synthetic gap does not appear when the cache already supports the answer. The KV-level comparison has only 6 pairs and is inconclusive. The revision will present the synthetic result as a controlled probe, report these natural-video results next to it, and drop the claim of a general retention–readout gap.

Reviewer_dyf5OpenReview ↗

Post-Rebuttal Official Comment by Reviewer dyf5

Thank you for the detailed experiments. While my verifiability concern is resolved, the rebuttal actually raises a more fundamental question of what exactly does the paper contribute. Given the authors agree that due to the per-benchmark tuning (program-search), the main tables are not apples-to-apples comparable evidence of StreamRSI being a better method compared to prior methods. Then it leaves only Table 3, which is essentially an ablation table for different variants of StreamRSI itself. An ablation table cannot serve as the main results without fair comparison with prior existing method. Additionally, the cost-performance tradeoff is not justified, given StreamRSI pays a substantial (nearly 1M GPU-s) one-time training cost, on top of a 20k GPU-s additional search budget per-benchmark program; and it is not even fairly comparable to any prior method. Given this limited scope and contribution, I maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.