- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper targets at Streaming Reference-to-Video (R2V), which is an autoregressive R2V generation in which reference images can be added or removed during rollout. The authors noticed two problems in this topic: (a) identities already rendered into history typically overpower a newly inserted reference; (b) the chunk where the reference set changes is especially error-prone.
Therefore, the authors propose two modules to address these problems:
- Reference reflective value gate: it reconstructs each history value token in every self-attention layer, and then gates the reconstruction per token.
- Virtual rollout error chain: it extends SVI's error recycling, and then injects a residual tube into the history. This can approximate multi-chunk error accumulation at one forward pass per iteration.
Based on the reported results, StreamingR2V is claimed to be best or near-best on most metrics (R2V metrics, timing metrics, interaction-compliance rates).
Critical strengths
[S1]
I like the idea of combining autoregressive video generation and R2V generation, which are typically treated as two independent topics. The paper really fills a gap between R2V and long/interactive generation.
Moreover, to adapt this new topic, the authors propose an evaluation protocol that actually targets it. Specifically: (a) R2V metrics are scored per condition group, so a subject is only required while its reference is active; (b) multi-subject matching in OpenS2V-Eval is fixed with Hungarian assignment, and missing subjects score zero instead of being dropped; and much more reasonable metrics.
A new topic + the corresponding evaluation protocol make this paper comprehensive.
[S2]
Based on the reported numbers, the proposed modules look promising. Specifically, from Table 3, it seems this cheap / drop-in module can (a) significantly raise post-interaction Face-Sim + DINO-I, (b) cut the before/after gap by about almost half (Face-Sim) to two-thirds (DINO-I), and (c) also lift pre-interaction fidelity. These numbers are consistent with the argument that drifted identity in history shouldn't serve as an identity condition.
Critical weaknesses
[W1]
I think the experiments are not enough to show that the proposed mechanisms are necessary and better than some simpler alternatives. They also don't show the proposed modules work for the claimed reasons. It is especially important because these two modules are the technical contribution.
For example, I can easily think of these alternatives which are untested in the paper:
- For the continuity trade-off, you could try to down-weight or mask history attention. Also, sanitized values could actually change hidden states, so later-layer attention is not actually untouched.
- You can try to mask subject regions in history using the SAM3 masks already in the pipeline.
- Stronger reference guidance.
- An R2V baseline that also conditions on history frames.
For the "Error chain", the l-max ablation in the paper is non-monotonic. l-max = 2 is worse than l-max = 1 on post-interaction Face-Sim & DINO-I. Since depth is not an input to the model, the chain mainly reshapes the mixture of injected errors. The "virtual rollout" approximation is never compared with real inference-time residuals.
Lastly, on the mechanism side, you can try to visualize the "gate" and "learned y values". Table 2 averages post-interaction fidelity over all subjects in the new set, so the claim that the gate specifically frees the newly inserted subject is not directly shown.
[W2]
I noticed that the interaction-compliance evidence is weaker than presented. This is consequential because interaction compliance and long-horizon consistency are the headline claims. There are a couple of concerns around this:
- The benchmark and metrics are self-constructed. Also, event counts are small (34 entries and 14 exits at 200 frames) so a single exit moves XSR by about 0.07.
- From my understanding, no non-commercial method sees a new reference before its cue. Two-stage pipelines invoke it only at the boundary, and SFT & StreamingR2V follow the schedule. However, at 200 frames, these baselines score PR of 0.08-0.13, and VACE+SVI scores 0.579.
- I am also confused on the PR definition. The paper defines PR as a fraction of entry events. ESR and XSR come out as multiples of 1/34 and 1/14, but the reported PR values do not. It seems like PR is computed differently than described.
Questions for authors
[Q1]
I am not sure how the authors produce non-zero PR for methods that never receive the new reference before its cue. Could the authors explain this? Also, could you report:
- the detector's false-positive rate on clips where a subject is known to be absent
- PR and RR under a sweep of tau, separately for the face and DINO backends
- how often a "premature" match is actually an existing subject
[Q2]
In Algorithm 1, is a tail produced at depth l-max dropped, or written back into the deepest bank? Is the l-max = 1 row literally SVI's self-recycling bank? Given the non-monotonic l-max results, how stable are Tables 3 and 7 across different random seeds?
[Q3]
Can Table 2 be split into newly inserted vs. already-present subjects, and into entry vs. exit boundaries? Can the Table 3 ablation also report ESR/PR/XSR/RR? For exits, the departing subject's reference is removed, so the gate has nothing to explain away its identity in history. How are exits represented in the curated training data, and does either module affect XSR or RR?
[Q4]
I think it is better to reconcile Figure 3 with Table 1 for MAGREF+SVI. How were the 600- and 1000-frame points computed, given that the benchmark has 200/400/800/1200-frame samples? Why is 1200 omitted, and why are VACE+SVI and Phantom+SVI not plotted?
2 comments
Dear Reviewer Pguk,
Thank you for your response and for recognizing that our rebuttal has resolved most of your concerns. We address your remaining concerns below.
1. More Detailed PR Analysis
PR covers both the pre-cue region and the tolerance window that follows the cue, so we now report the two parts separately, over all 120 samples of the benchmark (). The sampled frames of the pre-cue region and the tolerance window are in a ratio of about 3:1.
Table H: PR split into the pre-cue region and the tolerance window.
| Method | PR_precue | PR_window | PR |
|---|---|---|---|
| VACE+SVI | 0.155 | 0.673 | 0.285 |
| SVI+R2V SFT | 0.101 | 0.139 | 0.111 |
| StreamingR2V | 0.032 | 0.053 | 0.037 |
We have the following findings:
-
The gap does not depend on the window or the pre-cue length. StreamingR2V is lowest in both parts (0.032 vs. 0.155 / 0.101 before the cue; 0.053 vs. 0.673 / 0.139 in the window). As PR is their frame-weighted mean, the ranking holds for any pre-cue length.
-
The tolerance window exposes abrupt entry. The tolerance-window part of PR reflects an important failure that the other metrics hardly capture: the new subject is already fully visible right after the cue, instead of entering the scene gradually. Such an abrupt entry conflicts with the nature of streaming reference-to-video, yet ESR still counts it as a success. Every method has a higher PR in the window than before the cue. The increase is sharpest for VACE+SVI (0.673 vs. 0.155), whose new subject comes from a freshly generated clip and thus exhibits exactly this failure, while StreamingR2V stays lowest (0.053). In short, the window part shows how smoothly a new subject enters, and StreamingR2V introduces new subjects most smoothly.
We will add Table H and the definitions of both parts to the appendix of the revised paper.
2. Error Chain
The drop at . This drop appears only in post-cue DINO-I (0.612 to 0.578), while post-cue FS stays within its seed variation and both pre-cue metrics improve. is also the least stable setting across seeds (post-cue FS std 0.044, vs. at most 0.02 elsewhere). Our design does not rely on this setting, since and both exceed on all four metrics. We will add this discussion to the revised paper.
Check against real rollout errors. We clarify that the chain is a training-time strategy: each stored tube is a real model residual, but the history it is injected into is not a real rollout. Furthermore, a step-wise comparison with real inference-time residuals (as raised in your original W1) would need a ground-truth video at inference, which R2V does not have: the only inputs are scheduled reference images and prompts, and the references in our benchmark are not taken from any video. Real rollout errors are therefore observable only through their effect on the generated long videos, where the chain helps (Tab. 3 and Tab. 7 of the paper; Tables B and G of our first-round rebuttal). We will update this discussion in our revised paper.
We hope the clarifications above address your remaining concerns. Again, we sincerely thank you for your constructive suggestions and your continued positive evaluation of our submission.
Best regards,
Authors of Submission 753
Thanks to the authors for the detailed rebuttal. I have carefully read the rebuttal & the other reviewers' comments. First, I noticed that some of my concerns are shared by the other reviewers.
The rebuttal has addressed my [W1] & most of my questions. However, there are still two remaining concerns:
Overall, the rebuttal makes me more confident in the results. I agree with the other reviewers that the task is well motivated. Considering that two remaining concerns might affect how the interaction-compliance & error-chain claims should be read, I will keep my rating as weak accpetance.