- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
ViNAYA is a streaming non-autoregressive video captioner that fixes its read/write schedule in closed form before decoding, rather than learning it. A text frontier commits and emits spans left to right while a matched source horizon reveals the video at a coordinated pace. The authors show this gives a predictable fraction of the clip visible per emitted word, and that a single exponent cannot. Because the schedule is fixed in advance, emitted spans are final and never depend on future source content, regardless of training. On three video captioning datasets with frozen encoders, ViNAYA roughly halves Average Lagging against a block-diffusion streaming baseline at mostly comparable quality, and uses fewer forward passes than autoregressive decoding.
Critical strengths
Consistent, well-controlled latency gains. ViNAYA reduces lag substantially on all three datasets, across both short-caption and long-caption settings. All systems share the same architecture, training setup and encoders, so the improvement can be credibly attributed to the proposed schedule rather than to implementation differences.
Thorough empirical reporting. The authors show where the method loses, including the gap to an offline upper bound, and statistical significance tests. They also include ablations that isolate each design choice.
Critical weaknesses
The method is hard to follow and lacks an illustration. The core idea is two coordinated schedules, one for the text and one for the video, but the paper never shows how they interact. There is no figure or worked example of a single decoding run, such as which words are written at each step and how much of the video is visible at that moment. Readers must piece the method together from dense notation spread across several sections. This makes the contribution hard to understand, and verify, and it weakens the paper’s clarity even if the idea is sound.
The empirical evidence is narrower than the claims suggest. The streaming setup relies on pre-segmented clips with known boundaries and lengths, which is closer to windowed processing than to a true live feed. The paper does not discuss this limitation.
Questions for authors
Please refer to the weakness section.
3 comments
Official Comment by Reviewer
Thanks for the rebuttal and for revising the paper. I went through the updated version and Figure 1 really helped. It's much easier to see what's going on with the two schedules now. I would like to keep my rating at 3.
We thank the reviewer for reading the revised version and for the kind note on Figure 1.
We hope your second concern regarding pre-segmented setting is also sufficiently addressed by proper scoping in Sections 3.1 and 5. If anything on this point (or on anything else) is still open, please let us know and we will be happy to clarify it right away (as the discussion phase closes today).
Thanks again.
Response to Reviewer n1rq. It includes a worked decode illustration (Fig. 1, Table 11) and windowed streaming scope.
We thank the reviewer for raising important points. We address both the concerns as under:
W=Weakness
W1 (hard to follow, no illustration). We acknowledge this was a shortcoming and with an extra page we make up for it in Figure 1. The figure is built from a real ViNAYA decode on Charades. It shows for each pass how much of the clip is visible, which span is written, and the text emitted so far. It also shows a one-to-one comparison with the block baseline on the same window. Table 11 adds twelve more decodes, including failures.
W2 (pre-segmented clips). This is right. Our setting is windowed streaming rather than a live feed with unknown boundaries. We now explicitly scope that in Sections 3.1 and 5. The annotated segment length Fc sets the scale of the horizon, and within each window no feature beyond Ωk is read at pass k (Theorem 2). For a continuous stream with an unknown endpoint, Fc could either be supplied by an online segmenter (that might detect where an event ends) or replaced by a fixed chunk width. We have not evaluated this and leave it to future work.