acceptodds
Under review as a conference paper at ICLR 2027

ViNAYA: Yoked Schedules for Streaming Non-Autoregressive Captioning

Abstract

A streaming captioner must decide how much of a clip it has seen before writing every word. Autoregressive models answer this by construction at the cost of one sequential pass per token. Non-autoregressive models are faster but cannot stream because no prefix is guaranteed final until the last pass and nothing can leave early. We introduce ViNAYA (Vision Non-Autoregressive Yoked Architecture), which closes this gap by turning the write order into a declared schedule rather than a learned behaviour. A monotone text frontier commits and emits spans from left to right, while a yoked monotone source horizon reveals the clip at a matching pace. Both frontiers are closed mathematical forms fixed before decoding begins, making them provably safe under arbitrary arrival. Therefore, no emitted token can depend on source that has not yet arrived, and every emitted span is final the moment it leaves. ViNAYA's two-knob design permits separate control over both the source reveal and the text schedule, with the two knobs joined by a ratio grounded in corpus geometry. This coupling yields a predictable exposure budget of , which we validate against measured exposure on three video captioning corpora with three frozen encoders. On the same corpora, ViNAYA reduces Average Lagging by 43 to 50 percent against the strongest NAR streaming baseline, at quality statistically indistinguishable in most cases and with core architectural elements shared by construction. While an autoregressive wait- baseline retains a small quality edge on two corpora, ViNAYA compensates that with up to 2.7 times fewer forward passes per window and a sequential cost that stays bounded as captions grow. The per-token structure of autoregression forces its inference time to grow with output length, and on ActivityNet, where captions are longest, the autoregressive baseline requires roughly 408 seconds of test-split inference against 188 seconds for ViNAYA.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Non-autoregressive decoding, Streaming video captioning, Multimodal Generation

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_n1rqOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

ViNAYA is a streaming non-autoregressive video captioner that fixes its read/write schedule in closed form before decoding, rather than learning it. A text frontier commits and emits spans left to right while a matched source horizon reveals the video at a coordinated pace. The authors show this gives a predictable fraction of the clip visible per emitted word, and that a single exponent cannot. Because the schedule is fixed in advance, emitted spans are final and never depend on future source content, regardless of training. On three video captioning datasets with frozen encoders, ViNAYA roughly halves Average Lagging against a block-diffusion streaming baseline at mostly comparable quality, and uses fewer forward passes than autoregressive decoding.

Critical strengths

Consistent, well-controlled latency gains. ViNAYA reduces lag substantially on all three datasets, across both short-caption and long-caption settings. All systems share the same architecture, training setup and encoders, so the improvement can be credibly attributed to the proposed schedule rather than to implementation differences.

Thorough empirical reporting. The authors show where the method loses, including the gap to an offline upper bound, and statistical significance tests. They also include ablations that isolate each design choice.

Critical weaknesses

The method is hard to follow and lacks an illustration. The core idea is two coordinated schedules, one for the text and one for the video, but the paper never shows how they interact. There is no figure or worked example of a single decoding run, such as which words are written at each step and how much of the video is visible at that moment. Readers must piece the method together from dense notation spread across several sections. This makes the contribution hard to understand, and verify, and it weakens the paper’s clarity even if the idea is sound.

The empirical evidence is narrower than the claims suggest. The streaming setup relies on pre-segmented clips with known boundaries and lengths, which is closer to windowed processing than to a true live feed. The paper does not discuss this limitation.

Questions for authors

Please refer to the weakness section.

3 comments
AuthorsOpenReview ↗

Response to Reviewer n1rq. It includes a worked decode illustration (Fig. 1, Table 11) and windowed streaming scope.

We thank the reviewer for raising important points. We address both the concerns as under:

W=Weakness

W1 (hard to follow, no illustration). We acknowledge this was a shortcoming and with an extra page we make up for it in Figure 1. The figure is built from a real ViNAYA decode on Charades. It shows for each pass how much of the clip is visible, which span is written, and the text emitted so far. It also shows a one-to-one comparison with the block baseline on the same window. Table 11 adds twelve more decodes, including failures.

W2 (pre-segmented clips). This is right. Our setting is windowed streaming rather than a live feed with unknown boundaries. We now explicitly scope that in Sections 3.1 and 5. The annotated segment length FcF_c sets the scale of the horizon, and within each window no feature beyond Ωk\Omega_k is read at pass kk (Theorem 2). For a continuous stream with an unknown endpoint, FcF_c could either be supplied by an online segmenter (that might detect where an event ends) or replaced by a fixed chunk width. We have not evaluated this and leave it to future work.

Reviewer_n1rqOpenReview ↗

Official Comment by Reviewer

Thanks for the rebuttal and for revising the paper. I went through the updated version and Figure 1 really helped. It's much easier to see what's going on with the two schedules now. I would like to keep my rating at 3.

We thank the reviewer for reading the revised version and for the kind note on Figure 1.

We hope your second concern regarding pre-segmented setting is also sufficiently addressed by proper scoping in Sections 3.1 and 5. If anything on this point (or on anything else) is still open, please let us know and we will be happy to clarify it right away (as the discussion phase closes today).

Thanks again.

Reviewer_idtYOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes ViNAYA, a streaming non-autoregressive captioner with coupled text and source frontiers. The fixed schedule enables incremental span emission with structural guarantees on source dependence and finality. Experiments on three video-captioning datasets show lower latency than the constructed NAR baselines with comparable quality in most settings.

Critical strengths
  1. The method gives an explicit two-frontier schedule that jointly controls how much source is revealed and how much text is committed at each pass.
  2. ViNAYA reduces Average Lagging by roughly 45–50% over the blockwise NAR baseline while maintaining comparable quality in most reported settings.
Critical weaknesses
  1. The streaming setup may implicitly use future information. The source horizon depends on the full segment length (F_c), which appears to come from annotated segment boundaries. This assumption is unclear for a true online stream where the endpoint is unknown.
  2. The latency gains may partly come from the fixed decoding budget rather than the yoking mechanism itself. ViNAYA uses a bounded number of passes, while the competing AR and blockwise baselines scale with output length. A compute-matched fixed-pass NAR baseline is missing.
  3. Several practical factors affecting the schedule are insufficiently analyzed. The method depends on predicted target length and a manually chosen source floor (\phi), but no oracle-length or (\phi)-sensitivity analysis is provided.
  4. The schedule is content-agnostic and irreversible. Early spans are committed regardless of uncertainty, but the paper does not analyze failures caused by ambiguous early observations.
  5. The paper lacks qualitative streaming examples. Showing how source exposure and emitted captions evolve across passes would help illustrate the actual behavior of the proposed schedule, including early-commitment successes and failures.
Questions for authors
  1. Is (F_c) obtained from ground-truth segment boundaries? How would ViNAYA handle an unknown endpoint in a continuous stream?
  2. Can you compare against a fixed-pass NAR baseline with matched decoding cost and source exposure?
  3. How sensitive are the results to target-length prediction errors and the choice of (\phi)?
  4. How does the method behave when early observations are ambiguous but committed spans cannot be revised?
  5. How does the geometry-derived (\nu) compare with the fixed (\nu=0.5) used in the main experiments?
4 comments
AuthorsOpenReview ↗

Response to Reviewer idtY. It includes windowed streaming scope, matched-compute and exposure sweeps, sensitivity analysis and qualitative examples

We thank the reviewer for very specific questions. This feedback guided most of the experiments we ran during the rebuttal window. Below we address each question/concern one by one.

W=Weakness and Q=Question

W1/Q1 (segment boundaries). It is true that FcF_c is the annotated segment length and we now mention this in Sections 3.1 and 5. The only thing taken from the future is this length, which sets the scale of the horizon. The caption length N^\hat N is a separate quantity and is predicted from the previous caption only. Moreover, no feature beyond Ωk\Omega_k is read at pass kk (Theorem 2). Our setting is therefore windowed streaming. For a continuous stream with an unknown endpoint, FcF_c could either be supplied by an online segmenter (that might detect where an event ends) or replaced by a fixed chunk width. We have not evaluated this and leave it to future work.

W2/Q2 (matched cost and exposure). To separate the pass budget from the schedule we ran a new sweep on all three corpora (Table 3). It varies the number of passes for both systems and compares them at matched measured passes. ViNAYA has lower AL and LAAL at every matched pass count, with AL lower by 40 to 52 percent at the budgets of Table 1. Thus, the lag gain does not come from the pass budget. As for exposure, ViNAYA reads less of the clip per token than the block baseline in Table 3. To match exposure as well we take ν=0.3\nu=0.3 on Charades, where both systems read about the same share of the clip (0.75 against 0.76) at a similar cost (6.0 against 6.5 passes for L=4L=4, S=3S=3). There ViNAYA has BLEU-4 of 0.0623 against 0.0482 and CIDEr of 0.4918 against 0.4645, with AL of 7.36 against 7.89 (Tables 7, 9 and 12). The lag margin is smaller in this setting, which indicates that part of the gain in Table 3 comes from reading less. This is the trade that the ratio ν\nu controls.

W3/Q3 (length prediction and ϕ\phi). Our new experiments show that the length prior falls below the true length in 1.1 to 3.4 percent of windows (Table 8). An oracle length lowers BLEU-4 and raises AL because an exact length removes the early exit at EOS. For ϕ\phi we tried 1, 3, 5 and 8. It mainly moves latency (AL goes from 1.93 to 3.83 on ActivityNet) while quality changes little.

W4/Q4 (ambiguous early observations). It is true that a wrong early commit stays (Table 11 now shows real examples). However, to measure this effect we decoded the same weights with the entire clip visible (Table 10). We find the first-span word precision changes by at most 1.2 points on all three corpora, which indicates that the early commit is not where most of the quality is lost. We scoped the schedule as content-agnostic because it lets us state the budget and the dependency bound before decoding, and we discuss a learned read policy as a natural extension in Section 5.

W5 (qualitative examples). We acknowledge this was a shortcoming and with an extra page we make up for it in Figure 1. The figure is built from a real ViNAYA decode on Charades. It shows for each pass how much of the clip is visible, which span is written, and the text emitted so far. It also shows a one-to-one comparison with the block baseline on the same window. Plus Section 3.2 now walks through the same numbers right after Eq. 1. Finally, Table 11 adds twelve more decodes, including failures.

Q5 (geometry-derived ν\nu). The geometry-derived rule gives ν=0.71\nu=0.71 for Charades and YouCook2 and 0.580.58 for ActivityNet, which lies between the two values we sweep, i.e., 0.5 and 0.75. Table 9 now reports quality for this sweep. Going from 0.5 to 0.75 lowers AL by 2.6 to 4.1 but also lowers BLEU-4 on all three corpora. This shows that the geometry-derived rule points to a somewhat more aggressive setting and 0.5 is a conservative choice instead.

Dear Reviewer idtY, since your questions guided most of our revised submission, we shall be very grateful to know whether we addressed your concerns. We will be happy to clarify anything in the remaining time (as the discussion phase closes today).

Thanks.

Thank you for the detailed rebuttal and additional experiments. The matched-compute comparisons, sensitivity analyses, and qualitative examples have addressed most of my concerns. I appreciate the clarified scope of windowed streaming and the demonstrated latency benefits.

However, the method still relies on predefined segment boundaries, and its applicability to continuous online streaming with unknown endpoints remains unverified, which limits the generality of the contribution. Nevertheless, given the additional evidence and the value of the proposed framework within its stated scope, I am willing to raise my rating to 3 (Weak Accept).

Thank you for reading the revised work and raising the rating. We agree that application to continuous streams with unknown endpoints remain unverified. Sections 3.1 and 5 scope our work accordingly.

Thanks again.

Reviewer_Mf3TOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

It proposes a method for streaming non-autoregressive video captioning. The main idea is to divide the output into several spans that are generated in parallel within each span and committed sequentially from left to right. A separate source schedule determines how much of the video is visible when each span is generated. The paper further derives a relationship between the source/text schedules and the expected video exposure.

The core algorithm is relatively simple, while several theoretical results mainly formalize properties that follow directly from the construction. I am not yet convinced that the current technical and empirical contributions are sufficiently strong.

Critical strengths

1, The paper addresses an interesting problem of enabling incremental output for non-autoregressive captioning.

2, The predetermined read/write schedules provide clear guarantees on causality and prefix finality.

3, The method shows a reasonable quality-efficiency trade-off, particularly for longer captions.

Critical weaknesses

1, The method is conceptually close to blockwise/semi-autoregressive decoding with an additional source schedule.

2, Some of the theoretical results appear to formalize direct consequences of the algorithmic design. For example, prefix finality largely follows by construction because previously committed spans are explicitly frozen, while the bounded number of sequential passes follows directly from the fixed decoding schedule.

3, The predetermined schedule cannot adapt to individual video content or prediction uncertainty. Different videos may require different amounts of visual evidence before a caption prefix can be confidently committed.

4, The empirical evaluation is relatively small scale (a 4-layer decoder trained from scratch over frozen visual features).

5, Self-constructed weak NAR baseline. Comparisons with stronger adaptive or learned streaming policies would make the empirical conclusions more convincing.

Questions for authors

See weakness.

2 comments
AuthorsOpenReview ↗

Response to Reviewer Mf3T. It includes two-exponent budget theory, 66M pretrained backbone, and baseline sweep

We thank the reviewer for providing us a detailed review. Below we address each concern one by one:

W=Weakness

W1 (close to semi-autoregressive decoding). ViNAYA differs from block decoding in three ways: (i) spans follow a power law and span shape alone moves BLEU-4 by 10 to 42 percent at fixed exposure (Table 4) (ii) the source horizon runs on the same clock with a closed-form budget, and (iii) the sequential cost is bounded regardless of caption length (Proposition 2).

W2 (theory follows from the design). This is right for Theorem 2, Theorem 3 and Proposition 2, and it is by design (as these are our load-bearing guarantees). More importantly, they hold for any weights, which mask-predict cannot offer (as shown by our Proposition 1). Lastly, Theorem 1(i) does not follow by construction, because nothing in the design suggests that a single shared exponent fixes the exposure at 1/2 for every value. This result is the reason a second exponent was needed and it gives the method a controllable budget of 1/(1+ν)1/(1+\nu).

W3 (no adaptation to content). We agree that our schedule at present does not adapt to individual clips or to prediction uncertainty. We scoped it as such because it lets us state the exposure budget (Theorem 1) and the dependency bound (Theorem 2) before decoding. To measure this tradeoff we decoded the same weights with the entire clip visible and found that the first-span word precision changes by at most 1.2 points (Table 10). This tells us that the early commit is not where the quality is lost. Still, individual clips can be ambiguous early on, so we show such failure cases in Table 11. The amount of evidence before the first commit can also be raised through the source floor ϕ\phi at a measured latency cost (Table 8). And since content adaptation is a separate line of work, we discuss a learned read policy as a natural extension in Section 5.

W4 (scale). This is very useful feedback. To test whether our findings hold beyond a 4-layer decoder trained from scratch, we repeated the ActivityNet comparison (during the rebuttal window) with both systems on a pretrained DistilBERT backbone of 66M parameters (Appendix F, Table 13). We find that the lag advantage carries over, with AL lower by 51 percent and LAAL by 37 percent at a similar pass count. Moreover, the quality stays at parity. The measured exposure is also unchanged (0.678 against 0.674), which is in line with the budget being a property of the schedule and not of the weights. The visual features, however, still remain frozen in this experiment.

W5 (baseline strength). As there is no off-the-shelf streaming non-autoregressive baseline for video captioning, we built the one that shares every component with ViNAYA except the schedule. To check that it is not a weak baseline we now sweep its block size and rounds in Table 3 (during the rebuttal window). We find that it stays ahead of ViNAYA on YouCook2 at every budget, and that at L=2L=2 it reaches the best BLEU-4 and CIDEr on both Charades and YouCook2. Even in this setting ViNAYA has lower AL and LAAL at every matched pass count. On adaptive or learned policies such as NAST, our rationale is that they belong to a different line of work. NAST learns an adaptive policy (for text-to-text translation), so its latency is known only after training. ViNAYA, on the other hand, is fixed-policy (like the wait-kk family) with a latency budget set before decoding. Adapting NAST to video would also require re-deriving its alignment loss. For this reason we compare within the fixed-policy class and leave NAST to future work.

Dear Reviewer Mf3T, we would be grateful to know whether our response and the revised paper addressed your concerns. In particular, does the pretrained-backbone experiment added for weakness no. 4 (W4) answer your concern on scale? We will be happy to clarify anything in the remaining time (as the discussion phase closes today).

Thanks.

Reviewer_Dy6POpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ViNAYA, a streaming non-autoregressive video captioner that commits text spans while incrementally revealing source features. It proposes yoked source and text schedules with a closed-form exposure budget, enabling parallel decoding with guaranteed prefix finality. Evaluations on three video-captioning datasets show ViNAYA reduces Average Lagging by 45–50% over a block-diffusion baseline while maintaining comparable quality, with the largest efficiency gains on long captions.

Critical strengths
  • This paper addresses an interesting and practical research question: enabling non-autoregressive models to generate streaming captions.
  • The proposed yoked scheduling mechanism is clearly motivated and supported by theoretical guarantees on exposure, causality, and output finality.
  • The experiments are comprehensive and well controlled, covering three datasets, multiple baselines, and quality, latency, and inference-efficiency metrics.
Critical weaknesses
  • Concern about content-agnostic scheduling. ViNAYA fixes its source and text frontiers before decoding using corpus-level hyperparameters, so it cannot adapt to when important visual evidence appears in each video. This may cause premature emission for difficult clips or unnecessary waiting for simple ones. However, the paper does not compare this design with content-adaptive scheduling policies.

  • Concern about limited baseline coverage. The strongest streaming NAR baseline, BD3-LM, is constructed by the authors because no established baseline exists for this task. Although this enables a controlled comparison, it remains unclear whether BD3-LM represents the strongest possible block-diffusion implementation. Related learned streaming approaches such as NAST are discussed but not empirically evaluated.

  • Concern about the fairness of the fixed T=4T=4 comparison. ViNAYA uses four scheduled passes, while BD3-LM uses a fixed block size of L=4L=4, which does not ensure matched computational budgets. ViNAYA actually requires more forward passes on YouCook2 and Charades. A fairer evaluation should sweep TT and LL and compare quality and latency under matched pass counts or wall-clock costs.

Questions for authors

Please kindly refer to the weakness section.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Dy6P. It includes early-commitment diagnostic, baseline coverage, and matched-compute sweep

We thank the reviewer for their precise suggestions. Below we address each concern one by one:

W=Weakness

W1 (content-agnostic schedule). We agree that our schedule at present does not adapt to individual clips. We scoped it as such because it lets us state the exposure budget (Theorem 1) and the dependency bound (Theorem 2) before decoding. Now, in order to measure this tradeoff (during the rebuttal window) we decoded the same weights with the entire clip visible and found that the first-span word precision changes by at most 1.2 points (Table 10). We also show failure cases in Table 11. And since content adaptation is a separate line of work, we discuss a learned read policy as a natural extension in Section 5.

W2 (baseline coverage). The block baseline shares every component with ViNAYA except the schedule. The new sweep we present shows it is a strong baseline because at L=2L=2 it reaches the best BLEU-4 and CIDEr on two corpora. Our rationale for not evaluating NAST is that it belongs to a different line of work. And as it learns an adaptive policy (for text-to-text translation), its latency is known only after training. ViNAYA, on the other hand, is fixed-policy (like the wait-kk family) with a latency budget set before decoding. Adapting NAST to video would also require re-deriving its alignment loss. For this reason we compare within the fixed-policy class and leave NAST to future work.

W3 (T=4T=4 against L=4L=4 or Unmatched comparison). This is right for the originally submitted work. However, Table 3 now sweeps TT and (L,S)(L, S). ViNAYA has lower AL and LAAL at every matched pass count, with AL lower by 40 to 52 percent at the budgets of Table 1. Quality at matched passes depends on the corpus and the block baseline stays ahead of ViNAYA on YouCook2. Section 4.3 now also explains the higher pass count on short captions.

Thank you for the detailed response and additional experiments.

  • W1 is alleviated but not fully resolved. The first-span diagnostic and failure cases provide useful evidence that missing future context has a limited average effect. However, decoding streaming-trained weights with the full clip visible is not equivalent to evaluating a content-adaptive policy, and aggregate word precision may conceal instance-level semantic errors.
  • W2 is alleviated but not fully resolved. The broader BD3-LM sweep supports that the constructed baseline is reasonably strong. Nevertheless, it remains an internally constructed baseline, and no learned or content-adaptive streaming method such as NAST is empirically evaluated.
  • W3 is largely resolved. The matched-pass sweep directly addresses the original fairness concern and reports the corpus-dependent quality trade-offs transparently. ViNAYA consistently improves latency at comparable pass counts, although most sweep configurations use only one training run and wall-clock costs are not comprehensively reported.

Overall, the revision substantially strengthens the empirical evaluation, especially regarding matched compute, while the concerns about adaptive scheduling and external baseline coverage remain as limitations. I therefore maintain my original rating.

We thank the reviewer for a thorough assessment of our work. (Please disregard our earlier follow-up, which crossed with your comment - we have deleted it.)

We agree that the absence of a content-adaptive policy and of an external learned system like NAST are limitations of the present work and we state them in Sections 5 and 4.2.

On instance-level errors (W1), the right half of Table 10 compares streaming and full-clip decodes window by window. We agree this is a diagnostic and not an adaptive policy, which, as we also mention in the paper, is a separate line of work.

Lastly, we will report the sweep of Table 3 with three seeds in the final version.

Thanks again.

Comments on the paper

1 comment
AuthorsOpenReview ↗

A global response for all reviewers. It includes a worked example (Fig. 1), matched-compute and sensitivity sweeps, and a pretrained backbone

We thank reviewers for all the feedback and appreciate their careful reading. This is a global response for all reviewers. We also respond to each reviewer individually.

Acknowledging Feedback. We acknowledge that a worked example, a comparison at matched compute, sensitivity analysis and a clear scope were missing in the paper. We use this rebuttal period as an opportunity to address those concerns in the revised PDF of the paper. All changes are marked in blue.

For Presentation and Understanding. With an additional page provided by ICLR we have managed to add Figure 1, built from a real ViNAYA decode. We provide a pass by pass presentation where in each pass we show how much of the clip is visible, which span is written and what has been emitted. Section 3.2 also uses those same numbers in the text.

A word on NAST. NAST learns an adaptive read/write policy for text-to-text translation. It could be extended to a multimodal setting, but this requires re-deriving its alignment loss. ViNAYA is a fixed-policy method in the same family as wait-kk, and its latency budget is set before decoding. These are two different lines of work, and for this reason we compare ViNAYA against fixed streaming policies at matched compute. Section 4.2 now states this.

Scope. Our evaluation is windowed streaming. Windows are annotated segments, and the segment length sets the scale of the source horizon. Sections 3.1 and 5 now state this.

New experiments. Besides addressing presentation and NAST, we performed the ablation and sensitivity analyses suggested by the reviewers and report them in Section 4.3 and Appendices E and F. To provide a timely rebuttal these are single-run analyses on all three corpora, and Table 1 remains the three-seed reference. Quality at matched compute depends on the corpus, as detailed below. What stands out, in line with the main finding of the paper, is a consistent gain in lag (AL and LAAL).

  • Matched compute, Table 3 (full version in Table 12). We sweep the number of passes for both systems. ViNAYA has lower AL and LAAL at every matched pass count. At the budgets of Table 1, AL is lower by 40 to 52 percent and LAAL by 35 to 39 percent. Further, ViNAYA is at or near parity in quality on Charades and ActivityNet at these budgets, except for CIDEr at 4.3 passes on Charades. The block baseline stays ahead on YouCook2. At L=2L=2 it reaches the best BLEU-4 and CIDEr on the two short-caption corpora, at about 7.5 passes, which is roughly the cost of autoregression.
  • Sensitivity (Tables 8 and 9). We vary the length prior, an oracle length, the source floor ϕ\phi and the ratio ν\nu.
  • Early commitment (Tables 10 and 11). The schedule is content-agnostic by design, and we now measure what this costs. We decode the same weights with the whole clip visible. First-span word precision changes by at most 1.2 points, so the opening commit is not where quality is lost. The decode is a diagnostic. Table 11 shows real decodes including failures.
  • Pretrained backbone (Appendix F, Table 13). We repeated the ActivityNet comparison with both systems on a pretrained DistilBERT backbone (66M parameters). The lag advantage carries over, with AL lower by 51 percent and LAAL by 37 percent at a similar pass count, and quality stays at parity.

Corrections. ViNAYA runs at most T⋅S=8T \cdot S = 8 passes per window, not a constant 8. The offline rows of Table 5 are separately trained full-context models. To make space we moved the exposure sweep table to Appendix E.

We thank the reviewers again for giving us the opportunity to perform experimentation more thoroughly and to demonstrate that the low-lag advantage holds at matched compute.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.