acceptodds
Under review as a conference paper at ICLR 2027

Revise by Appending: Correcting Committed Chunks in Autoregressive Video Diffusion

Abstract

Chunk-wise autoregressive video diffusion models generate a video a few latent frames at a time and keep the key–value (KV) states of every finished chunk for the chunks that follow. Each chunk is therefore final once it is written. When a later chunk shows that an earlier one set up an event wrongly, the model can only continue from that mistake, and prompts that describe several consecutive events often end incomplete. Overwriting the earlier latent would invalidate every KV state computed from it. We observe that only the latent the Transformer attends to has to stay fixed, while the frames shown to the viewer can be decoded from the whole generated record. AmendAR uses this separation to correct earlier chunks by appending. After each video chunk, the same Transformer samples a small revision latent that describes how that exact chunk should change. The revision’s KV states are appended to the cache, so later chunks are generated in agreement with the correction, and a lightweight decoder scans the record from the last chunk to the first, adding a residual correction to each earlier latent before VAE decoding. Revisions are learned from enhanced versions of the model’s own drafts. We prove that finalizing a chunk early discards exactly the information that later chunks carry about it, and that the size of the decoder state bounds how many correction directions reach earlier chunks. On Wan2.1, MAGI-1, and HunyuanVideo 1.5 backbones, AmendAR improves event completion on VideoPhy-2, TC-Bench, and StoryEval by 4.8–8.1 points, including StoryEval from 24.9% to 33.0% on Wan, with 9–21% additional latency.

Primary area:
generative models
Keywords:
autoregressive video generation, transformer cache, history revision, video generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XJTJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes AmendAR, a method for correcting previously generated chunks in autoregressive video diffusion while preserving the KV cache. It appends compact revision tokens that guide subsequent generation and uses a backward decoder to refine earlier chunks with later information. Training uses enhanced versions of the model’s own generated videos as supervision. The paper provides theoretical analyses of cache invariance and information benefits, and experiments on three video backbones demonstrate improved event and temporal consistency with moderate inference overhead.

Critical strengths

The method addresses the difficulty of correcting committed video chunks without invalidating the KV cache. Compact revision tokens and a backward decoder allow later information to refine earlier outputs while preserving cached states, providing a meaningful extension to autoregressive video generation.

Critical weaknesses

The streaming benefit is limited by the backward decoder’s dependence on future chunks. The reported preview latency measures the uncorrected first chunk, whereas its final corrected version requires the complete sequence. This dependence conflicts with the streaming nature of autoregressive video generation, where chunks should be finalized and displayed incrementally without waiting for the entire video.

Questions for authors

1.Since the final corrected version of an early chunk depends on the complete sequence, how should AmendAR be used in live streaming, where previously displayed frames cannot be revised? Please clarify the intended playback protocol and report the quality and latency of corrected outputs under a fixed lookahead budget. 2. How does AmendAR compare with applying VEnhancer directly to the same generated drafts, in terms of both quality and total inference latency? This comparison would help establish the practical advantage of learning compact revisions over using the enhancement teacher as a post-processing method.

3 comments
AuthorsOpenReview ↗

(1/2) Live streaming: committed playback with a fixed lookahead

(1/2)

Thank you for the review, and for putting your finger on the exact gap in our streaming story. You are right that the submission timed an uncorrected first chunk and scored a fully corrected video, and never measured what a viewer gets when a chunk, once shown, can no longer change. We now measure exactly that.

How AmendAR should be used in live streaming

For live playback, AmendAR displays each chunk once and never replaces it; we call this committed playback with lookahead LL. Chunk ii is shown when chunk min⁡(i+L,N)\min(i+L,N) and its revision exist, rendered by the backward scan of that prefix. No retraining is needed, because the decoder is already trained on every prefix length (Eq. 4), and the cache is untouched. L=0L=0 corrects each chunk with its own revision only; L=N−1L=N-1 is the offline final render. The second mode, a revisable preview, is for settings where swapping a shown chunk is acceptable (an editor or a review tool). Sec. 4.2 now defines both.

Quality and latency under a fixed lookahead (Table 13, App. G.1; the scored video concatenates exactly the committed chunks a viewer saw; TC-Bench is shown where the readout was already scored)

Wan2.1 / Self ForcingVP-2TCStory1st chunk shown (s)
Native stream25.516.824.91.32
Time-matched video-only model30.621.230.71.32
Committed, L=0L=030.420.229.71.67
Committed, L=1L=131.8-31.02.96
Committed, L=2L=232.6-32.14.39
Committed, L=3L=333.0-32.65.95
Final render (L=6L=6)33.224.133.011.44

Three things follow, and they hold on MAGI and Hunyuan as well:

  • With no lookahead, committed playback is better than the native stream but only matches video-only training on the same targets. Your concern is therefore correct for L=0L=0: the part of the gain that beats matched training comes from later chunks.
  • One chunk of lookahead already puts committed playback at or above the time-matched control on both automatic benchmarks for all three backbones.
  • With L=2L=2 (1.5 s of video on Wan), it keeps 72-84% of the gap between L=0L=0 and the final render and exceeds the control by 1.4-2.2 points in every evaluated pair. Gains saturate faster on VideoPhy-2 (single actions) than on StoryEval, whose event chains are set up over more chunks.

On our A100s no backbone generates faster than real time, so the lookahead shows up as a start-up delay: on Wan, the first committed chunk appears after 4.39 s with L=2L=2, against 1.32 s for the native preview. On hardware fast enough for real-time generation, the same lookahead is simply a playback buffer of LL chunks.

Changes in the paper. Sec. 4.2 (two ways to use the stream), Sec. 5.2 ("Committed playback needs a short lookahead"), Table 13 with all three backbones, and the Limitations paragraph, which now states that live playback needs a short lookahead.

In short: the final render does depend on the whole record, as you say, and for streaming that never replaces a frame we now recommend L=2L=2, which keeps most of the gain at a small, measured delay.

AuthorsOpenReview ↗

(2/2) VEnhancer applied directly to the drafts: quality and latency

(2/2)

VEnhancer as a post-processor on native drafts

Applied directly at test time, VEnhancer reaches quality close to AmendAR's but costs far more time, needs the complete clip, and cannot inform later chunks. This was the right comparison to ask for. Table 14 (App. G.2) now reports it with the target-building settings (noise augmentation 200, 15 steps, CFG 7.5) at batch size one on the backbone's GPUs, on Wan and Hunyuan (on MAGI, generation alone already takes about 950 s per video):

SystemVP-2StoryFinal latency (s)
Wan: native25.524.911.63
Wan: native + VEnhancer32.431.6175.05
Wan: AmendAR33.233.014.06
Hunyuan: native + VEnhancer23.224.2112.41
Hunyuan: AmendAR24.625.862.20

Native + VEnhancer stays within 1.6 points of AmendAR. It adds 59.6-163.4 s per video, so the total is 1.8-12.5 times AmendAR's final latency; the factor is largest on Wan, where generation itself is fastest. We read this as close on quality and clearly different in cost and usability.

What learning compact revisions buys over the teacher

  • Time. On Wan, 14.06 s instead of 175.05 s per video, with the first chunk shown after 1.37 s (preview) or 4.39 s (committed, L=2L=2).
  • Streaming. Post-processing needs the finished clip; AmendAR's correction runs inside the stream with a short lookahead (part 1/2).
  • Later chunks follow the correction. Revisions sit in the cache, so the generator conditions on them. Blocking that access lowers the raw drafts by 1.9-2.6 points on every backbone and benchmark (Table 4). A post-processor cannot have this effect.

We do not claim that AmendAR beats the teacher at equal information. On AmendAR's own drafts the teacher is still the stronger corrector (Table 5), and the Limitations paragraph says so. We also added the learned, compact alternative that Reviewer 8DRV proposed: a text-conditioned refiner trained on enhanced versions of a time-matched video-only model's drafts. On Wan, AmendAR stays 1.1-1.3 points above it on both automatic completion benchmarks (Table 15).

Changes in the paper. Sec. 5.2 ("Teacher post-processing and bidirectional generators"), Table 14 and App. G.2, and the Limitations paragraph.

Thank you again for two questions that sharpened what the paper claims. If the streaming protocol and the teacher comparison resolve your concerns, we would be grateful if you would consider revisiting your score; we are happy to clarify anything further.

A major advantage of autoregressive over bidirectional video generation is the ability to display chunks as soon as they are generated. However, AmendAR’s additional gains require waiting for future chunks. At L=0L=0, it does not outperform the time-matched video-only baseline on Wan, while even L=1L=1 more than doubles the first-chunk latency. Therefore, my concern remains unresolved, and I will maintain my original score.

Reviewer_YzvbOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies chunk-wise autoregressive video diffusion. In these models each chunk is final once its KV states are cached, so early mistakes can’t be fixed. The authors note that only the latent the Transformer attends to must stay fixed, not the decoded frames. AmendAR samples a small revision latent after each chunk and appends its KV states, so later chunks can see the correction. A backward decoder then scans from the last chunk to the first and adds a residual to each earlier latent before VAE decoding. Training targets come from VEnhancer applied to the model’s own drafts.

Critical strengths
  • Separating "latent to attend" from "latent to decode" is a simple and new idea. It allows correction without breaking the KV cache.
  • The method is cheap in memory (+4.4–8.3% KV) and adds only a small decoder.
  • It is tested on three backbones of different sizes, with three seeds and confidence intervals.
  • The ablations are careful: forward and backward paths are tested separately, a single late revision is removed, and the state width is varied.
  • There is a fair matched-learning control that trains the native model on the same enhanced targets.
Critical weaknesses
  • There are no qualitative videos. no supplementary material, and no videos. Only videos can show whether re-rendering earlier chunks causes flicker or seams at chunk boundaries. Table 10 reports no flickering score, and Figs. 4 and 14 compare separate trajectories.
  • In Table 5, VEnhancer applied to the same drafts beats AmendAR on all backbones (Wan VP-2: 34.9 vs. 33.2). Native + VEnhancer at test time, and its latency, are not reported.
  • Later chunks add only 1.9-3.9 points: “No incoming state” still gets 30.4 vs. 33.2 (Table 3). Most of the correction is local, which weakens the main claim.
  • VEnhancer runs with noise augmentation 200 and text CFG 7.5, so it re-generates content. The event gains may come from the teacher’s text conditioning, not from appending revisions.
  • Training the native model on the same targets already gives 24.9 -> 30.7 on StoryEval. AmendAR adds 2.2-3.8 points, and the Wan VP-2 CI (-0.3 to 5.6) includes zero.
  • All scores use the final offline render. Preview quality is not evaluated, and there is no offline baseline. The final video can be decoded only after the last chunk is generated, so the output behaves like a bidirectional generator and loses the streaming benefit of AR models. Yet there is no comparison with bidirectional models (e.g. Wan2.1 1.3B, or a few-step distilled version) at similar latency.
  • Adaptation costs 522-3802 GPU-hours, mostly for building targets, and the targets must be rebuilt for each backbone.
Questions for authors

Can you provide side-by-side videos of native, raw draft and AmendAR on the same prompts, including failure cases?

What are the scores and latency of Native + VEnhancer at test time?

Can you report a temporal-flickering or boundary-consistency metric for the re-rendered videos?

Do the gains hold with a teacher that has no text conditioning, or with lower noise augmentation?

How good is the streaming preview compared with the final render?

If the final video needs all chunks, why not use a bidirectional model? Please compare with Wan2.1 1.3B and a few-step bidirectional model on the event-completion benchmarks, with latency.

AmendAR doesn’t improve VBench-Long (78.4 vs. 78.3). Does it combine with long-rollout methods such as LongLive or Infinity Rope?

3 comments
AuthorsOpenReview ↗

(1/3) Same-record comparisons, flickering and chunk boundaries; Native + VEnhancer at test time

(1/3)

Thank you for the most detailed review we received. Your weaknesses and questions read like a checklist of what a skeptical reader would need to see, and we used them that way: every item now has an experiment, an analysis or a text change in the revision. We were also glad that you singled out the separation of the attended latent from the decoded latent, the small memory cost, and the matched-learning control; those are the parts we most wanted readers to find convincing. This first part covers seams and side-by-side comparisons (W1, Q1, Q3) and the teacher at test time (W2, Q2).

Side-by-side comparisons on the same record (W1, Q1)

You are right that Figs. 4 and 14 compare separate trajectories; they compare generators, so that is unavoidable there, and we should have pointed readers to the figures that do isolate the decoder. Three figures decode one generation record two ways, so every difference between their rows comes from the backward decoder: Fig. 7 (raw draft and rendered output at eight matched stages, frisbee and clay-pot prompts), Fig. 12 (one late revision removed with record and cache fixed) and Fig. 21 (encoded vs. sampled revision on one juggling draft). Appendix G now opens by listing them. Because frames can hide temporal artifacts, we also measured seams directly, below.

Temporal flickering and chunk boundaries (Q3)

Table 18 (App. G.6) adds VBench temporal flickering (official implementation) and a boundary warping-error ratio, BWR: the RAFT-warped L1L_1 error between consecutive frames that straddle a chunk boundary, divided by the same error inside chunks (1 means boundaries are as smooth as the interior). Both use the first output of every VBench prompt. Raw draft and rendered output come from the same record, so their difference isolates the decoder.

Wan2.1 / Self ForcingTemporal flickeringBWR
Native98.961.06
AmendAR raw draft98.941.07
AmendAR rendered98.871.09

Across the three backbones, rendering lowers the temporal-flickering score (higher is smoother) by at most 0.07 points and raises BWR by at most 0.03 relative to the raw draft. So the decoder adds a small, measurable amount of flicker and boundary error, which we report rather than hide; the largest boundary ratio of any rendered output is 1.11, and the native generators are already above one (Table 18).

Native + VEnhancer at test time, with latency (W2, Q2)

You are right that the teacher beats AmendAR on the same drafts (Table 5: 34.9 vs. 33.2 on Wan VP-2); the Limitations paragraph now states this directly. Table 14 (App. G.2) adds the comparison you asked for, with the target-building settings at batch size one:

SystemVP-2StoryFinal latency (s)
Wan: native + VEnhancer32.431.6175.05
Wan: AmendAR33.233.014.06
Hunyuan: native + VEnhancer23.224.2112.41
Hunyuan: AmendAR24.625.862.20

Post-processing comes within 1.6 points of AmendAR but adds 59.6-163.4 s per video, needs the finished clip, and cannot reach later chunks. AmendAR's revisions do reach them: blocking the generator's access to revisions lowers raw drafts by 1.9-2.6 points (Table 4). We therefore position AmendAR as a correction that works inside streaming generation at 9-21% extra latency, not as a better offline enhancer.

Parts 2/3 and 3/3 follow.

AuthorsOpenReview ↗

(2/3) Local vs. later-chunk correction; matched training and its interval; the teacher's text conditioning

(2/3)

How much later chunks contribute (W3)

You are right that the no-incoming-state readout keeps much of the gain (Wan VP-2: 30.4 of 33.2), and the submission did not explain why that does not undercut the claim. Two clarifications, now in Sec. 5.3 and App. G.1:

  1. The "local" correction is itself a correction of a committed chunk by later information. Revision rir^i is sampled after chunk ii is fixed in the cache, from the prompt, the whole history and xix^i; rendering chunk ii from rir^i changes frames that a native generator has already finalized. It exists only because of the append-only revision.
  2. The part that later chunks add is exactly the part that beats matched training. Without the incoming state, the readout stays within 1.0 point of the time-matched video-only control in all nine pairs; with it, AmendAR exceeds that control by 2.2-3.8. Measured against the whole rendering gain (AmendAR minus raw draft), later chunks provide 39-88% of it (Table 3). Committed playback confirms this from the other side: one or two chunks of lookahead bring the margin over matched training back (Table 13).

Matched training and the interval that includes zero (W5)

You are right that aligned targets alone give a large gain (24.9 to 30.7 on Wan StoryEval), and we now say plainly that AmendAR's contribution is the margin on top of that. Three pieces of evidence for the margin:

  • It is positive in all nine backbone-benchmark pairs (2.2-3.8 points).
  • The interval for Wan VP-2 was wide because the tt multiplier with three runs is 4.30. We trained two more paired seeds (61, 79) under the unchanged protocol. With five seeds the margin is 2.7 points, 95% CI 1.5 to 3.9 (Table 19, App. G.9). The new seeds (2.15 and 3.35) fall within the spread of the first three. All other tables keep the original three seeds.
  • A stronger control on Wan, the time-matched model plus a text-conditioned refiner trained on its own enhanced drafts, stays 1.1-1.3 points below AmendAR on both VideoPhy-2 and StoryEval; for the smallest margin (StoryEval) the paired 95% CI is 0.1 to 2.1 (Table 15).

Is it the teacher's text conditioning? (W4, Q4)

This is a fair alternative explanation, and the right test is the one you suggested: change the teacher and retrain both AmendAR and the matched control on its targets. We ran the text-free variant on Wan with three paired seeds (Table 16, App. G.4):

Teacher (Wan)AmendAR VP-2 / StoryControl VP-2 / Story
Default: noise 200, text CFG 7.533.2 / 33.030.6 / 30.7
Text-free: noise 200, empty prompt29.4 / 29.027.5 / 27.3

Your suspicion is partly right, and we say so in the paper. With the text-free teacher, AmendAR's gain over the native model falls to 3.9-4.1 points (from 7.7-8.1 on these benchmarks), so part of the absolute gain is the teacher's text guidance. The margin over matched training, which is what the revisions add, persists at 1.7-1.9 points (2.3-2.6 by default). We chose the text-free teacher over the low-noise one because it removes the explanation you raised outright, while lower noise only weakens it. Sec. 4.3, Sec. 5.3 and the Limitations paragraph now separate these two statements.

AuthorsOpenReview ↗

(3/3) Preview quality, streaming and bidirectional models; adaptation cost; VBench-Long and long-rollout methods

(3/3)

Preview quality and the final render (W6, Q5)

The preview is AmendAR's raw draft, so its quality was in the paper but not labeled as such: on Wan, 27.6 / 18.5 / 26.3 on VP-2 / TC / Story, against 25.5 / 16.8 / 24.9 for the native model and 33.2 / 24.1 / 33.0 for the final render (Tables 4, 13). Between the two, committed playback scores what a viewer sees when shown chunks are final: with two chunks of lookahead it keeps 72-84% of the gap to the final render on all three backbones (Table 13; details in our reply to Reviewer XJTJ).

Why not a bidirectional model? (W6, Q6)

Generation stays causal with an append-only cache, so AmendAR keeps what the chunk-wise backbone offers: a first chunk after 1.37 s, committed playback with a short lookahead, and videos far past a fixed horizon (our ~30 s setting has 40 chunks). The backward scan is bidirectional, but it only reads stored latents. Table 17 (App. G.5) compares Wan2.1-1.3B and a public few-step bidirectional student of it (FastWan2.1-T2V-1.3B, 3 steps), timed on the same A100:

WanVP-2StoryFirst frame (s)Final (s)
Wan2.1-T2V-1.3B, 50 steps27.327.9218.82221.44
FastWan2.1-1.3B, 3 steps26.526.67.009.62
Self Forcing25.524.91.3211.63
AmendAR33.233.01.3714.06

Both bidirectional models complete more events than Self Forcing, most clearly on StoryEval, which agrees with our diagnosis that committing early chunks is what hurts. Both stay below AmendAR. The 3-step student is faster overall but shows nothing for 7.00 s and stops at 81 frames. We agree that for a fixed-length clip where streaming does not matter, a bidirectional model is a reasonable choice, and the Limitations paragraph says so.

Adaptation cost (W7)

The numbers are as you state, and we now put them in context (App. G.10). Parameter learning takes only 14-108 GPU-hours; the rest builds targets once per backbone, by the teacher on Wan and Hunyuan and by 64-step rollouts on MAGI. For scale, LongLive reports 32 GPU-days for its long-rollout fine-tuning and Self Forcing 1.5 hours on 64 H100s, so Wan's 522 A100 GPU-hours are in the same range. Rebuilding per backbone is the price of same-draft targets: reusing fixed initial drafts costs 1.5-2.6 points of rendered TC-Bench completion (Table 12). The refiner baseline pays a similar price (203.0 extra GPU-hours per seed on Wan) without matching AmendAR. The cost remains in Limitations.

VBench-Long and long-rollout methods (Q7)

AmendAR leaves VBench-Long at its backbone's level because it keeps the backbone's history policy, and that policy is exactly what long-rollout methods change: in Table 1, LongLive, Context Forcing and Deep Forcing lift VBench-Long while AmendAR lifts completion. The two act on different parts of generation, so they compose by construction. On a windowed method such as LongLive, each revision row is kept and evicted with its chunk, while the decoder reads the stored latents and still spans the whole record; a re-indexing such as Infinity-RoPE moves each revision row with its chunk (App. G.7). We prioritized the experiments above, which test the paper's central claims, and describe the combination in App. G.7 rather than report it as a result.

Thank you again for a review that made this paper substantially more complete. If the new results address your concerns about presentation and soundness, we would be very grateful if you would consider raising your scores, and we are glad to answer any remaining question.

Reviewer_8DRVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AmendAR separates immutable generation history from rendered video. It appends compact revision latents to the KV cache and uses a backward decoder to correct earlier chunks. Training uses enhanced versions of the model’s own drafts. Experiments across three backbones show improved event completion with moderate inference overhead.

Critical strengths
  1. The separation of cached history from rendered output is a useful idea. Earlier frames can be revised without recomputing old KV states, and the revisions can also guide later chunks.

  2. The experiments go beyond comparisons with the native models. Table 3 includes controls for extra learning time and for both revision paths, and the method is tested on three backbones.

Critical weaknesses
  1. My main concern is the missing comparison with a simpler approach: the video-only model trained for the same learning time, followed by a small text-conditioned refiner. On Wan StoryEval (Tables 3–4), that video-only model scores 30.7%, while AmendAR scores 26.3% before rendering and 33.0% afterwards. The no-local-revision ablation is useful, but it uses AmendAR’s drafts, not those of the stronger video-only model. A comparison at similar training and inference cost would help establish whether generating revisions is worth the added complexity.

  2. The streaming benefit is less clear. Table 2 distinguishes preview and final latency, but the preview is produced before revision sampling. The completion scores are measured after the full backward pass (Appendix E.3). These results show an improvement in the final video, but do not establish how much a viewer gains when earlier displayed frames cannot be replaced.

Questions for authors
  1. Could you compare against the video-only baseline plus a small text-conditioned refiner trained on enhanced versions of its own drafts, and report the final quality and latency?

  2. With the teacher seed fixed (Appendix D.3), yi=Gi+(x1:N,c)y^i=G_i^+(x_{1:N},c) is determined by the full draft and prompt, so VN,ri=0V_{N,r}^i=0. How should the last-revision ablation in Table 4 be interpreted in this case?

  3. Is the intended streaming use a revisable preview, or playback where displayed frames cannot be changed? For the latter, what delay is needed before committing a chunk to retain the reported gains?

2 comments
AuthorsOpenReview ↗

(1/2) Video-only model + text-conditioned refiner

(1/2)

Thank you for a review that went straight to the comparison that matters. You described the simpler competitor precisely (the time-matched video-only model followed by a small text-conditioned refiner trained on its own enhanced drafts) and explained why the no-local-revision ablation does not replace it, since that ablation runs on AmendAR's weaker drafts. We built exactly this baseline on Wan, the backbone and the StoryEval numbers you cited.

Setup (App. G.3). Starting from each seed's equal-learning-time A-only model of Table 3, we generate 4,000 drafts on the training prompts, enhance each with VEnhancer under the target settings, and train a refiner of the decoder's size: four Transformer blocks of width 512 on the same 32x32-pixel grid, reading the whole draft latent and the prompt embedding (cross-attention), adding a residual to every chunk before VAE decoding. It gets 1,000 updates at batch size 8 with the decoder's optimizer and selection rule, three seeds paired with the control's, and is scored on VideoPhy-2 and StoryEval, the benchmark you cited. Unlike AmendAR's decoder, it sees the prompt and every chunk at once, and it receives rollouts and teacher calls beyond AmendAR's budget, which favors the baseline.

Results (Table 15, Wan). Completion (%), three-batch means; final latency in seconds.

SystemVP-2StoryFinal (s)
time-matched A-only30.630.711.63
+ text-conditioned refiner31.931.911.69
AmendAR33.233.014.06

The refiner helps, by 1.2-1.3 points, and AmendAR stays 1.1-1.3 points above it on both benchmarks. The smallest margin, StoryEval, has per-seed paired differences of 1.50, 0.70 and 1.20 and a paired 95% interval of 0.1 to 2.1.

On cost, we want to be straightforward. The refiner itself is almost free at inference, so control + refiner is faster than AmendAR (11.69 vs. 14.06 s). At training time it needs its own drafts and teacher calls, 203.0 GPU-hours per seed on top of the control. So for less training compute than this baseline, the revisions buy the margin above for 21% extra inference time on Wan, plus streaming with a short lookahead, which a whole-clip refiner cannot offer.

Why AmendAR's raw drafts (26.3%) trail the video-only model (30.7%). The raw draft is not AmendAR's output. With an equal number of video updates, the A-only control writes the teacher's correction into its drafts, whereas AmendAR's generator leaves part of it to the revisions, which the decoder applies. What a viewer sees without waiting is committed playback: 29.7 at L=0L=0, 32.1 at L=2L=2, and 33.0 for the final render on Wan StoryEval (Table 13).

Changes in the paper. Sec. 5.3 now reports the refiner comparison; App. G.3 and Table 15 give the protocol and results.

Our reading is that a whole-draft refiner with the prompt recovers part of the margin, and the remainder needs revisions that the generator writes while it generates and that later chunks then follow.

AuthorsOpenReview ↗

(2/2) V_N,r=0 with a fixed teacher seed, and the delay needed before committing a chunk

(2/2)

VN,ri=0V^i_{N,r}=0 with a fixed teacher seed (Q2)

You are right, and thank you for checking the theorem against the implementation so carefully. With the teacher seed fixed, yi=Gi+(x1:N,c)y^i=G^+_i(x^{1:N},c) is a function of the prompt and the full draft, so μN,xi=yi\mu^i_{N,x}=y^i and VN,ri=0V^i_{N,r}=0. Our original sentence that the revision "contributes the terms Vk,riV^i_{k,r}" was misleading, and we corrected it in Sec. 3 and App. A. App. G.8 adds a short derivation:

  • The last revision adds nothing at the Bayes level. For any teacher randomness independent of the generator, yiy^i and r1:Nr^{1:N} are conditionally independent given (c,x1:N)(c,x^{1:N}), so VN,ri=0V^i_{N,r}=0.
  • Earlier revisions inform only through later chunks. For k<Nk<N, Vk,ri>0V^i_{k,r}>0 requires the forward path: later chunks read rkr^k and follow it, which makes rkr^k predictive of them.
  • Where revisions act on the final render. They lower the estimator error Eai\mathcal{E}^i_{\mathrm{a}} in Eq. (1), not the information sum. The decoder reads neither cc nor x<ix^{<i}, while rjr^j (j≥ij\ge i) is sampled from (c,H<j,xj)(c,\mathcal{H}^{<j},x^j) and can carry exactly that information.

How to read the last-revision test of Table 4. It sets uN=0u^N=0 in the trained decoder without refitting, so it measures how much the trained decoder relies on the last revision, not VN,riV^i_{N,r}. The 1.7-2.3 point drop shows that this reliance is real: the small decoder cannot recompute from its own inputs the prompt- and history-dependent information that the generator writes into rNr^N. The refitted counterpart is the no-uu decoder of Table 3, which loses 3.2-4.6 points even when trained without revisions. Sec. 5.3 now states this reading next to the result.

Revisable preview or fixed playback, and the delay needed (W2, Q3)

Both uses are supported, and Sec. 4.2 now defines them. A revisable preview shows each chunk at once and may later swap in its corrected version. For playback where displayed frames cannot change, we use committed playback with lookahead LL: chunk ii is shown once chunk i+Li+L and its revision exist, and never replaced. No retraining is needed, since the decoder is trained on every prefix length. Table 13 scores exactly the frames a viewer saw:

Wan StoryEvalL=0L=0L=1L=1L=2L=2L=3L=3final
Completion (%)29.731.032.132.633.0
First chunk shown (s)1.672.964.395.9511.44

For reference, the time-matched video-only model scores 30.7 and shows its first chunk after 1.32 s. The answer to your question is therefore two chunks: with L=2L=2 (1.5 s of video on Wan, 2 s on MAGI and Hunyuan), committed playback keeps 72-84% of the gap between L=0L=0 and the final render and exceeds the time-matched control by 1.4-2.2 points in every evaluated backbone-benchmark pair (VideoPhy-2 and StoryEval on all three backbones). L=1L=1 is already at or above the control on both automatic benchmarks everywhere. With L=0L=0, committed playback only matches matched training, which is consistent with your observation that the completion scores in the submission were measured after the full backward pass. The Limitations paragraph now states this.

Thank you again for two weaknesses and three questions that were each answerable with one clean experiment or derivation; the paper is more precise for them. If the refiner comparison and the streaming protocol address your concerns, we would be grateful if you would consider raising your score, and we are glad to discuss further.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the revision: streaming protocol, stronger baselines, teacher ablation

Dear Area Chair, dear Reviewers XJTJ, Yzvb and 8DRV,

Thank you for three careful and very concrete reviews. Each of you named the experiment that would settle your concern, which made the revision much easier to plan than a list of objections would have. The revised PDF is uploaded; new and revised passages are in blue, the new results are collected in Appendix G, and the main text still ends on page 10. Title and method are unchanged, and every added experiment tests a claim the submission already makes. Several comparisons run on Wan, the backbone with the external comparison, and each protocol in Appendix G states its scope.

What the reviews agreed on. Separating the latent the Transformer attends to from the latent that is decoded was called "a simple and new idea" that corrects "without breaking the KV cache" (Yzvb), "a useful idea" (8DRV) and "a meaningful extension to autoregressive video generation" (XJTJ). Yzvb and 8DRV also credited the controls: matched learning time, separate tests of the forward and backward paths, three backbones.

Shared concerns and what changed

  1. Streaming when displayed frames cannot be replaced (all three). Sec. 4.2 now defines two uses of the stream: a revisable preview, and committed playback with lookahead LL, which needs no retraining because the decoder is trained on every prefix length. Table 13 (App. G.1) scores committed playback for each LL on all three backbones, with the latency until the first chunk is committed.
  2. Simpler pipelines (XJTJ, Yzvb, 8DRV). VEnhancer on native drafts at test time (Table 14), the time-matched video-only model plus a text-conditioned refiner trained on its own enhanced drafts (Table 15), and bidirectional Wan2.1-1.3B and a public 3-step bidirectional student (Table 17).
  3. Does the gain come from the teacher's text guidance? (Yzvb). AmendAR and the matched control are both retrained with a text-free teacher (Table 16).
  4. The least certain margin (Yzvb). Two more paired seeds for Wan VideoPhy-2 (Table 19).
  5. Seams and flicker (Yzvb). Temporal flickering and a chunk-boundary warping-error ratio on the same record (Table 18), next to the same-record figures already in the paper (Figs. 7, 12, 21).
  6. Theory (8DRV). Sec. 3 and App. G.8 now state that VN,ri=0V^i_{N,r}=0 for our fixed-seed teacher and what the last-revision test of Table 4 measures instead.

New results at a glance

QuestionResultWhere
Committed playback, L=2L=2keeps 72-84% of the gap to the final render; +1.4 to +2.2 over the time-matched control in every evaluated pairTable 13
Native + VEnhancer at test timewithin 1.6 points of AmendAR, but +59.6 to +163.4 s per videoTable 14
Time-matched control + text refiner (Wan)AmendAR higher by 1.1-1.3 points on VideoPhy-2 and StoryEvalTable 15
Text-free teacher (Wan)AmendAR's margin over the control stays at 1.7-1.9 pointsTable 16
Bidirectional Wan2.1 / 3-step studentbelow AmendAR on VideoPhy-2 and StoryEval; student's first frame after 7.00 s (AmendAR preview 1.37 s)Table 17
Flicker / chunk boundariesflickering score (higher is smoother) at most 0.07 below, boundary ratio at most 0.03 above the raw draftTable 18
Wan VideoPhy-2 margin, 5 seeds2.7 points, 95% CI 1.5 to 3.9Table 19

Two of these results temper the paper rather than help it, and the Limitations paragraph (Sec. 6) now says so plainly: live playback needs a short lookahead to keep most of the gain, and when latency and streaming do not matter, running the teacher after generation is a strong corrector. Part of the absolute gain also reflects the teacher's text guidance; the part that the revisions add over matched training does not.

We also corrected transcription errors in a few table cells and the sentences that quote them. We reply to each reviewer separately below. Thank you again for the time you spent on this paper; the revision is clearer and better tested because of it.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.