- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In block-causal video diffusion with a KV cache, a later block reads an earlier token's Value (a projection of that token's layer input), so what the earlier token selected with its own Query reaches later blocks only one layer later, and later-block losses do not reach the earlier token's attention logits in the same layer. CausalFold keeps the cached Keys and stores, in place of the Value, the token's attention output . A token attends with the backbone's own weights over the Values of its own block and the cached outputs of earlier blocks, . The recursion terminates in block order; is a convex combination of Values (Thm. 1), and the Jacobian of a later output w.r.t. an earlier token's logits factorizes as visit probability times candidate differences (Thm. 2). No parameters are added. Each backbone is converted with 200 DMD updates on self-rollouts using a replay that sends gradients through every cache read; a Standard-KV control receives identical training. On Wan2.1-1.3B/14B, MAGI-1 4.5B, HunyuanVideo-1.5 8B and EchoWM 21.8B, TC-Bench transition completion rises by 2.3 to 4.9 points in all six T2V/I2V pairs (e.g. 37.9 to 42.7 T2V), StoryEval and temporal T2V-CompBench categories improve, VBench quality/consistency change by at most 0.3, and dynamic degree drops 0.4 to 0.7. Interventions show the trained model depends on reading outputs (restoring V drops TCR below Standard-KV) and that removing the later-block logit gradient removes about half the gain; MAGI-1, whose writer attends only within its block (zero visit probability), gains least. Long 60-s rollouts, WBench navigation, a 9,027-judgment human study, and 5 to 18% latency / up to 2x cache under GQA are reported.
Critical strengths
1. A simple, parameter-free change with a crisp mechanism, tested as a mechanism rather than as a number. The matched Standard-KV control shares checkpoint, data, sampler, updates and the replay-through-cache gradient path, so the comparison isolates what a cached read returns. The paper states predictions P1 to P4 and tests them with interventions that separate forward reads from the later-block gradient (history reads V; future-logit-off), and uses MAGI-1's self-only writer as a natural experiment for Theorem 2. The breadth (five backbones from 1.3B to 21.8B, T2V/I2V/camera control, 60-second rollouts, a large paired human study) is well beyond what is usual for a method of this size. This is the main reason I recommend acceptance.
2. The theory is modest but exact and does work. The path interpretation of , the convexity/contraction result, and the factorization are correct and are used to predict where the training signal vanishes and which factor limits it per backbone (Fig. 14).
Critical weaknesses
1. No uncertainty is reported for any headline comparison. Tables 1 to 4 and 7 give means over three training seeds with no standard deviation or interval. TC-Bench has 150 T2V / 120 I2V cases, the judges are VLM-based, the models receive only 200 fine-tuning updates, and the claimed differences are 2.3 to 4.9 points; seed-to-seed variation of that order is plausible. Appendix H describes the statistical units but never gives intervals for the main results. Since the conclusions rest on these differences, per-seed values or paired bootstrap CIs are needed; "all six comparisons positive" is weaker evidence than it sounds if the six are correlated through shared prompts and judges.
2. Trade-offs and training budget are under-examined. Dynamic degree decreases in every pair, VideoPhy-2 physical commonsense drops (3.50 to 3.40), T2V-CompBench object interaction drops, and the human preference for visual realism is only 57%. All comparisons use 200 generator updates after conversion; it is unknown whether the gap persists, grows or closes when both arms are trained longer. For the real-time use case that motivates block-causal generation, +5 to 18% latency and a doubled cache under grouped-query attention are not negligible and deserve more than an appendix table.
Questions for authors
Q1. Please add per-seed TCR (or mean, sd and a paired bootstrap CI over cases) for Tables 1, 2 and 4, and a prompt-level CI for the human-study preferences.
Q2. Please report a training-length sweep for the Wan 1.3B pair (e.g. 200 / 500 / 1000 updates for both arms). Does the TCR gap persist?
Q3. Why do dynamic degree and physical commonsense decrease? Is motion amplitude reduced, or are fewer videos crossing VBench's dynamic threshold while completing transitions?
Q4. Since retains information from blocks that have been evicted from the physical window, how does CausalFold behave in interactive settings where the user changes the action (EchoWM/WBench)? Any sign of over-persistence or stale context when the command changes?
Q5. In the "History reads: V" control, the drop below Standard-KV is a distribution shift for a model trained on R; it demonstrates dependence, not that R is intrinsically better. Please phrase it that way (Section 5.3 currently reads as additional evidence for CausalFold).
Q6. Is CausalFold compatible with the rolling-window and sink designs of LongLive / Rolling Forcing? The long-video comparison in Table 7 is unmatched, so the relative merit there cannot be read off.
Suggestion: Report cache memory and latency for the 14B model under GQA if applicable, and discuss whether R could be stored per KV group (averaging query heads) at some quality cost.
4 comments
Response to Reviewer jVbN (2/3): training length and metric trade-offs
(2/3)
Q2. Does the gap persist with longer training? Yes. Standard-KV can in principle relay what an earlier token selected through deeper layers, so this was the right control to ask for. We continued both arms of the Wan 1.3B pair from 200 to 500 updates for paired seeds 1 and 2, and compare them with the 200-update checkpoints of the same seeds (App. J.2, Table 12):
| Updates | Std-KV T2V | CausalFold T2V | Std-KV I2V | CausalFold I2V |
|---|---|---|---|---|
| 200 | 38.1 | 42.9 | 44.8 | 49.8 |
| 500 | 39.1 | 43.7 | 45.9 | 50.7 |
Both arms improve and Standard-KV gains slightly more, but the gap moves only from 4.8 to 4.6 points in T2V and from 5.1 to 4.8 in I2V (gaps are means of the per-seed gaps in Table 12, e.g. 5.2 and 5.0 for I2V at 200 updates, so they can differ by 0.1 from the difference of the rounded means above); every per-seed gap at 500 updates lies between 4.5 and 4.8, and Standard-KV at 500 updates is still below CausalFold at 200. This fits the mechanism: deeper-layer relay can recover part of the information, while the same-layer read and the later-block logit gradient are absent from Standard-KV at any training length. The final version will extend the sweep to 1,000 updates and all three seeds.
Q3. Why dynamic degree decreases. Your second hypothesis is the right one. VBench samples frames at about 8 fps and calls a video dynamic only if at least a quarter of frame pairs exceed the flow threshold, so a transition that completes early and then holds its end state can fail without moving less. Appendix J.3 measures this on VBench-I2V for all four backbones:
- Amplitude is unchanged: the mean peak frame-pair score moves by at most 0.03 of the threshold, and the share of videos that ever exceed it by at most 0.4 points.
- Duration changes: videos whose motion is too brief to count rise by 0.3 to 0.8 points, and the share of moving frame pairs falls by 0.7 to 1.8 points.
- On TC-Bench-I2V, completed transitions pass the test less often than incomplete ones in both models (56.3% vs 68.9% for Standard-KV), and within each stratum the two models differ by under one point. More completions lower the rate by composition.
HunyuanVideo, which has the largest drop, also shows the largest (still small) peak change, so a little of its drop is amplitude; the appendix says so.
Physical commonsense, object interaction, and realism. Splitting VideoPhy-2 by semantic adherence (App. J.4) places the drop from 3.50 to 3.40 in videos that do not follow the prompt (mean PC -0.27), i.e. attempted but incomplete actions. Among adherent videos mean PC changes by -0.03, and more CausalFold videos reach PC ≥ 4 (52.5% vs 49.4%), which is where joint accuracy rises. Object interaction (−0.7) scores contact outcomes such as a collision; that outcome is settled within a few blocks by contact dynamics, not by one entity's state carried across blocks, which is the relation CausalFold's reads add. The 57.43% realism preference is modest, as you say, and its prompt-level interval [55.74, 59.10] lies above 50%, so realism is preserved rather than traded away. Section 5.4 now summarizes these points and Section 6 lists dynamic degree and PC as limitations.
Response to Reviewer jVbN (3/3): cost under GQA, command changes, long-video protocols
(3/3)
W2. Latency and cache under GQA deserve the main text. Agreed. Section 5.4 now reports generation cost, training cost, and matched-budget results together, and the Limitations paragraph of Section 6 states the GQA growth directly. Two new results: on MAGI-1, Standard-KV given ten evaluations per block is slower than CausalFold (112.64 s vs 110.09 s) and still 1.8 TCR points lower (App. J.5), and training time rises by 4.7% to 14.9%, below the generation overhead because the score models are unchanged (App. J.8).
Suggestion: 14B under GQA, and R per Key/Value group. Wan2.1-14B uses multi-head attention (as many Key/Value heads as query heads), so its cache has the same size in both arms; its 2.53 GiB higher peak comes from transient buffers of the recursion. MAGI-1 is the GQA case (24 query, 8 Key/Value heads). We implemented your proposal at inference on the trained checkpoint: store one output per Key/Value group, the mean over its three query heads. This returns the cache to Standard-KV size and keeps 56% of the MAGI-1 I2V gain (TCR 49.4 vs 48.1 and 50.4) at 105.37 s instead of 110.09 s. Section 6 now names this as the option for GQA backbones.
Q4. Interactive settings: stale context after a command change? Yes, a small amount, and your question made us measure it (App. J.9). Splitting WBench turns by whether the command changed, CausalFold follows a new command 0.8 frames later on average (33 ms at 24 fps), and 0.9 points more of its changed-command turns still follow the old command in their first second. This is the persistence the path argument predicts. It stays small, and on changed-command turns the interaction score is still higher than Standard-KV's (86.6 vs 85.7), because the scene revealed after the turn stays consistent with what was seen before. Section 6 lists it as a limitation with the number.
Q6. Rolling windows, sinks, and an unmatched Table 7. Yes, they are compatible. CausalFold changes what a cache entry holds, not which entries are kept, so window, sink, and re-cache policies apply unchanged: a sink block caches its within-block aggregation, a re-cache recomputes (K, R) with the new prompt, and Rolling Forcing's jointly denoised frames are evaluated in block order, as for MAGI-1's overlapping chunks (App. J.6). The two act on different quantities. A sink or a longer window decides which Values stay readable; a cached output carries what its block selected from entries that were later evicted (App. B, Fig. 10). You are right that Table 7 cannot isolate this, so App. J.5 does it at matched memory inside one protocol: Standard-KV windows of 24 and 40 frames still have 37/384 and 35/384 failures against 20/384. Matched pairs inside LongLive and Rolling Forcing need each method's own long-rollout training recipe; the final version will add them, and Table 7's caption already marks the external rows as unmatched.
For the AC discussion, in short.
- The gains exceed seed and case variation, including jointly over the six correlated comparisons (J.1).
- The gap persists after 500 updates and against Standard-KV given more compute or a longer window (J.2, J.5).
- The metric trade-offs are explained and bounded: dynamic degree is motion duration, PC is failed attempts (J.3, J.4).
- Your GQA suggestion works on MAGI-1 even without retraining (J.5).
Thank you again for one of the most constructive reviews we have received. Your recommendation already highlights the mechanism-level evidence; if these additions resolve the concerns on uncertainty and trade-offs, we would be grateful if you would consider raising your score and your soundness rating.
Thank you for the thorough response and the additional analyses. The new results address my main concerns: the per-seed values and case-bootstrap intervals (J.1) show that the gains exceed seed and case variation; the 500-update sweep (J.2) shows the gap persists with longer training; and the decomposition of dynamic degree and physical commonsense (J.3, J.4) gives a convincing explanation of the metric trade-offs. I also appreciate the revised wording of the "history reads: V" control, the measurement of command-change lag (J.9), and the per-KV-group variant for GQA.
I encourage the authors to include the matched LongLive / Rolling Forcing comparisons and the 1,000-update sweep in the final version, as promised. Given these improvements, I maintain my positive assessment and recommend acceptance.
Response to Reviewer jVbN (1/3): uncertainty of the matched comparisons
(1/3) Thank you for a review that engages with the mechanism itself. You described the Standard-KV control as isolating "what a cached read returns", the P1–P4 interventions as testing a mechanism "rather than a number", and MAGI-1's self-only writer as a natural experiment for Theorem 2. That is the reading we hoped for, and your questions pushed us to back each of those points with the statistics and controls they needed. All changes are in blue; new analyses are in Appendix J.
W1 / Q1. Uncertainty for the headline comparisons. You were right that means alone were not enough, and that six positive comparisons sharing prompts and a judge are weaker evidence than they look. Appendix J.1 now reports both levels of variation, with the summary in Section 5.1.
Seeds. Each seed's TCR is a count over 750 T2V or 600 I2V videos. Seed SDs range from 0.6 to 1.2 points across the 12 arms, below the smallest gain (2.3). Every paired seed difference is positive:
Cases, with the correlation you raised. The interval resamples TC-Bench cases with all videos and seeds of a case kept together, and it resamples case indices jointly across comparisons that share a benchmark, so a prompt that favors CausalFold on several backbones counts once per draw. The last column is the lower end of a Bonferroni interval over all six comparisons. All six stay above zero. The two MAGI-1 bounds are the lowest, which is what the zero visit probability between its cache-writing calls predicts.
WBench (Table 10). Consistency, interaction, and physical plausibility exclude zero; the quality interval [-0.3, 0.5] contains it, so navigation quality is unchanged rather than improved.
Human study (Table 11). Resampling the 1,003 prompts with their nine judgments gives intervals that all exclude 50%, e.g. temporal consistency [73.51, 76.30] and visual realism [55.74, 59.10]. A conservative interval that uses only the published counts (variance bounded by m(1−m)) is also given and excludes 50% for every question.
Q5. Wording of the "history reads: V" control. We agree completely: restoring V in a model trained on R is a train–test mismatch. It shows that the trained model uses the outputs it reads, not that outputs are a better quantity to cache. Section 5.3 now says exactly this and points to the matched pairs of Tables 1–2 for the comparison of what to cache; the introduction was reworded the same way.
Parts (2/3) and (3/3) cover training length, the metric trade-offs, cost under GQA, command changes, and the long-video protocols.