- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents KineTra, an alternative representation for video diffusion transformers. Instead of generating a dense spatial latent grid for every frame, the model predicts an initial latent frame and segment-specific weight tokens. These weights parameterize a shared executor that transforms the carried boundary frame into a nonlinear latent trajectory, whose endpoint is passed to the next segment.
Training proceeds in two stages. The first stage jointly trains a slot encoder and the executor through predicted boundary frames, reducing the mismatch between training and generation. The second stage freezes this representation and trains the pretrained video transformer with rectified flow to generate the initial frame and transition weights. The paper also provides theoretical analyses of representation capacity and boundary-error propagation.
Critical strengths
- A meaningful representation-level contribution
KineTra replaces dense frame-wise generation with compact, segment-specific transition weights applied to a carried latent state. This directly addresses temporal redundancy and constitutes a substantive change to the generative representation.
- Strong empirical support
The evaluation is broad and carefully controlled. The results consistently support a favorable efficiency–quality trade-off and indicate that the method generalizes beyond a single architecture or setting.
- Clear analysis of scope and limitations
The paper explains why the representation can reduce computation and how recurrent boundary errors affect generation. It also identifies regimes in which the method is less effective, making the claims more credible and scientifically useful.
Critical weaknesses
- The quality-preservation claim lacks uncertainty and non-inferiority evidence
The paper mainly reports point estimates. Without paired confidence intervals and a non-inferiority margin, failure to detect a difference does not establish quality preservation.
Resolution condition: Report paired seed- and prompt-level intervals and test them against a meaningful degradation margin.
- The matched training budget cannot be independently verified
The “equal training budget” comparison is not auditable because absolute updates, processed samples, and compute are not reported. Since the stages have different per-step costs, the fairness of the comparison remains unclear.
Resolution condition: Report per-stage updates, samples, GPU time or FLOPs, and the exact alignment with Dense.
- The contribution of generated transition weights is not isolated from simpler alternatives
The ablations do not isolate dynamic weight generation from simpler alternatives: the interpolation baseline is weak, fixed-executor conditioning is absent, and the causal comparison lacks an official few-step Self-Forcing setup with KV-cache.
Resolution condition: Compare these alternatives under matched output formats, hardware, budgets, and quality–efficiency metrics.
Questions for authors
-
How is the “equal training budget” defined? Please report trainable parameters, updates, processed samples, and GPU hours or FLOPs for Stage 1, Stage 2, and Dense, and state how the budgets are matched.
-
How does KineTra compare with sparse keyframe generation followed by learned video completion? The linear-interior control is weak. Please add a matched learned completion baseline and report quality, motion, long-video consistency, and end-to-end runtime and memory.
-
Is dynamic weight generation necessary? Compare the generated matrices with a parameter- and compute-matched fixed executor conditioned through cross-attention or feature modulation. Similar performance would weaken the weight-generation claim.
-
Can code capacity be separated from executor capacity? The width sweep changes code size, executor capacity, and cost together. Please vary weight-token capacity and executor size independently, reporting quality, runtime, and memory.
-
Was KineTra fairly compared with Self-Forcing plus KV-cache? Clarify whether KV-cache and the official few-step sampler are used. Otherwise, add a same-hardware comparison with official Self-Forcing and report quality, time, memory, GPU count, and GPU hours.
-
How sensitive is KineTra to the Stage 1 budget and training-data scale? Vary the Stage 1 allocation and data fraction at fixed total compute, and report quality, long-video consistency, and training cost. Strong scale dependence should be stated as a limitation.
3 comments
Response to Reviewer gErz (2/3): simpler alternatives and code vs. executor capacity
(2/3) Simpler alternatives and code vs. executor capacity
W3 / Q2 / Q3. Is dynamic weight generation needed?
We agree this was the main gap, and it was the most useful request in the review. New Table 4 (§4.2; all columns in Table 24) trains the alternatives you named with the same GPU-hour budget, data order, output format, and hardware as Dense and KineTra, and reports quality, motion, long-video consistency, end-to-end time, and memory. These runs use SANA-Video 2.0 and one training seed; Dense and KineTra rows are the same seed.
| Variant (SANA, seed 1) | Tokens | DAVIS PSNR | Qual. 129 f | Dyn. binding 129 f | Qual. 961 f | Subj. 961 f | Sec. 961 f | GiB 961 f |
|---|---|---|---|---|---|---|---|---|
| Dense | 46,464 | 34.42 | 85.39 | 12.81 | 83.27 | 93.36 | 1108.2 | 35.79 |
| Keyframes + learned completion | 11,904 | 33.44 | 85.04 | 10.91 | 82.94 | 93.30 | 331.7 | 25.84 |
| Fixed executor + cross-attention | 8,064 | 33.69 | 85.19 | 12.26 | 83.06 | 93.28 | 251.5 | 23.58 |
| KineTra | 8,064 | 34.05 | 85.53 | 13.06 | 83.28 | 93.45 | 250.8 | 23.43 |
Learned completion (Q2). The keyframe model generates every boundary frame and fills the interior with a spatio-temporal transformer conditioned on both endpoints and the text, with parameters matched to our encoder and executor. It is a far stronger control than the linear interior of Table 3. It still needs 11,904 tokens, runs 1.32× slower than KineTra, and loses 2.15 dynamic binding: without a code for the segment, events between the endpoints can only be guessed.
Fixed executor (Q3). This variant keeps the slot encoder, the same 256 tokens, the carried frame, and the trajectory, and matches shared parameters and executor FLOPs to KineTra within 1%. It comes closest: it trails by 0.34 short-video and 0.22 long-video Quality, about twice the paired seed SD in Table 23, and more clearly by 0.36 dB of reconstruction and 0.80 dynamic binding. A generated 32×32 matrix transforms every feature of the carried frame at every position, while cross-attention adds a mixture of code vectors, and detail lost at one boundary is carried into all later ones. With one seed we treat the Quality gaps as indicative and the reconstruction gap as the firm result; we would extend these controls to more seeds and LTX for the final version.
Table 4 also includes temporal patching and a MAGViT-style compressor at KineTra's token count; both are fast but lose quality or reconstruction (details in our reply to Reviewer 5KnK).
Q4. Code capacity versus executor capacity
Table 25 separates the two where the question matters most, at R = 8, where quality drops. Starting from the existing R = 8 SANA Stage-1 checkpoint, we trained a quarter of the Stage-1 schedule more with the setup unchanged (control), with twice the generated tokens per segment, or with twice the shared executor, and measured reconstruction and executor time:
| R = 8 variant | DAVIS PSNR | Executor s (961 f) |
|---|---|---|
| R = 4 default | 34.05 | 7.2 |
| control (256 tokens, 33.8M) | 33.49 | 6.8 |
| 512 tokens | 33.97 | 7.3 |
| 2× executor (67.4M) | 33.57 | 13.6 |
Doubling the tokens recovers 0.48 of the 0.56 dB lost against R = 4; doubling the executor recovers 0.08 dB at twice its run time. Per-segment code capacity, the quantity in Theorem 1, is the binding constraint. A full code × executor grid with generation at every cell needs a training run per cell; this reconstruction-level version answers the question it was meant to answer.
Response to Reviewer gErz (3/3): official Self Forcing, Stage-1 share and data scale, summary
(3/3) Official Self Forcing, Stage-1 share and data scale, summary
Q5. Was KineTra compared with Self Forcing plus KV cache?
Not in the submission, and thank you for making us state this clearly. The causal Dense row of Table 2 uses the Self Forcing weights but denoises all frames jointly for 50 steps under a block-causal mask, without the KV cache or the few-step sampler; it isolates the representation, not the deployed pipeline. Table 29 now adds the released checkpoint with its official four-step chunk-wise sampler and rolling KV cache, on the same A100, output format, and VBench-Long protocol, next to KineTra with four-step distillation:
| Configuration (961 frames) | Qual. | Subj. | Sec. | GiB | GPUs | GPU-h per 1,000 videos |
|---|---|---|---|---|---|---|
| Dense, joint 50-step | 80.92 | 91.10 | 7275.7 | 62.50 | 4 | 8084.1 |
| KineTra, 50-step | 80.76 | 91.47 | 491.1 | 38.04 | 1 | 136.4 |
| Official Self Forcing, 4-step, KV cache | 79.37 | 89.82 | 131.1 | 21.64 | 1 | 36.4 |
| KineTra + 4-step DMD | 80.39 | 91.38 | 195.9 | 38.11 | 1 | 54.4 |
The two new rows use all 946 VBench-Long prompts (one sampling seed for Self Forcing, training seed 1 for KineTra + DMD). Official Self Forcing is 1.49× faster than four-step KineTra and streams its first chunk, which KineTra cannot, since it generates all weights jointly. Over 961 frames, far beyond its five-second training clips, its Quality is 1.02 points and its subject consistency 1.56 points lower. Both facts are in §4.4: the deployed causal pipeline wins on latency, and KineTra keeps long-video quality.
Q6. Sensitivity to the Stage-1 budget and data scale
Rather than retraining both stages at several splits and data fractions, Table 26 tests the part that Stage 1 controls, the representation, on SANA. The saved checkpoint at half of the Stage-1 updates reconstructs DAVIS at 33.83 dB instead of 34.05. Training Stage 1 from scratch on 2.5% of the unique sources costs 0.64 dB on out-of-domain DAVIS, more than KineTra's whole gap to its VAE, and 0.31 dB on held-out MiraData; at 10% most of the loss is gone. Since generated weights cannot reconstruct more than encoded ones, this bounds what Stage 2 can reach. As you suggested, §4.4 now states data scale as a limitation; each setting's training cost is that of a Stage-1 run in Table 21.
For the discussion with the AC
- Quality preservation is now tested, not inferred: 17/17 prompt-level and 16/17 seed-level intervals clear a 0.5-point margin set by the Quality cost of training-free caching (Table 23).
- The budget is auditable per stage, and the convergence curves show Dense is not stopped early (Tables 21–22).
- At the same budget, generated weights beat learned completion and a parameter- and FLOP-matched fixed executor, most clearly in reconstruction and dynamic binding (Table 4).
- The R = 8 limit is code capacity, not executor size (Table 25), as Theorem 1 predicts.
- The limits you asked about (Self Forcing streaming, data scale) are measured and stated in §4.4.
Your review turned the paper's central claim into something we could test, and the paper is better for it. If these results resolve your concerns, we would be grateful if you would consider raising your score and supporting the paper in the discussion. We are glad to answer anything further.
Response to Reviewer gErz (1/3): non-inferiority and the training budget
(1/3) Non-inferiority and the training budget
Thank you for a careful and generous review. You read KineTra as "a meaningful representation-level contribution", found the evaluation "broad and carefully controlled", and singled out the analysis of scope and limitations as what makes the claims credible. That is the reading we hoped for. Your three resolution conditions were concrete enough to turn directly into experiments; all of them are in the new Appendix H (blue), and the main text now states their outcome. Section and table numbers refer to the revised PDF.
W1. Quality preservation needs intervals and a margin
You are right that point estimates cannot establish preservation. Table 23 (Appendix H.2) now tests every Quality, Semantic, and VMBench change in Tables 1 and 2 against a non-inferiority margin of 0.5 points. The margin is about the Quality cost of the faster training-free caching presets on long LTX videos (Table 20: −0.39 to −0.54), a quality cost that published caching methods accept. We report both intervals you asked for: paired seed-level t intervals over the three within-seed differences, and paired prompt-cluster bootstrap intervals (2,000 resamples, seeds pooled).
We also rewrote Appendix D.2: with chance-level rater agreement, the human study is uninformative about parity, so the paper no longer leans on it.
W2 / Q1. The matched budget must be auditable
The budget is matched in A100 GPU-hours, counting video encoding, both stages, validation, and communication (Appendix A.4). Table 21 lists, per body and seed, trainable parameters, updates, processed clips, GPU-seconds per update, and GPU-hours. For LTX:
KineTra's total stays within 0.2% of Dense's GPU-hours on all four bodies. Because a Stage-2 update runs the transformer on 3–17× fewer tokens, Stage 2 makes more updates than Dense under the same GPU-hours (3.7× on LTX). This is the asymmetry you pointed to, so we checked whether it leaves Dense undertrained. Table 22 evaluates the saved SANA checkpoints of both models along the budget (three seeds, a fixed 236-prompt VBench subset): Dense gains only +0.04 Quality in the last quarter of its budget, so it is close to converged, and the parity in Table 1 does not come from Dense stopping early. Continuing Dense to KineTra's update count would need several times its budget; the flat end of its curve answers the same question.