acceptodds
Under review as a conference paper at ICLR 2027

MAX-Cache: Motion-Aligned Activation Reuse with Mismatch-Guided Selective Recomputation for Efficient Autoregressive Video Diffusion

Abstract

Interactive driving simulation requires video world models to generate future scenes continuously and with low latency. Existing acceleration methods reduce denoising steps, sparsify attention, or optimize key-value caches. We exploit a complementary source of redundancy: neighboring video chunks share scene content and have similar intermediate activations. Reusing these activations at unchanged token locations, however, is sensitive to ego motion, independently moving objects, parallax, and occlusion. We proposeMAX-Cache, an inference-acceleration framework with lightweight post-hoc calibration for motion-aligned activation reuse across chunks. MAX-Cache evaluates a safety gate before either reuse mechanism. When the gate passes, it directly reuses the entire motion-aligned output at , bypasses the prefix through , and repairs that feature with a lightweight residual corrector calibrated post hoc from training-split teacher pairs while the pretrained generator remains frozen. Otherwise, the executed blocks use a partial token mask: low-mismatch tokens reuse warped cached outputs, while high-mismatch tokens retain fresh block updates. Conservative reuse criteria and baseline K/V-state maintenance limit error accumulation. On the 21-frame single-view nuScenes setting, the resulting configuration increases end-to-end throughput from 3.90 to 5.48 FPS (1.41). Across both 21- and 211-frame settings, VBench, FID, and FVD remain close to the baseline, while the measured 211-frame end-to-end speedup reaches 1.30.

Primary area:
generative models
Keywords:
autoregressive video diffusion, driving world models, inference acceleration, motion-aligned cross-chunk reuse, token-level routing

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_YkC4OpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MAX-CACHE, a framework for accelerating cross-chunk inference in few-step autoregressive video diffusion models. Its main idea is to motion-align intermediate activations cached from the previous video chunk and then select between two mutually exclusive reuse paths using a safety gate. When the gate passes, MAX-CACHE reuses the aligned and residual-corrected output at DiT block 28 and skips the preceding 29 blocks. When the gate rejects whole-prefix reuse, the model executes the prefix and uses an input-side mismatch score to select high-risk tokens for recomputation, while the remaining tokens reuse warped cached outputs. The pretrained generator remains frozen, and only a lightweight residual corrector with approximately 295K parameters is trained. On single-view nuScenes using Wan2.1-T2V-1.3B and one NVIDIA H20 GPU, the method improves end-to-end throughput from 3.90 to 5.48 FPS for 21-frame generation, corresponding to a 1.41× speedup. For 211-frame generation, throughput improves from 4.27 to 5.57 FPS, corresponding to a 1.30× speedup. The reported VBench, FID, and FVD results remain close to those of the Causal Forcing++ baseline. The paper also provides a diagnostic showing that a direct transfer of X-CACHE is ineffective, activation-level analyses of motion alignment and correction, token-routing diagnostics, runtime decomposition, and several component and parameter ablations.

Critical strengths

The paper addresses an important and practically relevant problem. Driving world models require continuous, low-latency generation over long horizons. Cross-chunk computation remains expensive even after the number of denoising steps has been reduced. Exploiting redundancy between adjacent chunks is therefore a meaningful direction.

The method directly addresses spatial misalignment in cached activations. Instead of reusing activations at unchanged token positions, MAX-CACHE estimates per-view translations using phase correlation and warps the cached features accordingly. The X-CACHE compatibility diagnostic provides useful motivation for performing explicit alignment before assessing reuse reliability.

The two-level reuse design is coherent and reasonably complete. Whole-prefix reuse provides larger computational savings when the safety conditions are satisfied. When the gate rejects this path, mismatch-guided selective recomputation still exploits locally reliable cached features. Maintaining dense K/V history and performing clean recaching reduce the risk of recursively contaminating the persistent causal state.

Critical weaknesses

The experimental scope is limited. The main results cover only single-view nuScenes, one 1.3B backbone, one output resolution, and one NVIDIA H20 GPU. It is therefore unclear how well the method generalizes to other backbones, datasets, multi-view generation settings, GPU architectures, or closed-loop simulation.

The post-training cost is not included in the overall efficiency analysis. Although optimizing the corrector takes only about 8.7 minutes, exporting the teacher pairs takes approximately 10.4 hours, for a total calibration cost of about 10.6 GPU-hours. The paper does not report the break-even point at which inference-time savings compensate for this calibration cost. This is particularly relevant because a new backbone, hidden dimension, view layout, or reuse point requires a new teacher-pair export and calibration.

The statistical reliability of the quality results is insufficiently characterized. The 21-frame evaluation uses 150 samples, but the paper does not report results across multiple random seeds, confidence intervals, or paired variability. Because the differences in VBench, FID, and FVD between MAX-CACHE and Causal Forcing++ are relatively small, point estimates alone may not be sufficient to support the strong claim that quality is preserved. The number of samples used for the 211-frame quantitative evaluation should also be stated more clearly.

Questions for authors
  1. Generalization: How well does MAX-CACHE generalize to other datasets, backbones, multi-view settings, and GPU architectures?

  2. Baseline Fairness: Since some baselines are partially reimplemented or adapted, how do the authors ensure that the comparisons are fair?

  3. Practical Efficiency: How is selective token recomputation implemented to achieve actual computational savings? Could the authors also report its memory overhead and calibration break-even point?

Reviewer_PDneOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MAX-Cache, an inference acceleration framework for autoregressive video diffusion. It exploits temporal redundancy across adjacent video chunks through motion-aligned activation reuse and mismatch-guided selective recomputation. A lightweight residual corrector is trained while keeping the pretrained generator frozen. Experiments on nuScenes demonstrate approximately 1.3–1.4× end-to-end speedup with largely preserved generation quality.

Critical strengths
  1. Clear formulation. The problem definition and mathematical formulation are intuitive and easy to follow. The distinction between whole-prefix reuse and token-level selective recomputation is well explained.

  2. Reasonable empirical results. The method achieves meaningful inference acceleration while maintaining competitive generation quality, supported by component ablations and short- and long-horizon evaluations.

Critical weaknesses
  1. Insufficient novelty discussion and comparison with prior work. The paper does not adequately distinguish its contributions from existing video caching and activation reuse methods. Relevant works include FastCar [1], which investigates temporal activation reuse in autoregressive video generation; MotionCache [2], which introduces motion-aware token-level caching; and ARCache [3], which addresses history-guided caching and residual correction for autoregressive video diffusion. None of these are cited or compared. Although their architectures and reuse mechanisms differ, these works are closely related to the proposed approach and deserve more systematic discussion. Overall, the related-work coverage is relatively shallow.

  2. Moderate acceleration with additional training cost. While the reported speedups are useful, the improvements over the strongest competing methods are marginal: in Table 1, Light Forcing reaches a slightly higher end-to-end speedup at 21 frames (1.44× vs. 1.41×), and at 211 frames MAX-Cache leads Light Forcing and Forcing-KV by only 0.01–0.02×. Furthermore, the final method requires backbone-specific residual corrector training and teacher feature extraction. Its practical advantages over simpler or training-free alternatives (e.g., ARCache [3]) are not sufficiently established.

  3. Insufficient analysis of motion-dependent activation dynamics. Figure 3 groups transitions by ego motion (stationary / straight / turning) and reports reuse error across DiT blocks, but the alignment itself is a per-view global translation, and the whole-prefix gate relies on a yaw threshold and a reuse-streak limit. There is no analysis that separates independently moving objects from ego motion, or examines how their interaction affects intermediate representations across Transformer layers and denoising steps. A deeper analysis of motion-dependent feature changes would better justify the proposed routing criteria and heuristic thresholds.

  4. Limited generalization evidence. Experiments are restricted to single-view nuScenes with one backbone. The transferability across architectures, datasets, motion distributions, and multi-view generation remains unclear, particularly given the need for backbone-specific corrector calibration.

  5. Presentation issues. Figure 2 is visually overloaded, with numerous small equations and labels that are difficult to read. A simplified overview would improve clarity.

References

[1] Shen et al. FastCar: Cache Attentive Replay for Fast Auto-Regressive Video Generation on the Edge. ICLR 2026. https://arxiv.org/abs/2505.14709

[2] Xu et al. Motion-Aware Caching for Efficient Autoregressive Video Generation. ICML 2026. https://arxiv.org/abs/2605.01725

[3] Nan et al. Accelerating Autoregressive Video Diffusion via History-Guided Cache and Residual Correction. CVPR 2026. https://openaccess.thecvf.com/content/CVPR2026/html/Nan_Accelerating_Autoregressive_Video_Diffusion_via_History-Guided_Cache_and_Residual_Correction_CVPR_2026_paper.html

Questions for authors
  1. How does MAX-Cache differ conceptually and empirically from FastCar, MotionCache, and ARCache? A direct comparison (or a principled argument for why one is not applicable) against at least MotionCache and ARCache would materially affect my assessment of novelty.

  2. Given that Light Forcing is slightly faster at 21 frames and the 211-frame margin is 0.01–0.02×, what is the concrete practical advantage of MAX-Cache once the teacher export and corrector calibration cost are included? Table 2 reports the warp-only (training-free) variant only in 21-frame core FPS with degraded FID/FVD; how does it compare to the baselines in end-to-end FPS and at 211 frames?

  3. How does the reuse error behave for tokens on independently moving objects versus static background under the same ego motion, and does this vary across denoising steps?

  4. (Clarification / suggestion) Is there any evidence that the method transfers to another backbone, dataset, or the multi-view setting without re-running the full calibration pipeline?

  5. (Suggestion) Please consider simplifying Figure 2 for readability.

Reviewer_favyOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MAX-Cache to accelerate autoregressive video diffusion by reusing intermediate features across neighboring chunks. It aligns cached features using estimated motion, then either skips a block prefix or selectively recomputes tokens with large feature differences. A small trained corrector repairs reused features while the base generator remains frozen. On single-view nuScenes, the method achieves 1.41× and 1.30× end-to-end speedups for 21- and 211-frame generation, with quality close to the baseline.

Critical strengths
  • The paper addresses a practical problem: neighboring chunks contain repeated content, but motion makes direct feature reuse unreliable.
  • The design combines alignment, selective recomputation, and cache refresh in a reasonable way. It works with an existing few-step generator without retraining its main weights.
  • The paper reports end-to-end timing and includes useful component ablations. The results show a workable speed–quality tradeoff.
Critical weaknesses
  • Removing phase correlation only moves FID 12.09 -> 12.19 and FVD 135.13 -> 136.10 (Table 2). In Fig. 3b the warp reduces error by 0.2/4.2/7.2%, while the corrector reduces it by 23–31%. Most of the quality seems to come from the corrector and token routing.
  • Your baselines FlowCache-CD drops FlowCache's K/V compression and breaks down at 211 frames (FID 65.99, VBench 0.713), which looks more like an untuned port than FlowCache's real behavior. X-Cache is only run with its original thresholds, where it never triggers; that shows the gate doesn't transfer as-is, not how a tuned X-Cache would compare.
  • Evaluation scope is narrow. Results use one backbone and single-view nuScenes. The backbone has cross-view attention and the paper is framed around multi-view driving simulation, but there are no multi-view results.
Questions for authors
  • How often does the whole-prefix reuse gate pass in stationary, straight-driving, and turning scenes? Can the authors report quality and speed separately for these cases, including scenes with moving objects and occlusion?
  • Could X-Cache be evaluated with thresholds tuned for this backbone? A speed–quality curve would be more informative than showing that the original gate does not activate.
1 comment

Thank you for the detailed response and the additional experiments.

Points 1 and 2 address my questions. The re-run FlowCache-CD baseline is also a much fairer comparison than the original one.

The six-view evaluation is the most useful addition for me, since the paper is motivated by multi-view driving simulation. One thing is still missing: you report the dense Core FPS (3.10) but not the dense FID/FVD under the same six-view setting, so I can't tell how much quality is lost relative to 13.37/200.96. Please add the dense baseline quality to that table.

I also found the Light Forcing combination convincing. Getting an extra speedup (11.04 -> 12.73) with similar FID/FVD suggests the method is complementary to attention-side acceleration.

I will keep my score of acceptance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.