acceptodds
Withdrawn from ICLR 2027

MORCA: Offline-to-Online Reinforcement Learning for Adaptive Cache Reuse in Video Diffusion Acceleration

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion Transformers (DiTs) achieve remarkable performance in video synthesis, but their iterative denoising process suffers from high inference latency. To address this, caching has emerged as an effective acceleration strategy by capitalizing on inter-step redundancy during denoising. Existing dynamic caching methods typically estimate the error that cache reuse would introduce at each denoising step (step error) to guide cache decisions, whereas our concern is how much quality loss cache reuse would cause in the final generated video (terminal error). We show that step error does not directly correspond to terminal error and that latent information helps capture their relationship, thereby informing cache decisions. Moreover, existing threshold-based methods cannot provide precise speedup control, making it difficult to meet practical requirements for user-specified acceleration targets. To address these limitations, we introduce MORCA, a cache scheduling framework trained through offline-to-online reinforcement learning to make latent-aware reuse/recompute decisions under user-specified acceleration targets. Extensive experiments on different video generation models across multiple target acceleration ratios demonstrate that MORCA achieves better generation fidelity than state-of-the-art caching methods under comparable computational budgets.

Primary area:
generative models
Keywords:
Video generation, Diffusion models, Inference acceleration, Caching, Reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_c92HOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MORCA learns a cache scheduler for video diffusion acceleration using offline-to-online reinforcement learning. Built on SeaCache, it combines latent features with a reuse-budget constraint to make per-step reuse or recomputation decisions, using final-video PSNR as the training reward. Experiments on Wan2.1 and Wan2.2 show improved fidelity over the evaluated caching baselines at comparable inference latency.

Critical strengths
  1. The motivation is clear. Local caching errors do not directly reflect their impact on the final video. Incorporating latent information and optimizing a terminal reward is a reasonable approach to improving scheduling decisions.

  2. The method addresses a practical need for user-specified acceleration targets. Experiments across two video models and multiple speedup levels show consistent PSNR and SSIM improvements, with ablations supporting the latent features and training design.

Critical weaknesses
  1. The underlying motivation and observations have limited novelty. The fact that local caching errors do not directly reflect final generation errors has been discussed in LeMiCa, ERTACache, and related work, including the impact of caching decisions on final outputs and the accumulation and amplification of errors. The paper should discuss these studies and clarify what its observations and method add to prior work.

  2. Direct comparisons with learning-based caching methods are missing. OnlineCache also targets final generation quality, while BAG accounts for the computation budget. Comparisons with heuristic methods alone are insufficient to establish MORCA’s advantages. The IQL versus REINFORCE ablation does not replace comparisons between complete methods.

  3. Although the test prompts come from VBench, no VBench scores are reported. PSNR, SSIM, and LPIPS mainly measure similarity to the original videos. The quality of the generated videos themselves still needs evaluation, particularly motion and temporal consistency at higher speedups.

  4. The training cost is not clearly reported. The paper reports approximately 1% scheduler overhead during inference on Wan2.2, but does not provide the actual time or GPU-hours required for data collection and each training stage. Since separate training is needed for different models, these costs affect the method’s practical value.

Questions for authors

Please refer to the Critical Weaknesses section for questions. My main concerns are comparisons with learning-based caching methods and evaluation using VBench or other metrics that assess video quality. Please also report the additional training and inference costs introduced by the learning-based caching approach.

Reviewer_jNGWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MORCA, a learned cache scheduler for video diffusion transformers. Building on SeaCache’s caching mechanism, it replaces the threshold rule with a binary policy that decides at each denoising step whether to reuse the cached residual, with a state consisting of SeaCache’s signals, pooled latent features, and a reuse-budget counter. The target speedup is converted into an exact number of reuse steps enforced as a hard constraint. The policy is trained with IQL on offline trajectories collected from random threshold schedules and then fine-tuned online, with terminal PSNR to the no-cache output as reward. Experiments on Wan2.1-1.3B and Wan2.2-A14B compare against TeaCache, MagCache, DiCache and SeaCache at several target speedups.

Critical strengths
  • The observation that step error does not reflect final video quality (Fig. 1, Table 1) is a good motivation for learning the schedule rather than thresholding.

  • Formulating the target speedup as a hard reuse budget is a clean idea, and the realized speedups land close to the target without per-target tuning.

  • Fig. 6 shows the improvement over SeaCache is consistent across nine speedup levels rather than at a single operating point.

Critical weaknesses
  • The method is motivated by per-prompt, latent-aware decisions, but the learned schedules in Fig. 5 look largely the same across prompts: recompute early, reuse late, forced block at the end. And in Table 3 the variant with no latent features at all (A0) already captures 1.15 dB of the 2.17 dB gain over SeaCache. The natural control is a single fixed K-reuse schedule chosen on the training prompts and applied to all evaluation prompts at the same budget. Without it, the gain could come mostly from enforcing the budget rather than from the policy.

  • Also in Table 3, the full latent-aware state adds only +0.45 dB over A0, while some partial-latent variants are below A0 (A3: 19.27 vs 19.93). With one training seed per cell, the contribution attributed to latent features is not established.

  • The evaluation metric is the training reward. The reward is PSNR to the no-cache reference (Eq. 4) and the results are reported as PSNR/SSIM/LPIPS to the same reference, while the baselines were tuned with their own threshold rules and not for PSNR. No VBench-style quality metric or user study is provided, so it is unclear whether the 1–2 dB translates into visible quality.

  • The two evaluated models are both from the Wan family, and App. F notes that the scheduler does not transfer across models. While the scheduler itself is lightweight, training it requires 2,500 + 1,600 generations with Wan2.2-A14B and 4,000 + 1,600 generations with Wan2.1, and this has to be repeated for every new model. The total time for data collection and training is not reported, and I expect it to be considerable.

Questions for authors

Please refer to Weaknesses section.

Reviewer_mqShOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes MORCO, a learned cache-scheduling framework for accelerating video DiT. The author states that existing dynamic caching methods decide based on step error, while the terminal error actually matters. MORCA keeps SeaCache's caching granularity, reuse strategy, and proxy signals, but replaces its threshold rule with a learned policy. Cache scheduling is a budget-constrained finite-horizon MDP with a terminal PSNR reward. The policy is trained with offline IQL on randomly perturbed SeaCache threshold trajectories, then fine-tuned online. On Wan2.2-T2V-A14B and Wan2.1-T2V-1.3B, MORCA improves PSNR over SeaCache by roughly 1–2.2 dB at matched speedups.

Critical strengths
  • The paper is well motivated and supported by evidence. The step-error vs. terminal-error distinction is useful information.
  • The exact budget control is useful in practice.
  • MORCA outperforms SeaCache on PSNR, SSIM, and LPIPS at Wan2.2.
Critical weaknesses
  • RAPID 3^ 3 already formulates cache scheduling as an MDP solved with RL. OnlineCache already trains a latent-aware RL scheduler. BAG already targets budget-aware gating.
  • All baselines in Table 2 are training-free threshold methods, while MORCA trains a model-specific policy on thousands of rollouts.
  • The experiments do not show that RL is needed.
  • In Table 3, removing all latent features (A0) costs only a minor PSNR. It seems most of the improvement comes from learning a schedule from SeaCache's existing signals with the budget.

Zhao, Wangbo, et al. "RAPID 3^ 3: Tri-Level Reinforced Acceleration Policies for Diffusion Transformer." International Conference on Learning Representations. Vol. 2026. 2026.

Xie, Z., et al. "OnlineCache: Learning Dynamic Caching Policies with Error Correction for Efficient Diffusion Inference". arXiv preprint arXiv:2607.29398. 2026

Zhao, Tong, et al. "BAG: Budget-Aware Gating for Diffusion Caching." arXiv preprint arXiv:2608.09231 (2026).

Questions for authors
  • Which component of MORCA is new compared to RAPID 3^ 3, OnlineCache, and BAG?
  • Every baseline in Table 2 is a training-free threshold method, while MORCA trains a model-specific policy on thousands of rollouts. Can you compare against learned or search-based scheduler (e.g., any of OnlineCache, BAG) at matched speedups on Wan2.1?
  • The experiments compare RL variants against each other (Table 4) but never against non-RL ways of using the same data and budget. Could you report the best fixed, prompt-agnostic schedule per K?
  • Can you break down the gain step by step, from SeaCache, to SeaCache with budget masking, to a learned policy without latent features, to the full MORCA?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.