- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This study faces a practically significant application involving world models, thus the challenge of balancing inference speed with generation quality. To tackle this, the authors propose SpectralCache, a training-free spectral caching framework that utilizes linear extrapolation and scaling of singular values during generation process. Additionally, based on singular value analysis, the authors uncover two key insights regarding the world model: first, the singular value subspaces and singular vectors exhibit stability across consecutive steps; second, adjacent singular values demonstrate a linear relationship, allowing for a predictable estimate via linear operations. Experimental results show that the proposed framework outperforms competitive baselines, particularly regarding latency and generation quality.
Critical strengths
The paper clearly articulate the motivation behind this line, making their objectives easy to maser, given the limitations of existing research on caching and the growing demand for world models, inference speed is indeed a crucial factor worth addressing. Moreover, the experimental evaluation is adequate, ranging from the experimental settings to ablation studies, demonstrating the significant effort invested in this research. The authors also provide substantial qualitative evidence and algorithm derivations in the appendix to support the proposed framework.
Critical weaknesses
The base models selected by this study, HunyuanVoyager-13B and Aether-5B both generate videos based on image, text, and camera trajectory inputs. While the image condition remains relatively static, serving as the reference for the subsequent video sequence, the camera trajectories should ideally undergo significant dynamic changes rather than remaining fixed during generation and the semantic complexity introduced by different prompts also influences generation quality. I am aware that the authors' primary work focuses on DiT visual representations, but I am curious that how do the authors view and address the impact of these varying conditioning modalities on spectral stability. Could the authors provide a small robustness validation varying prompt complexity and camera trajectory magnitude to examine whether the observed spectral stability remains consistent across different conditioning settings?
Beyond the overall performance gains compared to other training-free methods, I am more curious in seeing a breakdown of performance improvements or quality degradation based on specific categories, e.g., indoor vs. outdoor or simple vs. complex scenes. The authors could design small-scale experiments for this aspect.
As the authors clarify in the introduction, this is a challenge regarding deployment practices. Therefore, it is important to honestly evaluate performance differences across hardware. The original experiments utilized two NVIDIA Pro 6000 96GB GPUs, a very powerful hardware setup, so how would the inference acceleration performance hold up under less powerful hardware conditions?
From a reader's perspective, although these are minor issues, I still feel it is necessary to point them out so that the author can review and revise them (including but not limited to):
- Please refer to Section 4.1 (CITATIONS WITHIN THE TEXT) in ICLR 2027 publicized template about the use of \citep{ } or \citet{ }
- The table caption should be placed at the top to maintain consistency with the template.
- The author should carefully check whether a comma or a period is used at the end of each equation.
- Lacks a discussion of limitations and future work for a complete paper.
Questions for authors
Please see the questions raised in the Critical Weaknesses, particularly regarding spectral stability, model performance, and stable inference efficiency. I hope to engage in a constructive discussion with the authors during the discussion phase.
[Direct Feature Extrapolation Ablation]
We thank the reviewer for this valuable suggestion. We conducted an additional ablation on HunyuanWorld-Voyager-13B by directly applying linear extrapolation to the feature F under the same scheduler. As shown below, direct feature extrapolation achieves only 22.81 PSNR and 0.738 SSIM, substantially lower than our method (25.06 PSNR and 0.818 SSIM) and the WorldCache baseline (23.49 PSNR and 0.770 SSIM). These results demonstrate that extrapolating singular values yields substantially better generation quality, whereas directly extrapolating the full features performs even worse than the WorldCache baseline.
Table: Comparison of direct feature extrapolation and spectral-based extrapolation on HunyuanWorld-Voyager-13B.
[Rank Sensitivity]
As shown above, increasing the rank from 32 to 256 consistently improves PSNR from 24.67 to 25.12 and SSIM from 0.806 to 0.820. Notably, even Rank=32 substantially outperforms direct feature extrapolation (24.67 vs. 22.81 PSNR; 0.806 vs. 0.738 SSIM), while higher ranks further improve reconstruction quality. Together, these results demonstrate the effectiveness of singular-value extrapolation over direct feature extrapolation, even at low ranks.
[Generalization to Diffusion-Based Video Generation]
We thank the reviewer for this valuable question. To evaluate the generalizability of SpectralCache beyond diffusion-based world models, we additionally apply our method directly to HunyuanVideo, a diffusion-based video generation model. We follow the same generative experimental settings as TaylorSeer for a fair comparison such as frames and denoising steps. As shown below, SpectralCache achieves a PSNR of 16.85, an SSIM of 0.732, and a 5.27× speedup, demonstrating its applicability to diffusion-based video generation models.
We further observe that the average similarities of the U and V subspaces between adjacent denoising steps on HunyuanVideo are 0.94 and 0.92, respectively, which are lower than those observed in diffusion-based world models (0.99 and 0.99, respectively). This relatively weaker subspace stability may explain the more modest PSNR improvement on HunyuanVideo, as our method relies on reusing highly stable singular subspaces across denoising steps. This observation further suggests that the particularly strong singular subspace stability identified in our paper is more pronounced in diffusion-based world models, highlighting the distinctive characteristics of their spectral dynamics and the suitability of SpectralCache for exploiting them.
[Effectiveness of Singular-Value Scaling]
While singular-value scaling introduces a modest reduction in generation quality compared with the variant without scaling, the resulting quality remains substantially higher than that of the strongest baseline, WorldCache (25.06 vs. 23.49 PSNR; 0.818 vs. 0.770 SSIM). More importantly, scaling significantly improves inference speed, achieving a 5.22× speedup compared with WorldCache's 3.65×. Without scaling, the speedup is only 3.84×, offering a relatively marginal improvement over WorldCache. Therefore, singular-value scaling plays a crucial role in achieving a substantially better speed–quality trade-off, delivering significantly faster inference while maintaining superior generation quality compared with existing best baselines.
[Scaling Approximation Overhead]
The Extrapolation Cache Step only requires linear extrapolation to reconstruct the features. In contrast, the Scaling Approximation Step involves additional operations beyond singular value scaling, including computing the velocity, acceleration, and curvature of each token, followed by sorting tokens according to their curvature to re-identify chaotic tokens [1], as described in our Method section and Figure 4. These additional operations account for the higher latency of the Scaling Approximation Step compared with the Extrapolation Cache Step, despite the simplicity of the scaling operation itself.
[1] Feng W, Fan G, Qin H, et al. Worldcache: Accelerating world models for free via heterogeneous token caching[J]. arXiv preprint arXiv:2603.06331, 2026.