acceptodds
Under review as a conference paper at ICLR 2027

LiteCa: Memory-Efficient Feature Caching for Diffusion Acceleration

Abstract

Feature caching accelerates Diffusion Transformers (DiTs) by reusing intermedi- ate features across timesteps. Recently Forecasting-based methods show that mod- ule features evolve predictably across timesteps, enabling temporal extrapolation for feature forecasting. However, storing module-level features and their histori- cal states introduces substantial cache memory overhead. To solve this problem, we propose LiteCa, a training-free and plug-and-play memory-efficient feature caching framework. We observe that the Adaptive Layer Normalization (AdaLN) modulation parameters exhibit a low-rank structure, enabling LiteCa to build a more compact cache representation with an adjustable rank for different memory budgets. We further introduce Test-Time Correction to improve generation qual- ity without changing the original forecasting method. Experiments on recent DiT models show that LiteCa reduces feature-cache memory by 90–99% compared with module-level caching, while improving both inference speed and generation quality and achieving state-of-the-art performance.

Primary area:
generative models
Keywords:
Diffusion Acceleration, Feature Caching, Memory Efficiency, Image generation, Video generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_VbDPOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes LiteCa, which uses low-rank AdaLN gate trajectories to aggregate per-module feature caches into a compact representation. It also introduces test-time correction to calibrate the aggregated components at full-computation steps.

Critical strengths
  1. To the best of my knowledge, using low-rank AdaLN gate trajectories to construct compact, cross-module feature caches is a novel contribution to diffusion caching.

  2. The method demonstrates substantial reductions in feature-cache memory across six image and video generation models, supporting its practical utility across different architectures.

Critical weaknesses

The gains over residual caching are not fully isolated. The main Wan2.1–HiCache configuration uses rank 0, where neither the gate-derived components nor TTC contributes. The reported results are obtained using different configurations across settings rather than a consistent configuration.

Questions for authors
  1. Could the authors investigate whether rank can be selected from the gate spectrum, for example using a fixed cumulative-energy threshold, rather than performance on the evaluation set? The existing spectra and rank sweeps could help assess this criterion.

  2. Could the authors analyze why nonzero ranks help some model–host pairs but not others? For example, comparing Qwen-Image and Wan2.1 in terms of gate reconstruction error and gains over rank-0 residual caching may clarify when additional components are useful.

  3. The current TTC analysis mainly demonstrates error reduction on FLUX, but does not explain how well coefficients fitted at an anchor transfer to subsequent skipped steps. Could the authors analyze TTC’s error reduction as a function of distance from the preceding anchor, using the existing per-step results?

8 comments
AuthorsOpenReview ↗

Response to W1: Isolating the gain over residual caching

We thank the reviewer for the constructive feedback.

Table R3A breaks down the transition from residual caching to LiteCa into two stages: adding one gate component and applying TTC. With TTC enabled, LiteCa outperforms both residual caching and the module-level host in PSNR and SSIM across all four settings.

Table R3A. DrawBench-200, N = 6. Rank 0 is residual caching under the host’s own schedule and predictor. PSNR↑ / SSIM↑ / LPIPS↓.

Model + hostHost (module-level)Rank 0 (residual caching)Rank 1, TTC offRank 1, TTC on
FLUX.1-dev + TaylorSeer16.03 / .650 / .39016.19 / .663 / .37915.99 / .654 / .39116.92 / .680 / .362
FLUX.1-dev + HiCache16.85 / .677 / .34817.00 / .688 / .34816.88 / .682 / .35117.25 / .692 / .354
Qwen-Image + TaylorSeer14.66 / .556 / .42411.73 / .413 / .57914.75 / .558 / .43017.83 / .608 / .319
Qwen-Image + HiCache18.06 / .650 / .28416.21 / .601 / .38218.43 / .681 / .29120.02 / .753 / .249
  • Gate component (rank 0 → rank 1, TTC off). Adding one gate component brings LiteCa closer to module-level caching. This increases PSNR by 3.0 dB on Qwen-Image with TaylorSeer, where module-level caching outperforms residual caching, but decreases PSNR by 0.2 dB on FLUX.1-dev, where residual caching performs better. This model-dependent behavior, also shown in Figure 2(d), highlights the importance of a unified representation that spans both caching granularities.

  • Test-Time Correction (TTC off → on). With the same rank-1 cache, enabling TTC improves both PSNR and SSIM across both models and both host methods. Under TaylorSeer, TTC increases PSNR by 0.9 dB on FLUX.1-dev and 3.1 dB on Qwen-Image. Additional comparisons across N=3,6,9N = 3, 6, 9 are provided in Table G1 of our general response.

AuthorsOpenReview ↗

Response to W2: Rank 0 on Wan2.1 with HiCache

We sincerely thank the reviewer for their careful review and recognition of our work.

A key advantage of LiteCa is that it uses a single cache representation for models with different caching preferences (Figure 2(d)). Rank 0 is also part of LiteCa, corresponding to residual caching. With coefficients fixed at 1, increasing the rank moves the representation closer to module-level caching. This allows LiteCa to work with models that favor different caching methods without changing its framework.

On Wan2.1, residual caching already performs similarly to module-level caching. The rank-1 component is also nearly orthogonal to the rank-0 error (Table R3E in our response to Q2), suggesting that adding more components provides little benefit. Thus, LiteCa at rank 0 achieves the best trade-off on Wan2.1: it matches the host in PSNR and SSIM (23.81 dB / .844 vs. 23.77 dB / .838), while using 97.8% less cache and achieving a higher speedup (3.09× vs. 2.85×). Increasing to rank 1 doubles the cache size without improving quality (23.76 dB / .842; Table 9). For models that benefit from additional gate components, Table R3A shows their contributions.

AuthorsOpenReview ↗

Response to W3: One configuration for all settings

We thank the reviewer for the constructive feedback.

We also evaluate LiteCa using a single configuration: rank 1 for every model and host (Table R3B; Appendix F, Tables 8–9). With this setting, LiteCa reduces cache memory by over 95% and runs faster than the host in all 10 model–host pairs. It also improves SSIM in all 10 pairs and PSNR in 9.

Table R3B. LiteCa with rank 1 across all model–host pairs, using the same schedules as Tables 1–2. PSNR↑ / SSIM↑ / LPIPS↓ are measured against full-step inference.

ModelHostHostLiteCa (rank 1)Cache reductionSpeedup (host → LiteCa)
FLUX.1-devTaylorSeer16.03 / .650 / .39016.92 / .680 / .36297.7%3.83× → 4.88×
FLUX.1-devHiCache16.85 / .677 / .34817.25 / .692 / .35497.7%3.81× → 4.88×
Qwen-ImageTaylorSeer14.66 / .556 / .42417.83 / .608 / .31998.3%2.73× → 3.20×
Qwen-ImageHiCache18.06 / .650 / .28420.02 / .753 / .24998.3%2.43× → 3.19×
HunyuanVideoTaylorSeer21.00 / .744 / .21922.07 / .772 / .19898.4%3.12× → 3.46×
HunyuanVideoHiCache22.12 / .763 / .19722.81 / .783 / .19398.4%3.21× → 3.45×
Wan2.1TaylorSeer21.93 / .808 / .13321.99 / .812 / .13295.6%3.01× → 3.07×
Wan2.1HiCache23.77 / .838 / .10623.76 / .842 / .10895.6%2.85× → 3.07×
MiniMax-H3TaylorSeer19.52 / .643 / .24520.04 / .654 / .24098.0%2.75× → 3.00×
MiniMax-H3HiCache20.33 / .664 / .22520.87 / .676 / .22398.0%2.86× → 3.02×
Reviewer_koY2OpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This work reduces cache memory overhead by exploiting the low-rank structure of AdaLN modulation parameters.

Critical strengths
  1. This work introduces a novel approach that applies low-rank decomposition to AdaLN gate.

  2. Table 1 demonstrates a substantial reduction in cache memory usage.

Critical weaknesses
  1. The most significant weakness of this work is the lack of adequate comparisons with methods that reduce the memory footprint of feature caching. In particular, the paper does not experimentally compare its performance with low-rank-based caching methods such as SVD-Cache and LearniBridge.

  2. I believe that the use of a shared basis is the primary factor enabling the substantial reduction in cached feature size. To my knowledge, prior work has already explored using basis representations to reduce cache memory. Please clarify the novelty and advantages of the proposed approach over these existing methods. Alternatively, if the shared basis is not the primary source of the memory savings, please provide a detailed explanation of what drives these reductions. A convincing response to these concerns could lead me to raise my score.

Questions for authors
  1. How does the proposed method perform compared with SVD-Cache and LearniBridge?

  2. While decomposition can reduce cache memory usage, it introduces additional computation during inference. What is the trade-off between memory savings and computational overhead?

7 comments
AuthorsOpenReview ↗

Response to W2: Source of the novelty and memory savings

We thank the reviewer for the constructive feedback.

Yes, the shared basis is the source of the memory savings. The novelty is threefold:

  1. The basis is built on the AdaLN gates, not on the features. The per-module weights wℓkw_{\ell k} of the low-rank gate decomposition are constant in time; only the shared temporal coefficients bk(t)b_k(t) vary, so the LL per-module caches can be summed into K+1K+1 caches. The saving is in the number of caches; features are neither projected nor truncated. On FLUX.1-dev at rank 1, the cache goes from 114 to 2 tensors and from 6.01 to 0.14 GiB (a 90–99% reduction overall).

  2. One representation covers both residual caching and module-level caching. Rank 0 is residual caching; with coefficients equal to 1, raising the rank approaches module-level caching. Models favor different granularities (Figure 2(d)), so the method does not need to change.

  3. Test-Time Correction. It fits one coefficient per gate component at each anchor, and the following skipped steps reuse those coefficients. This allows LiteCa to go beyond the interpolation between residual caching and module-level caching and further improve generation quality.

  • Against feature-basis methods: no training and no calibration. The alternative is calibration data, or an SVD of a large matrix at every anchor. The gates are known before denoising, so each image needs only one small SVD (about 3 ms, or 8 ms including the gates).

  • Against the module-level host: at a fixed rank of 1, all ten settings in Tables 1–2 use over 95% less cache and are faster, with better SSIM in all ten and better PSNR in nine.

To make these distinctions easier to follow, we have revised Figure 3 to provide a clearer overview of LiteCa’s framework and visually illustrate its two key components: Low-Rank Cache Aggregation and Test-Time Correction.

AuthorsOpenReview ↗

Response to Q2: Memory savings versus computational overhead

We sincerely thank the reviewer for their careful review and recognition of our work.

There is almost no trade-off: LiteCa improves memory, speed and quality at the same time.

  • Memory. LiteCa decomposes the AdaLN gates rather than the features, so the per-module caches are merged into K+1K+1 caches, which reduces the cache by 90–99%.

  • Speed. The decomposition runs once before denoising (≈ 8 ms). A skipped step forecasts only K+1K+1 tensors instead of one per module, so it is cheaper than in the host. LiteCa is faster than its module-level host in all ten model–host pairs of Tables 1–2.

  • Quality. Test-Time Correction (TTC) recalibrates the weight of each component online, at the anchor steps the host computes anyway, without extra network evaluations. At a fixed rank 1, LiteCa improves SSIM in all ten pairs and PSNR in nine.

Table R2A. Per-image latency breakdown on FLUX.1-dev (seconds; 1024², 50 steps, N=6N=6; median over 10 prompts). The columns after the total do not overlap; ×n is the number of executions per image. This breakdown is added to the main text of the revision as Figure 7(a) (Sec. 4.4).

MethodCache (GiB)Total (s)Anchor (×10)Skipped (×40)Build (×10)Gate (×1)TTC (×9)
TaylorSeer6.014.3103.2401.0140.054——
LiteCa (rank 1)0.143.4013.2590.0820.0290.0080.022
LiteCa (rank 2)0.213.4543.2540.1000.0460.0080.040
LiteCa (rank 3)0.283.5103.2480.1180.0690.0080.065
AuthorsOpenReview ↗

Response to Q1: Comparison with SVD-Cache and LearniBridge

We thank the reviewer for the constructive feedback.

We have added experiments against LearniBridge. For a fair comparison, both methods use FE=3 (the same anchor schedule and the same number of full steps) and BF16. LearniBridge first trains for 3–5 hours under every setting; LiteCa requires no training. In Table R2B, LiteCa is better on PSNR, SSIM and LPIPS at N = 3, 5, 6 and 8. On ImageReward and on CLIPScore, LiteCa wins three of the four comparisons. SVD-Cache has no executable code, so the structural comparison is in our response to W1 (Table R2C).

Table R2B. FLUX.1-dev, DrawBench-200. PSNR↑ / SSIM↑ / LPIPS↓ (against full-step inference), ImageReward↑, CLIPScore↑. LearniBridge uses its officially released code and default training settings, with the LoRA block switched from FP16 to BF16. LiteCa is applied to HiCache at rank 1. N = 5 and 8 are the intervals used in the LearniBridge paper.

NNMethodPSNR / SSIM / LPIPSImageRewardCLIPScoreTraining
3LearniBridge19.30 / .770 / .2340.99843.7632 adapters (3–5 h)
3LiteCa (rank 1)20.17 / .793 / .2110.99043.84None
5LearniBridge17.30 / .705 / .3240.99843.9738 adapters (3–5 h)
5LiteCa (rank 1)17.95 / .727 / .3051.00544.04None
6LearniBridge16.81 / .663 / .3750.97143.8940 adapters (3–5 h)
6LiteCa (rank 1)17.25 / .692 / .3540.98743.85None
8LearniBridge16.19 / .621 / .4400.93643.7442 adapters (3–5 h)
8LiteCa (rank 1)16.34 / .649 / .4230.94343.97None
Reviewer_SrRdOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes LiteCa, a training-free and plug-and-play framework designed to address the substantial memory overhead associated with module-level feature caching in Diffusion Transformers (DiTs). The authors identify a key structural property: the Adaptive Layer Normalization (AdaLN) modulation parameters exhibit a strong low-rank structure across denoising timesteps. Leveraging this observation, LiteCa decomposes the cache into one residual cache and K aggregated feature caches derived from the low-rank basis of the AdaLN gates. Extensive experiments demonstrate that LiteCa reduces feature-cache memory by 90–99% compared to module-level caching while maintaining or improving generation quality and inference speed.

Critical strengths
  1. Novel Insight into AdaLN Structure: The identification of the low-rank temporal structure in AdaLN gate trajectories is a well-motivated observation. It provides a theoretical basis for compressing module-level information without losing the fine-grained control that residual caching lacks. Figure 2 provides compelling empirical evidence for this low-rank property across different architectures.
  2. Effective Memory-Performance Trade-off: LiteCa successfully bridges the gap between memory-efficient residual caching and high-quality but memory-intensive module-level caching. The ability to tune the rank KK allows users to adapt the method to specific hardware constraints, which is crucial for practical deployment.
  3. Plug-and-Play Design: The method is compatible with existing linear extrapolation-based caching hosts (e.g., TaylorSeer, HiCache) without modifying their core prediction mechanisms or requiring retraining. This enhances its applicability and ease of integration into existing pipelines.
  4. Comprehensive Evaluation: The results consistently show massive memory savings (up to 99%) with competitive performance compared to baseline caching methods. The analysis of peak GPU memory in Figure 12 is particularly relevant for practitioners dealing with OOM issues in video generation.
Critical weaknesses
  1. Assumption Dependency: The method relies on two assumptions: (1) pre-computable low-rank AdaLN parameters and (2) linear temporal extrapolation. While valid for many current DiTs, it may not generalize to architectures that do not use AdaLN or employ non-linear extrapolation schemes.

  2. Reliance on local linearization of the readout: TTC estimates correction coefficients using a Jacobian-based linearization of the model's readout. This approximation may become inaccurate when the cached hidden state deviates substantially from the full-computation state, especially under aggressive step skipping. The paper does not sufficiently characterize the conditions under which the linearization remains reliable.

  3. Limited evaluation of generation quality: The reported primary metrics are frame/video similarity metrics rather than a comprehensive assessment, such as flickering, motion smoothness, identity consistency, or long-range temporal dependencies.

Questions for authors
  1. What are the advantages of LiteCa compared to feature cache offloading to host CPU memory or low-precision quantization storage schemes, and can this framework be integrated with these two paradigms?
  2. Could you illustrate the complete distribution of peak GPU memory consumption during a forward inference pass for video generation models?
  3. When dealing with models containing multiple distinct residual streams (e.g., video and audio streams), how can LiteCa allocate the optimal low-rank for each stream?
9 comments
AuthorsOpenReview ↗

Response to W1: Dependence on AdaLN gates and linear extrapolation

We thank the reviewer for the constructive feedback.

LiteCa requires two conditions: the gates can be computed before denoising, since they depend only on the timestep and the conditioning, and the extrapolator is linear in the cached features. Models without AdaLN gates can still use LiteCa: rank 0 is then equivalent to module-level caching, but stores only one cache. Existing feature-caching methods commonly use Taylor or Hermite extrapolation, both of which satisfy the linearity condition.

  • Models without AdaLN gates. If the gates are absent or do not vary with the timestep, the difference term in Eq. 4 is zero. Under a linear predictor, LiteCa at rank 0 is exactly equivalent to module-level caching while storing one aggregated cache, so the memory overhead of module-level caching disappears entirely.

  • Extrapolator. Existing feature-caching methods commonly use Taylor or Hermite extrapolation. The prediction is a linear combination of the cached features, so LiteCa applies directly. Nonlinear predictors are left as future work, and the revised conclusion states this under Limitations and future work.

AuthorsOpenReview ↗

Response to W2: Reliability of the readout linearization

We sincerely thank the reviewer for their careful review and recognition of our work.

Under more aggressive skipping, the closed-loop gain of TTC becomes larger : as N grows from 3 to 9, its PSNR gain rises from +0.30 to +1.47 dB on FLUX.1-dev and from +0.92 to +5.02 dB on Qwen-Image (TaylorSeer; Table G1 of the general response). Although the output gap at the anchor grows with N, the single JVP retains 86–99% of the iterated fit’s error reduction (Table R1A). At anchor fit of the closed-loop runs, we compare the single-JVP coefficient with an iterated fit that re-linearizes the readout until convergence.

Table R1A. Output gap at the anchor / share of the iterated fit’s error reduction kept by the single JVP (medians).

ModelNN = 3NN = 6NN = 9
FLUX.1-dev0.18 / 99%0.32 / 99%0.44 / 98%
Qwen-Image0.07 / 95%0.13 / 88%0.18 / 86%

The single JVP is also 2.9–4.8 times faster per fit than the iterated fit. The full results are in Appendix D.3 (Table 7, Figure 13) and D.4 (Figure 14) of the revision.

AuthorsOpenReview ↗

Response to W3: Quality evaluation beyond similarity metrics

We thank the reviewer for the constructive feedback.

We have added the ImageReward and CLIPScore results in Table 11 and the video VBench results in Table 12. On images, at N = 3 and 6 the differences from the host are within 0.03 ImageReward and 0.2 CLIPScore, except Qwen-Image with TaylorSeer at N = 6, where LiteCa raises ImageReward from 0.82 to 1.05 and CLIPScore from 42.74 to 44.56; at N = 9, LiteCa consistently scores higher than the host on both metrics in all four settings. On video, LiteCa performs comparably to the host. It is better on Motion Smoothness in all four settings, while using 95.6–98.4% less cache.

Table R1B. ImageReward↑ and CLIPScore↑ on DrawBench-200 (LiteCa at rank 1).

ModelNNIR: TaylorSeerIR: LiteCa-TSIR: HiCacheIR: LiteCa-HiCLIP: TaylorSeerCLIP: LiteCa-TSCLIP: HiCacheCLIP: LiteCa-Hi
FLUX.1-dev30.9960.9940.9890.99043.7843.8043.8343.84
60.9860.9650.9680.98743.8743.8043.7343.85
90.8350.8980.8600.90043.1843.8143.5743.75
Qwen-Image31.1031.0921.1081.08744.8844.8544.9544.88
60.8191.0541.0941.06642.7444.5644.5944.78
9−0.6670.7540.7720.99437.9242.8442.5044.02

Videos. LiteCa is faster in all four settings. Subject Consistency and Background Consistency are computed over all frames of a video, so they also reflect longer-range consistency.

Table R1C. VBench on HunyuanVideo and Wan2.1. LiteCa is at rank 1 throughout.

ModelMethodCache (GiB)↓Speedup↑Total↑Subject Consistency↑Background Consistency↑Temporal Flickering↑Motion Smoothness↑
HunyuanVideoTaylorSeer28.373.12×81.0295.8496.6099.2099.01
LiteCa-TS (rank 1)0.473.46×80.7795.7996.2599.2399.04
HiCache28.373.21×80.8095.7296.2299.1899.00
LiteCa-Hi (rank 1)0.473.45×80.5395.7196.2199.2599.10
Wan2.1TaylorSeer33.743.01×80.3295.4597.8899.0798.23
LiteCa-TS (rank 1)1.503.07×80.2995.4797.8299.0298.25
HiCache33.742.85×80.5595.6098.0298.9698.30
LiteCa-Hi (rank 1)1.503.07×80.3995.5098.0498.9398.31

Comments on the paper

2 comments
AuthorsOpenReview ↗

Official Comment by Authors

We thank all reviewers for their careful reading and constructive comments.

We are glad that the reviewers recognized the low-rank structure of the AdaLN gates as a novel insight, and its use to build compact cross-module caches for memory-efficient feature caching as a novel contribution (SrRd, koY2, VbDP). They also found the bridge between residual and module-level caching with a rank adjustable to the memory budget useful in practice (SrRd), the method plug-and-play with existing hosts such as TaylorSeer and HiCache without retraining (SrRd), the reduction in feature-cache memory substantial across image and video models (SrRd, koY2, VbDP), the peak-memory analysis relevant for practitioners (SrRd), and the paper clear and easy to follow (SrRd, VbDP).

Below we first address the concern raised by several reviewers, and then list the changes in the revised version. Each point is discussed in full in the individual responses.

Shared concern

Test-Time Correction under larger skipping intervals and across the skipped steps (SrRd, VbDP). Test-Time Correction (TTC) fits its coefficients once at each anchor step and reuses them for all following skipped steps, so its benefit in the closed loop shows whether the fitted coefficients carry over. Table G1 compares the same rank-1 cache with TTC on and off at NN = 3, 6 and 9 (App. D.3, Table 7). With TTC, LiteCa has a higher PSNR than both its module-level host and the same cache without TTC in all twelve settings, and the gain from TTC grows with NN for both models and both hosts (TaylorSeer: +0.30 to +1.47 dB on FLUX.1-dev and +0.92 to +5.02 dB on Qwen-Image). The accuracy of the readout linearization (SrRd) and the error reduction as a function of the distance from the anchor (VbDP) are analyzed in the individual responses.

Table G1. FLUX.1-dev and Qwen-Image, DrawBench-200, rank 1. Each cell is PSNR↑ / SSIM↑ / LPIPS↓ against full-step inference. "TTC off" is the same rank-1 cache with λ = 1. Bold marks the best value in each host block.

ModelNHostHost (module-level)LiteCa, TTC offLiteCa, TTC on
FLUX.1-dev3TaylorSeer19.77 / .784 / .21719.76 / .786 / .21720.06 / .789 / .213
HiCache20.00 / .789 / .21119.99 / .791 / .21220.17 / .793 / .211
6TaylorSeer16.03 / .650 / .39015.99 / .654 / .39116.92 / .680 / .362
HiCache16.85 / .677 / .34816.88 / .682 / .35117.25 / .692 / .354
9TaylorSeer14.45 / .582 / .50214.34 / .580 / .51315.81 / .633 / .441
HiCache15.47 / .638 / .42315.41 / .639 / .42916.15 / .650 / .436
Qwen-Image3TaylorSeer22.91 / .859 / .12822.93 / .861 / .12723.84 / .875 / .114
HiCache23.73 / .856 / .12723.85 / .865 / .12524.17 / .856 / .135
6TaylorSeer14.66 / .556 / .42414.75 / .558 / .43017.83 / .608 / .319
HiCache18.06 / .650 / .28418.43 / .681 / .29120.02 / .753 / .249
9TaylorSeer9.09 / .274 / .7199.37 / .282 / .71614.40 / .495 / .500
HiCache13.88 / .434 / .48614.26 / .477 / .50516.60 / .555 / .429

Main revisions in the uploaded PDF (new sections are marked in red)

  • Related-work clarifications, including SVD-Cache and LearniBridge (koY2). Sec. 2.

  • Improved method clarity (koY2). Figure 3 has been redesigned to illustrate how a single LiteCa representation spans residual and module-level caching.

  • Latency breakdown (koY2). Sec. 4.4 and Figure 7(a).

  • Modality-specific ranks and coefficients for MiniMax-H3 (SrRd). Sec. 4.4 and Figure 7(b); full results in App. C.3, Table 6.

  • Test-Time Correction across skipping intervals (SrRd, VbDP). App. D.3, Table 7 and Figure 13.

  • Readout linearization accuracy and cost (SrRd). App. D.4 and Figure 14.

  • Corrected rank labels in the native-setting peak-memory figure (SrRd). Figure 15 (Figure 12 in the submission).

  • Memory usage throughout the denoising loop (SrRd). App. E.2 and Figure 16.

  • Reference-free quality evaluation using ImageReward, CLIPScore, and VBench (SrRd). App. G and Tables 11–12.

AuthorsOpenReview ↗

Final Rebuttal Summary

Dear Area Chair,

Thank you for handling our submission. Below we summarize the post-rebuttal discussion.


1. Ratings and Emerging Consensus (4/3/2)

Current ratings (Rating / Confidence): VbDP (4/3), SrRd (3/4), koY2 (2/2)

  • VbDP (4): VbDP found their concerns "largely addressed" and raised the rating from 3 to 4 after we added an online rank selection that does not rely on the evaluation set (see 3(d)).
  • SrRd (3): SrRd stated that the revisions "fully address my concerns and significantly improve the quality of the paper."
  • koY2 (2): koY2 initially stated that a convincing response on novelty and the shared basis could raise the score (to 3/4). After the rebuttal, koY2 stated that "most of my concerns" had been addressed, with two points remaining: (1) no cache-compression comparison with LearniBridge, and (2) modest gains at N = 8, with LearniBridge better on ImageReward and CLIPScore in some settings. We responded to both with Table R2D (see 3(a)).

2. Strengths Recognized by the Reviewers

  • Novel insight: the low-rank structure of the AdaLN gates allows the per-module caches to be merged into K + 1 caches (SrRd, koY2, VbDP).
  • Substantial memory reduction: 90–99% less feature-cache memory than module-level caching across various image and video models (SrRd, koY2, VbDP).
  • Practicality: bridges residual and module-level caching, with K tunable to the memory budget; training-free and plug-and-play with TaylorSeer and HiCache (SrRd).

3. Resolution of the Main Technical Concerns

(a) Comparison with LearniBridge (koY2)

Including LearniBridge's per-step LoRA weights, LiteCa uses 52–62% less storage, and its storage does not grow with the skipping interval (Table R2D). LiteCa wins at least four of the five metrics at every interval (Table R2B), without LearniBridge's 3–5 GPU-hours of training per schedule.

(b) Source of novelty and computational cost (koY2; to 3/4)

Our core finding is that the AdaLN modulation parameters have a strong low-rank structure. SVD on the gates, not the features, yields time-invariant per-module weights and a few shared temporal coefficients, so the per-module caches can be combined into K + 1 caches without projecting or truncating the features. To our knowledge, LiteCa is the first method to exploit this structure for cache compression. The decomposition runs once before denoising (~8 ms), and LiteCa is faster than its host in all ten model–host pairs.

(c) Reliability of TTC (SrRd, VbDP)

With TTC, LiteCa has higher PSNR than the host in all 12 settings, and the gap gets larger as N grows (Table G1). TTC uses a single JVP to fit its coefficients. This keeps 86–99% of the error reduction of a full iterative fit (App. D.4).

(d) Rank selection (VbDP; rating raised to 4)

At rank 1, all ten model–host pairs use over 95% less cache and run faster (Table R3B). The online rank selection requires no evaluation set, outperforms fixed rank 3, and is comparable to the configurations tuned on the evaluation set (Table R3G).

(e) Applicability and quality evaluation (SrRd; fully addressed)

We clarified the applicability conditions and Limitations, and added ImageReward, CLIPScore, and VBench (App. G), per-step peak GPU memory (App. E.2), and multi-stream rank settings (App. C.3).


4. Conclusion

All three reviewers recognize LiteCa's core contribution: exploiting the low-rank structure of the AdaLN gates to reduce feature-cache memory by 90–99%, which our results show comes without sacrificing speed or quality. VbDP has raised the rating to 4, SrRd considers all concerns fully addressed, and we have responded to koY2's remaining concerns with Table R2D. All new content is in the revision (new sections marked in red).

We thank you and the reviewers for your time and valuable feedback.

Sincerely, The Authors of Submission 42206

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.