acceptodds
Withdrawn from ICLR 2027

DART: Diffusion Acceleration via Rectified Temporal Quantization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion models achieve high generative quality but incur substantial computational cost from iterative denoising. Feature caching and low-bit quantization each reduce this cost, and combining them promises multiplicative speedups. Yet we find that directly stacking the two strategies is unstable: cache reuse introduces a temporal state drift along the denoising trajectory, while quantization distorts the denoising operator, and the two errors interact in a way that local feature distance fails to predict. We propose DART, a training-free joint cache–quantization framework that addresses two questions explicitly—when to recompute and how to use cached quantized states—under a unified trajectory-sensitivity view. DART first measures a relative update sensitivity proxy along the sampling trajectory and partitions the timesteps into segments whose sensitivity fluctuation is bounded; the partitioning, formulated as a constrained dynamic program, decides where caching is safe. Within each segment, DART rectifies the cached quantized states through dual-domain affine correction, which decouples cache-induced input shift from quantization-induced operator distortion, and shares a segment-wise quantizer that adapts to temporal heterogeneity. The rectification parameters are absorbed into adjacent linear and dequantization operators whenever the required affine conditions hold, yielding negligible deployment overhead. Across class-conditional generation on ImageNet-256 with LDM and DiT-XL/2 and text-to-video generation on HunyuanVideo, DART consistently improves the quality–efficiency frontier over caching-only, quantization-only, and joint cache–quantization baselines under matched-speed and matched bit-width settings. For instance, on LDM-4 with W8A8 quantization DART improves FID from 4.03 to 3.83 at 8.12x versus CacheQuant's 7.87x, and improves FID from 7.21 to 6.53 at the most aggressive 18.93x setting where directly stacking DeepCache and W8A8 collapses to FID 15.36, an 8.83 FID gap at the same speed. On DiT-XL/2 at 50 sampling steps DART further improves FID from 5.43 to 5.08 at 13.2x versus the 12.7x of the recent joint baseline Q&C, and on HunyuanVideo it lowers LPIPS by 23 from 0.189 to 0.145 and lifts VBench from 80.35 to 80.42 at 4.12x versus DPCache's 4.05x.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Computer vision

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_hpHoOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes DART, a framework that jointly applies feature caching and quantization to accelerate diffusion models. The authors define a sensitivity proxy ρ^t\hat{\rho}_t as the relative change of the state within one step, and use it throughout all components. BPP partitions the trajectory into segments within which the fluctuation of ρ^t\hat{\rho}_t is bounded, and recomputes the network only at the start of each segment. DAR corrects the input shift caused by caching and the output distortion caused by quantization, each with a channel-wise affine map. TSQ shares quantization parameters within each segment, and NPF folds the correction parameters into adjacent operators. On LDM-4 and DiT-XL/2 (ImageNet-256) and HunyuanVideo, the authors report better speed and quality than existing caching, quantization, and joint methods.

Critical strengths
  1. The motivation is clear. The authors show that naively combining caching and quantization amplifies the error along the trajectory, and shows that ρ^t\hat{\rho}_t correlates with the joint error more strongly than the feature distance. This provides a useful starting point for joint acceleration.

  2. The method is evaluated on diverse backbones. DART is applied to a U-Net, a DiT, and a video model, using only a small calibration set.

Critical weaknesses
  1. The contribution over existing methods seems limited. Most components of DART appear to closely follow prior methods.

    • BPP. Eq. (2) uses the same objective, the cumulative L1 drift, and the same length constraint N/2≤sk+1−sk≤2NN/2 \le s_{k+1} - s_k \le 2N as the DPS of CacheQuant [A], with one additional constraint on the fluctuation of ρ^t\hat{\rho}_t. Moreover, the proxy ρ^t\hat{\rho}_t itself is taken from SenCache [B], as described by the authors (Ln206).
    • DAR. Eqs. (3) and (5) seem to match the DEC of CacheQuant, which also corrects the error with channel-wise affine maps fitted in closed form.
    • TSQ and NPF. Grouping timesteps and assigning separate quantization parameters to each group is a common practice in diffusion quantization, as in APQ-DM [C] and AccuQuant [D]. For NPF, CacheQuant also seems to fold its correction parameters into weight quantization (Sec. 4.3 of [A]), and merging batch normalization into dequantization parameters is a common technique in quantization. NPF thus does not seem to be new.
  2. Key settings are missing, and the description of retraining is inconsistent.

    • Missing settings. The paper does not describe which blocks or features are cached for LDM, DiT, and HunyuanVideo. Ln323 states that the DAR statistics are collected after weight reconstruction, but the reconstruction method and the settings in which it is used are not specified. Even if the code is provided, these settings should be stated in the paper, since they are needed to interpret the results and to see where the difference from the baselines comes from.
    • Inconsistent description of retraining. The paper claims that DART is training-free and requires no retraining (Ln128, Ln360, and Ln474). Table 1, however, marks the W4A8 results of DART as requiring retraining. It would be helpful to clarify this inconsistency.

References

[A] CacheQuant: Comprehensively Accelerated Diffusion Models, CVPR 2025

[B] SenCache: Accelerating Diffusion Model Inference via Sensitivity-Aware Caching, CVPR 2026

[C] Towards Accurate Post-Training Quantization for Diffusion Models, CVPR 2024

[D] AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Models, NeurIPS 2025

Questions for authors

The caching in DART reuses cached features, while recent methods such as TaylorSeer [E] extrapolate them. Do BPP and DAR remain effective when combined with such methods, given that the input-side correction of DAR may overlap with extrapolation?

[E] From Reusing to Forecasting: Accelerating Diffusion Models with TaylorSeers, ICCV 2025

Reviewer_7YkQOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper compose many methods for accelrating caching and low-bit diffusion, like timestep segmentation, correction of channels, quantizer shared by chunks

Critical strengths

This paper may have reported some incremental progress over CacheQuant [1] and I cannot find other strength

[1] Xuewen Liu, Zhikai Li, Qingyi Gu. CacheQuant: Comprehensively Accelerated Diffusion Models. CVPR 2025. arXiv:2503.01323.

Critical weaknesses
  1. First of all, In Fig. 5, the baseline columns are taken from CacheQuant's [1] paper without citation, this might be an integrity problem, since this paper reports different experimental settings to CacheQuant. And, this paper does not report that those images are from CacheQuant
  2. Everything reads very incremental to CacheQuant and I do not think this paper proposes any new methods.
  3. This paper is till use LDM which is really old and neeeds to introduce newer baselines
  4. Please cite: APQ-DM [2], TQ-DiT [3], HTG [4], MoDiff [5], ERTACache [6].

[1] Xuewen Liu, Zhikai Li, Qingyi Gu. CacheQuant: Comprehensively Accelerated Diffusion Models. CVPR 2025. arXiv:2503.01323.

[2] Changyuan Wang, Ziwei Wang, Xiuwei Xu, Yansong Tang, Jie Zhou, Jiwen Lu. Towards Accurate Post-training Quantization for Diffusion Models. arXiv:2305.18723, 2023.

[3] Younghye Hwang, Hyojin Lee, Joonhyuk Kang. TQ-DiT: Efficient Time-Aware Quantization for Diffusion Transformers. arXiv:2502.04056, 2025.

[4] Ning Ding, Jing Han, Yuchuan Tian, Chao Xu, Kai Han, Yehui Tang. Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Grouping. arXiv:2503.06930, 2025.

[5] Weizhi Gao, Zhichao Hou, Junqi Yin, Feiyi Wang, Linyu Peng, Xiaorui Liu. Modulated Diffusion: Accelerating Generative Modeling with Modulated Quantization. ICML 2025. arXiv:2506.22463.

[6] Xurui Peng, Chenqian Yan, Hong Liu, Rui Ma, Fangmin Chen, Xing Wang, Zhihua Wu, Songwei Liu, Mingbao Lin. ERTACache: Error Rectification and Timesteps Adjustment for Efficient Diffusion. arXiv:2508.21091, 2025.

Questions for authors

Why not report variance? FID improvement from 4.03 to 3.83 needs an error bar I think?

Reviewer_PYncOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies error accumulation when feature caching and post-training quantization are combined to accelerate diffusion models. It proposes cache scheduling, dual-domain affine rectification, and segment-shared quantization calibration based on a common temporal partition. BPP constrains the within-segment fluctuation of a relative-update sensitivity signal; Dual Affine Rectification (DAR) corrects input-domain shift and operator-domain distortion; the Temporally Stamped Quantizer (TSQ) shares quantization parameters within segments; and Normalization-Preserving Fusion (NPF) folds the rectification into inference operators. The authors report FID 3.83 at 8.12× speedup on LDM-4 with 250-step DDIM, together with gains on DiT and HunyuanVideo. However, the manuscript contains important inconsistencies in its theoretical claims, speedup comparisons, and description of the fusion implementation.

Critical strengths
  1. The joint optimization problem is meaningful. Caching and quantization reduce different sources of inference cost, but their errors interact. Coordinating cache scheduling, rectification, and quantization through a shared temporal structure is a worthwhile direction beyond independently combining the two techniques.

  2. The reported evaluation covers multiple backbones and includes direct comparisons with joint methods and component ablations. Tables 1–4 compare caching-only methods, quantization-only methods, CacheQuant, and Q&C, with quality improvements at some approximately matched-speed operating points. If the results and deployment implementation are confirmed, this would constitute a useful empirical contribution.

Critical weaknesses
  1. The stability claims are inconsistent with the mathematical meaning of the proxy. The abstract claims provably stable temporal segments, whereas Section 3.2 explicitly states that the proxy is not the Jacobian's spectral radius and does not provide a formal stability guarantee. Small or smoothly varying relative state updates do not, in general, bound perturbation amplification. The first-order expression used in the paper also requires additional assumptions and cannot be applied directly to an arbitrary nonlinear mapping with a nonzero offset. The correlations in Appendix A.8 provide useful empirical evidence, but do not establish the claim in Appendix A.9 that proxy fluctuations control the Jacobian norm. This concerns the foundation of the method, rather than wording alone.

  2. Key deployment and evaluation details are not sufficiently self-consistent to support reproducibility. DAR parameters vary with the timestep, and Equation (4) produces timestep-dependent effective weights, W′_t = W diag(a_1,t). The manuscript does not adequately explain the repacking, rescaling, or additional weight storage required beyond the O(TC) rectification table. Affine transformations also cannot generally be moved across normalization layers with input-dependent statistics. Appendix A.7 acknowledges some limitations, but the alternative of folding the rectification into the next linear layer still requires a concrete equivalent execution graph. In addition, the 50-step DiT column uses the 250-step baseline for speedup; Figure 5 shows text-conditioned Stable Diffusion outputs but is labeled as ImageNet-256; and 332.92/82.09 ≈ 4.06 in Table 3 does not match the reported 4.12× speedup. Collectively, these issues make the quality–speed conclusions difficult to independently verify.

Additional related-work comment: The manuscript discusses CacheQuant and Q&C but omits the directly relevant work QuantCache: Adaptive Importance-Guided Quantization with Hierarchical Latent and Layer Caching for Video Generation (Wu et al., 2025). QuantCache already combines hierarchical caching, adaptive importance-guided quantization, and structural redundancy-aware pruning to accelerate video generation. The authors should cite this work and explain DART's specific differences in cache scheduling, quantization granularity, and joint error handling. A quality–latency comparison on a comparable backbone and configuration would help establish DART's incremental contribution. This comment concerns technical relevance and contribution positioning; the missing citation alone is not a basis for rejection.

Questions for authors
  1. Theoretical clarification: Is BPP an empirical heuristic, or does it offer a formal stability guarantee? If the latter, could you provide the specific assumptions and proof connecting within-segment proxy fluctuations to a Jacobian-norm bound or a perturbation-growth bound?

  2. Implementation verification: How does NPF implement timestep-dependent input-channel scaling? Please provide the actual execution order of normalization, quantization, and linear layers, together with the costs of timestep-dependent weights, rescaling, kernels, and storage. Please also clarify the timing unit in Table 13 and its relationship to the main-table measurements.

  3. Algorithm and result verification: Equation (2) and Algorithm 1 select K without an explicit recomputation cost or a fixed segment-count budget. How is the target speedup controlled? Could you provide the actual recomputation timesteps, absolute latencies using baselines with the same number of sampling steps, and explanations for the figure and table inconsistencies noted above?

  4. Related work and contribution positioning: Please cite and discuss QuantCache (Wu et al., 2025, arXiv:2503.06545), which also combines quantization and caching for video generation. What are DART's central technical differences from this work? Could you provide a quality–latency comparison under comparable settings, or explain the specific obstacles to a direct comparison?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.