acceptodds
Under review as a conference paper at ICLR 2027

TeleQuant: Temporal Noise Shaping for Post-Training Quantized Diffusion Transformers

Abstract

Post-training quantization lowers the cost of the repeated denoiser evaluations that diffusion Transformers use for image and video generation. Because those evaluations are chained, the rounding errors they leave behind interact across steps. Correlated rounding errors can reinforce, accumulate into structured trajectory drift, and persist in the returned sample. This drift can degrade image and video fidelity. We propose TeleQuant, a training-free temporal noise-shaping framework that feeds a discarded activation-rounding residual into the next evaluation, with a gain matched to adjacent solver responses and activation scales. We derive a first-order telescoping identity: under ideal state storage, accumulated error separates into a terminal residual and a sensitivity-variation term, motivating selective layer placement and damped feedback. INT8 residual state and a fused activation operator add the compensation on the host's path. The feedback lowers accumulated latent drift and improves paired fidelity to matched FP16 outputs. Evaluations on DiT-XL/2, PixArt-, SDXL, and Open-Sora 1.2 show improved image fidelity and video consistency over frozen ViDiT-Q and SVDQuant hosts, with matched rounding ablations supporting the complete gain. The profiled image pipelines retain most of the host speedup, at – latency. Code is available at https://anonymous.4open.science/r/TeleQuant-8217/.

Primary area:
generative models
Keywords:
diffusion transformer, post-training quantization, temporal noise shaping, trajectory drift, efficient generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_KXWFOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TeleQuant reuses rounding residuals across denoising evaluations, with a feedback gain determined by solver coefficients, activation scales, and damping. It combines an error-cancellation analysis with calibrated site selection and INT8 residual storage. Experiments add the mechanism to frozen quantization hosts for image and video generation, reporting improved quality with modest overhead on the profiled image pipelines.

Critical strengths
  1. Coordinating rounding errors across denoising evaluations is an interesting way to improve quantized diffusion. The telescoping identity makes the idea easy to understand and explains why reducing the accumulated error can differ from improving each evaluation in isolation.
  2. Keeping each quantization host fixed makes the main comparisons informative. The PixArt and SDXL results improve both distributional quality and fidelity to matched full-precision outputs, providing stronger evidence than an isolated gain in one reconstruction metric.
  3. The implementation results let residual feedback look practical. INT8 state nearly matches the reported FP16-state quality, and the complete PixArt and SDXL pipelines incur roughly three to five percent additional latency over their quantized hosts.
  4. The analysis of placement is useful. Feedback helps some projection sites and hurts others, including the QKV sites in the displayed DiT study. This is an informative finding for applying temporal error feedback inside an existing denoiser.
Critical weaknesses
  1. The TCEC description appears incomplete. It rescales the current prediction, whereas the original method also uses the preceding prediction in its cumulative correction. If that term was omitted in implementation, Table 3 would compare against only part of TCEC. The actual update needs clarification before this result can support the claimed advantage.
  2. The unit-feedback ablation changes damping, solver matching, and scale matching together. A constant-damped feedback control is missing. Without it, I cannot tell whether the proposed matching rule improves generation beyond a simpler residual-feedback mechanism using the same quantization host and placement.
  3. The comparison set omits MoDiff, a published method for temporal activation quantization with public code. Its delta-based mechanism differs from TeleQuant, but a comparison on a common diffusion host is needed to establish what practical advantage this alternative offers over the existing approach.
  4. The smaller empirical gains are difficult to assess. The video fidelity improvements are tiny and lack paired uncertainty. Separately, the DiT full-precision reference uses a different sampler, so the reported quantization-gap recovery mixes quantization and sampling effects, despite the valid improvement over the fixed host.
Questions for authors
  1. Can you compute distributional scores for the matched DiT full-precision samples already used for fidelity evaluation? This would give a cleaner reference for the quantization-gap recovery without requiring another set of generations.
  2. Which model is differentiated for the site-selection JVPs, and how long does calibration take in practice? The separate shaped rollouts for candidate sites seem important to the offline cost.
  3. The damping ratio in Appendix E.2 minimizes a least-squares objective, rather than the induced norm in Equation 5. Could this be presented as a surrogate criterion? The central telescoping identity itself appears correct.
  4. Have you checked whether the image improvements also preserve prompt alignment? A comparison using an alignment or preference metric on the existing outputs would complement the distributional and reference-fidelity results.
4 comments
AuthorsOpenReview ↗

Response to Reviewer KXWF Part 1 of 2

Thank you for the review. We appreciate your constructive questions and suggestions.

W1 Correcting the cumulative TCEC baseline

The original draft used TCEC's [1] closed-form output rescaling and omitted preceding-error compensation, reporting 19.74 dB on DiT-XL/2 [2] and 15.68 dB on PixArt-Σ [3]. We implemented the cumulative correction from TCEC Eq. 13 on the same frozen ViDiT-Q [4] W4A8 hosts. In DDIM execution order, with preceding-error weight h=1h=1:

xi+1=x^i+1−Biei−haiai+1Bi−1ei−1x_{i+1}=\hat{x}_{i+1}-B_i e_i-h\sqrt{\frac{a_i}{a_{i+1}}}B_{i-1}e_{i-1}

Here aia_i is the cumulative noise schedule, x^i+1\hat{x}_{i+1} is the DDIM output computed from xix_i and ϵ^i\hat{\epsilon}_i, and ei=Ki⊙ϵ^ie_i=K_i\odot\hat{\epsilon}_i. The channel-wise KiK_i is fitted by the reconstruction objective in Appendix D.2, and Bi=1−ai+1−ai+1/ai1−aiB_i=\sqrt{1-a_{i+1}}-\sqrt{a_{i+1}/a_i}\sqrt{1-a_i}. The first evaluation has no history term. On PixArt-Σ, we adapted the correction to DPM-Solver++'s multistep formulation while caching preceding errors.

The corrected baseline improves to 19.77/15.74 dB on DiT/PixArt and improves all three metrics over the original version. Table 3 and Appendix D.2 are updated. Section 5.2 reports:

MethodDiT PSNR, dBDiT SSIMDiT LPIPSPixArt PSNR, dBPixArt SSIMPixArt LPIPS
Frozen ViDiT-Q Host [4]19.590.6800.28615.250.5900.448
TCEC [1]19.770.6950.27515.740.6030.428
MoDiff Adaptation [5]19.650.6850.28115.820.6050.425
TeleQuant, our method19.910.7080.26216.880.6110.400

TeleQuant improves all three metrics over TCEC on both backbones. The DiT margin is small, while PixArt gains 1.14 dB. TCEC corrects an error after propagation to the solver output. TeleQuant shapes activation-rounding residuals inside the denoiser before integer GEMM, changing their downstream nonlinear propagation.

W2 Separating damping from solver and scale matching

We use the same DiT-XL/2 ViDiT-Q W4A8 host, 38 selected sites, input latents, and matched FP16 references for all six configurations:

ConfigurationFeedback gainPSNR, dBSSIMLPIPS
1. Frozen hostNo feedback19.590.6800.286
2. Unit-delay feedback, α=1\alpha=1Unweighted residual reuse19.690.6960.282
3. Constant damping, λ=0.75\lambda=0.75Damped without matching19.710.6910.280
4. Solver matchingλct−1ct\lambda\frac{c_{t-1}}{c_t}19.810.6990.274
5. Scale matchingλst−1,lst,l\lambda\frac{s_{t-1,l}}{s_{t,l}}19.780.6970.277
6. Full TeleQuantλct−1ctst−1,lst,l\lambda\frac{c_{t-1}}{c_t}\frac{s_{t-1,l}}{s_{t,l}}19.910.7080.262

Each matching component improves all three metrics over constant damping, and their combination improves further. This supports the matching rule beyond residual reuse under the evaluated mask. Section 5.2 contains the table. Section 4.1 and Appendix E.2 clarify damping and matching.

References

[1] Liu, S., et al. Error propagation mechanisms and compensation strategies for quantized diffusion models. ICML, 2026, pp. 79701-79718.

[2] Peebles, W., and Xie, S. Scalable diffusion models with transformers. ICCV, 2023, pp. 4195-4205.

[3] Chen, J., et al. PixArt-Σ: Weak-to-strong training of diffusion transformer for 4K text-to-image generation. ECCV, 2024.

[4] Zhao, T., et al. ViDiT-Q: Efficient and accurate quantization of diffusion transformers for image and video generation. ICLR, 2025.

[5] Gao, W., et al. Modulated diffusion: Accelerating generative modeling with modulated quantization. ICML, 2025, pp. 18337-18362.

AuthorsOpenReview ↗

Response to Reviewer KXWF Part 2 of 2

W3 MoDiff state and latency

Our controlled MoDiff [5] adaptation caches compensated activation differences in every transformer block on the frozen PixArt-Σ ViDiT-Q W4A8 host:

MethodIntervention SitesPixArt PSNR, dBPersistent MemoryLatency Overhead
Frozen host0 sites15.250 MB1.00x
MoDiff Adaptation280 sites (All)15.82378 MB1.12x
TeleQuant (Eight-site)8 sites16.2432 MB1.05x
TeleQuant (Quality)42 sites16.88168 MB1.16x

The eight-site profile improves fidelity with less state and lower latency than the evaluated 280-site MoDiff recurrence. The 42-site profile trades higher latency for higher fidelity. Appendix F.5 reports these distinct operating points, with timing in Appendices E.6 and D.6.

W4 and Q1 Matched reference and video uncertainty

FP16, host, and TeleQuant use the same DDIM-50 schedule on DiT-XL/2 (Section 5.1, Appendix F.3):

Setting on DiT-XL/2 DDIM-50FIDsFIDInception ScorePSNR, dB
Matched FP16 Reference3.185.12268.4Reference
Frozen ViDiT-Q Host5.326.48248.519.59
TeleQuant, our method4.655.82254.119.91

TeleQuant recovers 31.3%, 48.5%, and 28.1% of the remaining FID, sFID, and Inception Score gaps. These replace the original Table 14 fractions of 21.97%, 35.11%, and 18.83%. Updated Table 20 uses the matched reference, removing the schedule discrepancy.

For Open-Sora 1.2, paired clip-level bootstrap with 10,000 resamples over 251 VBench clips gives a frame PSNR gain of +0.037 dB, 95% interval [0.012, 0.063], and temporal-difference PSNR gain of +0.058 dB, interval [0.021, 0.096]. The frame win rate is 70.0% (2,811/4,016 frames). Background consistency, subject consistency, and motion smoothness recover most of the remaining FP16 gap, while imaging quality and human action recover less. Frame and inter-frame fidelity improve (Appendices E.1 and E.4).

Q2 Offline calibration and JVP target

Screening and paired rollouts take 28.3 minutes for DiT and 40.9 minutes for PixArt on one NVIDIA A100 with 32 calibration instances and batch size 4. Forward automatic differentiation evaluates Jt,l(st,l⊙vl)J_{t,l}(s_{t,l}\odot v_l) using a fixed Gaussian probe vlv_l on a smooth host surrogate. Quantized weights, channel balancing, and outlier transforms are retained, while activation rounding is replaced with the identity. This measures first-order sensitivity from insertion sites to outputs without differentiating through rounding. The accepted mask stays fixed during inference, with no online JVP or site selection (Section 4.3, Appendix D.4).

Q3 Damping objective

The Frobenius-norm objective in Appendix E.2 is a computable least-squares surrogate, not a minimization of the induced ℓ∞→ℓ2\ell_\infty\to\ell_2 norm in Eq. 5. Appendix F.2 gives ∣M∣∞→2≤d,∣M∣F\\|M\\|_{\infty\to 2}\leq\sqrt{d}\\,\\|M\\|_F. Section 4.1 and Appendix E.2 now make this distinction explicit.

Q4 Text-prompt alignment

On PixArt-Σ coco_1024 outputs, CLIP ViT-B/32 [6] cosine similarity is:

ConfigurationCLIP cosine similarity ViT-B/32PSNR, dBFIDPrompt Alignment Status
PixArt-Σ Host0.313615.258.42Baseline host
TeleQuant, our method0.315416.887.55Preserved, mean change +0.0018

The mean change is +0.0018 with 95% paired bootstrap interval [0.00003, 0.00388]. Although the interval excludes zero, the mean change is small. We interpret this as preserved alignment, complementing reconstruction fidelity, rather than a substantive alignment gain. Section 5.2 summarizes the result, Appendix E.4 gives the table and interval, and Appendix D.5 defines the metric.

References

[5] Gao, W., et al. Modulated diffusion: Accelerating generative modeling with modulated quantization. ICML, 2025, pp. 18337-18362.

[6] Radford, A., et al. Learning transferable visual models from natural language supervision. ICML, 2021, pp. 8748-8763.

Thank you for the response and the new experiments. TeleQuant still outperforms the corrected TCEC baseline on both backbones, with a larger margin on PixArt. Solver matching and scale matching each improve on constant damping. The video gains remain small, but the image results now support the main contribution. I will raise my score from 2 to 3.

AuthorsOpenReview ↗

Thank You for Your Follow-up

Dear Reviewer KXWF,

Thank you for reviewing our responses and additional experiments, and for raising your score. Your constructive feedback helped us strengthen the experimental evidence and improve the clarity of our paper. We sincerely appreciate your time and thoughtful evaluation.

Reviewer_K89GOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a method to restore generation quality by modifying the rounding scheme in the inference process of the Diffusion Transformer with PTQ quantization. TeleQuant reduces error accumulation during continuous diffusion by feeding back the rounding quantization residual from the previous step while retaining the residual from the current step. Additionally, it determines the feedback gain of the previous step’s rounding quantization residual based on the solver response and the activation quantization scale. By analyzing the trajectory drift under selective feedback, TeleQuant attempts to express the quantization rounding residuals of consecutive diffusion steps as a telescoping sum, allowing the residuals to cancel each other out. By estimating sensitivity changes in adjacent denoising steps, the method identifies locations with gradual changes during the calibration phase, measures the improvement achieved through the feedback mechanism, and ultimately fixes the selected locations for inference. Experimental results show that, on the DiT-XL/2, PixArt-Σ\Sigma, SDXL, and Open-Sora models, TeleQuant, when combined with different PTQ methods, significantly improves generation quality. At the same time, the use of fusion operators provides a practically deployable solution with minimal hardware overhead.

Critical strengths
  1. The proposed residual feedback mechanism effectively improves the generation quality of Diffusion Transformer PTQ hosts with minimal hardware overhead.

  2. By analyzing the trajectory drift under selective feedback, clearly demonstrate when quantization rounding residuals across time steps cancel each other out, which guides the introduction of the damping coefficient λ\lambda and the selection of feedback positions.

Critical weaknesses
  1. The paper compares its method with TCEC on the same frozen host, but does not conduct comparative experiments with other methods, such as MoDiff, mentioned in related work.

  2. Appendix A.1 derives an upper bound on the error when using ctIc_t I to approximate HtH_t, but does not quantitatively measure the impact of this approximation in real networks.

Questions for authors
  1. Could the authors provide additional results comparing the performance of at least one other method on the same frozen host?

  2. Has TeleQuant been evaluated using more aggressive quantization setting (e.g., W2A4)? Does TeleQuant maintain its performance gains even at extremely low bit widths?

  3. Would Jacobian-vector product estimation and checking actual local errors on paired frozen and shaped rollouts introduce significant calibration overhead in the calibration of large models?

1 comment
AuthorsOpenReview ↗

Response to Reviewer K89G

Thank you for the review. We appreciate your constructive questions and suggestions.

W1 and Q1 Temporal baselines on common frozen hosts

We compared MoDiff [1] adaptations on PixArt-Σ [2] and DiT-XL/2 [3] using the same frozen ViDiT-Q [4] W4A8 hosts and schedules. TCEC [6] includes corrected preceding-error compensation. Section 5.2 and Table 3 report:

MethodDiT PSNR, dBDiT SSIMDiT LPIPSPixArt PSNR, dBPixArt SSIMPixArt LPIPS
Frozen ViDiT-Q Host19.590.6800.28615.250.5900.448
TCEC [6]19.770.6950.27515.740.6030.428
MoDiff Adaptation [1]19.650.6850.28115.820.6050.425
TeleQuant, our method19.910.7080.26216.880.6110.400

TeleQuant improves all metrics on both models, exceeding MoDiff by over 1 dB on PixArt-Σ. MoDiff reconstructs activations from quantized differences, caching compensated states in every block. On PixArt-Σ it uses 378 MB and 1.12x host latency. TeleQuant stores INT8 residuals selectively. Its eight-site profile gives 16.24 dB, 32 MB and 1.05x latency, reducing state by 91.5%. The 42-site quality profile gives 16.88 dB, 168 MB and 1.16x latency. DiT state is 28 MB. Appendix F.5 covers adaptation and costs, Appendix E.6 timing.

W2 Scalar solver-response approximation

The surrogate ctIc_t I separates the solver schedule from local denoiser sensitivity. Along real trajectories, we measured ∣Htv−ctv∣2/∣Htv∣2\\|H_t v-c_t v\\|_2/\\|H_t v\\|_2 using Gaussian probes vv. Execution index tt runs from initial noise (t=0t=0) to the final step (t=Tt=T):

Execution Sampling PhaseDiT-XL/2 DDIM-50PixArt-Σ DPM-Solver++-20
Coarse phase, steps t/T<0.3t/T < 0.33.12%2.85%
Structure phase, 0.3≤t/T≤0.70.3 \leq t/T \leq 0.74.85%4.42%
Refinement phase, steps t/T>0.7t/T > 0.76.24%5.91%

These sampled-direction errors support the surrogate, without a uniform operator-norm guarantee. Section 3 and Appendix A.1 clarify the scalar approximation and retain the Eq. 9 bound. Damping and site screening address sensitivity variation and realized feedback (Section 4.1, Appendix D.4).

Q2 Evaluated precisions and host inheritance

TeleQuant is a plug-and-play temporal wrapper orthogonal to host quantization. It inherits weight formats, codebooks, channel transforms and integer GEMM kernels [4], [5], feeding back activation residuals with weights frozen. SDXL uses INT4 weights with rank-32 FP16 branches [5], and DiT W4A4 follows this recipe:

Precision and Host SettingEvaluation ModelHost Baseline PSNRWith our TeleQuantFidelity Gain, Delta PSNR
W8A8, Uniform PTQDiT-XL/229.84 dB30.00 dB+0.16 dB
W4A8, ViDiT-Q HostPixArt-Σ15.25 dB16.88 dB+1.63 dB
W4A4, SVDQuant HostDiT-XL/222.48 dB22.89 dB+0.41 dB
W4A4, SVDQuant HostSDXL21.02 dB22.46 dB+1.44 dB

W4A4, our most aggressive evaluated setting, delivers steady gains of +1.44 dB on SDXL and +0.41 dB on DiT (Tables 1 and 6). SDXL FID drops from 14.86 to 13.95 across 5,000 matched prompts, confirming that temporal shaping remains effective as spatial noise intensifies. Mechanistically, TeleQuant coordinates cross-step activation residuals and cannot reconstruct missing weight capacity. In training-free PTQ, symmetric 2-bit weights collapse into a coarse ternary grid of -1, 0, and +1, causing degenerate outputs before temporal propagation begins. We ground our evaluation within validated hosts down to the available W4A4 limit (Section 5.1, Appendix F.4).

Q3 Offline calibration and JVP target

JVP screening and paired rollouts take 28.3 minutes for DiT and 40.9 minutes for PixArt on one NVIDIA A100, using 32 calibration instances and batch size 4. Forward automatic differentiation uses a smooth host surrogate that retains quantized weights, channel balancing, and low-rank branches but replaces activation rounding with the identity. It measures directional sensitivity from insertion sites to outputs without differentiating through rounding. The accepted mask stays fixed during generation, with no online JVP or site selection.

References

[1] Gao, W., et al. Modulated diffusion: Accelerating generative modeling with modulated quantization. ICML, 2025, pp. 18337-18362.

[2] Chen, J., et al. PixArt-Σ: Weak-to-strong training of diffusion transformer for 4K text-to-image generation. ECCV, 2024.

[3] Peebles, W., and Xie, S. Scalable diffusion models with transformers. ICCV, 2023, pp. 4195-4205.

[4] Zhao, T., et al. ViDiT-Q: Efficient and accurate quantization of diffusion transformers for image and video generation. ICLR, 2025.

[5] Li, M., et al. SVDQuant: Absorbing outliers by low-rank component for 4-bit diffusion models. ICLR, 2025.

[6] Liu, S., et al. Error propagation mechanisms and compensation strategies for quantized diffusion models. ICML, 2026, pp. 79701-79718.

Reviewer_LW7ZOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

TeleQuant coordinates activation-rounding errors across denoising evaluations in post-training quantized diffusion models. It carries bounded residuals between evaluations. Adjacent solver responses and activation scales determine feedback strength. A first-order telescoping identity motivates selective placement and damping. INT8 residual storage and a fused activation operator preserve the frozen quantization hosts. Experiments cover three image backbones and Open-Sora video generation. Image quality, fidelity to matched FP16 outputs, and video consistency improve. The profiled image pipelines show 3–5% inference overhead relative to their quantized hosts.

Critical strengths
  1. The method addresses error accumulation across the sampling trajectory while preserving frozen quantization hosts. Its explicit feedback rule connects temporal error coordination with a practical activation-quantization implementation.

  2. Gains cover two quantization hosts and four backbones. Image distributional metrics and matched-reference fidelity improve together. Open-Sora results provide complementary evidence on video consistency.

  3. The paper distinguishes ideal cancellation from deployment effects. Studies of INT8 state, damping, unit-delay feedback, and end-to-end timing provide useful evidence about accuracy and inference cost.

Critical weaknesses
  1. The main gain comparison changes damping, solver weighting, and scale matching together. The undamped setting provides partial support for matching. However, individual contributions remain unclear. This limits attribution of the additional fidelity gain to the proposed analytical rule.

  2. Deployment results emphasize low inference overhead, while site calibration requires separate rollouts for surviving candidates. Actual calibration time and candidate counts are unreported. These costs matter when adapting the method to another backbone or sampler (Section 4.3).

Questions for authors
  1. Figure 7 compares unit feedback with the complete gain, while Table 8 varies damping within TeleQuant. Were these runs performed with the same selected sites? Please explain which existing comparisons isolate response and scale matching from damping. This would clarify the source of the additional fidelity gain.

  2. Section 4.3 calibrates placement on 32 instances, and Algorithm 2 repeats shaped rollouts for surviving sites. How many candidates and sites were retained for each backbone? Please report the recorded calibration time, including Jacobian probes, if available. This would clarify the one-time cost behind the low inference overhead.

  3. TeleQuant’s gain depends on the sampler, while site selection uses calibration trajectories. If the step count or scheduler changes, which settings must be recomputed? Please distinguish analytical gain updates from site recalibration. This would clarify when the same quantized host and selected sites can be reused.

1 comment
AuthorsOpenReview ↗

Response to Reviewer LW7Z

Thank you for the review. We appreciate your constructive questions and suggestions.

W1 and Q1 Separating solver matching scale matching and damping

Original Figure 7 tested unit residual reuse (α=1\alpha=1), while original Table 8 varied damping within the matched gain. We now isolate the factors on DiT-XL/2 [2] with frozen ViDiT-Q [1] W4A8, DDIM-50, the same 38 sites, input noise latents, and matched FP16 references:

ConfigurationFeedback gainPSNR, dBSSIMLPIPS
1. Frozen hostNo feedback19.590.6800.286
2. Unit-delay feedback, α=1\alpha=1Unweighted residual reuse19.690.6960.282
3. Constant damping, λ=0.75\lambda=0.75Damped without matching19.710.6910.280
4. Solver matchingλct−1ct\lambda\frac{c_{t-1}}{c_t}19.810.6990.274
5. Scale matchingλst−1,lst,l\lambda\frac{s_{t-1,l}}{s_{t,l}}19.780.6970.277
6. Full TeleQuantλct−1ctst−1,lst,l\lambda\frac{c_{t-1}}{c_t}\frac{s_{t-1,l}}{s_{t,l}}19.910.7080.262

Rows 3-6 use λ=0.75\lambda=0.75. Damping improves PSNR and LPIPS over unit feedback while SSIM decreases. Either matching term improves all three metrics over damping alone, and combining them gives the best evaluated result. Section 5.2 contains the table. Section 4.1 and Appendix E.2 explain the roles of damping and matching.

W2 and Q2 Site screening and calibration

Algorithm 2 uses 32 calibration instances, class labels for DiT and prompts for PixArt-Σ [3]. Stage 1 measures sensitivity variation Vl/PlV_l/P_l with scale-aware Gaussian probes and threshold τtv=0.85\tau_{\mathrm{tv}}=0.85 (Eq. 6). DiT has 112 linear projections in 28 blocks. Ratios peak near 0.8 and remain below 0.85, so Stage 1 filters none. Stage 2 tests realized local error reduction ΔEl>0\Delta E_l>0 in paired rollouts (Eq. 7), retaining 38 sites concentrated in Attn-Out and FFN-FC2 (Figure 16). PixArt has 280 candidates. Excluding zero-denominator sites and applying the threshold retains 182 in Stage 1. Stage 2 accepts 42 sites for the quality profile, with an eight-site subset for the smaller profile.

Forward automatic differentiation computes Jt,l(st,l⊙vl)J_{t,l}(s_{t,l}\odot v_l) on a smooth host surrogate that retains quantized weights, channel balancing, and low-rank branches while replacing activation rounding with the identity. This measures first-order directional sensitivity from insertion sites to outputs without differentiating through rounding.

Sensitivity evaluation and paired rollouts take 28.3 minutes for DiT and 40.9 minutes for PixArt on one NVIDIA A100, with batch size 4 and 32 instances. The accepted mask stays fixed during inference, with no online JVP or site selection. Section 4.3 and Appendix D.4 document the procedure and counts.

Q3 Reuse across step counts and solvers

The host weights, scaling policies, and low-rank codebooks remain frozen. Solver coefficients ctc_t follow closed forms for DDIM (Eqs. 10 and 11) and DPM-Solver++ [4] (Eq. 13), and are recomputed for the target schedule in under 1 ms on CPU at initialization, without retraining or GPU calibration.

We tested reuse of the PixArt-Σ ViDiT-Q W4A8 mask calibrated on DPM-Solver++ at 20 steps, comparing direct reuse with separately recalibrated masks at 10 and 50 steps:

Target ScheduleHost BaselineDirect Mask ReuseRecalibrated Mask
DPM-Solver++ 10-step14.12 dB15.48 dB15.50 dB
DPM-Solver++ 20-step source15.25 dB16.88 dB16.88 dB
DPM-Solver++ 50-step15.82 dB17.38 dB17.40 dB

Direct reuse improves over the host and stays within 0.02 dB of recalibration. This evidence supports the evaluated step-count changes within the same solver family. Section 4.3 and Appendix F.2 report reuse, and Appendix A.1 gives the schedule-dependent coefficients.

References

[1] Zhao, T., et al. ViDiT-Q: Efficient and accurate quantization of diffusion transformers for image and video generation. ICLR, 2025.

[2] Peebles, W., and Xie, S. Scalable diffusion models with transformers. ICCV, 2023, pp. 4195-4205.

[3] Chen, J., et al. PixArt-Σ: Weak-to-strong training of diffusion transformer for 4K text-to-image generation. ECCV, 2024.

[4] Lu, C., et al. DPM-Solver++: Fast solver for guided sampling of diffusion probabilistic models. Machine Intelligence Research, 22(4), 730-751, 2025. doi:10.1007/s11633-025-1562-4.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response Part 1 of 2

Thank you for the careful reviews. The reviews highlighted TeleQuant's temporal formulation, training-free integration, fidelity gains on DiT-XL/2, PixArt-Σ, SDXL, and Open-Sora 1.2, and pipeline efficiency. We clarify the contribution and summarize the revisions below.

Novelty and Research Contribution

TeleQuant coordinates activation-rounding residuals across successive denoiser evaluations while keeping the quantized host, including its weight codebooks, channel transforms, and low-rank branches [2], fixed. Its feedback gain matches adjacent solver responses and activation scales. Under the stated first-order assumptions, the telescoping identity separates accumulated error into a terminal residual and a sensitivity-variation term. This motivates selective placement and damped feedback. INT8 residual storage and a fused activation operator support integration on the unchanged host GEMM path. Our contribution is the analytical matching rule and its connection to trajectory error, together with its implementation on frozen hosts.

Major Revisions in Our Paper

The revisions address the reviewers' questions through controlled comparisons, component ablations, and clearer theoretical and implementation descriptions. The two central experimental tables below appear in Section 5.2. Other supporting tables are placed in the corresponding appendices, with the main text explaining the relevant conclusions and methodological clarifications.

1. Comparison with temporal baselines

In response to Reviewers K89G and KXWF, Section 5.2 includes the following comparison with TCEC [3] and a controlled MoDiff adaptation [4] on frozen ViDiT-Q W4A8 hosts. The TCEC baseline now includes the preceding-error term in the cumulative correction. This corrected formulation is stronger than the version reported in original Table 3, improving TCEC from 19.74 dB to 19.77 dB on DiT-XL/2 and from 15.68 dB to 15.74 dB on PixArt-Σ, and TeleQuant remains better on all three metrics. Table 3 has been updated accordingly, and Appendix D.2 specifies the corrected baseline and the different intervention points, with Section 2 summarizing them. Appendix F.5 describes the MoDiff adaptation details.

MethodDiT PSNR, dBDiT SSIMDiT LPIPSPixArt PSNR, dBPixArt SSIMPixArt LPIPS
Frozen ViDiT-Q Host [1]19.590.6800.28615.250.5900.448
TCEC [3]19.770.6950.27515.740.6030.428
MoDiff Adaptation [4]19.650.6850.28115.820.6050.425
TeleQuant, our method19.910.7080.26216.880.6110.400

References

[1] Zhao, T., et al. ViDiT-Q: Efficient and accurate quantization of diffusion transformers for image and video generation. ICLR, 2025.

[2] Li, M., et al. SVDQuant: Absorbing outliers by low-rank component for 4-bit diffusion models. ICLR, 2025.

[3] Liu, S., et al. Error propagation mechanisms and compensation strategies for quantized diffusion models. ICML, 2026, pp. 79701-79718.

[4] Gao, W., et al. Modulated diffusion: Accelerating generative modeling with modulated quantization. ICML, 2025, pp. 18337-18362.

AuthorsOpenReview ↗

General Response Part 2 of 2

2. Matched-sampler full-precision reference

In response to Reviewer KXWF, Section 5.1 clarifies that the DiT-XL/2 distributional comparison uses the same DDIM-50 schedule for the FP16, host, and TeleQuant arms. The matched-reference results and recalculated gap-recovery table appear in Appendix F.3, with the sampling protocol in Appendix D.1. The DiT-XL/2 FP16 row of Table 1 is updated to this matched run (FID 3.18, sFID 5.12, IS 268.4). The comparison therefore isolates quantization from sampling-schedule effects. The matched-reference results update the DiT rows of original Table 14, now Table 20. The fractions change from 21.97/35.11/18.83% to 31.3/48.5/28.1%.

3. Disentangled ablation study

In response to Reviewers LW7Z and KXWF, Section 5.2 includes the following six-configuration ablation using the same 38 selected sites, initial latents, and FP16 references. Solver matching and scale matching each improve on constant damping, and their combination improves further. Section 4.1 explains their respective roles. The existing damping and residual-precision sweeps remain in Appendix E.2.

ConfigurationFeedback gainPSNR, dBSSIMLPIPS
1. Frozen hostNo feedback19.590.6800.286
2. Unit-delay feedback, α=1\alpha=1Unweighted residual reuse19.690.6960.282
3. Constant damping, λ=0.75\lambda=0.75Damped without matching19.710.6910.280
4. Solver matchingλct−1ct\lambda\frac{c_{t-1}}{c_t}19.810.6990.274
5. Scale matchingλst−1,lst,l\lambda\frac{s_{t-1,l}}{s_{t,l}}19.780.6970.277
6. Full TeleQuantλct−1ctst−1,lst,l\lambda\frac{c_{t-1}}{c_t}\frac{s_{t-1,l}}{s_{t,l}}19.910.7080.262

4. Schedule reuse

In response to Reviewer LW7Z, Section 4.3 explains how the solver-response coefficients are recomputed while the calibrated site mask is reused for the tested schedule changes. The direct-reuse versus recalibration table appears in Appendix F.2, with coefficient computation described in Appendix A.1. Reusing the PixArt-Σ DPM-Solver++ 20-step mask on the tested 10-step and 50-step schedules gives results close to recalibration, supporting reuse across the evaluated step counts within the same solver family.

5. Scalar solver-response approximation

In response to Reviewer K89G, Section 3 clarifies the distinction between the scalar first-order surrogate and the matrix-valued transfer. Appendix A.1 includes the phase-wise probe measurements alongside the approximation-error analysis. These measurements support the approximation on the evaluated trajectories, while the analysis retains the sensitivity-variation and nonlinear remainder terms.

6. Site screening and offline calibration

In response to the three reviewers, Section 4.3 specifies the JVP differentiation target and distinguishes sensitivity screening from the paired-rollout acceptance test. Appendix D.4 reports the screening counts, calibration configuration, and measured offline cost. The smooth surrogate retains the frozen host weights and transforms while replacing activation rounding with the identity. The accepted mask remains fixed during generation, so calibration adds no online screening cost.

7. Prompt alignment and video uncertainty

In response to Reviewer KXWF, Section 5.2 summarizes the complementary prompt-alignment and video-fidelity evidence. The PixArt-Σ CLIP [5] cosine-similarity table and paired bootstrap interval appear in Appendix E.4, with the metric defined in Appendix D.5. Video reconstruction results remain in Appendix E.1, with clip-level bootstrap uncertainty in Appendix E.4. These results indicate preserved prompt alignment, with a small positive mean CLIP cosine-similarity change, and consistently positive paired video-fidelity gains.

8. Theoretical precision and evaluation scope

In response to Reviewer KXWF, Section 4.1 distinguishes the induced-norm drift bound from the computable Frobenius-norm surrogate, with the damping derivation clarified in Appendix E.2. In response to Reviewer K89G, Section 5.1 states the evaluated host precisions, while the supporting bit-width table remains in Appendix E.2. The MoDiff state-memory and latency comparison appears in Appendix F.5, with the eight-site and 42-site quality profiles reported separately. Timing details are in Appendices E.6 and D.6.

The individual responses provide the detailed evidence for each revision. Supporting tables retained in those responses are incorporated into the appendices identified above.

References

[5] Radford, A., et al. Learning transferable visual models from natural language supervision. ICML, 2021, pp. 8748-8763.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.