acceptodds
Under review as a conference paper at ICLR 2027

SyRA-DiT: Synergistic Reconstruction-Aware Rotation for W4A4 Diffusion Transformers

Abstract

Post-training quantization (PTQ) enables low-precision deployment of pretrained Diffusion Transformers without retraining, yet aggressive W4A4 quantization remains challenging. Coarsely quantized weights are reused across denoising timesteps, while activation distributions vary over time, making layer reconstruction error difficult to minimize consistently throughout the denoising process. Group-wise quantization improves local range adaptation, but but its quantization fidelity remains highly sensitive to the within-group distribution. To this end, we propose SyRA-DiT, a reconstruction-driven PTQ framework that learns an orthogonal rotation to jointly reorganize weight and activation quantization groups across denoising timesteps, directly minimizing W4A4 layer reconstruction error on a limited calibration set via Cayley parameterization and straight-through estimation. Furthermore, SyRA-DiT uses the same per-timestep incoherence weights to construct the initialization covariance, the W4A4 reconstruction objective, and the GPTQ Hessian. The rotation is optimized to minimize the relative W4A4 reconstruction error, after which GPTQ quantizes the rotated weights using a Hessian constructed under the same temporal weighting. We evaluate SyRA-DiT on image, text-to-image, and video diffusion models under 4-bit weight and activation quantization, where it achieves state-of-the-art visual quality and quantitative performance. Notably, on ImageNet at resolution with DiT-XL/2, SyRA-DiT remains competitive with previously reported W4A8 results across diverse sampling and guidance settings, demonstrating that PTQ remains effective even in the challenging W4A4 regime.

Primary area:
generative models
Keywords:
Post-training quantization, Diffusion Transformer, learnable rotation, reconstruction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_vGivOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper directly optimizes the layer-output reconstruction error after quantization and learns an orthogonal rotation matrix. It uses per-timestep incoherence coefficients as weights, initializes the rotation using the product of the eigenvector matrix of the weighted covariance matrix and a fixed normalized randomized Hadamard transform, preserves orthogonality through Cayley parameterization. Experimental results further demonstrate the effectiveness of SyRA-DiT.

Critical strengths
  1. This paper uses the quantized layer output error as the optimization objective to learn an orthogonal rotation matrix. Orthogonality is ensured through the Cayley transformation. The problem of non-differentiability caused by quantization rounding during backpropagation is addressed using STE.

  2. This method uses timestep incoherence weights to directly perform a weighted average on the covariance of the activation matrix, obtaining the initial rotation matrix through eigenvalue decomposition combined with the Hadamard matrix. It also uses the same time-step weights to construct GPTQ’s second-order statistics, which guide the final weight rounding and error compensation. Experiments with DiT-XL/2, PixArt-Σ\Sigma, and Open-Sora demonstrate strong W4A4 performance across image and video generation tasks.

Critical weaknesses
  1. Missing comparisons to several recent, closely related methods, such as LRQ-DiT, DiRotQ, and TR-DQ, which are mentioned but did not compared. These are also rotation- or time-step-aware quantization methods aimed at achieving high-quality low-bit quantization. Given the similarities in their design philosophies, including comparisons with recent baselines would strengthen the paper’s positioning.

  2. Obtaining the initial rotation matrix requires solving the corresponding covariance matrices for all activation matrices in the calibration set. Additionally, optimizing the quantized layer output errors requires layer-by-layer learning. Since some models have a large number of layers, and certain layers may be very wide, this may introduce significant computational overhead for calibration.

Questions for authors
  1. Could the authors report at least one more comparison with a recent baseline method that takes rotation or time-step awareness into account?

  2. What is the calibration overhead associated with computing the covariance matrix of activations for each calibration set and learning with STE?

  3. What impact does using the normalized time-step weights ω~t\widetilde{\omega}_t mentioned in Appendix B have on the actual GPTQ calibration compared to using the original time-step weights ωt\omega_t?

Reviewer_p2BPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SyRA-DiT studies post-training W4A4 quantization of diffusion transformers. It learns an orthogonal rotation for each linear layer while keeping pretrained weights fixed. The objective reconstructs jointly quantized layer outputs across denoising timesteps. Timestep weights guide initialization, reconstruction, and subsequent GPTQ calibration. Experiments cover class-conditional images, text-to-image generation, and video generation. ImageNet FID improves over VETA-DiT in four sampling settings. PixArt fidelity also improves. The reported DiT deployment gains reach 1.74 times faster inference when both precision settings use CUDA graphs.

Critical strengths
  1. The objective directly reconstructs jointly quantized layer outputs across denoising steps. This addresses the interaction between fixed weights and changing activation distributions.

  2. ImageNet improvements hold across all four tested sampling settings. At 50 steps and CFG 1.5, FID improves from 7.90 to 6.28 over VETA-DiT (Table 1).

  3. Evaluation extends to PixArt and Open-Sora. The deployment table also compares matching execution modes, allowing the effect of CUDA graphs to be distinguished from quantization gains.

Critical weaknesses
  1. The deployment comparison leaves the rotation implementation unclear. Each layer learns a dense C-by-C transform, but its storage and execution costs are unspecified. An explicit account of these costs and higher-precision operations would clarify the reported W4A4 gains.

  2. Table 3 does not separate initialization, rotation learning, and temporal calibration choices. Its final variant changes timestep weighting across three stages together. The improvement supports the combined design, but leaves the individual contributions unclear.

  3. Section 5.1 does not summarize calibration sample counts or rotation optimization budgets. The supplementary algorithm also leaves the iteration count unspecified. This limits assessment of calibration efficiency and comparison with the reproduced baselines.

Questions for authors
  1. Section 4.2 learns a dense rotation for each quantized linear layer. Could you clarify how these rotations are stored and executed? Please identify higher-precision operations and confirm whether Table 4 includes rotation and dynamic quantization costs. This would establish the scope of the reported W4A4 deployment gains.

  2. Table 3 adds learned rotation and timestep weighting in two stages. What initialization and timestep weights are used in each row? Please clarify which stages change between rows, including reconstruction and GPTQ. This would help distinguish rotation learning from the combined temporal calibration design.

  3. Section 5.1 refers to prior calibration protocols but does not summarize the optimization budget. Could you provide the calibration sample count and rotation-learning iterations? Please include recorded calibration time and hardware, if available. This would clarify the cost of obtaining the reported W4A4 improvements.

Reviewer_3NEQOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper targets W4A4 post-training quantization of Diffusion Transformers. Weights and activations both use asymmetric group-wise min–max quantization, with dynamic per-token activation groups. SyRA-DiT has three components: learned rotation, learned rotation, and temporally weighted GPTQ. The experiment results using an A100 measurement report 1.74× latency over FP16 under CUDA graphs.

Critical strengths
  1. Strong W4A4 accuracy on class-conditional DiT.

  2. A sensible, principled consistency argument.The ablation shows that the timestep weighting contributes a sizable share of the final gain (FID 7.80 → 6.28).

Critical weaknesses
  1. The deployment cost of dense, per-layer learned rotations is not accounted for, so the efficiency and memory claims do not describe the proposed method. The paper presents W4A4 as a practical deployment path. As described, the method may be slower and larger than FP16.

  2. Limited novelty relative to learned-rotation/transform PTQ, and an incomplete experimental comparison. PTQ4DiT and Q-DiT collapse completely at W4A4 (FID=243–305, IS= 2), so they carry no information. ViDiT-Q was run without W4A4 tuning.

Questions for authors

Q1: Need more evaluations on rotation cost. Is R really dense per layer at inference? If so, please report: the storage of all rotations; the online rotation FLOPs and latency; end-to-end latency and peak memory with the rotations included; the W4A4 GEMM kernel and how it handles g = 128 asymmetric activation groups.

Q2: The ImageNet and PixArt accuracy numbers are encouraging. However, the method's online cost is not accounted for, which invalidates the efficiency claims as stated; the contribution is close to existing learned-rotation PTQ, which is not compared; the strongest W4A4 baselines are missing.

Q3: The theory does not match the implementation

Comments on the paper

3 comments
AuthorsOpenReview ↗

## Common response to all reviewers and AC(s)

Dear Area Chair and Reviewers,

We sincerely thank the three reviewers for their careful reading, thoughtful comments, and constructive suggestions, and the Area Chair for overseeing the review process. We are encouraged by the reviewers' recognition of SyRA-DiT's state-of-the-art W4A4 generation quality and its well-motivated design. We also appreciate their insightful feedback on methodological novelty, theoretical consistency, experimental comparisons, and practical efficiency, which has helped us further strengthen the manuscript. In response, we have made the following major revisions:

  1. Methodological clarification and component-wise analysis: We have clarified the novelty of SyRA-DiT relative to existing learned-rotation PTQ methods and introduced controlled ablations to isolate the contributions of rotation learning and temporal calibration.

  2. Expanded experimental comparisons: We have added comparisons with recent competitive W4A4 quantization methods across image, text-to-image, and video generation, including rotation-based PTQ and training-based approaches.

  3. Calibration and inference efficiency: We have quantified calibration budgets, dense rotation storage, and online computational overhead, and investigated structured rotations to improve practical deployment efficiency.

  4. Theoretical consistency and implementation clarity: We have clarified the relationship between timestep-weighted reconstruction and GPTQ calibration, improved the theoretical consistency claim, and refined the algorithm and implementation details.

These revisions provide a more comprehensive evaluation of SyRA-DiT's contributions, performance, and practical limitations. Below, we summarize the main shared concerns and our responses, followed by point-by-point replies to each reviewer.

Common questions and related Revisions

CQ1. Evaluations on rotation cost (by Reviewer p2BP, Reviewer 3NEQ, Reviewer vGiv)

Reply:

We thank the reviewer for this valuable comment. We have revised the efficiency analysis and Table 1 to explicitly account for activation rotation overhead. All configurations use CUDA Graph execution on an A100 GPU to ensure a consistent comparison.

Table 1. Inference cost of DiT-XL/2 on A100 (ImageNet-256, DDIM 50, CFG$=1.5$, batch size 1). Measured W4A4 results use a symmetric per-token INT4 timing prototype, distinct from the asymmetric group-128 quality implementation. †^\dagger denotes projected latency and estimated memory for the hybrid rotation design, which replaces dense fc2 rotations with structured transforms. Speedups are relative to FP16.

Methodms/img ↓\downarrowMemory (MB) ↓\downarrowSpeedup ↑\uparrow
FP16341.31671.71.00$\times$
W4A4, GEMM only193.31065.21.77$\times$
W4A4, dense RR370.44086.20.92$\times$
W4A4, hybrid R†R^{\dagger}263.42984.01.30$\times$

Specifically, SyRA-DiT applies dense FP16 rotations to 112 quantized linear layers. Although weight rotations are absorbed into GPTQ quantization, activation rotations still introduce additional computational and storage overhead, regardless of whether they are learned or fixed. Accounting for these operations, W4A4 incurs 370.4 ms/image latency and 4086 MB peak memory, compared with 341.3 ms/image and 1672 MB for FP16. The dense rotations introduce approximately 1.32 GiB of storage and 177 ms/image of latency overhead, with fc2 contributing approximately 84% of the rotation cost. We also clarify that the 1.77×1.77\times speedup (193.3 ms/image) corresponds to W4A4 GEMM-only execution without activation rotations, rather than the complete SyRA-DiT pipeline.

To mitigate this overhead, we evaluate a simple hybrid rotation design (denoted by hybrid R†R^{\dagger} in the table) that replaces dense rotations in fc2 layers with online Hadamard transforms and learned block-diagonal orthogonal rotations, while retaining dense rotations in the remaining layers. The hybrid design yields an estimated latency of 263.4 ms/image and a peak memory footprint of 2984 MB. This corresponds to a 1.30×1.30\times speedup over FP16, with a modest increase in FID from 6.28 to 6.57 relative to the dense-rotation design.

This initial experiment suggests that structured rotations can reduce inference overhead while largely preserving generation fidelity. Our primary aim is to use rotations to study how weight and activation quantization interact across diffusion timesteps. We leave a systematic comparison of low-cost alternatives, including sparse rotations and block-diagonal special orthogonal rotations, to future work. We thank the reviewers for raising this point and have revised the contribution statement and limitations section to clarify the trade-offs among quantization accuracy, rotation overhead, and inference efficiency.

AuthorsOpenReview ↗

## Common response to all reviewers and AC(s)

Continued

CQ2. More baseline comparison

**Reply:**We thank the reviewer for highlighting these concerns regarding baseline coverage and methodological novelty.

More baselines: We have expanded the evaluation to include recent W4A4 methods: TreeQ-PEFT [1], RobuQ [2], and JustQuant [3] for image generation (Table 2); LRQ-DiT [4] and DiRotQ [5] for text-to-image generation (Table 3); and DVD [6] for video generation (Table 4). These additions provide broader coverage of both PTQ and training-based quantization approaches.

Table 2. Cross-method comparison of W4A4 quantization on DiT-XL/2 for ImageNet 256×256256\times256 (CFG$=1.5$, 50 sampling steps, FID-10K). The best and second-best results are highlighted in bold and italics, respectively. “--” denotes unreported results.

MethodTypeIS$\uparrow$FID$\downarrow$sFID$\downarrow$Prec.↑\uparrow
VETA-DiTPTQ135.667.9019.110.73
SyRA-DiT (Ours)PTQ156.516.2818.960.75
QAT [0]QAT196.5310.7226.55--
TreeQ-PEFT [1]PEFT/QAT206.108.6923.150.7449
RobuQ [2]QAT222.028.2022.94--
JustQuant [3]QAD+QAT238.316.4819.31--

Table 3. Text-to-image generation performance of PixArt-Σ\Sigma [chen2024pixart] on the COCO dataset. The best and second-best results are highlighted in bold and italics, respectively.

MethodRFID$\downarrow$rsFID$\downarrow$LPIPS$\downarrow$CMMD$\downarrow$Key operator
Q-DiT (CVPR'25)120.6286.00.7351.275Automatic granularity allocation; dynamic activation quantization
ViDiT-Q (ICLR'25)136.6294.00.6651.493Fine-grained grouping; hybrid static--dynamic channel balancing.
VETA-DiT (NIPS'26)42.7247.80.4200.139Karhunen–Loève Transform (KLT) Alignment; incoherence-aware temporal adaptation.
DiRotQ (Arxiv'26)55.7385.40.7420.346PCA rotation-aware activation quantization; GPTQ
LRQ-DiT (TCSVT'26)63.3263.00.5270.587Twin-log quantization (TLQ); adaptive rotation scheme (ARS)
SyRA-DiT (Ours)35.6222.40.3320.103Learnable rotation; Cayley parameterization; per-timestamp adaptation; GPTQ

Table 4. Performance comparison of Open-Sora on VBench. The best and second-best results among W4A4 methods are highlighted in bold and italics, respectively. ∗^{*} denotes mixed precision.

MetricFP16ViDiT-Q$^{*}$Q-DiTViDiT-QVETA-DiTDVDSyRA-DiT
Subject Consistency0.9680.9600.9590.9580.9580.9570.960
Background Consistency0.9710.9620.9580.9570.9600.9590.959
Motion Smoothness0.9910.9890.9890.9890.9890.9890.989
Dynamic Degree0.2500.2500.2080.1940.1670.2360.222
Aesthetic Quality0.5310.5130.5060.5090.5120.5040.508
Imaging Quality0.5670.5460.5390.5290.5330.4990.550
Scene0.4040.4080.3430.3820.3360.3450.374
Overall Consistency0.2560.2570.2570.2550.2600.2510.257

Reference: [0] Jacob, Benoit, et al. "Quantization and training of neural networks ..." CVPR 2018.

[1] Yang, K., et al. "TreeQ: Pushing ... Mixed-Precision Search." arXiv preprint arXiv:2512.06353 (2025).

[2] Yang, K., et al. "Robuq: ... quantization." arXiv preprint arXiv:2509.23582 (2025).

[3] Yang, K., et al. "JustQuant: You Don't Need ... " arXiv preprint arXiv:2609.33601 (2026).

[4] Yang, L., et al. "Lrq-dit: Log-rotation ... generation." IEEE TCSVT, 2026.

[5] Sharify, S. , et al. "DiRotQ: Rotation-Aware ..." arXiv:2605.16732 (2026).

[6] Li, Z., , et al. "Dvd-quant: data-free video diffusion transformers quantization." ICLR 2026.

Methodological novelty:

  • Distinct motivation: Unlike methods optimizing weight and activation quantization separately, we examine their coupled effects in DiT layer products. Findings reveal reconstruction errors and outliers from coupling static weight and timestep-varying activation quantization.

  • Joint, temporally informed reconstruction: Beyond rotation learning, SyRA-DiT optimizes a shared orthogonal rotation via timestep-aware joint reconstruction. Temporal calibration coordinates rotation initialization, layer reconstruction, and GPTQ Hessian estimation, accounting for activation dynamics throughout calibration.

  • Performance and practical trade-offs: Experiments across image, text-to-image, and video diffusion models demonstrate state-of-the-art W4A4 generation quality under evaluated settings, competitive with W4A8 baselines. System-level evaluations quantify trade-offs between rotation overhead and low-bit acceleration.

Dear Reviewers,

Thank you for your efforts so far. As we are at the end of the discussion period, I would like to kindly encourage you to review the authors' rebuttals and engage in further discussions, particularly if the authors have addressed your questions or concerns.

Thank you for your time and contributions!

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.