SyRA-DiT: Synergistic Reconstruction-Aware Rotation for W4A4 Diffusion Transformers

Post-training quantization (PTQ) enables low-precision deployment of pretrained Diffusion Transformers without retraining, yet aggressive W4A4 quantization remains challenging. Coarsely quantized weights are reused across denoising timesteps, while activation distributions vary over time, making layer reconstruction error difficult to minimize consistently throughout the denoising process. Group-wise quantization improves local range adaptation, but but its quantization fidelity remains highly sensitive to the within-group distribution. To this end, we propose SyRA-DiT, a reconstruction-driven PTQ framework that learns an orthogonal rotation to jointly reorganize weight and activation quantization groups across denoising timesteps, directly minimizing W4A4 layer reconstruction error on a limited calibration set via Cayley parameterization and straight-through estimation. Furthermore, SyRA-DiT uses the same per-timestep incoherence weights to construct the initialization covariance, the W4A4 reconstruction objective, and the GPTQ Hessian. The rotation is optimized to minimize the relative W4A4 reconstruction error, after which GPTQ quantizes the rotated weights using a Hessian constructed under the same temporal weighting. We evaluate SyRA-DiT on image, text-to-image, and video diffusion models under 4-bit weight and activation quantization, where it achieves state-of-the-art visual quality and quantitative performance. Notably, on ImageNet at $256\times 256$ resolution with DiT-XL/2, SyRA-DiT remains competitive with previously reported W4A8 results across diverse sampling and guidance settings, demonstrating that PTQ remains effective even in the challenging W4A4 regime.