PyraQuant: Spatial Mixed-Precision Quantization for High-Resolution Diffusion Models

High-resolution diffusion generation remains expensive because denoising cost grows rapidly with image resolution. Progressive generation reduces this cost by first generating at a low resolution and then progressively increasing the resolution, while existing diffusion quantization methods mainly allocate precision across layers, channels, or denoising steps without exploiting the stage-wise and spatial structure of this process. We find that progressive refinement exhibits two forms of non-uniformity: quantization tolerance varies across resolution stages, and refinement changes differ across spatial regions. Based on these observations, we propose PyraQuant, a training-free precision allocation framework for quantized progressive diffusion models. PyraQuant keeps the base quantizer at the lowest resolution. At each subsequent resolution transition, it assigns each region a precision according to how much it changes, carrying these decisions to the next resolution. The lower-precision configuration is derived directly from the original quantized representation without additional calibration or model copies, and regions requiring limited refinement reuse cached predictions. Experiments show that on FLUX.1-schnell with the SVDQuant W4A4 base quantizer, PyraQuant reduces effective precision from 4.48 to 2.00 bits while maintaining comparable generation quality, and achieves a $1.81\times$ end-to-end speedup on UltraHR-eval4K.