- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes TurboQuantConv, a post-training quantization framework for Conv3D layers in video VAEs. It applies randomized Walsh–Hadamard rotations along input channels so activations can be transformed before overlapping spatiotemporal patches are gathered. The rotated weights and activations are then quantized using a polar/spherical codebook formulation or a sub-vector uniform backend, with proposed mappings to lookup-table and integer-GEMM hardware. Table 1 reports W4A4 reconstruction quality on two video VAEs, and Table 2 evaluates downstream video generation. The paper also makes strong claims about rate–distortion guarantees and efficient edge deployment.
Critical strengths
-
The channel-isolated construction addresses a real implementation issue in Conv3D. Proposition 1 in Section 3.1 uses orthogonal invariance of the channel contraction to move the rotation before im2col, avoiding repeated full-patch rotations across overlapping windows. The algebraic identity is elementary, but its use in a fused Conv3D implementation could offer practical value if realized and benchmarked on hardware.
-
The reconstruction results merit further examination. Table 1 reports substantially better PSNR and LPIPS on WAN 2.2 and LTX-Video VAEs, evaluated on TokenBench and DAVIS-2017. In Table 2, the quantized method has an FVD of 152.6, lower than the reported W4A4 comparators. Table 4 also investigates full-channel and sub-vector backends. These numbers are promising as empirical observations, provided the effective precision, scaling granularity, and actual execution paths can be verified.
Critical weaknesses
-
The central quantization representation and its claimed distortion rate are inconsistent. Section 3.3 and Algorithm 1 quantize an entire direction vector against a codebook of vector centroids; at , this describes one 4-bit index per -dimensional vector. Section 5 instead constructs a 256-entry multiplication LUT from separate 4-bit activation and weight indices at every contraction coordinate, which describes scalar or sub-vector quantization with a different bit budget, decoder, and attainable distortion. The theory has the same inconsistency. Proposition 2 states a distortion bound, while Appendix A.3, Eq. (20), gives for a spherical codebook with entries in dimensions. Equation (21) then switches to coordinate-wise scalar Lloyd–Max quantization. These expressions cannot be used interchangeably as a guarantee for the same quantizer. The sub-Gaussian concentration result in Theorem 1 also does not establish that an actual rotated activation distribution is jointly isotropic Gaussian, which the Gaussian-codebook argument assumes. The paper needs one explicit bitstream and reconstruction procedure, together with a proof appropriate to that procedure. This is a correctness issue, not merely a notation issue.
-
The reported advantage is confounded by scaling granularity, and the deployment claims lack measurements. Tables 3–4 report 29.49 dB for QuaRot-style rotation with shared per-tensor min–max scaling, 32.82 dB for rotated full-channel uniform quantization with per-voxel scaling, and 33.41 dB for the full-channel codebook. The paper itself decomposes the gain as +3.33 dB from changing the scaling scheme and only +0.59 dB from the codebook. Consequently, the headline +3.92 dB over QuaRot does not isolate the proposed spherical representation. Table 4 also reports 35.53 dB for the sub-vector uniform backend, which further calls the necessity of the spherical codebook into question. The baselines should be compared with matched effective bit budgets and per-voxel/group scaling. The claimed edge efficiency is even less substantiated: Section 5 and Appendix B describe operation counts, LUTs, and candidate NPU architectures, but provide no measured Conv3D kernel latency, full VAE decoding throughput, peak memory, or end-to-end speedup. Table 2 does not report BF16 FVD/FID, so the assertion of preserving unquantized video dynamics lacks a like-for-like reference on those metrics. Reconstruction fidelity and prospective hardware mappings do not establish deployment efficiency.
Questions for authors
- Does one 4-bit index represent an entire -dimensional direction, a scalar coordinate, or a sub-vector in the actual W4A4 implementation? Please specify the index tensor shapes, radial-scale precision, padding, inverse transforms, total bits, and GEMM operations, and reconcile Proposition 2 with Appendix Eqs. (20–21).
- With identical R-FWHT rotations, per-voxel/group scales, and effective bit budgets, what are the accuracy and execution costs of QuaRot-style uniform quantization, full-channel uniform quantization, the spherical/codebook method, and the sub-vector backend? How much of the reported +3.92 dB is specifically due to the codebook?
- Can the authors provide actual GPU or target-NPU Conv3D timings, complete VAE decoding latency/throughput, peak memory, and BF16-to-W4A4 end-to-end speedups? Please also include BF16 FVD/FID under the same downstream evaluation protocol.
2 comments
I appreciate the authors' efforts in addressing my concerns. However, most of the reported 3.92 dB improvement comes from finer scaling, with only 0.32 dB attributable to the codebook itself. The proposed LUT-based execution also remains unimplemented, and its efficiency on edge hardware has not been verified. These issues still leave me with some doubts about the method's specific contribution and practical deployment claims.
Considering the substantial improvements made during the rebuttal, I've decided to raise my rating to 2, although some concerns remain.
Thank you for a careful and precise review. It led us to rewrite the quantizer description and theory, re-run matched comparisons and add measurements.
Weakness 1.
Weakness 2.
Q1. A scalar coordinate. All results use one R-FWHT per layer along Cin (zero-padded to m=2⌈log2Cin⌉, shared by weights and activations), one L2 radius per vector, and 16 fixed Lloyd–Max levels of N(0,1/m) per coordinate.
Q2 (WAN 2.2, 492 TokenBench clips, R-FWHT, 4 bits; Tables 3–4).
Q3 (A100, 480p × 81 frames, Table 5; throughput = 81 frames / latency).
We hope this addresses your concerns and are happy to discuss further.