acceptodds
Under review as a conference paper at ICLR 2027

TurboQuantConv: Channel-Isolated Incoherence Processing for 3D Convolutions in Video Generative AI

Abstract

Three-dimensional Convolutional Variational Autoencoders (3D-VAEs) serve as the fundamental compression backbone of modern video generative models, yet their deep 3D convolutions (Conv3D) impose prohibitive computational and memory burdens. Pushing post-training quantization (PTQ) down to 4-bit weights and activations (W4A4) typically causes substantial quality degradation due to non-stationary spatiotemporal activation bursts. In this paper, we propose TurboQuantConv, a channel-isolated incoherence processing and quantization framework for deep multi-dimensional convolutions. By harmonizing outlier suppression with convolutional receptive field geometry, TurboQuantConv effectively suppresses spatiotemporal bursts and achieves high-fidelity quantization without requiring fine-tuning or retraining. Under W4A4 compression, TurboQuantConv outperforms prior state-of-the-art PTQ methods by  4 to 8 dB PSNR across 3D-VAE benchmarks. In downstream 81-frame video generation with the 5B Wan 2.2 Diffusion Transformer, TurboQuantConv achieves the lowest FVD while closely preserving unquantized visual dynamics, avoiding the severe motion freeze and chromatic collapse from which existing PTQ methods suffer. These results establish a practical foundation for efficient edge deployment of video generative models.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Quantization, Video Generative AI, 3D Convolution, Hardware Optimization, Edge Deployment, W4A4

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_FnALOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes TurboQuantConv, a post-training quantization framework for Conv3D layers in video VAEs. It applies randomized Walsh–Hadamard rotations along input channels so activations can be transformed before overlapping spatiotemporal patches are gathered. The rotated weights and activations are then quantized using a polar/spherical codebook formulation or a sub-vector uniform backend, with proposed mappings to lookup-table and integer-GEMM hardware. Table 1 reports W4A4 reconstruction quality on two video VAEs, and Table 2 evaluates downstream video generation. The paper also makes strong claims about rate–distortion guarantees and efficient edge deployment.

Critical strengths
  1. The channel-isolated construction addresses a real implementation issue in Conv3D. Proposition 1 in Section 3.1 uses orthogonal invariance of the channel contraction to move the rotation before im2col, avoiding repeated full-patch rotations across overlapping windows. The algebraic identity is elementary, but its use in a fused Conv3D implementation could offer practical value if realized and benchmarked on hardware.

  2. The reconstruction results merit further examination. Table 1 reports substantially better PSNR and LPIPS on WAN 2.2 and LTX-Video VAEs, evaluated on TokenBench and DAVIS-2017. In Table 2, the quantized method has an FVD of 152.6, lower than the reported W4A4 comparators. Table 4 also investigates full-channel and sub-vector backends. These numbers are promising as empirical observations, provided the effective precision, scaling granularity, and actual execution paths can be verified.

Critical weaknesses
  1. The central quantization representation and its claimed distortion rate are inconsistent. Section 3.3 and Algorithm 1 quantize an entire direction vector u∈Sm−1u\in S^{m-1} against a codebook of 2b2^b vector centroids; at b=4b=4, this describes one 4-bit index per mm-dimensional vector. Section 5 instead constructs a 256-entry multiplication LUT from separate 4-bit activation and weight indices at every contraction coordinate, which describes scalar or sub-vector quantization with a different bit budget, decoder, and attainable distortion. The theory has the same inconsistency. Proposition 2 states a Cb2−2bC_b2^{-2b} distortion bound, while Appendix A.3, Eq. (20), gives Cm2−2b/mC_m2^{-2b/m} for a spherical codebook with 2b2^b entries in mm dimensions. Equation (21) then switches to coordinate-wise scalar Lloyd–Max quantization. These expressions cannot be used interchangeably as a guarantee for the same quantizer. The sub-Gaussian concentration result in Theorem 1 also does not establish that an actual rotated activation distribution is jointly isotropic Gaussian, which the Gaussian-codebook argument assumes. The paper needs one explicit bitstream and reconstruction procedure, together with a proof appropriate to that procedure. This is a correctness issue, not merely a notation issue.

  2. The reported advantage is confounded by scaling granularity, and the deployment claims lack measurements. Tables 3–4 report 29.49 dB for QuaRot-style rotation with shared per-tensor min–max scaling, 32.82 dB for rotated full-channel uniform quantization with per-voxel scaling, and 33.41 dB for the full-channel codebook. The paper itself decomposes the gain as +3.33 dB from changing the scaling scheme and only +0.59 dB from the codebook. Consequently, the headline +3.92 dB over QuaRot does not isolate the proposed spherical representation. Table 4 also reports 35.53 dB for the sub-vector uniform backend, which further calls the necessity of the spherical codebook into question. The baselines should be compared with matched effective bit budgets and per-voxel/group scaling. The claimed edge efficiency is even less substantiated: Section 5 and Appendix B describe operation counts, LUTs, and candidate NPU architectures, but provide no measured Conv3D kernel latency, full VAE decoding throughput, peak memory, or end-to-end speedup. Table 2 does not report BF16 FVD/FID, so the assertion of preserving unquantized video dynamics lacks a like-for-like reference on those metrics. Reconstruction fidelity and prospective hardware mappings do not establish deployment efficiency.

Questions for authors
  1. Does one 4-bit index represent an entire mm-dimensional direction, a scalar coordinate, or a sub-vector in the actual W4A4 implementation? Please specify the index tensor shapes, radial-scale precision, padding, inverse transforms, total bits, and GEMM operations, and reconcile Proposition 2 with Appendix Eqs. (20–21).
  2. With identical R-FWHT rotations, per-voxel/group scales, and effective bit budgets, what are the accuracy and execution costs of QuaRot-style uniform quantization, full-channel uniform quantization, the spherical/codebook method, and the sub-vector backend? How much of the reported +3.92 dB is specifically due to the codebook?
  3. Can the authors provide actual GPU or target-NPU Conv3D timings, complete VAE decoding latency/throughput, peak memory, and BF16-to-W4A4 end-to-end speedups? Please also include BF16 FVD/FID under the same downstream evaluation protocol.
2 comments

Thank you for a careful and precise review. It led us to rewrite the quantizer description and theory, re-run matched comparisons and add measurements.

Weakness 1.

  • One quantizer. Each 4-bit index encodes one scalar coordinate of the rotated vector.
    • At bb bits per coordinate, the optimal quantizer, a vector codebook over the whole direction, needs 2mb2^{mb} codewords (220482^{2048} at m=512m=512), so it cannot be stored or searched.
    • We use its practical instance, the product codebook Cbm\mathcal C_b^m: same size, mm scalar lookups, and MSE within 2.43× of the lower bound at b=4b=4.
    • The submitted text presented the vector form (with size 2b2^b). The revision keeps it as the reference (Eq. 5) and states the product codebook we use (Eq. 6), which the §5 lookup table implements.
  • Prop. 2 vs. Eqs. (20)–(21).
    • Former Eq. (20) is the distortion of a 2b2^b-word code for the whole direction (b/mb/m bits per coordinate), not of our quantizer; we removed it.
    • Former Eq. (21) is the correct scalar derivation and now underlies Prop. 2. The distortion depends only on coordinate marginals (identical under R-FWHT), tends to DbD_b (D4=0.0095D_4=0.0095) and is at most 3π24−b\tfrac{\sqrt3\pi}{2}4^{-b} for every mm: near-optimal up to a constant, not optimal.
  • Theorem 1. Agreed, it does not show joint Gaussianity; the revised bound needs only the marginals.
  • Bitstream. Algorithm 1 now outputs it (indices plus one radius per vector); Prop. 2 / A.3 bound exactly that quantizer (details in Q1).

Weakness 2.

  • +3.92 dB. With Table 4's full-channel uniform entry corrected (32.82 → 33.09 dB), the gap to QuaRot is +3.60 dB from finer scales plus +0.32 dB from the codebook (§4.4). QuaRot already rotates; at matched per-voxel scales, rotation itself adds +13.39 / +19.37 dB (uniform / codebook).
  • Sub-vector backend. 35.13 dB under the TokenBench protocol (corrected from 35.53 dB): +1.72 dB over ours, at ≈12.5% vs. 0.4–1.6% metadata. The codebook serves the low-metadata point.
  • Baselines, measurements, claims. Table 3 adds per-voxel baselines; Q2–Q3 give the matched comparison and measurements. We did not measure edge NPUs or implement the §5 lookup-table datapath, and limit the claims to the A100.

Q1. A scalar coordinate. All results use one R-FWHT per layer along CinC_{in} (zero-padded to m=2⌈log⁡2Cin⌉m=2^{\lceil\log_2 C_{in}\rceil}, shared by weights and activations), one L2 radius per vector, and 16 fixed Lloyd–Max levels of N(0,1/m)\mathcal N(0,1/m) per coordinate.

  • Shapes. Activations (N×T×H×W×C_in) become indices (N×T×H×W×m) plus one radius per voxel; weights become indices (k_t×k_h×k_w×m×C_out) plus one radius per (tap, output channel).
  • Precision and bits. 4+s/m4+s/m bits per coordinate for an ss-bit radius. The tables use FP32 radii; BF16 / FP16 radii (0.4–1.6% metadata) give 33.40 / 33.39 vs. 33.41 dB (§3.3). Preliminary FP8 radius runs cost little accuracy with a suitable format; we can add them if needed. Decoder weights take 265 vs. 1056 MiB (BF16).
  • Padding (§3.3). None in the decoder (Cin∈{256,512,1024}C_{in}\in\{256,512,1024\}). Encoder layers with Cin∈{160,320,640}C_{in}\in\{160,320,640\} store 6.4 index bits per input channel.
  • Inverse transforms. None: the shared Π\Pi preserves ⟨x,w⟩\langle x,w\rangle (Prop. 1), so the conv contracts integer codes in the rotated basis. Section 4 simulates it in floating point; the fused integer kernels reproduce it (33.95 vs. 33.86 dB, 6 clips, decoding only; App. B.3).
  • GEMM. Implicit GEMM, one integer MMA per tap (INT4 for the uniform grid, INT8 codes for Lloyd–Max); INT32 partial sums are rescaled by sXsWs_Xs_W and accumulated in FP32.

Q2 (WAN 2.2, 492 TokenBench clips, R-FWHT, 4 bits; Tables 3–4).

  • Accuracy (metadata). QuaRot-style uniform: 29.49 dB per tensor (≈0); per voxel, i.e. full-channel uniform: 33.09 dB (0.4–1.6%). Codebook, per voxel: 33.41 dB (0.4–1.6%). Sub-vector, g = 32: 35.13 dB (≈12.5%). The codebook's share of +3.92 dB is +0.32 dB.
  • A100 cost, 188 decoder convs at 480p×17f (Table 6). cuDNN BF16 612.1 ms; per-voxel uniform 310.4 + 11.0 ms (1.90×); Lloyd–Max 556.6 + 27.1 ms (1.05×).

Q3 (A100, 480p × 81 frames, Table 5; throughput = 81 frames / latency).

  • Latency. cuDNN BF16 5.66 s; BF16 on our call path 4.38 s; W4A4 uniform 2.01 s (2.82× / 2.18×); Lloyd–Max 3.35 s (1.69× / 1.31×).
  • 720p. cuDNN BF16 runs out of memory; W4A4 takes 4.65 s.
  • Peak memory at 480p (XLA estimate). 11.6 / 7.7 / 7.2 / 6.9 GiB (cuDNN / our BF16 / Lloyd–Max / uniform). Only the weights (1.0 of the 7.7 GiB) shrink. The other 6.7 GiB, activations and causal-conv state, stays in floating point for normalization, SiLU, residuals and upsampling (App. B.3).
  • FVD / FID, Table 2 protocol: BF16 98.5 / 16.14 (seed-to-seed floor), ours 125.4 / 25.32, QuaRot 221.0 / 34.81.

We hope this addresses your concerns and are happy to discuss further.

I appreciate the authors' efforts in addressing my concerns. However, most of the reported 3.92 dB improvement comes from finer scaling, with only 0.32 dB attributable to the codebook itself. The proposed LUT-based execution also remains unimplemented, and its efficiency on edge hardware has not been verified. These issues still leave me with some doubts about the method's specific contribution and practical deployment claims.

Considering the substantial improvements made during the rebuttal, I've decided to raise my rating to 2, although some concerns remain.

Reviewer_HuEfOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper does W4A4 PTQ for 3D-VAE convolutions, using a Hadamard rotation along input channels plus a per-voxel norm/direction split and a Lloyd-Max codebook. It reports large PSNR gains on the Wan 2.2 and LTX VAEs.

Critical strengths

VAE decoding is a real latency bottleneck in video generation, and this target is under-explored.

Critical weaknesses

Baselines all use per-tensor activation scales while the method uses per-voxel scales. The paper's own Tab. 4 shows granularity accounts for +3.33 dB and the codebook only +0.59 dB, so the comparison is unfair, and channel-wise rotation plus Proposition 1 are standard. There are no measured latency or memory results, the codebook description contradicts itself (16 vectors in R^m vs. per-coordinate quantization), and the downstream FVD seems to use only a handful of videos.

Questions for authors

What do the baselines get with per-token or group-wise scales? What is the actual codebook form, what is the measured speed on real hardware, and how many videos are used for FVD?

1 comment

Thank you for the detailed review. It led to several corrections and clarifications in the revision. We quote each point briefly.

"Baselines all use per-tensor activation scales" / "What do the baselines get with per-token or group-wise scales?" Table 3 now re-evaluates the baselines with one activation scale per voxel, the per-token analogue for convolutions (weights get one scale per kernel tap and output channel). PSNR, per-tensor → per-voxel:

MethodPer-tensorPer-voxel
RTN17.85 dB19.70 dB
SVDQuant (R = 32)24.53 dB26.07 dB
Rotated uniform grid (QuaRot-style)29.49 dB33.09 dB
TurboQuantConv–33.41 dB

With group-wise scales (one per 32 rotated channels, same rotation), the uniform grid reaches 35.13 dB (Table 4). The Table 1 caption and §4.1 now state that Table 1 uses per-tensor scales.

"granularity accounts for +3.33 dB and the codebook only +0.59 dB". You are right. Correcting the full-channel uniform entry of Table 4 (32.82 → 33.09 dB, its value under the TokenBench protocol) changes the split to +3.60 dB from finer scales and +0.32 dB from the codebook. QuaRot already rotates, so rotation is not part of this gap. Measured against no rotation at the same per-voxel scales, rotation is the largest single factor: +13.39 dB with the uniform grid and +19.37 dB with the codebook (§4.4). §4.2 and §4.4 now state that most of the gain over QuaRot comes from scale granularity, and §4.4 reports the rotation gain separately.

"channel-wise rotation plus Proposition 1 are standard". We agree that Hadamard rotation and its orthogonal invariance are standard (QuIP, QuaRot, SpinQuant), and that Proposition 1 is elementary. Our contribution is where the rotation sits in a convolution:

  • What we rotate. Each voxel's channel vector is rotated once, before im2col, instead of each unfolded patch. For a 3×3×3 kernel, every voxel appears in 27 patches.
  • What this enables. The rotation stays an O(mlog⁡m)O(m\log m) per-voxel pass that can be fused with the preceding normalization, and an implicit-GEMM kernel can contract integer codes with per-voxel and per-tap scales. Our fused A100 kernels (Table 6) depend on this.

"no measured latency or memory results" / "measured speed on real hardware?" The revision adds A100 measurements (Table 5, Appendix B.3):

  • 480p × 81 frames. Full WAN 2.2 decoding takes 2.01 s with fused W4A4 kernels (uniform grid, INT4). That is against 5.66 s for cuDNN BF16 (2.82×) and 4.38 s for a BF16 kernel on the same call path (2.18×). With Lloyd–Max levels (INT8 codes) it takes 3.35 s (1.69× / 1.31×).
  • 720p × 81 frames. cuDNN BF16 runs out of memory; we decode in 4.65 s.
  • Kernel level. Over all 188 decoder convolutions at 480p × 17 frames, the INT4 kernel takes 310.4 ms and its fused quantizer adds 11.0 ms, vs. 612.1 ms for cuDNN (1.90×).
  • Memory. Weights take 265 MiB as 4-bit codes vs. 1056 MiB in BF16. Peak memory is 6.9 GiB vs. 7.7 GiB for our BF16 kernel, not 4× lower, because the weights are only 1.0 GiB of that peak. The other 6.7 GiB, activations within a decoding step and the causal-convolution state carried between chunks, is the same for both kernels. Each W4A4 kernel quantizes its input to 4-bit codes internally, but the tensors that persist between layers stay in floating point, since normalization, SiLU, residual additions and upsampling need them (Appendix B.3).
  • Scope. Edge NPUs were not measured.

"the codebook description contradicts itself" / "What is the actual codebook form?" The levels are scalar. After rotation, each voxel's vector is split into one radius and a unit direction. Every coordinate of the direction is quantized independently to one of 16 Lloyd–Max levels of N(0,1/m)\mathcal N(0,1/m): one 4-bit index per coordinate, one scale per vector. This is the quantizer in all experiments, and the "16 vectors in Rm\mathbb R^m" wording was corrected. §3.3, Algorithm 1 and Appendix A.3 now describe it as the product codebook Cbm\mathcal C_b^m, which is also why the lookup table of §5 works per coordinate.

"FVD seems to use only a handful of videos" / "how many videos are used for FVD?" The submitted Table 2 used 6 videos. We replaced it with a full rerun on 256 VBench-I2V pairs (704×1280, 81 frames). FD is measured against BF16 videos of the same inputs with a different seed.

MethodFVDFID
BF16 (seed-to-seed floor)98.516.14
TurboQuantConv125.425.32
QuaRot221.034.81
SVDQuant224.844.44

We hope these changes address your concerns and are happy to answer follow-up questions.

Reviewer_n1trOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TurboQuantConv, a channel-isolated incoherence processing and quantization framework for multidimensional convolutions. By aligning outlier suppression with convolutional receptive-field geometry, it effectively mitigates spatiotemporal outliers and enables high-fidelity quantization without fine-tuning or retraining.

Critical strengths
  1. The research problem is important and well-motivated. As generative backbones continue to accelerate, the encoding and decoding costs of 3D VAEs, along with low-bit quantization errors, are becoming increasingly significant bottlenecks. Video VAEs involve channel, spatial, and temporal dimensions, making quantization schemes designed for linear layers or 2D models unsuitable for direct adoption. Therefore, this problem warrants dedicated investigation.
  2. The channel-isolated rotation design is well-motivated and tailored to the problem. Rather than mechanically applying rotations to the full unfolded convolutional patches, the authors exploit the fact that the same voxel is reused across multiple sliding windows and apply a shared rotation only along the channel dimension. This preserves the convolutional inner-product relationships while avoiding redundant processing of the same activation. The method's motivation is therefore well aligned with the structural properties of 3D convolutions.
  3. The main experiments support the effectiveness of the proposed method. The evaluation covers two 3D VAEs with different scales and architectures, two datasets, and complementary metrics including PSNR, LPIPS, and CIE76. TurboQuantConv achieves substantial and consistent improvements across these evaluations.
Critical weaknesses
  1. The attribution of contributions could be clarified further. The tables suggest that fine-grained scaling contributes substantially to the results. Thus, my understanding is that the paper's most robust contribution lies in the overall combination of channel-isolated rotation, dynamic fine-grained scaling, and distribution-aware quantization, rather than in any single component independently accounting for the full improvement. Could the authors more precisely delineate the novelty of each component and clarify their respective contributions?
  2. The generalizability of the method could be evaluated more thoroughly. The current evaluation on two models is more convincing than a single-model study, but the experiments primarily focus on short-video and low-resolution reconstruction. Further experiments are needed to establish its generalizability to longer videos, higher resolutions, and other causal and non-causal VAEs.
  3. Beyond the overall FVD, FID, and VBench scores, I am curious whether the quantization errors have a more pronounced impact on color, fine details, or temporal consistency. Are these observations consistent with the paper's motivation regarding spatiotemporal activation bursts and channel-wise rotation?
Questions for authors

See Weaknesses.

2 comments

Thank you for the positive assessment and for your questions. Changes are highlighted in the revised PDF.

Weakness 1 — "more precisely delineate the novelty of each component and clarify their respective contributions". We agree that the robust contribution is the combination. The revision now measures each part under matched conditions (Table 3, §4.4; WAN 2.2, all 492 TokenBench clips):

  • Channel-isolated rotation. With one activation scale per voxel, R-FWHT raises PSNR from 19.70 to 33.09 dB under a uniform grid, and from 14.04 to 33.41 dB under the Lloyd–Max codebook. It is better on all 492 clips. Without rotation, no tested configuration exceeds 26.07 dB.
  • Per-voxel scaling. Replacing QuaRot's per-tensor scales with one activation scale per voxel (and one weight scale per kernel tap) on the rotated features adds +3.60 dB (29.49 → 33.09 dB).
  • Lloyd–Max codebook. It adds +0.32 dB over a uniform grid at identical rotation, scales and bits (33.09 → 33.41 dB; LPIPS 0.0035 → 0.0029).

Of the +3.92 dB over QuaRot, which already rotates, finer scales therefore account for +3.60 dB and the codebook for +0.32 dB.

Channel isolation is what makes rotation and per-voxel scaling compatible with convolution. Each voxel's channel vector is rotated and scaled once, before im2col, so a fused implicit-GEMM kernel can contract the integer codes directly and rescale each kernel tap. The measured A100 kernels in §5 rely on this structure.

Weakness 2 — "longer videos, higher resolutions, and other causal and non-causal VAEs".

  • Reconstruction (Table 1) uses 17-frame 240p clips from TokenBench and DAVIS-2017.
  • Generation. The revised downstream evaluation (Table 2) runs the quantized VAE inside Wan 2.2 TI2V-5B at 704×1280 and 81 frames, on 256 VBench-I2V inputs. TurboQuantConv reaches FVD 125.4, against 221.0 for QuaRot and a BF16 seed-to-seed floor of 98.5.
  • Decoding at 720p. Our A100 kernels also decode 720p × 81 frames (Table 5), but we report only latency at that size.

We agree that reconstruction on longer, higher-resolution clips and on more VAE families would strengthen the paper.

Weakness 3 — "a more pronounced impact on color, fine details, or temporal consistency".

  • Color. In our results, coarse activation scaling degrades color and fine detail together. CIE76 is 4.82 with QuaRot's per-tensor scales (Table 1), 2.76 with per-voxel scales on the rotated features (Table 4) and 2.42 with the Lloyd–Max codebook (Table 1). PSNR moves 29.49 → 33.09 → 33.41 dB over the same steps.
  • Link to our motivation. A per-tensor scale is set by the few voxels with burst activations, which leaves the other voxels too few levels. Per-voxel radii decouple them, and the rotation spreads channel outliers within each voxel (+13.39 dB at per-voxel scales).
  • Fine detail. It follows the same ranking (LPIPS 0.0066 → 0.0035 → 0.0029, Table 3).

We hope this answers your questions and are happy to discuss further.

Thank you for the authors' response. My concerns regarding the generalizability of the method and its practical performance have been partly addressed. I will keep my score unchanged.

Comments on the paper

1 comment
Area_Chair_QHtwOpenReview ↗

Please join the discussion before it closes

Thank you to the authors for the rebuttal and to the reviewers for engaging in the discussion. As the discussion period draws to a close, I encourage everyone to keep the conversation going. I’d appreciate a brief final assessment from each reviewer. After reading the rebuttal and the discussion so far, what is the main reason you lean toward acceptance or rejection, and what evidence supports that view? Please note which concerns the rebuttal resolved and which remaining concerns, if any, are decisive for your recommendation.

If your assessment differs from another reviewer’s, please point to the specific claim or evidence you interpret differently. Authors are welcome to clarify any factual points that remain open. This will help us identify where we agree and understand the reasons for any remaining disagreement.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.