acceptodds
Under review as a conference paper at ICLR 2027

BASS: A Block-wise Assessment of Sensitivity and Selective Smoothing Framework for Quantization of Branch-Conditioned Video Diffusion Transformers

Abstract

Video Diffusion Transformers (VDiTs) have substantially advanced high-resolution and controllable video generation, but their considerable computational and memory costs make practical deployment prohibitively expensive. Post-training quantization (PTQ) has proven to be an effective method for reducing memory usage and computational costs, yet existing methods primarily target VDiTs with a single denoising backbone and suffer from accuracy degradation when applied to branch-conditioned architectures with auxiliary control signals. Wan2.2-Animate-14B is a representative example: its auxiliary face and body adapters inject motion and identity conditioning signals into the main DiT, and its conditioning pathways exhibit quantization sensitivities that differ markedly from those of the backbone. Applying a uniform low-bit configuration across these components can therefore corrupt the conditioning signals, leading to motion degradation and identity drift. To address this problem, we propose BASS, a Block-wise Assessment of Sensitivity and Selective Smoothing framework for mixed-precision post-training quantization of branch-conditioned VDiTs. BASS identifies and protects sensitive layers across different branches, particularly those within the adapters, with minimal manual intervention. We validate BASS on advanced branch-conditioned VDiTs, using a mixed-precision policy combining W8A8 and W4A4, and show that it preserves motion fidelity, identity consistency, and overall visual quality under a mixed low-bit quantization policy. Furthermore, we implement efficient GPU kernels to achieve practical memory savings and inference speedups.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
video generation, low-bit quantization, diffusion model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_g2NFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BASS constructs an offline mixed-precision policy for video diffusion models with auxiliary conditioning branches. It combines selective smoothing, sensitivity measured at enclosing block outputs, and a greedy allocation of W4A4 and W8A8 execution across structural groups and timestep contexts. Experiments on two Wan models report improved fidelity to BF16 generations and measured memory and latency savings.

Critical strengths
  1. Treating the conditioning branches as part of precision allocation is a useful problem to study. It is sensible to assess adapters together with the backbone, since their importance to control need not follow their parameter count.
  2. The reported improvements in fidelity to BF16 are substantial on Wan-Animate. The TikTok and Fashion UBC results agree across several paired metrics, and the visual examples support preservation of the reference appearance better than the reported baseline configurations.
  3. The smoothing and random-selection ablations are informative. Selective smoothing outperforms applying smoothing everywhere, while sensitivity-based allocation improves over random selection at the stated budget. These results give useful initial support for the complete recipe.
  4. The Wan-VACE results in the appendix add a useful second conditioning architecture. The paper also describes an actual GPU execution path and measures end-to-end latency, which helps assess whether the quantization recipe can be used in practice.
Critical weaknesses
  1. The main comparisons do not establish equivalent compression costs. Uniform W8A8 is also slightly faster than BASS in Figure 4, but its quality is not reported. I would like to see whether BASS offers a better quality and memory tradeoff than this simple alternative and comparably costly mixed-precision baselines.
  2. The allocation cost is defined as byte savings, but its relation to resident memory is unclear when one layer uses different transformed weights across timestep contexts. The reported memory reduction needs accounting for cached weight versions and FP8 expansion buffers. Otherwise the optimized budget and measured deployment benefit remain disconnected.
  3. The ablations support selective smoothing and sensitivity-based allocation, but leave the central structural choices less clear. In particular, keeping adapters uniformly at higher precision and measuring sensitivity at layer outputs are missing controls. These would help show whether the gains come from the proposed branch-aware block assessment.
  4. The evaluation primarily measures agreement with BF16 generations and general video quality. It does not directly test reference-identity preservation or adherence to the driving pose, despite these motivating the branch-conditioned setting. Targeted identity and motion-control measurements are needed to establish that the most important conditioning behavior survives compression.
Questions for authors
  1. Which quality results correspond to the ten-step configuration used for timing? The appendix describes twenty steps, so please clarify how the allocation is mapped or recalibrated for the reported deployment workload.
  2. How many clips enter the evaluation, and were calibration and budget selection separated from those clips? The small VACE differences are especially difficult to interpret without these counts and the corresponding variability.
  3. What is the offline profiling cost relative to the evaluated baselines? Measuring groups separately across timestep contexts could make policy construction expensive, even if the resulting policy is fixed during inference.
  4. How does the block-output sensitivity measure differ from that in 6Bit-Diffusion? The application to control branches is interesting, and a clearer account of what this assessment adds would sharpen the contribution.
4 comments

We thank the reviewer for the thoughtful and constructive feedback. Your comments have guided our supplementary analyses and revisions. We provide detailed responses to each weakness and question below.


W1: The main comparisons do not establish equivalent compression costs. Uniform W8A8 is also slightly faster than BASS in Figure 4, but its quality is not reported.

A1: We thank you for the catch on the missing report of W8A8. We report the performance of W8A8 and compare it with our BASS as below:

Table 1. Uniform W8A8 and BASS on TikTok and Fashion UBC.

DatasetMethodSubject Consistency ↑Background Consistency ↑Aesthetic Quality ↑Imaging Quality ↑Temporal Flickering ↑Motion Smoothness ↑VBench Average ↑LPIPS ↓FVD ↓PSNR ↑SSIM ↑
TikTokUniform W8A80.90020.94740.41360.45450.98860.99230.78270.06234.1025.560.93
TikTokBASS0.91010.95090.41320.45300.98960.99290.78500.0687.6826.430.94
Fashion UBCUniform W8A80.96810.95640.47320.58510.99720.99580.82930.0124.6932.560.96
Fashion UBCBASS0.98910.97590.47110.58380.99870.99620.83580.0117.0432.820.97

On both datasets, BASS outperforms W8A8 mainly in Subject Consistency, Background Consistency and Temporal Flickering, with similar results in Aesthetic Quality and Imaging Quality. For frame-level agreement metrics, BASS performs better than W8A8. We also conduct experiments and show the BASS's advantage over W8A8 in motion and identity detection metrics, you can check it in A4 Table 3.

What's more, we appreciate you point out the missing compression cost clarification in our paper. For mixed-precision baselines in our paper, we set them with the same actual W4A4/W8A8 ratio as BASS, so the results in Table 1 in our paper are the quality results of comparably costly mixed-precision baselines. We add this clarification in our revised Appendix A.3.


W2: The allocation cost is defined as byte savings, but its relation to resident memory is unclear when one layer uses different transformed weights across timestep contexts.

A2: Thank you for pointing out this distinction. Our allocation cost measures the quantized-weight bit-width cost, normalized to uniform W8A8. It provides a common precision budget for comparing allocation policies. We agree that it is not an reflection of the real resident or peak GPU memory.

Our implementation manages GPU memory through CPU offloading, stage-wise unloading of inactive modules, and on-demand transfer of quantized weights for computation. The low-precision transform is shared across timestep contexts for each physical layer, avoiding a separate transformed weight copy for every timestep.


W3: In particular, keeping adapters uniformly at higher precision and measuring sensitivity at layer outputs are missing controls. These would help show whether the gains come from the proposed branch-aware block assessment.

A3: We conduct a comparison experiment as you suggest. For the comparator, we fix the adapter at W8A8 and only conduct BASS's quantization mechanism on the backbone. The comparator shares a same overall ratio of W4A4/W8A8 with BASS. The results are shown below:

Table 2. Fixed-adapter versus BASS on TikTok and Fashion UBC.

DatasetMethodSubject Consistency ↑Background Consistency ↑Aesthetic Quality ↑Imaging Quality ↑Temporal Flickering ↑Motion Smoothness ↑VBench Average ↑LPIPS ↓FVD ↓PSNR ↑SSIM ↑
TikTokAdapter W8A8 + backbone MP0.90650.94450.40040.42960.98960.99290.77730.07175.5725.420.95
TikTokBASS0.91010.95090.41320.45300.98960.99290.78500.0687.6826.430.94
Fashion UBCAdapter W8A8 + backbone MP0.97690.97020.46810.58020.98960.99620.83020.0232.1632.330.98
Fashion UBCBASS0.98910.97590.47110.58380.99870.99620.83580.0117.0432.820.97

Compared with the fixed-adapter policy, BASS shows better performance of VBench Average on both datasets. Moreover, BASS improves LPIPS and PSNR, while reducing FVD. This comparison provides evidence for the effectiveness of branch-aware assessment mechanism.


W4: Targeted identity and motion-control measurements are needed to establish that the most important conditioning behavior survives compression.

A4: Thank you for the suggestion on conducting experiments to test the preservation of identity and motion. We add experiments for BF16, uniform W8A8, BASS, and the three adapted baselines on TikTok dataset. The TikTok clips contain severe body movements, making them a challenging test of motion adherence and identity detectability in generated videos. We evaluate identity against the input reference image and pose against the frame-aligned driving control. The mixed-precision baselines (ViDiT-Q, Q-VDiT) adopt the same ratio of mixed W4A4/W8A8 quantization as BASS.

CSIM is the cosine similarity of ArcFace identity embeddings on detected faces. AKD is the mean Euclidean error of jointly detected keypoints in pixels, reported separately for the body and hands. MKR is the ratio of valid driving keypoints missing from the generated video. Face coverage reports the ratio of generated frames with a detected face.

Table 3. Identity and motion preservation on TikTok.

MethodCSIM ↑Body AKD ↓ (px)Hand AKD ↓ (px)Body MKR ↓ (%)Hand MKR ↓ (%)Face coverage ↑ (%)
BF160.48897.8817.661.3711.9599.48
W8A80.46227.9114.063.2411.8699.22
ViDiT-Q0.46597.9814.153.8813.1899.48
Q-VDiT0.46637.9113.742.9811.9999.22
SmoothQuant0.011930.5742.5541.9859.1810.91
BASS0.46647.7115.082.1711.1299.48

Among all baselines, BASS performs better in the similarity of identity and detected body keypoints while falls a little behind in detected hand keypoints. What's more, BASS's identity similarity as well as body and hand error remain closely to BF16, demonstrating the preservation of motion and identity.


Q1: Which quality results correspond to the ten-step configuration used for timing? The appendix describes twenty steps, so please clarify how the allocation is mapped or recalibrated for the reported deployment workload.

A5: We apologize for the mismatch of our experiment configurations and the configuration used for timing. All of the proposed quality results are conducted with a 20-step inference procedure, which is the default setting of Wan2.2-Animate. The timing experiment is conducted under 20-step configuration and stops at step 10 for a quick feedback. Since the inference time cost for each step remains stable, it can also reflect the speed of proposed methods. We report the actual allocation of the policy used in our supplemental experiment at target cost ratio 0.85. The quantizable scope contains 480 backbone Linear layers and 24 adapter Linear layers.

What's more, we report the W4A4 ratio of actual layer calls in the whole 20-step inference.

ComponentW4A4 ratio of actual layer calls
DiT backbone3010/9600 (31.35%)
Adapter90/480 (18.75%)
Overall3100/10080 (30.75%)

Q2: How many clips enter the evaluation, and were calibration and budget selection separated from those clips?

A6: For each dataset, we choose 100 clips to enter the evaluation. The 5 clips used for calibration and budget selection is separated from the evaluation clips. We think the number of the clips we use for evaluation is large enough to avoid variability in results.


Q3: What is the offline profiling cost relative to the evaluated baselines?

A7: We thank you for the concerns on the offline profiling cost. Among the baselines, the cost of the offline profiling procedure of BASS is similar to ViDiT-Q which also conducts an assessment of sensitivity, larger than SmoothQuant and Q-VDiT. Though the offline profiling stage costs high, it is conducted once or few times for practical video generation.


Q4: How does the block-output sensitivity measure differ from that in 6Bit-Diffusion?

A8: Thank you for pointing out this connection. First, 6Bit-Diffusion defines sensitivity by quantizing one linear layer at a time, while our BASS evaluates structural groups, including simultaneous Q/K quantization, at the enclosing block output. This directly measures the combined perturbation of coupled projections through attention and subsequent block operations.

Second, 6Bit-Diffusion fits a predictor using the preceding timestep’s block input–output difference to route activation precision online. BASS aggregates calibration errors and then combines these risk estimates with cost savings to construct an offline policy under a shared backbone–adapter budget. This provides a measured basis for deciding where low precision is acceptable across conditioning and backbone pathways, without uniformly protecting or compressing all adapters. Inference executes the resulting group–context assignments through policy lookup and no extra calculation.


Thank you for the response and the additional comparisons. The W8A8 results clarify the quality trade-off, and the identity and pose results address my concern about conditioning. The fixed-adapter comparison supports joint precision allocation across the backbone and adapters. The benefit of measuring sensitivity at the block output rather than at individual layers remains unclear. I will maintain my score.

We are encouraged by your acknowledgement of our additional comparisons, and appreciate you for pointing out the missing illustration. We would like to clarify the advantages of adopting block-output sensitivity over layer-output sensitivity.

The distinction is between measuring the quantization error introduced at a layer and measuring its effect after propagation through the remaining operations within the block. Let h denote an intermediate layer output, δh its quantization error, and F the subsequent computation within the block. Layer-output error measures ‖δh‖², whereas block-output error measures ‖F(h + δh) − F(h)‖². Under a first-order approximation, the latter becomes ‖Jδh‖², where J is the Jacobian of F evaluated at h. This captures how subsequent computations can increase or reduce the effect of quantization errors on the block output. Consequently, similar layer-output errors can have different effects on the representation passed to the next block.

This distinction is particularly relevant to attention: errors in Q/K projections affect attention logits, but their effect on the block output also depends on softmax, value aggregation, and subsequent projections. Measuring at the enclosing block output evaluates the error after these operations. We therefore adopt block-output error in BASS as a computationally tractable sensitivity proxy.

Reviewer_PrZSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BASS is a mixed-precision post-training quantization framework for video diffusion models with auxiliary conditioning branches. It allocates W4A4 and W8A8 precision across structural groups, branches, and timestep buckets. Experiments show improved agreement with BF16 generations and higher VBench averages than the evaluated quantization baselines. The paper also reports 3.09× lower peak GPU memory and 1.74× faster inference than BF16.

Critical strengths
  1. The design addresses a consequential architectural problem. Conditioning adapters and the denoising backbone serve different functions, making uniform precision allocation potentially wasteful or damaging. Jointly evaluating query–key projections and measuring distortion after the enclosing block are sensible ways to account for attention and residual interactions. These choices give the framework a coherent technical motivation.

  2. The reported preservation of BF16 outputs is promising across differnt architectures.

Critical weaknesses
  1. The paper does not establish a clear, comparable quality–resource trade-off.

The main configuration uses a target cost ratio of 0.85 relative to W8A8, but the actual W4A4 allocation and its relationship to deployed memory are not reported. Section 4.3 defines savings for individual group–context pairs, whereas resident weight storage is shared across timesteps. A group that switches precision across buckets may require multiple packed weight representations, so additive savings across contexts do not directly establish memory savings. The relationship between this objective and the reported peak-memory reduction therefore remains unclear.

More importantly, the mixed-precision baselines are not shown to have matched byte budgets or comparable effective precision. Consequently, their quality differences cannot be confidently attributed to better allocation. Figure 4 also shows uniform W8A8 running slightly faster than BASS, but its quality and memory results are absent. Without that comparison, it is difficult to determine what practical advantage the additional W4A4 machinery delivers.

  1. The evidence does not sufficiently validate the branch-aware sensitivity criterion or its claimed protection of conditioning.

Equation (7) measures the effect of quantizing one candidate on its enclosing block. This captures interactions within that block, but it does not establish that normalized errors from adapter blocks and backbone blocks are comparable predictors of final generation damage. Nevertheless, these scores are directly compared in the global allocation policy, which then applies many quantization decisions simultaneously.

Table 2 establishes an advantage over random allocation, but does not isolate the value of block-output scoring, timestep-dependent allocation, or learned protection of adapters. In particular, a simple policy that retains adapters at W8A8 is an important missing comparator. Furthermore, agreement with BF16 outputs is appropriate for quantization evaluation, but the reported metrics do not directly test the motivating claims about pose adherence, facial motion, and reference identity. Temporal consistency and motion smoothness can remain high even when the conditioning is followed incorrectly. Thus, the results demonstrate improved reconstruction more clearly than they demonstrate the proposed explanation for that improvement.

Questions for authors
  1. Clarification of the cost objective: What exactly do C0C_0 and ΔCi\Delta C_i count? How are weights reused across timestep buckets and multiple precision/smoothing caches accounted for?

  2. Clarification of comparative efficiency: Are ViDiT-Q, Q-VDiT, and BASS evaluated under matched resource budgets? What are their actual W4A4 proportions, stored model sizes, and peak-memory requirements? What quality does uniform W8A8 achieve under the same generation settings, given its slightly better latency in Figure 4?

3 comments

We deeply appreciate the reviewer for the detailed and profound feedback. We have carefully considered your insightful comments on memory cost and the comprehensiveness of our experiments, and we provide our detailed responses below.


W1: The main configuration uses a target cost ratio of 0.85 relative to W8A8, but the actual W4A4 allocation and its relationship to deployed memory are not reported. More importantly, the baselines are not shown to have matched byte budgets or comparable effective precision.

A1: We thank you for pointing out the missing reports and experiment configurations. Firstly, we report the actual allocation of the policy used in our supplemental experiment at target cost ratio 0.85. The quantizable scope contains 480 backbone Linear layers and 24 adapter Linear layers. We report the W4A4 ratio of actual layer calls in the whole 20-step inference.

ComponentW4A4 ratio of actual layer calls
DiT backbone3010/9600 (31.35%)
Adapter90/480 (18.75%)
Overall3100/10080 (30.75%)

Secondly, we would like to claim that the target ratio specifies a constraint on the quantized-weight cost. Under this constraint, BASS seeks a precision allocation that minimizes the estimated quantization risk. This cost is not a precise calculation of resident or peak memory. We agree that a switching layer may retain both packed representations, and in the settings in our paper we find that among all 504 layers, 304 layers only use W8A8, 108 layers only use W4A4 and the rest 92 switch during time buckets. Thus, for about 18.25% of the layers, BASS needs to store both W4A4/W8A8 precision.

Thirdly, we appreciate you for pointing out the missing compression cost clarification in our paper. For mixed-precision baselines in our paper, we set them with the same actual W4A4/W8A8 ratio as BASS, so the results in Table 1 in our paper are the quality results of comparably costly mixed-precision baselines. We add this clarification in our revised Appendix A.3.

W2: The evidence does not sufficiently validate the branch-aware sensitivity criterion or its claimed protection of conditioning.

A2: We thank you for reminding us of the missing evidence. Firstly, evaluating end-to-end generation damage for every candidate group–context decision would require repeatedly running the remaining denoising trajectory, making exhaustive assessment prohibitively expensive. We therefore use Eq. (7) as a computationally surrogate for quantization sensitivity, measuring the perturbation induced by a candidate at its enclosing block’s output.

Secondly, we conduct the comparison experiment of fixed-adapter and BASS and report our results as follows:

Table 1. Fixed-adapter versus BASS on TikTok and Fashion UBC.

DatasetMethodSubject Consistency ↑Background Consistency ↑Aesthetic Quality ↑Imaging Quality ↑Temporal Flickering ↑Motion Smoothness ↑VBench Average ↑LPIPS ↓FVD ↓PSNR ↑SSIM ↑
TikTokAdapter W8A8 + backbone MP0.90650.94450.40040.42960.98960.99290.77730.07175.5725.420.95
TikTokBASS0.91010.95090.41320.45300.98960.99290.78500.0687.6826.430.94
Fashion UBCAdapter W8A8 + backbone MP0.97690.97020.46810.58020.98960.99620.83020.0232.1632.330.98
Fashion UBCBASS0.98910.97590.47110.58380.99870.99620.83580.0117.0432.820.97

Compared with the fixed-adapter policy, BASS shows better performance of VBench Average on both datasets. Moreover, BASS improves LPIPS and PSNR, while reducing FVD. This comparison provides evidence for the effectiveness of branch-aware assessment mechanism.

Thirdly, we also conduct experiments measuring the preservation of identity and motion. We add experiments for BF16, uniform W8A8, BASS, and the three adapted baselines on TikTok dataset. The TikTok clips contain severe body movements, making them a challenging test of motion adherence and identity detectability in generated videos. We evaluate identity against the input reference image and pose against the frame-aligned driving control. The mixed-precision baselines (ViDiT-Q, Q-VDiT) adopt the same ratio of mixed W4A4/W8A8 quantization as BASS.

CSIM is the cosine similarity of ArcFace identity embeddings on detected faces. AKD is the mean Euclidean error of jointly detected keypoints in pixels, reported separately for the body and hands. MKR is the ratio of valid driving keypoints missing from the generated video. Face coverage reports the ratio of generated frames with a detected face.

Table 2. Identity and motion preservation on TikTok.

MethodCSIM ↑Body AKD ↓ (px)Hand AKD ↓ (px)Body MKR ↓ (%)Hand MKR ↓ (%)Face coverage ↑ (%)
BF160.48897.8817.661.3711.9599.48
W8A80.46227.9114.063.2411.8699.22
ViDiT-Q0.46597.9814.153.8813.1899.48
Q-VDiT0.46637.9113.742.9811.9999.22
SmoothQuant0.011930.5742.5541.9859.1810.91
BASS0.46647.7115.082.1711.1299.48

Among all baselines, BASS performs better in the similarity of identity and detected body keypoints while falls a little behind in detected hand keypoints. What's more, BASS's identity similarity as well as body and hand error remain closely to BF16, demonstrating the preservation of motion and identity.


Q1: Clarification of the cost objective.

A3: Specifically, C₀ counts the weight payload of all quantizable backbone and adapter layers under the W8A8 reference. Each weight contributes one byte. For a candidate i = (g, c), ΔCᵢ counts the reduction when group g switches from W8A8 to W4A4 in context c: half a byte per weight. As we state in A1 above, this cost is not a precise calculation of resident or peak memory, but a constraint for BASS to seek a precision allocation that minimizes the estimated quantization risk.

Additionally, in the settings in our paper we find that among all 504 layers, 304 layers only use W8A8, 108 layers only use W4A4 and the rest 92 switch during time buckets. During inference, BASS only stores multiple precision for those who switch between buckets. Thus, for about 18.25% of the layers, BASS needs to store both W4A4/W8A8 precision. Smoothing-scale metadata has a small storage footprint relative to the weight tensors.


Q2: Clarification of comparative efficiency.

A4: Again we appreciate you for point out the missing clarification of experiment configurations. As we state in A1, for mixed-precision baselines in our paper, we set them with the same actual W4A4/W8A8 ratio as BASS, so the results in Table 1 in our paper are the quality results of comparably costly mixed-precision baselines.

We report the performance of W8A8 as below:

Table 3. Uniform W8A8 and BASS on TikTok and Fashion UBC.

DatasetMethodSubject Consistency ↑Background Consistency ↑Aesthetic Quality ↑Imaging Quality ↑Temporal Flickering ↑Motion Smoothness ↑VBench Average ↑LPIPS ↓FVD ↓PSNR ↑SSIM ↑
TikTokUniform W8A80.90020.94740.41360.45450.98860.99230.78270.06234.1025.560.93
TikTokBASS0.91010.95090.41320.45300.98960.99290.78500.0687.6826.430.94
Fashion UBCUniform W8A80.96810.95640.47320.58510.99720.99580.82930.0124.6932.560.96
Fashion UBCBASS0.98910.97590.47110.58380.99870.99620.83580.0117.0432.820.97

On both datasets, BASS outperforms W8A8 mainly in Subject Consistency, Background Consistency and Temporal Flickering, with similar results in Aesthetic Quality and Imaging Quality. For frame-level agreement metrics, BASS performs better than W8A8. We also conduct experiments and show BASS's advantage over W8A8 in motion and identity detection metrics, you can check it in Table 2 in A2.


Reviewer_MKs6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BASS, a block-wise assessment of sensitivity and selective smoothing framework for video diffusion. Post-training quantization (PTQ) is an effective method to reduce the memory footprint and computation cost. However, applying PTQ to the branch architecture hurts performance. BASS, a mixed-policy PTQ framework, adaptively allocates quantization configurations across structural groups, branches, and denoise steps. Extensive experiments using various branch architectures show better performance. A custom kernel is also implemented for practical use.

Critical strengths
  • Studying PTQ for branch architectures in video diffusion is an important topic since it accelerates synthesis.
  • Reasonable performance improvement with actual speedup via a custom kernel.
Critical weaknesses
  • While effective, BASS seems incremental. First, it creates an execution map. Then smoothing is only applied when it helps (i.e., Gl>τGG_l>\tau_G), and block-sensitive scores are computed. These results are used in stage 3. The quality of the method seems to depend on the execution map.
  • The motivation presented in Section 3 is weak. A direct experiment is preferred. For example, provide an analysis of why applying PTQ to the adapter hurt the performance.
  • In Figure 4, BASS achieves similar latency to W8A8. I am curious about the performance of the baseline or adapter kept at W8A8.
  • The relationship between adapter sensitivity and applying low-bit quantization to the adapter, leading to poor performance, is weak. There is no references nor counter example.
  • Lack of reporting on which layer/group uses W4A4. Are there any trends such as earler layer prefer W4A4 and vice versa?
  • BASS requires many hyperparameters, which may hinder practical use when applying to other datasets and architectures.
Questions for authors

See weaknesses. Besides that, I have several questions.

  1. Can the author provide a direct experiment/analysis that explains low bit quantization to adapter hurt performance? And link it to layer/group sensitivity.
  2. Provide the performance of baselines at W8A8.
  3. In the Appendix, 81 frames are used for offline calibration. How does this number affect the performance? Do fewer frames reduce performance, and do more frames improve performance? If so, when does BASS saturate?
2 comments

We really appreciate the reviewer for the detailed and constructive feedback. Your thoughtful concerns motivate us to further investigate in this work. Below, we provide detailed responses for every weakness and question.


W1: BASS seems incremental since the quality of the method seems to depend on the execution map.

A1: We would like to clarify that the execution map is constructed during offline profiling by tracing the model’s actual execution paths under the specified inference configuration. Thus, it is not a heuristically assumed dependency structure. It records the observed execution structure and provides the structural basis for defining quantization groups and identifying the branch and timestep contexts in which they are evaluated.


W2 & W4 & Q1: Can the author provide a direct experiment/analysis that explains low bit quantization to adapter hurt performance? And link it to layer/group sensitivity.

A2: We response to the second and fourth weakness and the first question who share a lot of common together, and we appreciate the reviewer for pointing out these questions and missing evidence.

Our claim in Section 3.1 is that adapter precision should be determined through explicit sensitivity measurement, not that applying PTQ to an adapter necessarily fails. By stating that the small size of an adapter does not imply lower sensitivity to quantization in Section 3.1, our point is that applying PTQ to an adapter without careful measurement may fail. Figure 1 in our paper provides evidence motivating this claim. With the same input and the same quantization ratio, the baseline adapted to a mixed W8A8/W4A4 policy on Wan2.2-Animate-14B exhibits severe degradation of facial structure and visual details, whereas BASS preserves these details more faithfully relative to BF16. The example illustrates the need to assess quantization decisions carefully in a branch-conditioned architecture.

We apologize for the missing illustration of the relationship between low-bit quantization and layer sensitivity in our paper. ViDiT-Q (Zhao et al., ICLR 2025) observes that layers have varying quantization sensitivity and further shows that quantizing different layer types affects different aspects of generation. [1] This supports the principle that different components should receive different precision allocations according to their measured sensitivity. This principle motivates our treatment of the backbone and adapters in branch-conditioned models. Applying low precision to a sensitive adapter group can damage generation quality, while less sensitive groups may tolerate the same reduction in precision. BASS therefore measures sensitivity for candidate groups in both branches and across timestep contexts, and uses these measurements to guide W8A8/W4A4 allocation. We also add this illustration to our revised paper in Section 3.1.

[1] ViDiT-Q: Efficient and accurate quantization of diffusion transformers for image and video generation. Zhao et al., ICLR 2025.


W3 & Q2: Provide the performance of W8A8.

A3: We thank you for the catch on the missing report of W8A8. We report the performance of W8A8 and compare it with our BASS as below:

Table 1. Uniform W8A8 and BASS on TikTok and Fashion UBC.

DatasetMethodSubject Consistency ↑Background Consistency ↑Aesthetic Quality ↑Imaging Quality ↑Temporal Flickering ↑Motion Smoothness ↑VBench Average ↑LPIPS ↓FVD ↓PSNR ↑SSIM ↑
TikTokUniform W8A80.90020.94740.41360.45450.98860.99230.78270.06234.1025.560.93
TikTokBASS0.91010.95090.41320.45300.98960.99290.78500.0687.6826.430.94
Fashion UBCUniform W8A80.96810.95640.47320.58510.99720.99580.82930.0124.6932.560.96
Fashion UBCBASS0.98910.97590.47110.58380.99870.99620.83580.0117.0432.820.97

On both datasets, BASS outperforms W8A8 mainly in Subject Consistency, Background Consistency and Temporal Flickering, with similar results in Aesthetic Quality and Imaging Quality. For frame-level agreement metrics, BASS performs better than W8A8. We also add the performance of W8A8 in our revised paper. You can check it in Table 1 in our paper.


W5: Lack of reporting on which layer/group uses W4A4. Are there any trends such as earlier layer prefer W4A4 and vice versa?

A4: Thank you for pointing out this missing report. We report the actual allocation of the policy used in our supplemental experiment at target cost ratio 0.85. The quantizable scope contains 480 backbone Linear layers and 24 adapter Linear layers. We report the W4A4 ratio of actual layer calls in the whole 20-step inference.

ComponentW4A4 ratio of actual layer calls
DiT backbone3010/9600 (31.35%)
Adapter90/480 (18.75%)
Overall3100/10080 (30.75%)

Allocation by depth. W4A4 is distributed across the backbone rather than confined to early or late blocks. Dividing the 40 backbone blocks into four equal depth ranges gives the following call-weighted proportions:

Backbone blocksW4A4 calls / total Linear-layer callsW4A4 ratio
0–9820/240034.17%
10–19620/240025.83%
20–29680/240028.33%
30–39890/240037.08%

There is no preference for earlier or later layers in this policy.


W6: BASS requires many hyperparameters, which may hinder practical use when applying to other datasets and architectures.

A5: We appreciate your reasonable concern on the practical application. The main hyperparameters are the cost budget (τ = 0.85), smoothing exponent (α = 0.5), gain threshold (τ_G = 0), risk coefficient (λ_σ = 1). For a new model or a new dataset, we recommend starting with these defaults, recalibrating on samples. We validate the settings in the WAN-VACE experiment in Appendix B. Your concern motivates us to explore the application to other diffusion-transformer architectures in future work.


Q3: The selection of number of frames

A6: Our calibration video length follows the default Wan2.2-Animate setting. Wan2.2-Animate allocates 21 latent temporal positions, one of which is reserved for the reference-image condition. We adopt the default setting to avoid potential negative influence from the settings of the backbone model.


Comments on the paper

1 comment

For clarity and simplicity, we will refer to Reviewer MKs6, PrZS and g2NF as reviewer R1, R2 and R3.

We sincerely thank all the reviewers for thoughtful and constructive feedback. We have carefully considered each comment provided by the reviewers and believe we have responded to all their concerns. In our revised paper, we have added the suggested experiments, additional discussions or evidence. Below we summarize the updates of our paper.


updates of the experimental results during rebuttal

Main text: Add the quality performance of W8A8 in Table 1. (R1, R1, R3)

Main text: Add the experimental result of the ablation study illustrating the contribution of branch-aware mechanism in Table 2. (R2, R3)

Appendix A.4: Report the actual allocation of W4A4 in BASS. (R1, R2, R3)

Appendix B.1: Report the experimental result of motion control and identity metrics. (R2, R3)


updates of the additional discussions or evidence

Main text: Add references to link low-bit quantization to layer sensitivity. (R1)

Appendix A.3 Add clarifications on the experiment settings of baselines. (R2, R3)


From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.