acceptodds
Under review as a conference paper at ICLR 2027

SP-QAT: Spectral-Preserving Quantization-Aware Training for Extreme Low-Bit Vision-Language-Action Models

Abstract

Vision-language-action (VLA) models are rapidly advancing and becoming central to embodied AI, yet their deployment on edge devices remains challenging. Since in VLA model, LLM backbone dominates memory consumption and inference latency, aggressive quantization of this LLM module presents a straightforward solution. Unfortunately, existing VLA quantization techniques mostly operate at 4-bit or higher bit widths and avoid fixed sub-4-bit quantization because of its catastrophic performance collapse, leaving it largely underexplored. To bridge this gap, we propose SP-QAT, an efficient quantization-aware training framework (QAT) that enables fixed 2-bit quantization of the Large Language Model (LLM) backbone in VLA, to our knowledge, for the first time. Through theoretical and experimental analysis, we identify that preserving the cognitive feature spectral subspace is the key to extreme low-bit quantization for the LLM backbone of VLAs. Therefore, we first constrain the weight-update subspace to preserve the pretrained model priors. Second, we propose Soft-Mask Gradients with Severity-Aware Scale Regulation to mitigate outlier-driven step size expansion from causing quantized weights to collapse into noise. Finally, we propose Difficulty-Aware Curriculum Learning with SNR-Guided Static Reweighting to emphasize hard samples and low-noise timesteps, thereby preserving the action-sensitive energy in the cognition feature spectrum. Extensive experiments on SIMPLER with CogACT and show SP-QAT only results in a small drop of compared to FP16 baselines, with 75%–84% memory savings and inference speedup. Code is available as supplementary material.

Primary area:
applications to robotics, autonomy, planning
Keywords:
VLA, Quantization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_pUJpOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents SP-QAT, a QAT framework for extreme low-bit quantization of VLA models, with a particular focus on fixed 2-bit weight quantization of the LLM backbone. The paper first analyzes why directly applying standard LSQ to VLA models causes severe performance degradation at 2-bit precision. The authors attribute the failure to distortion of the singular spectrum of the LLM’s cognition features, especially the loss of energy in the action-sensitive subspace. Based on this diagnosis, SP-QAT introduces three components: Prior-Anchored Low-Rank Quantization, which freezes the pretrained weights and constrains QAT-induced weight updates to a low-rank subspace; Soft-Mask Gradients with Severity-Aware Scale Regulation, which provides non-zero pull-back gradients for clipped outliers and modifies the step-size gradient to alleviate outlier-driven quantization-resolution collapse; and Difficulty-Aware Curriculum Learning with SNR-Guided Static Reweighting, which emphasizes low-noise timesteps and quantization-sensitive hard samples. The method is evaluated on CogACT and π0 using SimplerEnv, including in-domain and cross-domain settings, together with component ablations and real-world Franka Panda experiments. The paper reports a success-rate drop of at most 2.6% in the main in-domain setting; cross-domain and real-world W2A16 results show larger drops (5–6%). It also reports 75%–84% memory savings and a 1.6× inference speedup.

Critical strengths

1.A systematic analysis of why standard QAT fails under extreme low-bit VLA quantization. Through theoretical and experimental analysis, the authors identify that preserving the cognition-feature spectral subspace, particularly the concentration of action-sensitive energy in the singular spectrum, is critical for extreme low-bit quantization of the VLA LLM backbone. The analysis shows that W2A16-LSQ causes spectral flattening of cognition features, with energy leaking from a few dominant directions into many tail directions, while the action-sensitive subspace is selectively degraded. The authors further trace this distortion to three possible causes. These observations provide a coherent motivation for the three corresponding components of SP-QAT. 2.Strong empirical performance across multiple evaluation settings. The empirical results are compelling, particularly under the challenging W2A16 setting. SP-QAT is evaluated on two VLA policies with different action-generation mechanisms, CogACT and π0, and the paper includes in-domain evaluation, zero-shot cross-embodiment evaluation, extensive component/hyperparameter ablations, and real-world Franka Panda experiments. The W2A16 models remain close to their FP16 counterparts in several settings where the evaluated conventional PTQ/QAT methods degrade severely or collapse. The real-world results additionally show that W2A16 CogACT retains a 60% average success rate compared with 65% for FP16 while reducing the reported inference latency from 241 ms to 151 ms.

Critical weaknesses

The paper’s central claim that preserving action-sensitive energy in the cognition-feature spectrum is key to extreme low-bit VLA quantization is well motivated, but the current evidence is not yet sufficient. First, the construction of the action-sensitive subspace is insufficiently described in the main text. Second, the experiments mainly show a correlation between SP-QAT, improved spectral preservation, and improved success, rather than isolating the mechanism. It would strengthen the claim to report action-sensitive energy preservation and effective rank after progressively adding each component, and to examine whether these metrics consistently track the corresponding success-rate improvements.

Questions for authors

1.Clarification on the action-sensitive subspace and spectral evidence. The authors could provide a precise definition and implementation procedure for constructing the action-sensitive subspace.

2.Given that the real-world experiments perform inference on a remote NVIDIA A100 GPU, could the authors provide measurements on a representative edge platform, or otherwise clarify the scope of the edge-deployment claim.

  1. The related-work discussion identifies ActQuant as a sub-4-bit VLA quantization approach and BitVLA as a native 1.58-bit model, but the main empirical comparison focuses on GPTQ, Ω-QVLA, LSQ, and SQIL. A comparison or at least a controlled discussion with ActQuant at comparable BPW would be useful.

4.The Figure 1 caption and the main text are run together “...with ω-QVLA.” is immediately followed by “in one shot without task-level dynamic adjustment...” making the boundary between caption and body text unclear. Please separate them.

  1. The abstract claim of “at most 2.6%” should be scoped to the main in-domain setting, since the cross-domain and real-world W2A16 results in Tables 2 and 4 show larger drops (6.1%, 5.3%, and 5.0%). The authors should clarify the scope of this claim.
3 comments

Thanks for your deep and constructive comments!

Q1: Clarification on the action-sensitive subspace and spectral evidence. The authors could provide a precise definition and implementation procedure for constructing the action-sensitive subspace.

A1: The action-sensitive subspace is defined as the span of the top-k right singular vectors of the Jacobian matrix of the action output with respect to the cognition feature: Ja=∂a∂h\mathbf{J}_a = \frac{\partial \mathbf{a}}{\partial \mathbf{h}}, where a\mathbf{a} denotes the action output by the action head, and h\mathbf{h} is the cognitive feature fed as the input to the action head. It identifies the cognition feature directions most critical to action prediction. In practice, we project both FP16 and quantized cognition features onto this subspace to compute the energy preservation ratio. We have already included the precise definition and implementation procedure in the paper (Sec.3.2).

Q2: Given that the real-world experiments perform inference on a remote NVIDIA A100 GPU, could the authors provide measurements on a representative edge platform, or otherwise clarify the scope of the edge-deployment claim.

A2: We thank the reviewer for this valuable comment. Due to hardware access constraints, we are currently unable to provide end-to-end inference latency measurements on physical edge platforms. Nevertheless, our method produces fixed-bit quantized weights conforming to the standard GPTQ packing format. This design guarantees compatibility with many widely adopted inference backends and hardware accelerators, and cleanly decouples our quantization training algorithm from low-level kernel implementations. We have revised the manuscript to clarify this scope.

Q3: The related-work discussion identifies ActQuant as a sub-4-bit VLA quantization approach and BitVLA as a native 1.58-bit model, but the main empirical comparison focuses on GPTQ, Ω-QVLA, LSQ, and SQIL. A comparison or at least a controlled discussion with ActQuant at comparable BPW would be useful.

A3: We have added controlled comparisons against ActQuant and BitVLA for π0.5\pi0.5 on LIBERO. We evaluate our fixed 2-bit quantized variant alongside ActQuant at 2 bpw and BitVLA, where only the LLM backbone is quantized for all methods. Since BitVLA builds upon BitNet and natively supports only 1.58-bit quantization without flexible bitwidth adjustment, we report its published results from the original work.

SettingMethodSpatialObjectGoalLongAvg. ↑\uparrowΔ\Delta
FPBaseline98.4%98.4%97.6%93.6%97.0%-
--------
2bpwActQuant94.6%97.2%81.8%92.4%91.5%-5.5%
2bitSP-QAT97.4%99.0%96.2%93.0%96.4%-0.6%
1.58bitBitVLA96.4%99.0%95.4%92.8%96.0%-1.0%

The results show that our SP-QAT maintains strong task performance without relying on mixed-precision layouts. It outperforms ActQuant by 4.9% in average success rate and exceeds BitVLA by 0.4%. Unlike BitVLA, which demands an expensive 14-day multi-stage pre-training before robotic fine-tuning, our pipeline does not require heavy dedicated pre-training and finishes training in roughly 4 hours on only two H100 GPUs. In addition, our method imposes no hard restriction on quantization bitwidth and theoretically flexibly supports arbitrary uniform bitwidth configurations.

Q4: The Figure 1 caption and the main text are run together “...with ω-QVLA.” is immediately followed by “in one shot without task-level dynamic adjustment...” making the boundary between caption and body text unclear. Please separate them.

A4: We have fixed this formatting issue in the revised version.

Q5: The abstract claim of “at most 2.6%” should be scoped to the main in-domain setting, since the cross-domain and real-world W2A16 results in Tables 2 and 4 show larger drops (6.1%, 5.3%, and 5.0%). The authors should clarify the scope of this claim.

A5: We incorporate the restriction “under in-domain setting” in the revised version to bound this statement.

Q6: Effective Rank and Action-Sensitive Energy Preservation across Components.

A6: Thank you for your suggestion. We have revised Sec.4.3 in the main text by reporting the action-sensitive energy preservation RsR_s and effective rank reffr_\mathrm{eff} after progressively adding each component. Partial results from Tab.3 are reproduced below:

MethodAvg. ↑\uparrowΔ\Deltareffr_{eff}R_s
LSQ (Baseline)0%—122.20.142
+ PA-LoQ17.5%+17.5%102.70.354
+ SMG-SASR (w/o SMG)33.1%+33.1%84.30.472
+ SMG-SASR (w/o SASR)35.2%+35.2%78.90.512
+ SMG-SASR44.0%+44.0%61.80.708
+ DACL-SNR (w/o DACL)46.5%+46.5%57.50.877
+ DACL-SNR (SP-QAT)52.3%+52.3%54.80.908

It shows that progressive integration of our components reduces the effective rank while raising the action-sensitive energy preservation RsR_s, along with the increase in success rate. Thus these metrics consistently track the corresponding success rate improvements.

Hope this message finds you well. Thank you again for your constructive comments on our paper. We have posted our point-by-point responses and revised the manuscript accordingly. With the discussion period closing in less than two days, we would like to ensure we have addressed all your concerns.

For quick reference, the key changes are:

  1. Provided a precise definition and implementation procedure for constructing the action-sensitive subspace in Sec.3.2.

  2. Clarified the scope of the edge-deployment claim in Abstract, Introduction and Appendix H.

  3. Added visualization of spectral changes and damage to action-sensitive directions on π0\pi0 in Appendix J.

  4. Added a comparison with ActQuant and BitVLA in Q3.

  5. Fixed this formatting issue and incorporate the restriction “under in-domain setting”.

  6. Added additional results on effective rank and action-sensitive energy preservation across components in Sec. 4.3.

Your insights are valuable to us, and we hope that our responses and the revised manuscript have addressed your concerns. Thank you again for your time, effort and help in improving the quality of our paper.

Reviewer_EFdQOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies extreme low-bit quantization of VLA models, focusing on fixed 2-bit weight quantization of the LLM backbone. The authors investigate the performance degradation caused by directly applying LSQ and discuss possible causes through changes in feature spectra and energy loss in action-sensitive directions. They then propose SP-QAT, which combines low-rank quantization-aware training, soft-mask gradients (SMG), severity-aware scale regulation (SASR), and loss weighting based on SNR and sample difficulty. The method is evaluated on CogACT and π0, with additional cross-robot evaluation, real-world experiments, and inference speed measurements for CogACT. The overall results are promising, but the technical advance over existing methods and the connection between the VLA analysis and the proposed designs remain insufficiently clear.

Critical strengths
  1. The results at very low precision are practically valuable. In the in-domain SIMPLER evaluation in Table 1, both 2-bit models perform close to their respective FP16 baselines and substantially outperform several quantization methods included in the comparison. Compressing the LLM backbone to this precision while retaining good closed-loop task performance is a worthwhile result.

  2. The evaluation covers different action-generation methods and includes real robot execution. CogACT and π0 use diffusion and flow matching, respectively, so results on both provide evidence that the overall approach is applicable across different models. The cross-robot evaluation, real-world results, and measured inference speedup on an A100 further support its practical value.

  3. The paper investigates quantization failures and includes a substantial set of ablations. The feature-spectrum and action-sensitive subspace analyses offer useful observations about how quantization affects action generation. Table 3 and the supplementary experiments also demonstrate gains from low-rank updates, gradient modifications, and loss weighting.

Critical weaknesses
  1. The methodological novelty is limited, and the differences from existing low-rank QAT methods are not sufficiently clear.
  • PA-LoQ closely resembles existing low-rank QAT formulations. Like L4Q, it freezes the original weights, adds trainable low-rank updates before quantization, and learns quantization parameters. The paper should clarify the substantive differences.
  • The limited-data argument motivates the approach but does not establish novelty. Avoiding excessive weight updates with limited training data is reasonable, but similar concerns arise in LLM and VLM quantization and fine-tuning.
  • The contributions of the new gradient rules need clearer validation. The main additions are the gradient modifications and loss weighting. The ablations demonstrate their effectiveness, but SMG and SASR are always introduced together, despite accounting for a substantial improvement. Separate results would help identify what each rule contributes beyond the low-rank QAT baseline.
  1. The connection between the method design and the VLA-specific analysis is not sufficiently developed.
  • The analysis does not clearly explain the choice of gradient rules. The observations about action-sensitive directions are interesting, and the authors motivate their designs through clipped weights and scale expansion. However, SMG and SASR depend only on local quantization quantities, such as weights, quantization bounds, and rounding errors. Action-task information enters mainly through upstream loss gradients. It remains unclear how the spectral and action-sensitivity analyses informed these particular designs.
  • The underlying quantization problems are not specific to VLA models. With the same clipped STE, weights outside the quantization range would also have zero local gradients in a VLM. The trade-off between quantization range and resolution likewise does not depend on action prediction. The paper does not establish whether these issues are more severe or behave differently in VLA models. As presented, SMG and SASR appear to be general QAT improvements applied to VLA models, with an indirect connection to the action-related representation analysis.
  • The supporting analysis is concentrated on CogACT. Most detailed diagnostics and component ablations use this model. The overall results on π0 support the effectiveness of the complete method, but do not establish that the same explanations and component-level benefits hold for a different action-generation approach.
Questions for authors
  1. What are the substantive differences between PA-LoQ and existing low-rank QAT methods such as L4Q? Does the “LSQ + PA-LoQ” baseline in Table 3 use the same trainable parameters, initialization, and quantization formulation as these methods?

  2. How did the spectral and action-sensitive subspace analyses inform the choice of SMG and SASR? What evidence connects these two designs to the preservation of action-related representations?

  3. Could the authors report results using SMG alone, SASR alone, and both together? Given the substantial improvement from this combination, separating their contributions would be informative.

  4. Are similar spectral changes and damage to action-sensitive directions observed in π0? Do the two gradient modifications also provide substantial gains on that model?

  5. The problems addressed by SMG and SASR appear likely to arise in VLMs under the same QAT setup. How do these problems differ in VLA models? Is there analysis or a controlled comparison that helps determine whether the gains mainly come from general quantization improvements, or whether properties of action prediction also play a role?

6 comments

Thank you for the deep and constructive comments!

Q1:What are the substantive differences between PA-LoQ and existing low-rank QAT methods such as L4Q? Does the “LSQ + PA-LoQ” baseline in Tab. 3 use the same trainable parameters, initialization, and quantization formulation as these methods?

A1: We acknowledge that PA-LoQ shares the fundamental low-rank QAT parameterization with L4Q. Both methods freeze the pre-trained weights, introduce trainable LoRA matrices, and jointly optimize the low-rank parameters and quantization parameters by quantizing the merged weights. However, our method differs fundamentally in its underlying motivation and VLA-specific technical choices.

Different goals from prior low-rank QAT. PA-LoQ serves as a constrained weight-update mechanism designed to mitigate pre-trained representation drift under 2-bit VLA quantization. Existing low-rank QAT methods are designed for memory reduction and lack explicit mechanisms to constrain quantization-induced weight drift. In contrast, due to expensive annotations and cross-environment/robot distribution shifts, VLAs face more severe data scarcity than LLMs and degrade much more easily in generalization. We found that while full-parameter ultra-low-bit QAT can succeed on LLMs, it fails in VLA. PA-LoQ therefore uses the low-rank branch not for memory efficiency, but as a prior anchor to restrict weight updates to a subspace, preserving pretrained vision-language priors.

Concrete technical differences from L4Q:

1) Quantization parameter initialization. L4Q adopts a symmetric initialization anchored on the maximum absolute weight value. This forces the negative bound to match the magnitude of extreme positive outliers and wastes quantization resolution, which is exactly the problem our SMG-SASR seeks to avoid. PA-LoQ instead uses asymmetric min-max initialization to derive tight bounds from the true weight extremes, providing a better starting point for VLA QAT. Detailed comparative analysis is in Appendix I.7.

2) Quantization granularity. L4Q uses group-wise quantization. PA-LoQ uses per-channel quantization (common setting in VLA), which yields lower overhead for learnable quantization parameters.

Regarding the “LSQ + PA-LoQ” baseline in Tab. 3: It uses exactly the PA-LoQ formulation described in Sec. 3.3 (Eq. 9): frozen Wbase\mathbf{W}_\mathrm{base}, trainable low-rank αBA\alpha \mathbf{B}\mathbf{A}, and learnable per-channel δ,β\delta, \beta with min-max asymmetric initialization. It does not use L4Q's symmetric L4Qinit or group-wise quantization. Thus, the "LSQ + PA-LoQ" baseline strictly uses our specific PA-LoQ design, rather than a generic low-rank QAT configuration.

Q2: How did the spectral and action-sensitive subspace analyses inform the choice of SMG and SASR? What evidence connects these two designs to the preservation of action-related representations?

A2: The spectral analysis directly motivated SMG and SASR by identifying that the accumulation of weight quantization errors is one of the major cause that leads to the degradation of action-sensitive subspace in cognition feature, which is uniquely catastrophic for VLA models.

Since for any backbone layer, the quantized cognition feature satisfies Hq=XWq=X(W+ΔW)=H+Eq\mathbf{H}_q = \mathbf{X}\mathbf{W}_q = \mathbf{X}(\mathbf{W}+\Delta\mathbf{W}) = \mathbf{H}+\mathbf{E}_q, its singular value spectrum is determined by its covariance matrix: Hq⊤Hq=H⊤H+H⊤Eq+Eq⊤H+Eq⊤Eq\mathbf{H}_q^\top \mathbf{H}_q = \mathbf{H}^\top \mathbf{H} + \mathbf{H}^\top \mathbf{E}_q + \mathbf{E}_q^\top \mathbf{H} + \mathbf{E}_q^\top \mathbf{E}_q. The positive semi-definite term Eq⊤Eq\mathbf{E}_q^\top \mathbf{E}_q imposes an artificial noise floor across all feature directions. For the original cognition feature with exponentially decaying singular spectrum, this noise overwhelms tail singular directions and erodes the dominance of leading principal components, lifting its tail and flattening the spectrum.

While this spectral flattening may merely cause performance drops in standard VLM, it is destructive for VLA models. The directions concentrating projected action energy EsE_s exhibit a significant drop in retained energy under this flattening, whereas random subspaces do not show such a drop. For our W2A16-LSQ quantized CogACT model, Rs≈0.14R_s \approx 0.14, while random subspaces retain Rrand≈1.36R_{\text{rand}} \approx 1.36.

To reduce weight quantization errors, a natural idea is to refine gradient computation. Derivations in Appendix A reveal that curbing uncontrolled expansion of the outlier-driven step size effectively suppresses these errors. We thus devise SMG and SASR to address this along two complementary lines: weight gradient and step-size gradient computation.

Q3: Could the authors report results using SMG alone, SASR alone, and both together? Given the substantial improvement from this combination, separating their contributions would be informative.

A3: We have added ablation results for SMG alone and SASR alone in Sec.4.3 Component Study. Partial results from Tab.3 are reproduced below:

MethodSpoon on TowelCarrot on PlateStack CubeEggplant in BasketAvg. ↑\uparrowΔ\Delta
LSQ (Baseline)0%0%0%0%0%-
+ PA-LoQ24.2%16.7%3.3%25.8%17.5%+17.5%
+ SMG-SASR (w/o SMG)35.8%50.0%5.8%40.8%33.1%+33.1%
+ SMG-SASR (w/o SASR)49.2%45.0%9.2%37.3%35.2%+35.2%
+ SMG-SASR62.5%56.7%15.0%41.7%44.0%+44.0%

Notably, applying only SMG or only SASR yields moderate average success rates of 35.2%, and 33.1% respectively, demonstrating their individual effectiveness in mitigating partial quantization noise. However, neither alone is sufficient, especially on complex tasks. On the Stack Cube task, the standalone module achieves only roughly half the success rate of the full SMG-SASR pipeline. Incorporating the full SMG-SASR yields over 38% gains on the Put Spoon on Towel and Put Carrot on Plate tasks, with a 26.5% rise in average success rate.

Q4: Are similar spectral changes and damage to action-sensitive directions observed in π0? Do the two gradient modifications also provide substantial gains on that model?

A4: Yes, similar spectral perturbations and degradation of action-sensitive directions are observed on π0\pi0, and the corresponding visualizations are added to Appendix J.

Here are the ablation studies of SMG-SASR on π0\pi0.

MethodSpoon on TowelCarrot on PlateStack CubeEggplant in BasketAvg. ↑\uparrowΔ\Delta
w/o SMG-SASR36.3%27.1%3.3%34.2%25.2%-
only SMG41.7%50.0%7.5%59.6%39.7%+14.5%
only SASR40.8%41.7%8.3%55.8%36.7%+11.5%
SMG-SASR (Ours)53.3%61.7%16.7%83.3%53.8%+28.6%

The standalone SMG and SASR individually bring Avg. success-rate gains of 14.5% and 11.5% over the baseline. The joint application of SMG and SASR delivers a cumulative average improvement of 28.6%, confirming that these two gradient modifications yield substantial and mutually complementary performance gains on π0\pi0, consistent with our findings on CogACT.

Q5: The problems addressed by SMG and SASR appear likely to arise in VLMs under the same QAT setup. How do these problems differ in VLA models? Is there analysis or a controlled comparison that helps determine whether the gains mainly come from general quantization improvements, or whether properties of action prediction also play a role?

A5: How do these problems differ in VLA models?

First, VLA backbones exhibit significantly more severe weight outliers than standard VLMs. For CogACT, the largest outlier magnitude in the weights of Layer 0 of its Llama2-7B backbone is 8.33×8.33\times larger than the average weight magnitude, versus 7.42×7.42\times for vanilla Llama2-7B. This ratio climbs to 23.94 in the final layer for CogACT’s backbone, compared to merely 20.11 for vanilla Llama2-7B. These heavier outliers make VLA model more susceptible to outlier-driven step-size expansion under standard LSQ.

Second, the downstream impact of this spectral flattening differs fundamentally. For classic VLM tasks such as open-ended question answering, pure VLMs conduct discrete token discrimination, which relies only on the relative ordering of logits. Predictions remain reliable so long as semantic classes remain linearly separable. In contrast, the LLM backbone of a VLA supplies conditioning signals for continuous action generation. Continuous control requires high-fidelity information along dominant principal components, so minor feature distortions coupled with closed-loop trajectory execution can induce irreversible action drift.

Controlled comparison Prismatic-7B.

To isolate action-prediction effects, we apply part of our method to the language backbone of Prismatic-7B [1], a VLM model whose LLM backbone and vision encoder are homologous to those adopted in CogACT. We use a fixed 15k random subsample of the LLaVA-Instruct-158K instruction tuning dataset for QAT, with pre-computed fixed visual embeddings to preserve the original multimodal distribution. Vision encoders and the projector are frozen throughout, identical to our VLA training. We evaluate on POPE [2], a widely adopted benchmark for VLM quantization that targets visual grounding and cross-modal alignment, and report both classification accuracy and the effective rank reffr_{eff} of cognition features.

MethodSettingAcc.reffr_{eff}
BaselineFP1688.28%48.6
----
LSQ2/1676.24%89.3
+PA-LoQ2/1676.81%82.7
+SMG2/1677.93%74.1
+SASR2/1677.56%76.4
+SMG-SASR2/1678.62%70.2

The table shows that per-channel W2A16 LSQ quantization only moderately raises the effective rank of VLM cognition features from 48.6 to 89.3 and yields a graceful degradation of POPE accuracy, without catastrophic collapse. By contrast, the identical quantization recipe pushes the effective rank of CogACT cognition features from 52.3 to 122.2 and collapses action success rate to nearly zero, as reported in Tab.3. While SMG and SASR yield only a moderate reduction in reffr_{eff} (from 17.9% to 39.8%) and marginal POPE improvements, peaking at 78.62%, the same methods reduce reffr_{\text{eff}} by 10.4% to 15.5% and, when used together, brings more than 2×2\times task success rate for both π0\pi0 and CogACT.

[1] Karamcheti S, Nair S, Balakrishna A, et al. Prismatic vlms: Investigating the design space of visually-conditioned language models[J]. arXiv preprint arXiv:2402.07865, 2024.

[2] Li Y, Du Y, Zhou K, et al. Evaluating object hallucination in large vision-language models[C]//Proceedings of the 2023 conference on empirical methods in natural language processing. 2023: 292-305.

Reviewer_5FSZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SP-QAT, a quantization-aware training (QAT) framework to enable 2-bit weight quantization of vision-language-action (VLA) models. SP-QAT introduces three techniques: (1) Prior-Anchored Low-Rank Quantization, (2) Soft-Mask Gradients with Severity-Aware Scale Regulation, and (3) Difficulty-Aware Curriculum Learning with SNR-Guided Static Reweighting. Experiments on CogACT and π0 using SimplerEnv demonstrate that SP-QAT substantially outperforms existing quantization baselines under 2-bit settings.

Critical strengths
  • I appreciate the authors' efforts on investigation of why conventional quantization-aware training fails under 2-bit widths.
  • The paper also includes a real-world test case beyond just simulation.
Critical weaknesses
  • One major concern is that the VLA models used in the study -- CogACT and Pi-0 are quite dated. Therefore, the results cannot represent the state of the arts.
  • There is a lack of explanation on why SP-QAT could even outperform FP baseline under CogACT.
Questions for authors
  • It is highly suggested that the authors could use the state-of-the-art VLA models and reports results on them.
  • Could the authors provide detailed explanation on why SP-QAT could even outperform FP baseline under CogACT?
3 comments

Thanks for your deep and constructive comments!

Q1: It is highly suggested that the authors could use the state-of-the-art VLA models and reports results on them.

A1: Thank you for the suggestion. We further report results on LIBERO for π0.5\pi0.5, the state-of-the-art open-source model in the π\pi series, and GR00T N1.5, NVIDIA’s 2025 dual-system robot foundation model.

ModelSettingSpatialObjectGoalLongAvg. ↑\uparrowΔ\DeltaMem. (GB) ↓\downarrow
pi0.5FP98.4%98.4%97.6%93.6%97.0%—6.7
---------
4/1698.4%99.4%97.4%94.0%97.3%+0.3%2.4
2/1697.4%99.0%96.2%93.0%96.4%-0.6%1.7
GR00T N1.5FP92.0%92.0%86.0%76.0%86.5%—5.5
---------
4/1691.6%92.8%85.8%76.2%86.6%+0.1%3.6
2/1691.0%91.6%84.6%74.8%85.5%-1.0%3.3

Specifically, for π0.5\pi0.5, the W4A16 setting slightly improves the average success rate from 97.0% to 97.3% while reducing memory from 6.7 GB to 2.4 GB, and the W2A16 setting further reduces memory to 1.7 GB (74.6% off) with only a 0.6% average drop. And for GR00T N1.5, W4A16 maintains an average success rate of 86.6% with a 0.1% gain and reduces memory consumption from 5.5 GB to 3.6 GB, whereas W2A16 lowers memory usage to 3.3 GB at the cost of only a 1.0% drop in average success rate.

It shows that our SP-QAT also performs well on the state-of-the-art VLA π0.5\pi0.5 and GR00T N1.5. We have added these new results to Appendix.M of the revised version.

Q2: Could the authors provide detailed explanation on why SP-QAT could even outperform FP baseline under CogACT?

A2: A plausible explanation is that SP-QAT forces the 2-bit model to focus its limited optimization capacity on preserving the action-sensitive subspace. SP-QAT not only mitigates the representation distortion caused by extreme low-bit quantization, but also introduces task-aware optimization through difficulty-aware curriculum learning and SNR-guided reweighting, which may improve action prediction on certain manipulation tasks beyond simply recovering the FP16 performance.

Hope this message finds you well. Thank you again for your constructive comments on our paper. We have posted our point-by-point responses and revised the manuscript accordingly. With the discussion period closing in less than two days, we would like to ensure we have addressed all your concerns.

For quick reference, the key changes are:

  1. Added Pi0.5 results (the state-of-the-art open-source model in the π\pi series) to Appendix M.

  2. Added a plausible explanation about why SP-QAT outperform FP baseline under CogACT in Sec.4.2.

Your insights are valuable to us, and we hope that our responses and the revised manuscript have addressed your concerns. Thank you again for your time, effort and help in improving the quality of our paper.

Hope this message finds you well. Thank you again for your constructive comments on our paper. Regarding Q1, we have additionally included results on GR00T N1.5 and updated them in Appendix M.

For quick reference, all key changes are summarized as follows:

  1. Added results for π0.5\pi0.5 (the state-of-the-art open-source model in the π\pi series) and GR00T N1.5 (NVIDIA’s 2025 dual-system robot foundation model) to Appendix M.
  2. Added a plausible explanation for why SP-QAT outperforms the FP baseline under CogACT in Sec. 4.2.

We hope that our responses and the revised manuscript have addressed your concerns. Thank you again for your time, effort, and help in improving the quality of our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.