- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies quantization of vision-language-action models, aiming to reduce storage while preserving task success. Its motivation is that local reconstruction error does not fully capture the effect of different weights on the final action. SACQ first fits vector codebooks according to the downstream operations of each projection layer and allocates mixed bit widths based on action error per bit saved. It then refines codeword assignments with second-order error compensation and calibrates the codebooks and scales against the BF16 model’s action outputs. Experiments show that the method reduces storage while largely preserving task performance.
Critical strengths
The proposed method achieves a great trade-off between the storage and the task execution success rate of VLA, the paper is clear and easy to follow, and the results are promising, with Table 1 retaining nearly the reported BF16 success at approximately 1.5 L/A bpw with substantially lower resident memory.
Critical weaknesses
- Lack of experimental details: it is unclear what supervision, what calibration samples, and what optimization budget are used when adapting ActQuant to the GR00T/DiT, since the original ActQuant uses the ground truth actions while SACQ uses the output of the BF16 teacher.
- More analysis on the performance: the performance improvements are relatively minor when removing structured codebook fitting mixed bit width allocation, 92.60% and 93.25% respectively, compared to 93.85% of the full model. I suggest providing more analysis or per-task results to help determine whether these differences are stable.
Questions for authors
- What supervision, calibration data, and optimization budget are used for the adapted ActQuant baseline? How are its statistics computed for the DiT, and how does this differ from the original ActQuant?
- How do the 576 training requests per suite in Appendix C correspond to the 2,880 observations in Table 8? Please clarify the units and the training and validation split. It would also be helpful to report the calibration time and memory cost.
- Can the authors provide per-task results or uncertainty estimates for the structural codebook fitting comparison at 1.5 bpw? Results from a few different calibration seeds would also help show whether the improvement is stable.
4 comments
Follow-up: Calibration Time and Memory Cost
As mentioned in our previous response, we measured the wall-clock time and peak GPU memory usage of SACQ's quantization and calibration stages using two NVIDIA RTX 4090 GPUs (24 GB each).
The measurement covers activation statistics collection, codebook fitting, GPTQ preparation, and mixed-bit allocation across all four LIBERO suites. The four suites are jointly used to determine a shared bit allocation, while the subsequent refinement is performed separately for each suite. Here, we measured index reassignment and action-level calibration for the Spatial suite at 1.5 L/A bpw.
Table R9. Wall-clock time and peak GPU memory of SACQ's quantization and calibration stages.
| Stage | Wall-clock time | Peak GPU memory (GB) |
|---|---|---|
| VLM activation statistics & codebook fitting | 5m 15s | 10.13 |
| DiT activation statistics collection | 5m 33s | 11.73 |
| DiT codebook fitting | 27m 4s | 1.21 |
| GPTQ preparation | 22m 58s | 12.59 |
| Mixed-bit allocation | 70m 22s | 16.56 |
| Index reassignment | 4m 52s | 11.04 |
| Action-level calibration | 29m 12s | 23.38 |
| Total | 2h 40m 7s | 23.38 |
Peak GPU memory represents the maximum PyTorch allocated memory on either GPU. Some stages were executed concurrently, so individual stage times do not sum to the total wall-clock time. GPTQ preparation includes both allocation candidates and fixed background weights.
These measurements are based on a single run using existing calibration data and BF16 teacher actions. Calibration dataset collection, BF16 teacher action generation, and rollout evaluation are excluded.
Thank you again for your valuable feedback.
Gentle Reminder: Author–Reviewer Discussion Period Ending Soon
Dear Reviewer kZED,
Thank you again for your thoughtful review and valuable feedback.
As the author–reviewer discussion period is approaching its end, we would like to briefly follow up on our rebuttal and additional results.
In our rebuttal, we provided the following clarifications and analyses:
Protocol of the adapted ActQuant baseline: We clarified that the adapted ActQuant baseline was evaluated using the same calibration requests as SACQ, with the actions predicted by the BF16 policy as targets. We also described how HSIC-based sensitivity and Fisher importance were adapted to GR00T's DiT action head.
Sample units and data split: We explained that each training request contains five observations, so 576 training requests correspond to 2,880 observations per suite, with 64 separate validation requests.
Uncertainty of the 1.5-bpw comparison: Using the existing paired outcomes, we reported an exact McNemar test (p = 0.043) and a task-level paired t-test (p = 0.040), with a 95% confidence interval of [+0.06, +2.44] percentage points.
Calibration time and memory cost: We additionally reported the measured wall-clock time and peak GPU memory for each stage of SACQ's quantization and calibration pipeline in a separate follow-up response.
If these results address your concerns, we would greatly appreciate it if you could acknowledge this and consider reassessing your rating. We would be happy to clarify any remaining questions.
Thank you again for your time and consideration.
Best regards,
The Authors
The authors’ response addresses my concerns. I maintain my score.
Response to Reviewer kZED
We thank the reviewer for the careful and constructive review, and for recognizing the useful low-bit operating point of SACQ and the motivation of the structural metrics. We address each point below and will add these clarifications to the revised version.
[W1 & Q1] Protocol of the adapted ActQuant baseline
The original ActQuant targets OpenVLA-OFT and π0.5. For GR00T N1.7, we used the official ActQuant code for bit allocation and quantization, and adapted the model-specific parts, such as activation collection, the Fisher loss, and the layer mapping, to GR00T. To keep the calibration data equivalent across methods, ActQuant uses the same 576 training requests per suite as SACQ and the other baselines (2,880 observations, see our answer to Q2), and uses the actions predicted by the BF16 policy as target actions instead of ground-truth demonstration actions, as in SACQ. Both the HSIC-based sensitivity and the Fisher importance are computed with these BF16 actions.
ActQuant performs HSIC-based bit allocation followed by Fisher-weighted selection of the quantization scales and quantized values, both computed over the same 576 requests. For the DiT action head, which is executed over four denoising steps, we compute the HSIC sensitivity at each step and average the four scores with equal weights. Following the original method for models with only an action-head pathway, the Fisher importance is computed from the action loss, which is the flow-matching loss of GR00T’s action head, with the noise and time step sampled once per request. The quantization scope covers the same 338 language and action linear layers as SACQ, including the attention, feed-forward, AdaLN modulation, and output projections, with the bit budget allocated jointly across the language backbone and the action head. In addition, bit budgets are matched in nominal bpw as for the other methods, and the quantized weights are evaluated in the same BF16 evaluation path as the other methods rather than in the original deployment runtime.
[W1 & Q2] Sample units and data split
In our LIBERO action-calibration dataset, each saved request contains a batch of five observations, where each observation consists of the third-person and wrist camera images, the robot state, and the language instruction. The 576 training requests per suite in Appendix C therefore correspond to 2,880 observations per suite, which is the unit used in Table 8. Accordingly, the calibration budgets of 90, 180, 360, 720, 1,440, and 2,880 observations in Table 8 correspond to 18, 36, 72, 144, 288, and 576 training requests per suite, respectively.
For each suite, we use 576 training requests (2,880 observations) and 64 separate validation requests (320 observations). The validation requests are used only to select the action-level calibration checkpoint.
The action-level calibration runs for 12 epochs with a global batch size of 6 requests, corresponding to 1,152 optimizer updates per suite. We are currently measuring the wall-clock time and peak GPU memory of each stage under a consistent setting, and will report them in a follow-up comment.
[W2 & Q3] Uncertainty of the 1.5-bpw comparison
Using the existing paired outcomes, we quantified the uncertainty of the comparison between structure-aware and layer-local fitting at 1.5 L/A bpw under the full refinement budget. Both models are evaluated on the same 2,000 episodes, where each episode uses the same initial state and policy noise for both models, so their outcomes can be paired. At the episode level, structure-aware fitting succeeds on 83 episodes where layer-local fitting fails, and layer-local fitting succeeds on 58 episodes where structure-aware fitting fails (exact McNemar test, p = 0.043). At the task level, a paired t-test over the 40 tasks gives t(39) = 2.12 and p = 0.040, with a 95% confidence interval of [+0.06, +2.44] percentage points for the mean difference of +1.25 points. The advantage of structure-aware fitting is therefore small but statistically significant at the 0.05 level.