- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces Execution-Level Quantization Sensitivity (EQS), a criterion for allocating mixed-precision PTQ in VLA action-generation modules. EQS converts quantization-induced action differences into horizon-level SE(3) motion discrepancies, evaluates them with a robot-configuration-dependent operational-space inertia metric, balances translation, rotation, and gripper components, and uses gradient attribution to rank layers for BF16 retention. The authors evaluate EQS on π0.5/LIBERO and CogACT/SimplerEnv under multiple precision budgets, compare against action-level, drift-aware, and layer-statistics baselines, and conduct real-robot experiments on four Airbot tasks. EQS generally achieves higher task success with comparable quantization and inference costs.
Critical strengths
1.EQS addresses a meaningful gap between easily computed action deviation and expensive closed-loop task evaluation. The combination of ordered SE(3) composition and configuration-aware operational-space weighting is technically well motivated, and the ablations support the contribution of both components.
2.The empirical validation is broad and controlled: two VLA models, two simulation settings, multiple BF16 retention budgets, calibration-overhead measurements, and real-robot experiments. The method consistently outperforms the evaluated sensitivity baselines under the same retention configurations.
Critical weaknesses
1.The physical execution proxy is under-validated. EQS evaluates a horizon discrepancy using the operational-space inertia at the horizon-start configuration and keeps it fixed over the horizon, while real execution also depends on controller dynamics, contacts, joint limits, saturation, and changing configurations. The paper does not establish that the proposed cost transfers across robots or reliably correlates with closed-loop task degradation beyond the reported benchmarks.
2.The mapping from EQS scores to the final jointly quantized model is not sufficiently specified or validated. Equation (15) uses gradient magnitudes, but it is unclear how gradients pass through W4A4 quantization, whether scores are computed with one candidate layer quantized at a time, and how interactions among simultaneously quantized layers are handled. Without rank stability or sensitivity-to-final-task-loss analysis, the layer-allocation mechanism remains difficult to assess.
Questions for authors
1.How is the EQS gradient in Eq. (15) computed through W4A4 quantization? Is a straight-through estimator used, and are scores obtained by quantizing each candidate layer individually while all other layers remain full precision?
2.Does the operational-space matrix use the actual configuration at every execution step or only the horizon-start configuration? How sensitive are the rankings to the horizon length, damping coefficient, and robot/controller dynamics?
3.Are calibration chunks strictly disjoint from evaluation episodes and tasks? Please report sensitivity-ranking stability across calibration resamples and confidence intervals for the 40-trial real-robot results.