- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Quantile Head for vision-language-action models, which predicts ordered marginal action quantiles in a single forward pass. The method uses a median-centered representation with positive gaps and trains the quantiles jointly through a masked pinball loss. The median is used for default action decoding, while the additional quantiles enable optional sampling. The authors also provide a theoretical analysis of joint quantile supervision. Experiments on LIBERO, LIBERO-Plus, LIBERO-Pro, and real-robot tasks demonstrate promising performance.
Critical strengths
-
The paper explores an interesting and practical direction for VLA action heads, combining efficient single-pass action prediction with quantile-based distribution modeling. The overall method is simple and well motivated.
-
The experimental results are strong across multiple benchmarks. In particular, the matched ablations show that joint quantile supervision improves the median policy over median-only training. The real-robot experiments further support the practical effectiveness of the approach.
Critical weaknesses
-
Unclear source of improvement from joint quantile supervision. Although jointly supervising multiple quantiles improves the median policy, it remains unclear whether the gains come from reduced gradient variance or more general auxiliary supervision. Theorem 2 assumes calibrated quantiles and fixed gaps, while the actual model learns these gaps jointly. The connection between the theoretical analysis and the empirical improvements could be better established.
-
Limited modeling of joint action distributions. The method predicts marginal quantiles for individual action coordinates without explicitly modeling their joint distribution across dimensions and time steps. This may limit its ability to capture coordinated multimodal action trajectories. The obstacle-avoidance experiments in Figure 6 also show limited success with quantile sampling, raising questions about the practical advantages of the learned distribution beyond median prediction.
Questions for authors
-
Can the authors provide further evidence that joint quantile supervision reduces the variance of median updates in practice, rather than simply acting as additional auxiliary supervision?
-
How well do the assumptions of Theorem 2, particularly calibrated quantiles and fixed gaps, reflect the actual training dynamics with jointly learned gaps?
-
Since the method models marginal action quantiles, how are dependencies across action dimensions and time steps handled during sampling, especially in multimodal action settings?
5 comments
Question 3 (Weakness 2): Since the method models marginal action quantiles, how are dependencies across action dimensions and time steps handled during sampling, especially in multimodal action settings?
Response:
We address coordination through shared action representations and structured sampling, supported by new diagnostics in Appendix B.6 (Fig. 21).
(1) Shared representations support coordinated predictions. Action tokens exchange information through bidirectional attention across the entire action chunk, while action coordinates are predicted from shared observation-conditioned features. Coordinate-wise quantile losses therefore jointly update a shared action expert, allowing supervision across dimensions and time steps to shape coordinated predictions. This design is consistent with OpenVLA-OFT, which combines bidirectional attention, parallel action-chunk prediction, and coordinate-wise L1 supervision for effective robot control [1].
(2) Structured sampling preserves pooled correlations in the tested settings. During stochastic decoding, each coordinate's sampled quantile rank is held fixed throughout the ten-step chunk, and correlated latent variables can control how sampled ranks vary across replanning calls. Our new diagnostics also vary the degree of rank sharing across coordinates, from independent to fully shared. Using a single checkpoint evaluated on 1,024 training inputs, all 99 tested stochastic settings closely preserve the demonstrations' pooled correlation structure: preservation scores are 0.9429–0.9787 across dimensions and 0.9913–0.9916 across time steps. These scores measure correlation errors relative to a shuffled-label reference, with a score of 1 indicating a match to the demonstration correlations. Median decoding also achieves high scores of 0.9787 and 0.9916, respectively. These diagnostics assess pooled correlations in the tested setting.
(3) Quantile Head retains multiple marginal action modes. In the controlled experiments with only two demonstration trajectories per setting, Figs. 15 and 18 show peaks near both demonstrated action values in the ambiguous coordinates. In these experiments, flow matching produces broader distributions, while L1 and L2 each produce a single intermediate value. This illustrates the benefit of retaining marginal distributional information alongside the coordination supported by the shared expert and sampling scheme. For task performance, Fig. 4 shows that the best tested sampling configuration improves LIBERO-Long success from 98.6% to 99.6%, providing a modest additional benefit over median decoding.
Dear Reviewer,
We would like to follow up on our response above as the discussion deadline approaches. Could you let us know whether our response addresses your main concerns, or if any points need further clarification? We would be happy to provide additional details.
Thank you for your time and consideration.
We thank the reviewer for recognizing the simplicity and practical value of our approach, as well as the strong experimental evidence supporting its effectiveness. We appreciate the constructive suggestions and have revised the manuscript and appendix accordingly, with all revisions highlighted in blue.
Question 1 (Weakness 1): Can the authors provide further evidence that joint quantile supervision reduces the variance of median updates in practice, rather than simply acting as additional auxiliary supervision?
Response:
We have added training-time diagnostics in Fig. 8 that provide empirical support for Theorem 2's prediction that joint quantile supervision reduces the variance of median updates.
Specifically, we track B/a2, the normalized center-score variance, for Quantile Head, Median-Only, and Detached Median Anchor. This quantity estimates the local median-update noise coefficient in Theorem 2. Normalization by a2 accounts for differences in local correction strength; the calculation is detailed in Appendix B.2. At 12,000 training steps, Quantile Head has the lowest normalized variance among the three variants at all three displayed finite-difference step sizes, with reductions of 20.8%, 36.5%, and 51.3% relative to Median-Only at h=0.02,0.03,0.05, respectively. Here, h is the finite-difference step size used to estimate the local correction slope.
To examine the role of auxiliary supervision, the Detached Median Anchor control retains auxiliary quantile supervision of the shared features while blocking its direct gradients to the median output. The median-gradient rescaling used in this control is described under “Ablation Settings” in Appendix B. Quantile Head also exhibits lower normalized variance than this control. This comparison supports a contribution from directly coupling quantile supervision to median updates, in addition to the auxiliary supervision of shared features.
Question 2 (Weakness 1): How well do the assumptions of Theorem 2, particularly calibrated quantiles and fixed gaps, reflect the actual training dynamics with jointly learned gaps?
Response:
We have added diagnostics in Fig. 7 that support treating these assumptions as local approximations during late-stage training, even when the gaps are learned jointly.
(1) Calibrated quantiles. Figure 7(a) shows that the pooled empirical coverage of the median and its neighboring quantiles stabilizes near their nominal levels. At 12,000 training steps, the absolute coverage errors are at most 0.71 percentage points, providing evidence of approximate calibration at the pooled level on the diagnostic observations.
(2) Fixed gaps. Theorem 2 holds the gaps fixed when analyzing the local median-update direction, while the implemented model learns them jointly. Fig. 7(b) shows that per-update changes in the learned adjacent gaps remain small during the second half of training. At the final update (step 12,000), the mean absolute gap change is only 0.0496% of the mean gap size. These small per-update changes support treating the gaps as approximately fixed within a single update for the local analysis.
The diagnostic procedures are detailed in Appendix B.2. We also report variance diagnostics in Fig. 8, where the observed reduction in the normalized variance of median updates is consistent with Theorem 2's prediction.
Reference
[1] Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success. arXiv preprint arXiv:2502.19645.