- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes an accuracy-aware training method for VLA in high-precision scenarios that does not require changes to the model architecture or inference process. The authors found that existing Flow Matching methods apply similar training penalties to prediction errors in the VLA model across different operational phases and flow times, without fully considering the impact of these errors on the final task success rate. To this end, the authors propose PRISM, a VLA training method designed for fine-grained operations. The method estimates accuracy requirements for different operational stages based on the robot’s taught trajectory and adjusts the training loss accordingly, considering flow time and signal-to-noise ratio. Based on RLBench, LIBERO, and real-world robot experiments, PRISM has demonstrated strong performance and practical value. They address a significant and meaningful issue with current VLA models in fine-motor tasks. I believe the main contribution of this paper lies in incorporating accuracy requirements into the VLA training process and demonstrating the effectiveness of this design through experiments. The overall structure of the essay is clear, allowing readers to grasp the very issue the author raises. However, the paper still raises some issues that warrant further discussion. We raise these concerns in the Critical Weaknesses section and hope the authors will address them further.
Critical strengths
-
The greatest strength of this article is that the research question is representative of real-world applications. The author links training supervision to the accuracy requirements of different operational phases, making the approach highly targeted. Compared to simply scaling up the model or increasing the amount of training data, this method offers an alternative approach to improving the VLA’s operational capabilities.
-
This method does not increase inference overhead and can be easily integrated into existing VLA models. PRISM's PPS labels are generated offline before training, and CPR's prediction head is used only during training. It is a very flexible solution that offers a direction for future exploration.
-
Results from experiments on real devices show that the improvements are concentrated in the fine-tuning phase, but gains were also observed on RLBench and LIBERO. This suggests that the authors did not observe any significant trade-off between fine-tuning performance and general performance.
Critical weaknesses
I think there is one crucial concern regarding the methodology. The core premise of PRISM is that different operational stages have different accuracy requirements. Therefore, different training weights should be assigned. However, the PPS in Section 3.2 is primarily based on the accuracy requirements for estimating the robot’s motion speed, end-effector displacement, and gripper state. This method is more about determining whether the robot is in a slow, stable operational phase rather than actually measuring physical accuracy requirements. In real-world applications, certain high-speed contact operations may also have strict accuracy requirements. In Figure 5(a), the authors report a high correlation between the CPR predictions and the PPS labels. However, this only demonstrates that the model can predict the accuracy labels designed by the authors. I don't think this proves that the PPS labels accurately reflect the actual physical tolerances. If PPS merely identifies slow-motion phases, then PRISM’s core mechanism could also be interpreted as a training weighting method that takes motion phases into account. I believe this issue is well worth considering in terms of future efforts to truly generalize these methods to the real world for performing precise operations. It also raises the question of whether this paper has overstated its contributions.
In addition, the authors need to provide results for two further experimental sections. This would further supplement the results that the cumulative ablation in Table 4 cannot demonstrate. First, we hope the authors will provide performance results using only Flow-time Weighting without PPS. Second, how effective would it be to apply weighting directly to the low-speed operation phase without using the full PPS?
Questions for authors
PPS primarily relies on motion speed and accuracy requirements to estimate gripper states. So, there are 2 questions.
- Could the authors further validate the relationship between PPS and real-world physical tolerances?
- Does this method remain effective when motion speed and accuracy requirements do not align?
4 comments
Author Response 2/2 | W2: Flow-time-only and low-speed trained controls
-
Both requested controls are now trained: On drive insertion, we trained the two requested variants and several further controls with the same data, initialization and update budget as PRISM (Appendix G.2, Table 16). Each uses 20 trials with 95% Wilson intervals, and time and attempts average successful trials.
Variant SR [95% CI] Time (s) Attempts π0.5 baseline 0.35 [0.18, 0.57] 47 4.1 Flow time only, no PPS/CPR/SNR 0.45 [0.26, 0.66] 46 4.1 Flow time + SNR, no PPS/CPR 0.45 [0.26, 0.66] 47 4.2 PRISM without CPR 0.75 [0.53, 0.89] 44 3.2 Randomized PPS 0.40 [0.22, 0.61] 51 4.3 Unmasked low speed 0.70 [0.48, 0.85] 41 2.7 Full PRISM 0.95 [0.76, 0.99] 38 2.1 -
Flow-time weighting alone: Clean-end weighting without PPS, CPR or the SNR gate gives 45%, and adding the SNR gate also gives 45%. Flow-time emphasis therefore yields only a modest gain over the 35% baseline, and reaching 95% requires context-dependent demand.
-
Weighting slow phases directly: Replacing PPS with the unmasked low-speed score in both PSW and CPR gives 70% with 2.7 attempts. Full PRISM reaches 95% with 2.1 attempts and the shortest time (38 s). This 25-point gap comes from the gripper cues and the active mask, which keep extra supervision away from slow preparation and handling (Figure 13).
-
Further controls: Shuffled PPS gives 40%, and removing CPR gives 75%. Mean-normalized weights keep LIBERO at 97.45% versus 97.50% and mating at 16/20 (Appendix G.1). The gain therefore depends on correct demand assignment, not on loss scale.
-
Why PPS serves both PSW and CPR: PPS tells CPR when local precision is needed and tells PSW where to add supervision. The cumulative ablation raises success from 35% to 45/70/85/95% as CPR, precision weighting, clean-end emphasis and SNR gating are added (Table 4). Table 5 shows the same benefit without VLM state inputs, with π0.5 rising from 30% to 95% and π0 from 25% to 80%.
Thanks for the response, which has addressed my concerns. I will maintain my positive assessment.
Official Comment by Authors
We sincerely appreciate your response and the positive feedback. Thank you for your time and valuable insights!
Author Response 1/2 | W1 & Q1 & Q2: PPS beyond slow motion and its link to tolerance
We thank the reviewer for the thoughtful review and for recognizing the practical value of precision-aware training. All revisions are highlighted in blue in the updated paper.
PPS is not a slow-motion detector: PPS multiplies a within-trajectory motion rank by gripper engagement and by two gates, one that suppresses rapid closing and one that suppresses isolated gripper motion while the arms are still (Section 3.2). Slow motion alone therefore does not give a high score. Among 24 manually labeled slow preparation or free-motion samples, PPS selects none, while speed only and masked speed select 9 and 8 (Appendix G.4, Figure 13). In training, weighting slow phases directly reaches 70% insertion success, versus 95% with PPS (Appendix G.2, Table 16).
Link to physical tolerance (Q1): Figure 5(a) tests whether CPR learns PPS, so we additionally validated PPS against references it never sees.
Appendix C, Figure 7, shows the same pattern across nineteen tasks.
Fast precision actions (Q2): Three properties keep PRISM effective.
Contribution: As you summarize, PRISM brings precision demand into VLA training without changing the architecture or inference. PPS estimates this demand from signals present in every demonstration, and its validity rests on manual labels, measured tolerances and trained controls. With it, end-to-end success rises from 35/35/45% to 95/80/85% on the three MSSM tasks, while simulation performance is preserved or improved.