- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies Noise Space Learning (NSL), which adapts a frozen flow-matching VLA by learning a context-conditioned distribution over its input noise. It implements this interface using PPO, SAC, and success-filtered AWR, and evaluates their tradeoffs across interaction and update budgets. Experiments cover LIBERO, RoboTwin, and four real-robot tasks. An information-theoretic analysis characterizes when unrestricted noise reweighting can achieve a desired behavioral change with minimum KL cost.
Critical strengths
- Useful empirical comparison. The paper studies how optimizer choice, update budgets, rollout counts, and training components affect noise-space adaptation.
- Substantial update-time savings. Unlike the previous paper, it directly measures update time against adaptation baselines.
- Meaningful experimental coverage. Results include two VLA backbones, two simulation benchmarks, and four real-robot tasks.
- Carefully qualified theoretical analysis. The appendix explicitly discusses restricted coverage, deterministic-kernel singularities, and why behavioral anchoring does not establish physical safety.
Critical weaknesses
- Limited statistical evidence for superiority over strong baselines. Table 4 reports one training seed, without variability across independent runs. The average improvements over RLT are modest: 2.5 percentage points for NSL-SAC on LIBERO and 1.3 points for autonomous NSL-AWR on real robots. Table 1 also reports the best checkpoint within 60 epochs, leaving the robustness of these margins uncertain.
- Different evaluation modes complicate the base-policy comparison. Appendix B.1 evaluates NSL using the actor’s deterministic mean, while the base policy samples Gaussian noise. A matched evaluation ablation would help isolate the contribution of learned noise adaptation.
- Efficiency measurements cover model updates only. Tables 1 and 3 demonstrate substantial update-time savings, but exclude the full adaptation process. End-to-end wall-clock time, including rollout collection and evaluation, is needed to establish practical speedups.
- The comparative safety claim has limited support. The claim of “safer execution” relies partly on an earlier SAC experiment with unintended table contact. Systematic collision or intervention rates under comparable conditions are needed to substantiate AWR’s safety advantage (Section 4.7; Appendix B.6).
Questions for authors
- Are the LIBERO training and evaluation reset states disjoint? Can you provide comparisons under matched stochastic and deterministic noise-selection protocols, including a zero-noise base-policy reference? Could you provide LIBERO Pro evaluation results?
- Does repeating a 32-dimensional noise vector across the action horizon reproduce the native VLA prior? What exactly are the noise space and reference distribution in the theoretical analysis and implementation?
- Which parts of the minimum-information result remain applicable to the restricted Gaussian actors and deterministic evaluation used experimentally? Can you clarify the practical interpretation of the comparison when alternative deterministic interventions have infinite joint KL?
- What quantitative evidence supports the safer-execution recommendation—for example, intervention counts or unintended-contact rates from the existing experiments under comparable conditions?
5 comments
Q1-1: Are the LIBERO training and evaluation reset states disjoint?
No. We use the same 50 benchmark-provided reset states per task for LIBERO training and evaluation, consistent with prior work such as SimpleVLA-RL [1] and LaST-R1 [2]. Evaluation covers all 500 task–reset pairs across the ten tasks.
For performance with disjoint reset states, we refer the readers to RoboTwin experiments in the appendix, where each task and difficulty setting uses 50 training scenarios and 50 held-out evaluation scenarios with disjoint reset seeds. All three NSL algorithms consistently improve performance under this setting. At epoch 30, averaged equally across the eight tasks and the Easy/Hard settings, NSL-PPO, NSL-AWR, and NSL-SAC improve SR over the base policy reference by 37.75, 37.13, and 48.63 percentage points, respectively.
[1] SimpleVLA-RL. SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning. arXiv:2509.09674. https://arxiv.org/abs/2509.09674
[2] LaST-R1. LaST-R1: Reinforcing Robotic Manipulation via Adaptive Physical Latent Reasoning. arXiv:2604.28192. https://arxiv.org/abs/2604.28192
Q1-2: Can you provide comparisons under matched stochastic and deterministic noise-selection protocols, including a zero-noise base-policy reference?
Yes, please refer to our response to W2 for the evaluation settings and detailed results. The zero-noise base policy achieves similar performance to the native Gaussian reference, and trained NSL-AWR policies perform similarly under deterministic and stochastic evaluation.
Q1-3: Could you provide LIBERO Pro evaluation results?
Yes, we evaluate the best checkpoints selected on LIBERO-10 for the five adaptation methods on the LIBERO-10 suite of LIBERO-Pro-1.0, and present the results in the following table. We find that NSL consistently improves performance over the base policies on LIBERO-Pro, with NSL-SAC achieving the highest success rate on both models.
| Model | Base | Action-AWR | RLT | NSL-PPO | NSL-AWR | NSL-SAC |
|---|---|---|---|---|---|---|
| SmolVLA-Full | 10.40% | 21.40% | 20.15% | 17.80% | 22.30% | 24.20% |
| π₀.₅-One | 14.05% | 22.55% | 24.05% | 19.60% | 23.45% | 27.35% |
| Avg. | 12.23% | 21.98% | 22.10% | 18.70% | 22.88% | 25.78% |
Q2-1: Does repeating a 32-dimensional noise vector across the action horizon reproduce the native VLA prior?
No. Native VLA noise is sampled independently across the action horizon, whereas repetition shares one 32-dimensional vector across time. Our base policy reference uses native Gaussian noise.
Q2-2: What exactly are the noise space and reference distribution (\rho) in the theoretical analysis and implementation?
The noise space contains the generator’s noise inputs, and is a fixed reference distribution on that space. Our theory does not require a specific form or dimensionality of , under the stated assumptions. The 32-dimensional implementation can be represented using a standard Gaussian reference, with repetition and fixed noise transformations included in the generator map.
Q3-1: Which parts of the minimum-information result remain applicable to the restricted Gaussian actors and deterministic evaluation used experimentally?
The information-cost lower bound remains applicable to stochastic Gaussian actors, while deterministic mean evaluation leaves the framework-level analysis unchanged. The minimum-information equality characterizes the NSL framework over unrestricted noise distributions; a Gaussian actor attains this bound when it can represent an optimal noise distribution.
Q3-2: Can you clarify the practical interpretation of the comparison when alternative deterministic interventions have infinite joint KL?
Infinite joint KL reflects a support mismatch caused by changing a deterministic generation mechanism; it does not imply infinite training cost. Our result establishes conditions under which NSL attains the behavioral information lower bound while preserving the pretrained generator. Appendix A provides a shared Gaussian smoothing formulation for finite information-cost comparisons at a specified scale. Computational efficiency is evaluated separately through measured training times.
Thank you again for your careful review and valuable suggestions. We hope our responses can clarify the points raised and address your concerns.
Thank you again for your thoughtful review and feedback. As the discussion period is approaching its end, we wanted to follow up and ask whether our response has addressed your main concerns. We sincerely appreciate the time and care you have given to reviewing our work.
Thank you for taking the time to review our paper in detail and share your thoughtful comments. Below, we respond to your questions and concerns with additional explanations and experimental results.
W1: Limited statistical evidence for superiority over strong baselines
[Seed] Reporting the training seed in Table 4 is for strict reproducibility. Based on our prior experiments, our method works quite robust for different seeds. Since re-running experiments under different seeds for different algorithms takes long time, we will add these results in our revision.
[Advantage] Compared with baselines such as RLT, the main advantage of NSL is training efficiency. Specifically, NSL-SAC achieves an 11.2× speedup in model training in simulation, and NSL-AWR achieves a 15.2× speedup in real-robot experiments, while achieving better performance.
[Robustness] To further assess the robustness of our results, we compare the average success rates of all five methods over the final five epochs of the 60-epoch training runs, and show the results in the table, which demonstrate that NSL-SAC consistently outperforms RLT across both models, with an average improvement of 2.82 percentage points.
W2: Different evaluation modes complicate the base-policy comparison.
To address the reviewer’s concern, we provide a matched evaluation ablation on SmolVLA-Full and π₀.₅-One. For the untrained noise actor, deterministic evaluation at its zero mean yields performance similar to the native Gaussian reference. For the trained noise actor, stochastic and deterministic evaluation also produce similar success rates.
We compare the native Gaussian reference with zero-noise evaluation. For trained NSL-AWR policies, we compare mean and stochastic evaluation using the same best checkpoint selected by mean evaluation for each model.
W3: Efficiency measurements cover model updates only.
To address the reviewer’s concern, we additionally report rollout collection time and evaluation time for different methods in simulation. We use model-update time for comparison in the paper because all methods share identical rollout and evaluation settings, resulting in similar time costs for these two stages. Model-update time therefore directly captures the computational differences between training methods. Including rollout collection and evaluation, NSL-SAC reduces the total time from 406.70 to 286.53 seconds per batch compared with RLT, a 29.5% reduction. NSL-AWR reduces the total time from 511.40 to 280.49 seconds per batch compared with Action-AWR, a 45.2% reduction.
W4&Q4: The comparative safety claim has limited support.
In the additional real-robot bead-placement experiment, NSL-AWR improves task success without triggering emergency stops, whereas NSL-SAC’s emergency-stop rate increases over successive update rounds. To address the reviewer’s concern, we report emergency-stop statistics during online adaptation, with 48 collection attempts scheduled per round.
Round 0 denotes initial collection before any model update. After two update rounds, all 23 NSL-SAC collection attempts triggered emergency stops, prompting early termination of the experiment.
Emergency stops include both operator-initiated stops in response to unsafe behavior and automatic stops triggered by excessive torque. Unsafe behaviors include unintended table collisions, large-amplitude hazardous motions, and knocking away beads or the bead-placement board.