- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes SCOUT, an online RL module that keeps a pretrained VLA frozen and learns a lightweight refiner which transports the VLA's reference action chunk toward a critic-weighted target distribution via a point-source Schrödinger bridge. Experiments on LIBERO, LIBERO-PRO, and four real-world bimanual tasks show gains in success rate and execution throughput over online-refinement baselines.
Critical strengths
-
The paper attempts to apply a new mathematical approach to online RL for VLAs, which looks interesting.
-
The real-world experiments report results over 50 trials, reducing random error.
Critical weaknesses
-
The motivation is not convincing, and the framing limits the paper's perceived scope. The claim that many failures can be handled by small-magnitude refinement is not well supported. And as a result, the proposed method appears applicable only to local small-magnitude correction, which suggests a low ceiling and weakens the overall motivation.
-
The use of the Schrödinger bridge is insufficiently justified and not directly compared against simpler alternatives. While there is a mathematical basis for the Schrödinger bridge, it is hard to see why this particular formulation is required. Moreover, weight-adjustment-based RL methods such as AWR, CFG are not directly compared against, which makes the use of the Schrödinger bridge appear unmotivated. In addition, it is unclear why DSRL_RLT matches SCOUT on LIBERO but underperforms it on LIBERO-Pro. The paper offers no experimental phenomenon to help the reader understand this discrepancy.
Questions for authors
-
Why are existing online refinement methods insufficient, such that the Schrödinger bridge formulation is needed?
-
Why does the Schrödinger bridge yield better performance under distribution shift (OOD)? Have you tried other benchmarks with OOD settings, such as LIBERO-Plus, RoboTwin Hard, etc.?
6 comments
Response (2/3)
W2. The Schrödinger bridge is insufficiently justified against simpler alternatives like AWR or CFG. Unclear why matches in LIBERO but underperforms in LIBERO-PRO.
Thank you, Reviewer xZ7K, for requesting clarification on the importance of the Schrödinger bridge in SCOUT. We first conduct a thorough ablation comparison with Conditional Flow-Matching (CFM) and compare with AWR on LIBERO and LIBERO-PRO to show the advantage of the reference-source Schrödinger bridge. Then, we provide analysis on why SCOUT performs better in out-of-distribution robustness compared to .
Ablation and Simpler Alternatives. Starting from Gaussian-source CFM, we add one component at a time until reaching full SCOUT, and report success rates on simulation and real-world tasks. Due to computational resource limitations, the ablation uses 10 LIBERO tasks and 10 LIBERO-PRO layouts from our original evaluation set, and two real-world tasks. Reference-centered CFM centers CFM's Gaussian source at the VLA reference action; SCOUT, no bridge noise uses SCOUT's point source, endpoint objective, and two-step readout but removes the interpolation noise (σ = 0).
| Actor variant | LIBERO (10 tasks) | LIBERO-PRO (10 layouts) | Hardware (2 tasks) |
|---|---|---|---|
| Gaussian-source CFM | 83.4 | 54.3 | 63 |
| Reference-centered CFM | 76.3 | 75.4 | 64 |
| SCOUT, no bridge noise (σ = 0) | 68.9 | 76.2 | 70 |
| SCOUT | 88.0 | 90.8 | 79 |
This ablation shows that both the reference-action start and the bridge noise are needed. While Gaussian-source CFM performed well in LIBERO, centering the source at the reference and switching to SCOUT's point source and endpoint objective contributed more than 20 percentage points in LIBERO-PRO layouts, and the Brownian bridge noise contributed another 14.6 points. Real-world evaluation success rates support this conclusion.
| Method | 150 ep. | 300 ep. |
|---|---|---|
| Frozen | 36.8 | 36.8 |
| AWR (deterministic / stochastic readout at 300 ep.) | 32.6 | 34.7 / 37.3 |
| SCOUT | 74.5 | 89.4 |
Compared to AWR, SCOUT shows a large performance advantage and continues to improve with training. On the same 10 LIBERO tasks and 10 LIBERO-PRO layouts, SCOUT improves mean success over the frozen policy by 52.6 percentage points, while AWR remains near the frozen policy (32.6–37.3% vs. 36.8%). AWR only reweights actions that were already executed, so it cannot propose corrections outside them, whereas out-of-distribution VLA refinement often requires small corrections outside the VLA's output distribution. We updated Section 4.3 (Table 3a) and Appendix M (Table 29) of the manuscript to include these ablations, and Table 3b of the updated manuscript also includes direct critic-weighted regression onto the same candidates.
Regarding CFG, we found several methods named CFG in related contexts. We interpreted CFG as classifier-free guidance, which is closely related to the conditional flow matching (CFM) actor we evaluate above. We would be happy to provide additional experiments if you had a different method in mind.
SCOUT outperforms when successful actions lie outside of the VLA action distribution. steers the input noise of the frozen policy, so it can only select actions the frozen policy can already generate. This suffices when in distribution. However, under a layout shift, the needed correction may lie outside that set. We report simulation success rates as layout shift and difficulty increase:
| Task set | SCOUT | |
|---|---|---|
| LIBERO, all 26 tasks | 87.3 | 86.5 |
| LIBERO-PRO, all 36 layouts | 87.7 | 83.7 |
| LIBERO-PRO, 13 hard layouts (frozen ≤ 30%) | 78.2 | 71.2 |
SCOUT's lead over grows from 0.8 points on in-distribution LIBERO to 4.0 points under LIBERO-PRO layout shifts and 7.0 points on the hardest LIBERO-PRO layouts. We have added these analyses to Section 4.1 (Table 1), with per-layout results and paired statistics in Appendices G and J (Tables 10 and 22).
Response (3/3)
Q1. Why are existing online refinement methods insufficient, such that the Schrödinger bridge formulation is needed?
Thank you, Reviewer xZ7K, for this question. Refining a frozen VLA for precise manipulation tasks requires (i) starting from the VLA's reference action to preserve its competence in high-success scenarios, (ii) reaching corrections the VLA cannot generate itself when task success requires so, and (iii) adaptively balancing critic-guided corrections against the reference and exploratory alternatives. Each existing family lacks one of these properties, while SCOUT's point-source bridge enables all three:
| Method | (i) Starts from VLA action | (ii) Beyond VLA outputs | (iii) Actor update | LIBERO-PRO mean ↑ | Worst layout ↑ | Tasks below frozen ↓ |
|---|---|---|---|---|---|---|
| DSRL | No | No | SAC on input noise | 72.6 | 4 | 8 |
| Partly (via RL token) | No | SAC on input noise | 83.7 | 17 | 7 | |
| RLT | Yes | Yes | Critic gradient (TD3+BC) | 69.5 | 1 | 6 |
| CFM | No | Yes | Flow matching from Gaussian noise | 74.3 | 2 | 9 |
| SCOUT | Yes | Yes | Critic-weighted bridge | 87.7 | 41 | 1 |
Noise-space steering (DSRL, ) can only select actions the frozen VLA already generates, which is why nearly matches SCOUT on LIBERO but falls behind under layout shift, as we analyze for W2. RLT can move beyond the VLA's outputs, but its deterministic actor follows the critic's gradient under a fixed critic/behavior-cloning weighting, so it cannot adjust per context between staying at the reference and following the critic; as a result, it does not consistently preserve the base model's competence (regresses on 6 of 62 tasks by up to 19 points). CFM learns from the same critic-weighted candidates as SCOUT but generates actions from Gaussian noise and uses the reference only as conditioning, so it does not start from the VLA's competent action (13.4 points below SCOUT). SCOUT, by construction, weights the reference, exploratory Gaussian, and critic-gradient proposals per context, so how far it moves from the reference is set by the critic for each context rather than by a fixed regularization weight, achieving the highest mean and worst-layout success with only one regression. This distinction is briefly discussed in Section 2, with supporting results in Section 4.1 (Table 1, Figure 2).
Q2. Why does the Schrödinger bridge yield better performance under distribution shift (OOD)? Have you tried other benchmarks with OOD settings, such as LIBERO-Plus, RoboTwin Hard, etc.?
Thank you, Reviewer xZ7K, for this question and suggestion to evaluate on additional benchmarks. SCOUT's robustness under distribution shift comes from its two bridge components. The point source at the reference action preserves the VLA's competence, and the bridge noise trains the actor on a neighborhood of each reference-to-target path, so it tolerates the shifted reference actions and intermediate states that arise under new layouts. The ablation in our response to W2 isolates both effects on LIBERO-PRO.
Additional OOD benchmarks. Thank you for proposing two additional simulation benchmarks to evaluate on. We agree that LIBERO-Plus and RoboTwin-Hard results would further strengthen our OOD claim. However, with limited computational budget for evaluation, we prioritized expanding LIBERO-PRO from 10 to 36 layouts (W1; Appendices G and I) during this discussion period. We believe that a comparison between LIBERO and LIBERO-PRO, two benchmarks with clear lineage, can demonstrate SCOUT's OOD performance through relative performance lead against baselines.
Summary. Thank you again for your recognition of our innovative mathematical formulation, and your insightful feedback on presenting our motivation and the need for our method. We hope our clarifications, evaluation on simpler alternative baselines, and additional ablation studies address your concerns. We are happy to answer any further questions.
Response (1/3)
Thank you, Reviewer xZ7K, for your detailed and constructive review. We appreciate your recognition of our method innovation in mathematical formulation for VLA refinement and extensive evaluations. We now address each of your comments below in detail.
Thank you, Reviewer xZ7K, for raising this point. We agree the original framing of the motivation was broader than intended, and we have revised it in Section 1. We specifically study the improvement of precise, contact-rich manipulation using a fairly competent frozen VLA policy. In precise, contact-rich manipulation, pretrained policies often fail because of small errors at precision-critical stages, such as near-miss grasps and last-millimeter misalignment (Ankile et al., 2024; Parés-Morlans et al., 2026; Xu et al., 2026; Zhang et al., 2025). Our experiments show that critic-guided corrections around the VLA's reference action are effective, and this guides SCOUT's training target design. Despite refinements being locally guided, we show large gains in success rates, including on tasks where the frozen policy almost never succeeds.
Following Reviewer n7Tn's question on task selection, we expanded our evaluation during the discussion period using the same frozen-policy screening as the originally reported tasks, which excludes tasks where π0.5 always or never succeeds. This grows LIBERO from 20 to all 26 eligible tasks and LIBERO-PRO from 10 to 36 layouts (Appendix I). We report simulation success rates (%):
SCOUT achieves large success rate gains under LIBERO-PRO distribution shifts, even when the frozen policy almost never succeeds. The last row shows that a moderate directed step works best, as both a smaller and a larger step reduce success, suggesting that critic guidance is most useful within a moderate neighborhood of the reference action. We have added the full per-task results to Appendix G (Tables 9–10) and the step-size study to Appendix H.2 (Table 15) of the updated manuscript.
On the ceiling of the method, we agree that tasks the base policy never completes require acquiring a new skill that local refinement cannot sufficiently address. This is outside of SCOUT's scope of study, and we now state this in Section 5 (Limitations and Future Work).
References