acceptodds
Under review as a conference paper at ICLR 2027

SCOUT: CRITIC-GUIDED SCHRÖDINGER TRANSPORT FOR ONLINE VLA REFINEMENT

Abstract

Contact-rich and dynamic manipulation often requires well-crafted refinement methods that can learn from sparse rewards while preserving efficient execution beyond a large pretrained policy. We introduce SCOUT, a lightweight framework for refining pre-trained vision-language-action (VLA) policies online through critic-guided Schrödinger transport. SCOUT formulates policy improvement as a Schrödinger bridge from the VLA's reference action chunk to a distribution of promising corrections, where a critic proposes and reweights multiple candidate action chunks, thereby constructing a training target that captures coordinated changes across action space and time. We exploit the point-source structure induced by conditioning on the VLA reference action, enabling tractable bridge regression toward an online critic-weighted target. This enables simulation-free learning through endpoint regression on Brownian interpolants without iterative coupling training. We evaluate SCOUT on simulated benchmarks and real-world precise manipulation tasks. SCOUT improves success rates over RLT and DSRL baseline methods. It achieves comparable measured inference latency to RLT and improves real-world active-execution throughput over RLT and both DSRL variants. Together, these results demonstrate that reference-conditioned action transport strengthens effective online refinement with low deployment overhead. More details and visualizations can be found on our website: https://scout-vla.github.io.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot learning, vision-language-action models, online reinforcement learning, Schrödinger bridges, action-space refinement

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_xZ7KOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SCOUT, an online RL module that keeps a pretrained VLA frozen and learns a lightweight refiner which transports the VLA's reference action chunk toward a critic-weighted target distribution via a point-source Schrödinger bridge. Experiments on LIBERO, LIBERO-PRO, and four real-world bimanual tasks show gains in success rate and execution throughput over online-refinement baselines.

Critical strengths
  1. The paper attempts to apply a new mathematical approach to online RL for VLAs, which looks interesting.

  2. The real-world experiments report results over 50 trials, reducing random error.

Critical weaknesses
  1. The motivation is not convincing, and the framing limits the paper's perceived scope. The claim that many failures can be handled by small-magnitude refinement is not well supported. And as a result, the proposed method appears applicable only to local small-magnitude correction, which suggests a low ceiling and weakens the overall motivation.

  2. The use of the Schrödinger bridge is insufficiently justified and not directly compared against simpler alternatives. While there is a mathematical basis for the Schrödinger bridge, it is hard to see why this particular formulation is required. Moreover, weight-adjustment-based RL methods such as AWR, CFG are not directly compared against, which makes the use of the Schrödinger bridge appear unmotivated. In addition, it is unclear why DSRL_RLT matches SCOUT on LIBERO but underperforms it on LIBERO-Pro. The paper offers no experimental phenomenon to help the reader understand this discrepancy.

Questions for authors
  1. Why are existing online refinement methods insufficient, such that the Schrödinger bridge formulation is needed?

  2. Why does the Schrödinger bridge yield better performance under distribution shift (OOD)? Have you tried other benchmarks with OOD settings, such as LIBERO-Plus, RoboTwin Hard, etc.?

6 comments
AuthorsOpenReview ↗

Response (1/3)

Thank you, Reviewer xZ7K, for your detailed and constructive review. We appreciate your recognition of our method innovation in mathematical formulation for VLA refinement and extensive evaluations. We now address each of your comments below in detail.

W1. The motivation is not convincing, and many failures can be addressed with small-magnitude refinement is not well supported.

Thank you, Reviewer xZ7K, for raising this point. We agree the original framing of the motivation was broader than intended, and we have revised it in Section 1. We specifically study the improvement of precise, contact-rich manipulation using a fairly competent frozen VLA policy. In precise, contact-rich manipulation, pretrained policies often fail because of small errors at precision-critical stages, such as near-miss grasps and last-millimeter misalignment (Ankile et al., 2024; Parés-Morlans et al., 2026; Xu et al., 2026; Zhang et al., 2025). Our experiments show that critic-guided corrections around the VLA's reference action are effective, and this guides SCOUT's training target design. Despite refinements being locally guided, we show large gains in success rates, including on tasks where the frozen policy almost never succeeds.

Following Reviewer n7Tn's question on task selection, we expanded our evaluation during the discussion period using the same frozen-policy screening as the originally reported tasks, which excludes tasks where π0.5\pi_{0.5} always or never succeeds. This grows LIBERO from 20 to all 26 eligible tasks and LIBERO-PRO from 10 to 36 layouts (Appendix I). We report simulation success rates (%):

Task setFrozenSCOUT
LIBERO-PRO, original 10 layouts33.490.8
LIBERO-PRO, 26 new layouts44.986.5
8 LIBERO/LIBERO-PRO tasks with frozen success < 10%2.966.4
Directed step η = 0.05 / 0.15 (default) / 0.3, 10 LIBERO/LIBERO-PRO tasks—39.8 / 81.7 / 53.0

SCOUT achieves large success rate gains under LIBERO-PRO distribution shifts, even when the frozen policy almost never succeeds. The last row shows that a moderate directed step works best, as both a smaller and a larger step reduce success, suggesting that critic guidance is most useful within a moderate neighborhood of the reference action. We have added the full per-task results to Appendix G (Tables 9–10) and the step-size study to Appendix H.2 (Table 15) of the updated manuscript.

On the ceiling of the method, we agree that tasks the base policy never completes require acquiring a new skill that local refinement cannot sufficiently address. This is outside of SCOUT's scope of study, and we now state this in Section 5 (Limitations and Future Work).

References

  • Ankile et al. From Imitation to Refinement: Residual RL for Precise Assembly. arXiv:2407.16677, 2024.
  • Parés-Morlans et al. Demystifying When and Why VLAs Fail in Contact-Rich Tasks and How to Fix Them. arXiv:2608.01402, 2026.
  • Xu et al. RL Token: Bootstrapping Online RL with Vision-Language-Action Models. arXiv:2604.23073, 2026.
  • Zhang et al. Experiences from Benchmarking Vision–Language–Action Models for Robotic Manipulation. arXiv:2511.11298, 2025.
AuthorsOpenReview ↗

Response (2/3)

W2. The Schrödinger bridge is insufficiently justified against simpler alternatives like AWR or CFG. Unclear why DSRLRLTDSRL_{RLT} matches in LIBERO but underperforms in LIBERO-PRO.

Thank you, Reviewer xZ7K, for requesting clarification on the importance of the Schrödinger bridge in SCOUT. We first conduct a thorough ablation comparison with Conditional Flow-Matching (CFM) and compare with AWR on LIBERO and LIBERO-PRO to show the advantage of the reference-source Schrödinger bridge. Then, we provide analysis on why SCOUT performs better in out-of-distribution robustness compared to DSRLRLTDSRL_{RLT}.

Ablation and Simpler Alternatives. Starting from Gaussian-source CFM, we add one component at a time until reaching full SCOUT, and report success rates on simulation and real-world tasks. Due to computational resource limitations, the ablation uses 10 LIBERO tasks and 10 LIBERO-PRO layouts from our original evaluation set, and two real-world tasks. Reference-centered CFM centers CFM's Gaussian source at the VLA reference action; SCOUT, no bridge noise uses SCOUT's point source, endpoint objective, and two-step readout but removes the interpolation noise (σ = 0).

Actor variantLIBERO (10 tasks)LIBERO-PRO (10 layouts)Hardware (2 tasks)
Gaussian-source CFM83.454.363
Reference-centered CFM76.375.464
SCOUT, no bridge noise (σ = 0)68.976.270
SCOUT88.090.879

This ablation shows that both the reference-action start and the bridge noise are needed. While Gaussian-source CFM performed well in LIBERO, centering the source at the reference and switching to SCOUT's point source and endpoint objective contributed more than 20 percentage points in LIBERO-PRO layouts, and the Brownian bridge noise contributed another 14.6 points. Real-world evaluation success rates support this conclusion.

Method150 ep.300 ep.
Frozen π0.5\pi_{0.5}36.836.8
AWR (deterministic / stochastic readout at 300 ep.)32.634.7 / 37.3
SCOUT74.589.4

Compared to AWR, SCOUT shows a large performance advantage and continues to improve with training. On the same 10 LIBERO tasks and 10 LIBERO-PRO layouts, SCOUT improves mean success over the frozen policy by 52.6 percentage points, while AWR remains near the frozen policy (32.6–37.3% vs. 36.8%). AWR only reweights actions that were already executed, so it cannot propose corrections outside them, whereas out-of-distribution VLA refinement often requires small corrections outside the VLA's output distribution. We updated Section 4.3 (Table 3a) and Appendix M (Table 29) of the manuscript to include these ablations, and Table 3b of the updated manuscript also includes direct critic-weighted regression onto the same candidates.

Regarding CFG, we found several methods named CFG in related contexts. We interpreted CFG as classifier-free guidance, which is closely related to the conditional flow matching (CFM) actor we evaluate above. We would be happy to provide additional experiments if you had a different method in mind.

SCOUT outperforms DSRLRLTDSRL_{RLT} when successful actions lie outside of the VLA action distribution. DSRLRLTDSRL_{RLT} steers the input noise of the frozen policy, so it can only select actions the frozen policy can already generate. This suffices when in distribution. However, under a layout shift, the needed correction may lie outside that set. We report simulation success rates as layout shift and difficulty increase:

Task setSCOUTDSRLRLTDSRL_{RLT}
LIBERO, all 26 tasks87.386.5
LIBERO-PRO, all 36 layouts87.783.7
LIBERO-PRO, 13 hard layouts (frozen ≤ 30%)78.271.2

SCOUT's lead over DSRLRLTDSRL_{RLT} grows from 0.8 points on in-distribution LIBERO to 4.0 points under LIBERO-PRO layout shifts and 7.0 points on the hardest LIBERO-PRO layouts. We have added these analyses to Section 4.1 (Table 1), with per-layout results and paired statistics in Appendices G and J (Tables 10 and 22).

AuthorsOpenReview ↗

Response (3/3)

Q1. Why are existing online refinement methods insufficient, such that the Schrödinger bridge formulation is needed?

Thank you, Reviewer xZ7K, for this question. Refining a frozen VLA for precise manipulation tasks requires (i) starting from the VLA's reference action to preserve its competence in high-success scenarios, (ii) reaching corrections the VLA cannot generate itself when task success requires so, and (iii) adaptively balancing critic-guided corrections against the reference and exploratory alternatives. Each existing family lacks one of these properties, while SCOUT's point-source bridge enables all three:

Method(i) Starts from VLA action(ii) Beyond VLA outputs(iii) Actor updateLIBERO-PRO mean ↑Worst layout ↑Tasks below frozen ↓
DSRLNoNoSAC on input noise72.648
DSRLRLTDSRL_{RLT}Partly (via RL token)NoSAC on input noise83.7177
RLTYesYesCritic gradient (TD3+BC)69.516
CFMNoYesFlow matching from Gaussian noise74.329
SCOUTYesYesCritic-weighted bridge87.7411

Noise-space steering (DSRL, DSRLRLTDSRL_{RLT}) can only select actions the frozen VLA already generates, which is why DSRLRLTDSRL_{RLT} nearly matches SCOUT on LIBERO but falls behind under layout shift, as we analyze for W2. RLT can move beyond the VLA's outputs, but its deterministic actor follows the critic's gradient under a fixed critic/behavior-cloning weighting, so it cannot adjust per context between staying at the reference and following the critic; as a result, it does not consistently preserve the base model's competence (regresses on 6 of 62 tasks by up to 19 points). CFM learns from the same critic-weighted candidates as SCOUT but generates actions from Gaussian noise and uses the reference only as conditioning, so it does not start from the VLA's competent action (13.4 points below SCOUT). SCOUT, by construction, weights the reference, exploratory Gaussian, and critic-gradient proposals per context, so how far it moves from the reference is set by the critic for each context rather than by a fixed regularization weight, achieving the highest mean and worst-layout success with only one regression. This distinction is briefly discussed in Section 2, with supporting results in Section 4.1 (Table 1, Figure 2).

Q2. Why does the Schrödinger bridge yield better performance under distribution shift (OOD)? Have you tried other benchmarks with OOD settings, such as LIBERO-Plus, RoboTwin Hard, etc.?

Thank you, Reviewer xZ7K, for this question and suggestion to evaluate on additional benchmarks. SCOUT's robustness under distribution shift comes from its two bridge components. The point source at the reference action preserves the VLA's competence, and the bridge noise trains the actor on a neighborhood of each reference-to-target path, so it tolerates the shifted reference actions and intermediate states that arise under new layouts. The ablation in our response to W2 isolates both effects on LIBERO-PRO.

Additional OOD benchmarks. Thank you for proposing two additional simulation benchmarks to evaluate on. We agree that LIBERO-Plus and RoboTwin-Hard results would further strengthen our OOD claim. However, with limited computational budget for evaluation, we prioritized expanding LIBERO-PRO from 10 to 36 layouts (W1; Appendices G and I) during this discussion period. We believe that a comparison between LIBERO and LIBERO-PRO, two benchmarks with clear lineage, can demonstrate SCOUT's OOD performance through relative performance lead against baselines.

Summary. Thank you again for your recognition of our innovative mathematical formulation, and your insightful feedback on presenting our motivation and the need for our method. We hope our clarifications, evaluation on simpler alternative baselines, and additional ablation studies address your concerns. We are happy to answer any further questions.

Reviewer_n7TnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to improve robot foundation models such as VLAs with online reinforcement learning. The authors propose SCOUT, a lightweight framework for refining pre-trained vision-language-action (VLA) policies online through critic-guided Schrödinger transport. SCOUT formulates policy improvement as a Schrödinger bridge from the VLA's reference action chunk to a distribution of promising corrections, where a critic proposes and reweights multiple candidate action chunks, thereby constructing a training target that captures coordinated changes across action space and time. Experiments on LIBERO, LIBERO-Pro, and real robot tasks demonstrate the proposed method's superior performance.

Critical strengths
  1. The paper studies online reinforcement learning improvement of robot foundation models such as VLAs, which is an important and timely research topic.

  2. The authors conduct extensive experiments, especially real robot experiments, to demonstrate the effectiveness of their proposed method.

Critical weaknesses
  1. The authors need to discuss action candidate set generation more. For example, how they design it and how different design schema impacts final performance of the proposed method. It is very important to understand whether and how much action candidate set generation impacts final performance.

  2. In the ablation studies, Schrodinger transport performs much better than CFM, and the authors do not provide much analysis in these experiment results. It is important to understand what makes Schrodinger transport so different from other diffusion or flow matching methods.

  3. The authors only perform experiments on part of LIBERO and LIBERO-pro tasks. How these tasks are selected ?

  4. It seems that the proposed method does not have much performance advantage compared to baselines. For example, it performs the same as DSRL (RLT) on LIBERO tasks, and it does not show much advantage on real robot long-horizon and precise manipulation tasks compared to best performing baseline.

Questions for authors

See the weakness points I list.

12 comments
AuthorsOpenReview ↗

Response (1/3)

Thank you, Reviewer n7Tn, for your thoughtful review and constructive feedback. We appreciate your recognition of our extensive experiments, especially real robot experiments, that highlight the strengths and real-world effectiveness of our method. We now address each of your comments below in detail.

W1. Discuss action candidate set generation and how it impacts final performance.

Thank you for requesting clarification on the candidate-set generation design. We agree that candidate-set selection is an important engineering design of SCOUT and our earlier explanation was brief. We now report formal ablation studies on both candidate composition and candidate-set size:

1. Candidate composition. We vary the ratio of gradient-guided and Gaussian proposals while holding K=12K=12 fixed. Across 10 LIBERO/LIBERO-PRO tasks, the chosen 4G+6N configuration achieves 81.7% mean success, compared with 62.6% for gradient-only (10G+0N), 34.6% for Gaussian-only (0G+10N), and other mixtures evaluated. We find that the two types of proposals are complementary and both contribute to final performance improvement. We updated Section 4.3 (Figure 5b) of the manuscript with more analysis, with the full candidate-set study in Appendix H.2 (Table 15) and Appendix L (Table 26).

2. Candidate-set size. We carry out studies with K ∈ {4, 6, 7, 12, 17, 22, 42} to find a sufficiently good candidate-set size. Mean success on simulation increases from 22.3% at K=4K=4 to 81.7% at K=12K=12. Improvements largely saturate beyond K=12K=12. We updated Section 4.3 (Figure 5a) of the manuscript with details of this ablation, with configurations in Appendix L.

These experiments demonstrate that our 4G+6N mixture candidate design achieves the highest mean performance with a relatively modest candidate-set size budget, providing a favorable tradeoff between proposal-evaluation cost and final success rate.

W2. Schrödinger transport performs much better than CFM in ablation, but more analysis on why it differs from CFM is important.

Thank you, Reviewer n7Tn, for pointing this out. We agree that the original analysis on why point-source Schrödinger transport performs better than CFM was too brief. We substantially expanded Section 4.3 (Table 3a) with controlled ablations that identify two properties that reflect why Schrödinger transport outperforms standard CFM in online VLA refinement: 1. Reference-anchored transport (SCOUT) instead of generation from Gaussian noise (CFM).; 2. Stochastic (Brownian) bridge training instead of a deterministic interpolation path.

We evaluate the effectiveness by removing these two components from SCOUT and compare against an otherwise identically constructed Gaussian CFM baseline. The average success rate on sim and real tasks are reported in the table below:

Actor variantLIBERO (10 tasks)LIBERO-PRO (10 layouts)Hardware (2 tasks)
Gaussian-source CFM83.454.363
Reference-centered CFM76.375.464
SCOUT, no bridge noise (σ = 0)68.976.270
SCOUT88.090.879

These ablations establish two findings. First, anchoring transport at the VLA reference substantially improves adaptation under distribution shift, as reference-centered CFM improved by 21.1 points in LIBERO-PRO but decreased by 7.1 points in LIBERO. Second, stochastic bridge training is essential, as removing it reduces SCOUT's success by 19.1 points on LIBERO and 14.6 points on LIBERO-PRO. Real-world ablation results reinforce this conclusion.

In summary, our experiments identify two key components that explain why SCOUT substantially outperforms standard CFM formulation: anchoring on the action prior from a pre-trained VLA and stochastic bridge training that produces a robust transport. We have updated the manuscript with more detailed analysis in Section 4.3, with per-task results in Appendix K (Tables 24–25).

AuthorsOpenReview ↗

Response (2/3)

W3. The authors only perform experiments on part of LIBERO and LIBERO-pro tasks. How these tasks are selected?

Thank you, Reviewer n7Tn, for raising the candid concern regarding our simulation task selection. We excluded tasks where the frozen base policy (π0.5\pi_{0.5}) achieved either 0% or 100% success during initial evaluation. Tasks with 0% success require learning new skills rather than action refinement, while tasks with 100% success leave no room for improvement. Of the 100 LIBERO tasks, 26 are eligible, and we randomly selected 20 for the original evaluation; the original 10 LIBERO-PRO layouts came from earlier screens with the same criterion. To address potential selection bias, we additionally evaluated the remaining 6 LIBERO tasks, and screened and evaluated 26 additional LIBERO-PRO object-swapped layouts with the same criterion.

Task setFrozenSCOUTDSRLRLTDSRL_{RLT}CFMRLTDSRL
LIBERO, original 2053.192.392.387.381.283.5
LIBERO, 6 additional50.871.067.365.363.562.5
LIBERO, all 2652.687.386.582.277.178.6
LIBERO-PRO, original 1033.490.883.154.359.174.5
LIBERO-PRO, 26 additional44.986.583.982.073.571.9
LIBERO-PRO, all 3641.787.783.774.369.572.6

SCOUT achieves the highest mean success rate across both original and newly evaluated task subsets. The improvements are not dependent on the original task selection. We have clarified the selection criteria and added the expanded evaluations to Section 4.1 (Table 1), Appendix G (Tables 9–10), and Appendix I (Tables 19–21) of the updated manuscript.

W4. Proposed method does not have much performance advantage in sim or real-world tasks compared to the best baseline.

Thank you, Reviewer n7Tn, for this feedback. We agree that in the original manuscript, SCOUT's success rate leads DSRLRLTDSRL_{RLT} by only a few percentage points in simulation and holds minor throughput advantages.

However, we respectfully emphasize that DSRLRLTDSRL_{RLT} is a baseline we created to combine two popular published baselines, RLT and DSRL. DSRLRLTDSRL_{RLT} has a uniquely strong performance that, to the best of our knowledge, is not studied in existing literature. However, DSRLRLTDSRL_{RLT} requires two VLA inference passes whereas SCOUT only requires one. When comparing against DSRL and RLT, two popular VLA refinement baselines that have a similar computational budget to SCOUT (one VLA pass), SCOUT outperforms both baselines with much bigger advantages.

To further address your concern, even when comparing against DSRLRLTDSRL_{RLT}, we conduct further experiments to highlight that SCOUT's advantage extends beyond in-distribution success rates to stronger generalization, more reliable refinement, and substantially better real-world execution efficiency on challenging tasks. We report an extended 62-task simulation study, as well as two additional challenging long-horizon manipulation tasks in the real-world. Our evaluations highlight SCOUT's three key performance advantages:

1. SCOUT matches DSRLRLTDSRL_{RLT} in distribution and extends the lead under distribution shift. Mean success rate (%) on 26 LIBERO tasks and 36 LIBERO-PRO layouts:

BenchmarkFrozen π0.5\pi_{0.5}RLTDSRLDSRLRLTDSRL_{RLT}SCOUT
LIBERO (26)52.677.178.686.587.3
LIBERO-PRO (36)41.769.572.683.787.7
LIBERO-PRO Hard (13; frozen ≤30%)13.544.453.071.278.2

SCOUT outperforms DSRLRLTDSRL_{RLT} by 4.0 points across all LIBERO-PRO layouts (p = 0.017), and the margin grows to 7.0 points on the 13 hard layouts where the frozen policy succeeds at most 30% of the time (Table 1). The lead holds across three independent training runs on five layouts (84.7% ± 1.2% vs. 80.5% ± 2.2%; Appendix H.5). Against RLT and DSRL, which use one VLA pass like SCOUT, SCOUT leads in LIBERO-PRO by 18.2 and 15.1 points (both p < 0.001). We update Section 4.1 and Appendix J (Table 22) of the manuscript with these results.

AuthorsOpenReview ↗

Response (3/3)

(Continuing our response to W4.)

2. SCOUT rarely degrades the frozen policy performance. Average success can hide regressions on tasks the frozen policy already solves. However, the uncertainty of a large performance regression often hinders real-world deployment of refinement methods. We compare each method with the frozen policy on all 62 simulated tasks/layouts to measure the amount and magnitude of success rate regressions:

62 simulated tasks/layoutsRLTDSRLDSRLRLTDSRL_{RLT}SCOUT
Tasks SR below frozen policy ↓6871
Worst SR change (points) ↑−19−25−21−3
Worst LIBERO-PRO SR (%) ↑141741
Mean on 8 tasks with frozen < 10% ↑31.443.156.066.4

SCOUT falls below the frozen policy on only one of 62 tasks, by 3 points, whereas the baselines regress on 6–8 tasks by up to 19–25 points (Figure 2). Most baseline regressions occur on tasks where the frozen policy is already strong, but SCOUT's formulation allows the refiner to stay close to the reference. On the other hand, SCOUT also reaches the highest success on the tasks where the frozen policy almost always fails. SCOUT consistently raises performance lower bounds without sacrificing existing competence. We update Section 4.1 (Figure 2) of the manuscript with this analysis, with per-task results in Appendix G (Tables 9–10).

3. On challenging precise, long-horizon manipulation, SCOUT achieves up to 4× the success rate and 2.1× the throughput of the strongest baseline (DSRLRLTDSRL_{RLT}). We added two multi-stage tasks that require precise placement at every stage, Cup-Stacking (four stacking stages, then inversion onto a bottle) and Shape-Insertion (close a box, insert a shape, rotate the box, insert a second shape; success counted only when all steps succeed) and report the success rate and throughput of each method:

Success (%) / throughputFrozen π0.5\pi_{0.5}RLTDSRLDSRLRLTDSRL_{RLT}SCOUT
Cup-Stacking20 / 330 / 832 / 564 / 970 / 19
Shape-Insertion4 / 00 / 00 / 010 / 040 / 6
All six tasks48.3 / 9958.3 / 16554.7 / 13053.3 / 8073.7 / 196

In two new tasks where refinement must coordinate several stages, SCOUT demonstrates the greatest lead in success-rate and throughput. It achieves 4× DSRLRLTDSRL_{RLT}'s success on Shape-Insertion and 2.1× throughput on Cup-Stacking. In the extremely challenging Shape-Insertion task, SCOUT is the only method among those compared with a positive throughput. Across all six tasks, it leads the strongest baseline (RLT) by 15.3 points in success rate and has the highest throughput among the compared methods on every task (Figure 4; Table 11). This includes the two precise insertion tasks from the original manuscript, where SCOUT's success is within one trial of the best baseline (Ethernet-Insertion: 33 vs. 34 of 50; LEGO-Click: 46 vs. 45 of 50) while completing more insertions per unit time. Importantly, DSRLRLTDSRL_{RLT} has less than half of SCOUT's aggregated throughput across all six tasks (80 vs. 196), due to its VLA inference overhead.

In summary, the strongest baseline DSRLRLTDSRL_{RLT} matches SCOUT on in-distribution LIBERO, but clearly falls behind in settings that motivate VLA refinement: SCOUT leads under distribution shift (LIBERO-PRO), regresses on 1 of 62 simulated tasks versus baselines' 6–8, and achieves up to 4× the success rate and 2.1× the throughput on challenging real-world tasks, all while using one VLA request instead of two. We have added these results to Sections 4.1–4.2 and Appendices G–J.

Summary. Thank you again for recognizing the importance of online VLA refinement and our extensive real-robot experiments, and for your constructive questions on candidate-set design, the advantage over CFM, task selection, and performance gains. We hope our candidate-set ablations, component-wise actor ablation, transparently screened and expanded evaluation, and new long-horizon real-world tasks address your concerns. We are happy to answer any further questions.

Reviewer_VvZ8OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SCOUT learns to refine the action chunks of a frozen vision-language-action policy using sparse task rewards. During training, a critic scores 12 candidate chunks, and a small network learns to predict a critic-weighted endpoint from a noisy interpolation starting at the reference action. At deployment, two evaluations of this network refine a single VLA prediction. The authors evaluate SCOUT on 20 LIBERO tasks, 10 LIBERO-PRO layouts, and four real-robot tasks.

Critical strengths
  1. Table 13 makes a persuasive case for the actor design. Across five LIBERO-PRO layouts and three training seeds per layout, SCOUT achieves 82.5% success, compared with 67.7% for direct critic-weighted regression and 74.1% for the hard-target bridge. The largest gains occur on the two hardest layouts. This comparison shows that how the actor learns from the candidates matters even when the candidate-generation procedure is shared.
  2. The method improves adaptation while keeping deployment inexpensive. After 300 online episodes per layout, SCOUT reaches 90.8% success on held-out initial states in LIBERO-PRO, compared with 83.1% for the strongest baseline. It requires one VLA pass per decision. The hardware experiments also demonstrate a useful gain on a precise task: Ethernet insertion improves from 18/50 successes with the frozen policy to 33/50 with SCOUT. These results motivate my recommendation of weak acceptance.
Critical weaknesses
  1. Table 1 leaves the source of the 36.5-point advantage over Gaussian-source flow matching on LIBERO-PRO unresolved. The comparison changes the starting action, the interpolation noise, and the prediction target together. Starting from a competent VLA action could itself explain much of the difference. Table 13 supports the full actor design, but it leaves open how much the Brownian interpolation contributes.
  2. The main comparisons generally use one training run per method and task. Table 15 documents repeated SCOUT runs on five layouts, but the training variability of the strongest baseline remains unclear. This matters when interpreting the 7.7-point lead on LIBERO-PRO. On hardware, several differences between learned methods are small: SCOUT and RLT both succeed on 45/50 stacking trials, and their average episode times in the separate timing study are 13.29 and 13.20 seconds, respectively.
Questions for authors
  1. Do you have a comparison that keeps the reference-action start, endpoint objective, candidate weighting, and two-step readout fixed, while removing the interpolation noise in Eq. (6) during training? Results on the two hardest layouts in Table 13 would help establish what the Brownian interpolation contributes.
  2. Do you have paired training-seed results for SCOUT and DSRL-RLT on the difficult LIBERO-PRO layouts, using the same held-out initial states for each pair? I would like to know whether the relative advantage persists across independently trained policies. The repeated SCOUT runs in Table 15 establish only one side of this comparison.
2 comments
AuthorsOpenReview ↗

Response (1/2)

Thank you, Reviewer VvZ8, for your thoughtful review. We appreciate your recognition of the effectiveness of the Schrödinger bridge formulation and the meaningful improvements in real-world performance. We address each of your comments below.

W1. The source of 36.5-point advantage over CFM on LIBERO-PRO is unresolved, with many simultaneous changes.

Thank you, Reviewer VvZ8, for raising this point in our original report. To pin down the contributions you refer to, we conducted a comprehensive ablation that adds one component at a time, both on the simulator and in real-world tasks:

Actor variantLIBERO (10 tasks)LIBERO-PRO (10 layouts)Hardware (2 tasks)
Gaussian-source CFM83.454.363
Reference-centered CFM76.375.464
SCOUT, no bridge noise (σ = 0)68.976.270
SCOUT88.090.879

Of the 36.5-point gap on LIBERO-PRO, centering the source at the reference action accounts for 21.1 percentage points, switching to SCOUT's point source, endpoint objective, and two-step readout for 0.8, and the Brownian interpolation for 14.6. As anticipated, a competent reference action explains more than half of the difference, and the interpolation noise contributes most of the remainder. On in-distribution LIBERO, the reference-centered source alone lowers success by 7.1 points, and the gain comes from the interpolation noise (+19.1 points over the noise-free variant). The hardware results follow the same ordering. We have added this decomposition to Section 4.3 (Table 3a), with per-task results in Appendix K (Tables 24–25).

W2. Training variability of the strongest baseline remains unclear, which matters when interpreting 7.7-point lead on LIBERO-PRO. Several differences between methods on hardware experiments are small: SCOUT and RLT both succeed on 45/50 stacking trials, and their average episode times in the separate timing study are 13.29 and 13.20 seconds, respectively.

Thank you, Reviewer VvZ8, for raising the concern regarding training variability and real-world performance advantages. We address the two in turn.

The LIBERO-PRO advantage holds across independent training runs. As you suggested in Q2, we trained SCOUT and DSRLRLTDSRL_{RLT} three times each on five LIBERO-PRO layouts, using matched held-out evaluation states for each pair (success %, mean ±\pm SD over runs):

LIBERO-PRO layoutFrozen π0.5\pi_{0.5}SCOUTDSRLRLTDSRL_{RLT}
T1: Cream cheese and butter into basket (hardest)15.756.3 ± 3.128.3 ± 2.3
T2: Cream cheese into bowl (bottle swap)40.073.3 ± 4.578.3 ± 4.9
T3: Cream cheese into basket (milk swap)57.096.0 ± 2.097.3 ± 2.5
T4: Salad dressing into basket59.097.7 ± 2.198.7 ± 2.3
T5: Butter into basket53.0100.0 ± 0.0100.0 ± 0.0
Mean over five layouts44.984.7 ± 1.280.5 ± 2.2

SCOUT achieves higher aggregate success in all three runs, driven by the hardest layout, where the frozen VLA struggles most (56.3% vs. 28.3%). We acknowledge that DSRLRLTDSRL_{RLT} slightly exceeds SCOUT on three of the other four layouts (T2–T4), by at most 5 points, and the two methods tie on T5. Note that DSRLRLTDSRL_{RLT} is a particularly strong baseline that we constructed by combining DSRL and RLT, and it requires two VLA requests per decision, twice SCOUT's inference cost. We have added these paired runs to Appendix H.5 (Table 18) of the updated manuscript.

Beyond mean success, SCOUT is more robust under distribution shift and rarely degrades the frozen policy. During this discussion period, we expanded our evaluation to 26 LIBERO tasks and 36 LIBERO-PRO layouts using the same task-selection criterion (Appendix I). SCOUT leads DSRLRLTDSRL_{RLT} by 7.0 points on the 13 hard LIBERO-PRO layouts where the frozen policy succeeds at most 30% of the time (Section 4.1, Table 1), and falls below the frozen policy on only 1 of 62 simulated tasks, versus 6–8 for the baselines (Figure 2). We have updated Section 4.1 (Table 1, Figure 2) of the manuscript with these results, with the full per-task results, task screening, and paired statistics in Appendices G, I, and J.

AuthorsOpenReview ↗

Response (2/2)

(Continuing our response to W2.)

In the real world, the stacking tie is not representative of the six-task results. Success rate (%) and throughput (successful completions per 10 minutes of execution) on our two new multi-stage tasks and across all six real-world tasks:

Success (%) / throughputFrozen π0.5\pi_{0.5}RLTDSRLDSRLRLTDSRL_{RLT}SCOUT
Cup-Stacking20 / 330 / 832 / 564 / 970 / 19
Shape-Insertion4 / 00 / 00 / 010 / 040 / 6
All six tasks48.3 / 9958.3 / 16554.7 / 13053.3 / 8073.7 / 196

SCOUT and RLT tie on Cube-Stacking, but SCOUT has the highest aggregate success across the six real-world tasks (73.7% vs. 58.3% for RLT and 53.3% for DSRLRLTDSRL_{RLT}), with the largest gaps on the new multi-stage tasks. The near-identical stacking episode times (13.29 s vs. 13.20 s) are expected: both SCOUT and RLT use critic-guided action refinement that speeds up physical execution, and both roughly halve the frozen policy's episode time (26.13 s) with one VLA request per decision. In contrast, DSRLRLTDSRL_{RLT} needs two VLA requests per decision (2.0–2.6$\times$ higher measured VLA latency), and its stacking episodes take 33.28 s, slower than the frozen policy. Across all six tasks, SCOUT achieves a higher aggregate throughput than every baseline (196 vs. 165 for RLT). We have updated Section 4.2 (Figure 4) of the manuscript with these real-world results, with per-task success, throughput, timing, and inference cost in Appendix G (Tables 11–13).

Q1. Do you have a comparison that keeps the reference-action start, endpoint objective, candidate weighting, and two-step readout fixed, while removing the interpolation noise in Eq. (6) during training?

Thank you, Reviewer VvZ8, for this suggestion. Yes: this is the no bridge noise (σ=0\sigma=0) variant in our response to W1, which keeps the reference-action start, endpoint objective, candidate weighting, and two-step readout fixed and only replaces the Brownian interpolation in Eq. (4) of the updated manuscript (Eq. (6) in the original) with the straight line Xt=(1−t)aref+tX1X_t=(1-t)a_{\mathrm{ref}}+tX_1. Removing the interpolation noise lowers success on LIBERO, LIBERO-PRO, and real-world tasks; please see W1 for the analysis. On the two layouts from W2 where the frozen policy is weakest (T1 and T2), which W1 does not break out separately, removing the noise has the same effect:

LIBERO-PRO layoutSCOUT (Eq. 4)No interpolation noise (σ=0\sigma=0)Improvement
T1: Cream cheese and butter into basket (hardest)66%13%+53 pp
T2: Cream cheese into bowl (bottle swap)61%30%+31 pp
Mean (2 layouts)63.5%21.5%+42.0 pp

We have updated Section 4.3 (Table 3a) of the manuscript with this ablation, with per-task results for all 20 simulated tasks in Appendix K (Table 24).

Q2. Do you have paired training-seed results for SCOUT and DSRL-RLT on the difficult LIBERO-PRO layouts, using the same held-out initial states for each pair?

Thank you, Reviewer VvZ8, for this suggestion. Following your suggestion, we conducted three independent training runs for SCOUT and DSRLRLTDSRL_{RLT} on five LIBERO-PRO layouts, using matched held-out evaluation states for each pair. Please see our response to W2 for the results and analysis. We have added these paired runs to Appendix H.5 (Table 18) of the updated manuscript.

Summary. Thank you again for your recognition of our actor design and SCOUT's efficient deployment, and for your suggestions on isolating the contribution of the Brownian interpolation and on training variability. We hope our component-wise ablation, paired training runs with DSRLRLTDSRL_{RLT}, and expanded simulation and real-world evaluations address your concerns. We are happy to answer any further questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

To the Area Chair and Reviewers: We thank the reviewers for their thoughtful feedback and constructive engagement. We appreciate the reviewers for recognizing online refinement of VLAs as an important and timely research topic, the novel mathematical approach, and our extensive real-robot experiments. We summarize comments and responses on motivation, the Schrödinger bridge, evaluation, and efficiency.

Motivation and Scope

  • Is local refinement sufficiently motivated? (xZ7K): We revised Section 1 to focus on precise, contact-rich manipulation with a competent frozen VLA, where failures concentrate in small errors at precision-critical stages (near-miss grasps, last-millimeter misalignment). Existing pretrained VLA policies commonly exhibit such limitation in fine-manipulation, and the focus of our study is consistent with RLT.
  • Excluding 0%-success tasks (n7Tn): During the rebuttal period, we trained and evaluated all methods on 40 additional zero-success tasks/layouts (App I.2). SCOUT recovers success (27.5% vs. 17.9% for DSRL), while DSRLRLT\mathrm{DSRL}_{\mathrm{RLT}} is higher (32.9%) and makes larger corrections on the tasks it wins. Importantly, 38 out of 40 tasks randomly sampled cannot be corrected by local refinement since they involve picking up wrong objects or approaching wrong locations. We state this regime as a limitation (Sec 5) and not the focus of our study.

Why the Schrödinger Bridge

  • Which component explains the gain over CFM? (VvZ8, n7Tn, xZ7K): A component-wise ablation (Sec 4.3, Tab 3a) shows that starting from the VLA reference adds 21.1 points on LIBERO-PRO and the Brownian bridge noise in Schrödinger Bridge training process adds another 14.6 (+42 points on the two hardest layouts).
  • Simpler alternatives such as AWR (xZ7K): AWR stays near the frozen policy (32.6–37.3% vs. 36.8%) because it only reweights executed actions, while SCOUT reaches 89.4% (App M).
  • Candidate-set design (n7Tn): We report more detailed ablation studies, showing that our reported 4 gradient + 6 Gaussian mixture reaches 81.7%, versus 62.6% gradient-only and 34.6% Gaussian-only. Gains saturate at K=12K=12 (Sec 4.3, Fig 5), which we choose for efficient training and deployment.

Evaluation and Advantage over Baselines

  • Task selection (n7Tn): We expanded to all 26 eligible LIBERO tasks and 36 LIBERO-PRO layouts from our base-policy success rate screening process (App I). SCOUT has the highest mean on both (87.3%, 87.7%; App G).
  • Margin over DSRLRLT\mathrm{DSRL}_{\mathrm{RLT}} (n7Tn, xZ7K): The lead grows from 0.8 points on LIBERO to 4.0 on LIBERO-PRO (p = 0.017) and 7.0 on the hardest layouts (Tab 1). Against single-pass RLT and DSRL, SCOUT leads by 15–18 points on LIBERO-PRO.
  • Training variability (VvZ8): Across three paired training runs on five LIBERO-PRO layouts, SCOUT leads DSRLRLT\mathrm{DSRL}_{\mathrm{RLT}} (84.7 ± 1.2% vs. 80.5 ± 2.2%; App H.5).
  • Robustness: SCOUT falls below the frozen policy on only 1 of 62 simulated tasks, versus 6–8 for every baseline (Sec 4.1, Fig 2).
  • Real-world gains (n7Tn, VvZ8): We added two multi-stage tasks. Across six tasks, SCOUT achieves the highest throughput on every task, up to 2.2× that of DSRLRLT\mathrm{DSRL}_{\mathrm{RLT}}. In two long-horizon tasks that also require precise manipulation at multiple stages, SCOUT exhibits the greatest success rate and throughput advantages over all baselines.

Efficiency and Training Dynamics

  • Inference cost: SCOUT needs one VLA pass per decision; DSRLRLT\mathrm{DSRL}_{\mathrm{RLT}} needs two (2.0–2.6× higher latency, App G.4).
  • RL Training curves (n7Tn): Added in App G.3. SCOUT completes training 1.4× faster than DSRLRLTDSRL_{RLT} in wall-clock time and reaches DSRLRLT\mathrm{DSRL}_{\mathrm{RLT}}'s best training success rate with 49–76% less time (Tab 11–12).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.