acceptodds
Under review as a conference paper at ICLR 2027

Acting at the Moment of Consequence: Physical Foresight for VLA Action Refinement

Abstract

Vision-language-action (VLA) models can produce coherent manipulation behavior yet fail during physical interactions that require precise control. Refining their actions requires identifying when local adjustments can still affect an operation’s outcome. We introduce PREEMPT, a framework for selectively refining the actions of a fixed VLA policy. A V-JEPA-based anticipator foresees physical decision windows and initiates local correction. A latent world model predicts short candidate trajectories, and a completion-time evaluator relates these forecasts to the likelihood and timing of operation completion. These estimates guide smooth corrections constrained by the policy’s intended motion, while predicted trajectory diversity guides further search allocation. We evaluate PREEMPT on LIBERO and RoboCasa with , on LIBERO with OpenVLA-OFT, and on four real-robot tasks with an Agilex dual-arm robot. With , PREEMPT increases success from 96.7% to 97.1% on LIBERO and from 64.9% to 68.6% on RoboCasa. Real-robot success rises from 72.5% to 82.5%, a gain of 10 percentage points. In a LIBERO-10 scheduling comparison, window-based search uses 58.4% fewer candidate evaluations than searching at every replan.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Test-time Optimization, World Models, Physical Reasoning, Robot Manipulation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_DPUUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes PREEMPT, an inference-time framework that refines the actions of a frozen VLA policy. It has four components:

  1. A V-JEPA + GRU anticipator (PTA) predicts “physical decision windows” to decide when to search.
  2. A dual-view latent world model predicts 5-step consequences of candidate actions. The Vendi Score of these predicted trajectories schedules the interval before the next search.
  3. A completion-time evaluator, trained on completed, failed, and right-censored attempts, maps short-horizon predictions to the probability and timing of operation completion.
  4. A 12-D spline correction in the policy’s intent coordinates is optimized with CEM, using structural costs that penalize deformation and drift across replans.
Critical strengths

Strengths

  1. The motivation is well grounded. The observation that the value of correction depends on the interaction stage is intuitive, and concentrating search around critical windows is a clean idea.
  2. The completion-time evaluator is a novel design. Modeling discretized completion-time distributions with a partial-label likelihood handles completed, failed, and censored samples in a principled way, and is better motivated than direct progress regression.
  3. The intent-constrained correction balances smoothness with fidelity to the policy’s motion structure. The cumulative-offset cost addresses consistency across replans, which similar work often overlooks.
Critical weaknesses

Weaknesses

  1. Small simulation gains with unclear significance. On LIBERO, PREEMPT adds only 0.4 points for π0.5 and 0.2 points for OpenVLA-OFT, and it degrades OpenVLA-OFT on the Object suite. The appendix mentions paired bootstrap, but the main text reports no confidence intervals, so it is unclear whether the gains are significant.
  2. No component ablations. The paper does not isolate the contribution of each module. Useful comparisons would include replacing the completion-time evaluator with a simple progress or success classifier, removing the structural costs, replacing spline CEM with sampling from the policy itself, and removing Vendi-based scheduling. Without these, the necessity of the core design choices is unsupported.
  3. Problematic RoboCasa baseline. The H=5 baseline itself degrades sharply (from 454 to 319 successes), so the reported 42.3% failure reduction overstates the effect. Against the default H=50 baseline, the gain is only 26 successes, and Double Door performance decreases.
  4. Narrow scope of the window-guided computation result. Refinement is enabled on only 3 LIBERO-10 tasks (150 episodes), yet results are aggregated over all 10 tasks. Both the 58.4% reduction and the 95% vs. 92% success comparison are therefore easy to misread. Stronger baselines, such as fixed-interval or random triggering, are also missing.
Questions for authors

Questions

  1. How well calibrated is the completion-time evaluator? Do its predicted Q values match empirical completion rates?
  2. How often is a correction actually executed (J > 0), and how many executed corrections prevent a failure?
  3. How much training data do PTA, the world model, and the evaluator require on the real robot? What is the annotation cost for transferring to new tasks?
3 comments
AuthorsOpenReview ↗

Response to Reviewer DPUU (1/3)

We appreciate your recognition of the motivation for physical decision windows, the completion-time formulation, and the role of intent-constrained correction. Thank you for the constructive suggestions that prompted the component comparisons and additional analyses below.


W1: LIBERO gains and statistical uncertainty

We report paired outcomes and confidence intervals to quantify the LIBERO gains. We use LIBERO to evaluate general manipulation, where π0.5 and OpenVLA-OFT already achieve 96.7% and 96.5% success, leaving limited room for improvement. PREEMPT raises these rates to 97.1% and 96.7%, respectively. The table below reports results for 2,000 episodes per policy comparison. The confidence intervals describe the success-rate difference in percentage points. A rescue changes a baseline failure into success; a harm changes a baseline success into failure.

MethodBaselineRescuesHarms95% CI (pp)
π0.5 + PREEMPTπ0.5102[+0.06, +0.74]
OpenVLA-OFT + PREEMPTOpenVLA-OFT106[-0.19, +0.59]
OpenVLA-OFT + PREEMPTOpenVLA-OFT, H=5388[+0.84, +2.16]

We also evaluated PREEMPT on LIBERO-Plus [1], where π0.5 has more room for improvement. Average success rises from 85.2% to 88.6%, and PREEMPT exceeds the reported OpenVLA-OFT result by 17.2 percentage points. These results show a larger benefit under the variations tested by LIBERO-Plus.

MethodCameraRobotLanguageLightBackgroundNoiseLayoutAverage
OpenVLA0.83.523.08.134.815.228.516.3
OpenVLA-OFT56.431.979.588.793.375.874.271.4
π0.578.473.680.896.294.189.084.585.2
π0.5 + PREEMPT89.073.682.096.597.096.086.088.6

W2: Contributions of the individual components

We conducted the requested component ablations on RoboCasa. Each tested component contributes to the full result: success falls from 68.6% to 60.7% without the cumulative-offset cost, to 58.0% without all structural costs and to 60.3% without both predicted futures and Vendi adaptation. The completion-time evaluator improves success by 11.0 percentage points over either the binary success classifier or the progress predictor. Replacing spline CEM with policy sampling, using 256 candidates per search, reduces success by 17.6 points. The scheduling comparison in W4 tests PTA and Vendi adaptation: full PREEMPT achieves 93.6%, compared with 88.2% for fixed-interval scheduling within PDWs, which omits Vendi adaptation.

VariantSuccess (%)
Binary success classifier57.6
Progress predictor57.6
PREEMPT w/o cumulative-offset cost60.7
PREEMPT w/o structural costs58.0
PREEMPT w/o Predicted futures and Vendi adaptation60.3
Policy sampling instead of spline CEM51.0
PREEMPT (full)68.6

(1/3)

AuthorsOpenReview ↗

Response to Reviewer DPUU (2/3)

W3: RoboCasa execution horizon and continuity

Thank you for asking us to clarify the source of the RoboCasa improvement and the drop in baseline success when H changes from 50 to 5. We explain the purpose of the H=5 comparison and the policy's sensitivity to replanning below:

  • Native policy configuration. We use the publicly released LeRobot π0.5 RoboCasa checkpoint, whose prediction and default execution lengths are both 50. With H=5, the policy still predicts 50 actions but discards the remaining 45 after executing the first five. This increases replanning frequency tenfold and can disrupt temporal consistency across independently generated chunks.

  • Fairness of comparison. PREEMPT executes local corrections over five steps. We also evaluate π0.5 with five-step replanning to examine whether more frequent feedback alone can explain the improvement. This baseline matches the local correction granularity.

  • RoboCasa task characteristics. Door manipulation requires sustained contact along a constrained hinge trajectory. Interruptions from frequent replanning may help explain the decline in H=5 success, particularly on door tasks in the original Table 2.

We evaluated H=5 + continuity correction to separate continuity effects from predictive action selection:

MethodSuccess
π0.5, H=50454/700
π0.5, H=5319/700
π0.5, H=5 + continuity correction409/700
PREEMPT (full)480/700

PREEMPT retains a 10.14-percentage-point advantage over the continuity baseline. Together with the structural-cost ablations in W2, this supports a contribution beyond smoothing execution across replans.


W4: Search allocation and stronger scheduling baselines

Our original allocation study covered a limited set of tasks. To strengthen the scheduling evidence, we compared Every replan, fixed-interval scheduling, random scheduling, and fixed-interval scheduling within PDWs. Each search evaluates 4 × 64 = 256 sampled candidates, and the fixed interval is 20 steps:

Search scheduleSuccess (%)Candidate evaluations per episode
Every replan80.814,748.16
Fixed-interval scheduling86.23,660.80
Random scheduling87.25,767.68
Fixed-interval scheduling within PDW88.21,886.72
PREEMPT (full)93.63,706.88

PREEMPT uses 74.9% fewer candidate evaluations than every replan and outperforms fixed-interval scheduling by 7.4 percentage points at nearly the same search cost. Restricting fixed-interval scheduling to PDWs achieves higher success with approximately half the search cost of unrestricted fixed-interval scheduling.


Q1: Calibration of the completion-time evaluator

We thank you for asking the completion-time evaluator's estimation reliability. Q combines the probability and timing of local operation completion. As defined in Section 4.3, it is the expected discounted completion utility: Q=∑bpb exp⁡(−ub/60).Q=\sum_b p_b\,\exp(-u_b/60). The completion-time bin upper bounds are 0, 5, 15, 30, 60, 120, and 240 control steps; non-completion has zero utility. Accordingly, we compare completion probabilities with observed completion frequencies, and Q with observed discounted utilities using the same time-bin weights.

We conducted the evaluator evaluation on 208 RoboCasa parent trajectories, disjoint from evaluator and target-domain WM training/model selection, as well as score-scale calibration. Probability metrics use 2,646 windows with resolved outcomes, and utility MSE uses 1,990 windows with resolved utility targets. We further compared predictions and observations in ten equal-width prediction bins. In the most populated probability bin, 0.9–1.0, the mean prediction is 0.9384, closely matching the observed completion frequency of 0.9453 across 676 windows. In the most populated Q bin, 0.2–0.3, predicted Q averages 0.2512, versus an observed discounted utility of 0.2639 across 586 windows.

EvaluatorCompletion Brier ↓Completion ECE ↓Discounted-utility MSE ↓
PREEMPT0.18990.10670.0890

These results show particularly close agreement at high predicted completion probabilities. Lower-probability predictions exhibit greater underestimation, while the high-probability region provides encouraging evidence of reliable completion estimates.


(2/3)

AuthorsOpenReview ↗

Response to Reviewer DPUU (3/3)

Q2: Frequency and benefit of executed corrections

We report both the frequency of executed corrections and their effect on task outcomes. The table below gives paired outcomes for the original LIBERO and RoboCasa results in Tables 1 and 2, using 2,000 and 700 episodes per policy comparison, respectively. The confidence intervals describe the success-rate difference in percentage points. A rescue changes a baseline failure into success; a harm changes a baseline success into failure:

Refined policyBaselineRescuesHarms95% CI (pp)
LIBERO
π0.5 + PREEMPTπ0.5102[+0.06, +0.74]
OpenVLA-OFT + PREEMPTOpenVLA-OFT106[-0.19, +0.59]
OpenVLA-OFT + PREEMPTOpenVLA-OFT, H=5388[+0.84, +2.16]
RoboCasa
π0.5 + PREEMPTπ0.5, H=518120[+19.41, +26.59]
π0.5 + PREEMPTπ0.5, H=504418[+1.53, +5.90]

We also measured correction frequency and compared individual interventions from the same states on LIBERO-10. One continuation executes the nominal five-step prefix and the other executes the correction; both then follow the same π0.5 policy. Correction frequency is the percentage of replanning decisions at which a correction is actually executed.

Method comparisonState pairsRescuesHarmsBoth succeedBoth failCorrection frequency (%)
PREEMPT vs. π0.550024144441836.3

Results show that 24 corrections change failure to success, while 14 change success to failure in this branch study. These counts measure the outcome of the selected intervention directly.


Q3: Real-robot training data and annotation cost

The three auxiliary models used in the reported real-robot evaluation share 565 demonstrations, split as shown below.

ModelTraining trajectoriesValidation trajectoriesTest trajectories
PTA4445863
World model4445863
Completion-time evaluator4445863

Annotation cost for new tasks: We annotate physical decision windows in approximately 141 trajectories per task. Human annotation, verification, and correction require approximately 0.783 person-hours per task. The world model learns from recorded actions and observations and requires no additional manual labels.


Concluding Remarks

Thank you for recognizing the motivation and completion-time formulation novelty, and for suggesting more direct tests of the components and baselines. We have provided:

  • Component and continuity comparisons: the completion-time evaluator gains 11.0 points over either simpler scorer, and PREEMPT exceeds continuity correction by 10.14 points.
  • Stronger scheduling controls: PREEMPT achieves 93.6% success and uses 74.9% fewer candidate evaluations than Every replan.
  • LIBERO uncertainty and broader evaluation: we report paired confidence intervals and achieve 88.6% on LIBERO-Plus, up from 85.2%.
  • Calibration and deployment details: we quantify evaluator calibration, correction frequency and outcomes, and real-robot training data and annotation cost.

We sincerely appreciate your constructive feedback. We are glad to hear of your willingness to reconsider the rating and look forward to further discussion.


Reference

[1] Fei, S., Wang, S., Shi, J., Dai, Z., Cai, J., Qian, P., Ji, L., He, X., Zhang, S., Fei, Z., Fu, J., Gong, J., & Qiu, X. (2025). LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models. arXiv preprint arXiv:2510.13626.


(3/3)

Reviewer_tVFxOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PREEMPT refines the actions of a frozen VLA around contact-critical moments during policy execution. A V-JEPA-based anticipator predicts "physical decision windows". Within the predicted decision windows, PREEMPT searches over smooth spline deformations of the policy's intended path. A latent world model predicts five-step futures for each candidate, and each future is scored using a completion-time evaluator. The CEM picks a correction regularized toward the policy's motion. Experiments show incremental performance improvements on simulated benchmarks and substantial improvements in real robot tasks.

Critical strengths
  • The 10% improvement on real robot tasks is substantial.
  • The completion-time evaluation approach is innovative and principled.
  • Searching over spline deformations of the policy's own intended path keeps corrections smooth and close to the policy's behavior.
  • Compute is reported transparently (parameters, memory, and latency).
Critical weaknesses
  • Missing component ablations for the anticipator, evaluator, etc. It is unclear if the contributions are directly correlated with the performance gains.
  • RoboCasa is confounded. The baseline run with 5-step execution collapses (454 → 319 successes), and PREEMPT (480) mostly restores the default-horizon level (+26 trials). The continuity costs may account for most of the gain, and baselines that enforce consistency between chunks are missing.
  • The real-robot setup is under-documented. It is unclear what is the training data for the auxiliary models and the baseline's execution horizon.
  • Libero performance gains are negligible
Questions for authors
  1. What execution horizon did the real-robot baseline use, and what data trained the auxiliary models for the robot?
  2. How are decision windows labeled?
  3. Why would searching at every replan hinder performance?
3 comments
AuthorsOpenReview ↗

Response to Reviewer tVFx (1/3)

Thank you for recognizing the completion-time formulation, the smooth action corrections, and the real-robot improvement. We report the additional comparisons and address your questions below.


W1: Contributions of the individual components

We conducted component ablations on RoboCasa. Each tested component contributes to the full result: success falls from 68.6% to 60.7% without the cumulative-offset cost, to 58.0% without all structural costs and to 60.3% without both predicted futures and Vendi adaptation. The completion-time evaluator improves success by 11.0 percentage points over either the binary success classifier or the progress predictor. Replacing spline CEM with policy sampling, using 256 candidates per search, reduces success by 17.6 points.

VariantSuccess (%)
Binary success classifier57.6
Progress predictor57.6
PREEMPT w/o cumulative-offset cost60.7
PREEMPT w/o structural costs58.0
PREEMPT w/o Predicted futures and Vendi adaptation60.3
Policy sampling instead of spline CEM51.0
PREEMPT (full)68.6

To assess PTA, we compared fixed-interval scheduling, random scheduling, fixed-interval scheduling within physical decision windows (PDWs), and full PREEMPT on LIBERO-10:

Search scheduleSuccess (%)Candidate evaluations per episode
Every replan80.814,748.16
Fixed-interval scheduling86.23,660.80
Random scheduling87.25,767.68
Fixed-interval scheduling within PDW88.21,886.72
PREEMPT (full)93.63,706.88

Each search evaluates 4 × 64 = 256 sampled candidates. The fixed interval is 20 steps. PREEMPT uses 74.9% fewer candidate evaluations than Every replan and outperforms fixed-interval scheduling by 7.4 percentage points at nearly the same search cost. Restricting fixed-interval scheduling to PDWs achieves higher success with approximately half the search cost of unrestricted fixed-interval scheduling.


W2: RoboCasa improvement beyond action continuity

Thank you for asking us to clarify the source of the RoboCasa improvement and the drop in baseline success when H changes from 50 to 5. We explain the purpose of the H=5 comparison and the policy's sensitivity to replanning below:

  • Native policy configuration. We use the publicly released LeRobot π0.5 RoboCasa checkpoint, whose prediction and default execution lengths are both 50. With H=5, the policy still predicts 50 actions but discards the remaining 45 after executing the first five. This increases replanning frequency tenfold and can disrupt temporal consistency across independently generated chunks.

  • Fairness of comparison. PREEMPT executes local corrections over five steps. We therefore additionally evaluate π0.5 with five-step replanning to examine whether more frequent feedback alone can explain the improvement. This baseline matches the local correction granularity.

  • RoboCasa task characteristics. Door manipulation requires sustained contact along a constrained hinge trajectory. Interruptions from frequent replanning may help explain the decline in H=5 success, particularly on door tasks in the original Table 2.

We evaluated H=5 + continuity correction to separate continuity effects from predictive action selection:

MethodSuccess
π0.5, H=50454/700
π0.5, H=5319/700
π0.5, H=5 + continuity correction409/700
PREEMPT (full)480/700

PREEMPT retains a 10.14-percentage-point advantage over the continuity baseline.


(1/3)

AuthorsOpenReview ↗

Response to Reviewer tVFx (2/3)

W3 & Q1: Real-robot execution and training data

In the reported real-robot evaluation, π0.5 generates 50 actions and executes 10 before replanning. PREEMPT uses the same 50-action policy output, with a nominal execution horizon of 10 steps and a correction horizon of up to 5 steps.

The auxiliary models used in the reported evaluation share 565 demonstrations across the four real-robot tasks, split as shown below.

ModelTraining trajectoriesValidation trajectoriesTest trajectories
PTA4445863
World model4445863
Completion-time evaluator4445863

W4: Size of the LIBERO improvement

We use LIBERO to evaluate general manipulation. The π0.5 and OpenVLA-OFT baselines already achieve 96.7% and 96.5% success, leaving limited room for improvement. PREEMPT raises these rates to 97.1% and 96.7%, respectively. Below, we report paired outcomes for the original LIBERO results in Table 1, using 2,000 episodes per policy comparison. A rescue changes a baseline failure into success; a harm changes a baseline success into failure.

MethodBaselineRescuesHarms
π0.5 + PREEMPTπ0.5102
OpenVLA-OFT + PREEMPTOpenVLA-OFT106
OpenVLA-OFT + PREEMPTOpenVLA-OFT, H=5388

We also evaluated PREEMPT on LIBERO-Plus [1]. PREEMPT achieves 88.6% average success, improving over π0.5 by 3.4 percentage points and exceeding the reported OpenVLA-OFT result by 17.2 points.

MethodCameraRobotLanguageLightBackgroundNoiseLayoutAverage
OpenVLA0.83.523.08.134.815.228.516.3
OpenVLA-OFT56.431.979.588.793.375.874.271.4
π0.578.473.680.896.294.189.084.585.2
π0.5 + PREEMPT89.073.682.096.597.096.086.088.6

(2/3)

AuthorsOpenReview ↗

Response to Reviewer tVFx (3/3)

Q2: How decision windows are labeled

We label decision windows offline from recorded trajectory videos with visible control-step indices. Each physical decision window (PDW) is represented by an enter–commit interval. Enter marks the beginning of the final local interaction, when small changes in pose, alignment, or gripper timing can directly affect the operation's outcome; distant approach and ordinary transport are excluded. Commit marks the earliest point at which that attempt's outcome is established. For example, a grasp is confirmed by the object moving stably with the gripper, rather than merely by gripper closure or contact. Failed attempts are also annotated, with commit indicated by a clear miss, slip, or the need to reset and retry.

These intervals supervise PTA’s enter and commit outputs. At deployment, PTA predicts windows from current and past observations, without access to annotations or future frames.


Q3: Why searching at every replan can hurt

Searching at every replan can introduce corrections at times when the nominal action already suffices. More frequent search therefore creates more opportunities to act on imperfect score estimates. PREEMPT retains the nominal prefix unless a candidate improves the regularized objective, J > 0, and uses PTA and predicted consequence diversity to concentrate search.

As reported in W1, Every replan achieves 80.8% success, versus 93.6% for PREEMPT. We also compared nominal and corrected actions from the same states on LIBERO-10. Each branch executes five actions and then continues with the same π0.5 policy; window membership is determined before the intervention.

MethodBaselineState pairsRescuesHarms
PREEMPT (full)π0.55002414
Every replan, inside PDWπ0.55002216
Every replan, outside PDWπ0.55001116

A rescue changes a nominal failure into success; a harm changes a nominal success into failure. Every replan produces 22 rescues and 16 harms inside PTA windows, compared with 11 and 16 outside them. The more favorable balance inside the windows supports concentrating intervention at these moments.


Concluding Remarks

Thank you again for recognizing the completion-time formulation and real-robot gains, and for suggesting the additional controls. We have strengthened the evidence in three areas:

  • Component contributions and continuity: ablations support the evaluator, structural costs, and spline search; PREEMPT retains a 10.14-point advantage over continuity correction.
  • Search timing: PREEMPT achieves 93.6% versus 80.8% for Every replan, with same-state comparisons showing a more favorable balance of rescues and harms inside PDWs.
  • Evaluation and reproducibility: LIBERO-Plus success increases from 85.2% to 88.6%. We also report paired outcomes and explain the real-robot execution horizons, auxiliary training data, and window annotations.

We appreciate your careful review and hope these results and clarifications address your questions. We welcome any further discussion.


Reference

[1] Fei, S., Wang, S., Shi, J., Dai, Z., Cai, J., Qian, P., Ji, L., He, X., Zhang, S., Fei, Z., Fu, J., Gong, J., & Qiu, X. (2025). LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models. arXiv preprint arXiv:2510.13626.


(3/3)

Reviewer_sTqoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PREEMPT refines the actions of a frozen vision-language-action policy at inference time. A V-JEPA-based anticipator identifies physical decision windows, and a latent world model predicts the five-step consequences of candidate action corrections. A completion-time evaluator scores the candidates, and the cross-entropy method searches over spline corrections while penalizing deviations from the policy’s intended motion. The paper evaluates the system on LIBERO, RoboCasa, and four tasks on an Agilex robot.

Critical strengths
  • The completion-time objective addresses a concrete limitation of short-horizon prediction. A five-step forecast may capture initial contact while missing whether the operation eventually succeeds. The partial-label likelihood provides a principled way to use completed, failed, and right-censored attempts.

  • The spline parameterization and cumulative-offset penalty give the optimizer a structured way to make local corrections while accounting for drift across replans.

  • The real-robot gain is meaningful on the tested setup.

Critical weaknesses
  • The evaluation does not establish that the completion-time scorer ranks candidate corrections reliably. The evaluator is trained on executed prefixes and their observed outcomes (Appendix C), while CEM searches over spline-modified prefixes. Table 18 shows improved latent prediction on recorded trajectories, but it does not test whether the selected correction actually outperforms the nominal action from the same state. A matched comparison against a simpler success classifier, using the same candidate set and search budget, would also help establish the value of modeling completion time.

  • The scheduling experiment does not isolate the value of physical decision windows. Table 4 enables refinement on three LIBERO-10 tasks covering 150 of the 500 episodes. Both methods share this scope, so the 58.4% candidate reduction is valid for that comparison. An equal-budget fixed-interval baseline and a fixed cadence within PTA windows would help distinguish the effects of window localization and Vendi-based scheduling. The reported overlap with model-generated reference windows in Appendix F.1 leaves open whether the detected windows are the times at which intervention helps most.

  • The RoboCasa comparison leaves the source of the improvement unclear. Shortening the baseline execution horizon from 50 to five steps reduces success from 454/700 to 319/700. PREEMPT reaches 480/700, a gain of 26 successes over the default policy. The five-step comparison controls for execution horizon, but without a continuity-aware baseline or a structural-cost ablation, it is hard to tell how much of the gain comes from predictive scoring versus smoother execution across replans.

  • The LIBERO gains are small and their uncertainty is unreported. Table 1 shows gains of 0.4 and 0.2 percentage points for π₀.₅ and OpenVLA-OFT, respectively, with a decline on the OpenVLA-OFT Object suite. Appendix D.2 describes a paired bootstrap protocol, but Table 1 omits confidence intervals for these gains. The paired outcomes would show how often refinement fixes a failure and how often it introduces one.

  • The strongest result is difficult to interpret without the real-robot protocol. Section 5.3 does not specify the baseline’s execution horizon, and Appendix C does not describe the real-robot training and calibration data for the auxiliary models. These details matter because execution horizon has a large effect in RoboCasa and because the amount of task-specific data determines what the real-robot result demonstrates about transfer.

Questions for authors

see weakness.

3 comments
AuthorsOpenReview ↗

Response to Reviewer sTqo (1/3)

We sincerely appreciate your thoughtful assessment and your recognition of the completion-time objective, structured corrections, and strong real-robot performance. We address your questions with additional experimental evidence and clarifications below.


W1: Does the evaluator select better corrections?

We compared actions selected by the binary success classifier and completion-time evaluator from the same RoboCasa states. Both scorers use the same 257 candidates (256 sampled candidates plus the nominal prefix) and a shared budget of 257 candidate evaluations. After executing the selected prefix, all branches continue with the same π0.5 policy, using an H=50 execution schedule and the same remaining episode budget, without further corrections.

Action selectionSuccess (%)
Nominal60.1
Binary success classifier61.4
Completion-time evaluator68.0

The results show that the completion-time evaluator improves success by 7.9 percentage points over the nominal action and 6.6 points over the binary success classifier. This comparison directly evaluates the outcomes of selected spline corrections, including candidates that differ from the executed prefixes used for training.


W2: Physical decision windows and search scheduling

We compared fixed-interval scheduling, random scheduling, fixed-interval scheduling within physical decision windows (PDWs), and full PREEMPT on LIBERO-10:

Search scheduleSuccess (%)Candidate evaluations per episode
Every replan80.814,748.16
Fixed-interval scheduling86.23,660.80
Random scheduling87.25,767.68
Fixed-interval scheduling within PDW88.21,886.72
PREEMPT (full)93.63,706.88

Each search evaluates 4 × 64 = 256 sampled candidates. The fixed interval is 20 steps. PREEMPT uses 74.9% fewer candidate evaluations than Every replan and outperforms fixed-interval scheduling by 7.4 percentage points at nearly the same search cost. Restricting fixed-interval scheduling to PDWs achieves higher success with approximately half the search cost of unrestricted fixed-interval scheduling.

To test whether corrections help more inside PDWs, we compared interventions inside and outside these windows on LIBERO-10. From each saved state, we execute the nominal and corrected five-step prefixes in separate branches and then continue with the same π0.5 policy. Window membership is determined before the intervention, and states are selected independently of their outcomes:

MethodBaselineState pairsRescuesHarms
PREEMPT (full)π0.55002414
Every replan, inside PDWπ0.55002216
Every replan, outside PDWπ0.55001116

Rescues change a nominal failure into success; harms change a nominal success into failure. The balance between rescues and harms is more favorable inside PTA windows, linking the detected intervals to the effectiveness of intervention.


(1/3)

AuthorsOpenReview ↗

Response to Reviewer sTqo (2/3)

W3: Separating predictive correction from continuity

Thank you for asking us to clarify the source of the RoboCasa improvement and the drop in baseline success when H changes from 50 to 5. We explain the purpose of the H=5 comparison and the policy's sensitivity to replanning below:

  • Native policy configuration. We use the publicly released LeRobot π0.5 RoboCasa checkpoint, whose prediction and default execution lengths are both 50. With H=5, the policy still predicts 50 actions but discards the remaining 45 after executing the first five. This increases replanning frequency tenfold and can disrupt temporal consistency across independently generated chunks.

  • Fairness of comparison. PREEMPT executes local corrections over five steps. We therefore additionally evaluate π0.5 with five-step replanning to examine whether more frequent feedback alone can explain the improvement. This baseline matches the local correction granularity.

  • RoboCasa task characteristics. Door manipulation requires sustained contact along a constrained hinge trajectory. Interruptions from frequent replanning may help explain the decline in H=5 success, particularly on door tasks in the original Table 2.

We also evaluated H=5 + continuity correction, which smooths transitions between chunks without predictive scoring. As shown below, PREEMPT retains a 10.14-percentage-point advantage over this baseline.

MethodSuccess
π0.5, H=50454/700
π0.5, H=5319/700
π0.5, H=5 + continuity correction409/700
PREEMPT (full)480/700

We also conducted the requested structural-cost ablations on RoboCasa. Success falls from 68.6% to 60.7% without the cumulative-offset cost, to 58.0% without all structural costs and to 60.3% without both predicted futures and Vendi adaptation. Together with the scorer comparison in W1, these results support the contributions of predictive selection and motion regularization:

VariantSuccess (%)
Binary success classifier57.6
Progress predictor57.6
PREEMPT w/o cumulative-offset cost60.7
PREEMPT w/o structural costs58.0
PREEMPT w/o Predicted futures and Vendi adaptation60.3
Policy sampling instead of spline CEM51.0
PREEMPT (full)68.6

W4: LIBERO gains and paired outcomes

We use LIBERO to evaluate general manipulation. The π0.5 and OpenVLA-OFT baselines already achieve 96.7% and 96.5% success, leaving limited room for improvement. PREEMPT raises these rates to 97.1% and 96.7%, respectively. Below, we report the requested paired outcomes for the original LIBERO results in Table 1, using 2,000 episodes per policy comparison. The confidence intervals describe the success-rate difference in percentage points.

MethodBaselineRescuesHarms95% CI (pp)
π0.5 + PREEMPTπ0.5102[+0.06, +0.74]
OpenVLA-OFT + PREEMPTOpenVLA-OFT106[-0.19, +0.59]
OpenVLA-OFT + PREEMPTOpenVLA-OFT, H=5388[+0.84, +2.16]

We also evaluated PREEMPT on LIBERO-Plus [1]. PREEMPT achieves 88.6% average success, improving over π0.5 by 3.4 percentage points and exceeding the reported OpenVLA-OFT result by 17.2 points.

MethodCameraRobotLanguageLightBackgroundNoiseLayoutAverage
OpenVLA0.83.523.08.134.815.228.516.3
OpenVLA-OFT56.431.979.588.793.375.874.271.4
π0.578.473.680.896.294.189.084.585.2
π0.5 + PREEMPT89.073.682.096.597.096.086.088.6

(2/3)

AuthorsOpenReview ↗

Response to Reviewer sTqo (3/3)

W5: Real-robot execution and auxiliary-model data

In the reported real-robot evaluation, π0.5 generates 50 actions and executes 10 before replanning. PREEMPT uses the same 50-action policy output, with a nominal execution horizon of 10 steps and a correction horizon of up to 5 steps.

The auxiliary models used in the reported evaluation share 565 real-robot demonstrations, split as shown below. For PTA, the validation set is further partitioned deterministically into 35 trajectories for decoder-threshold calibration and 23 for checkpoint selection.

ModelTraining trajectoriesValidation trajectoriesTest trajectories
PTA4445863
World model4445863
Completion-time evaluator4445863

For adaptation to new tasks, the base VLA remains frozen, pretrained visual representations and auxiliary-model weights are reused, and all three modules share demonstration data. Query-conditioned local operations also support reuse across tasks composed of familiar skills. This reduces the need to train models and collect data separately for each component.


Concluding Remarks

We thank you again for the thoughtful feedback and the suggestions to test action selection and search timing more directly. Our additional evidence includes:

  • Same-state scoring: the completion-time evaluator achieves 68.0% success versus 61.4% for the binary classifier on identical candidates.
  • Continuity and component controls: PREEMPT retains a 10.14-point advantage over continuity correction, with ablations supporting the evaluator and structural costs.
  • Window-guided search: PREEMPT reaches 93.6% versus 86.2% for fixed-interval scheduling at comparable cost; interventions are more beneficial inside PDWs.
  • Broader evaluation and protocol details: LIBERO-Plus success increases from 85.2% to 88.6%. We also report LIBERO paired outcomes and clarify real-robot execution, training data, and calibration.

We appreciate your openness to reconsidering the evaluation. We would be more than grateful if you would consider a higher score.


Reference

[1] Fei, S., Wang, S., Shi, J., Dai, Z., Cai, J., Qian, P., Ji, L., He, X., Zhang, S., Fei, Z., Fu, J., Gong, J., & Qiu, X. (2025). LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models. arXiv preprint arXiv:2510.13626.


(3/3)

Comments on the paper

2 comments
AuthorsOpenReview ↗

To AC: Rebuttal Summary

Dear Area Chair,

Thank you very much for your time and your support for coordinating the review and considering our responses. We appreciate the reviewers’ constructive feedback. During the rebuttal, we addressed all concerns and updated the manuscript accordingly.

The reviewers recognized our completion-time objective, intent-constrained corrections design, and strong performance across simulation and real-world evaluation. Their main questions were whether the gains come from the proposed components and whether the evaluation sufficiently supports them. We provide the following additional evidence:

  • Attributing gains to the proposed components. We conducted component ablations, a continuity-aware RoboCasa comparison, same-state scorer comparisons, and scheduling controls (Tables 2, 5–6, 23). PREEMPT retains a 10.14-percentage-point advantage over continuity correction, while the ablations support the core values of the evaluator, structural costs, and spline search.
  • Strengthening and explaining the evaluation. We added complete evaluation on LIBERO-Plus, improving success from 85.2% to 88.6%, reported paired LIBERO outcomes and confidence intervals, and specified the real-robot execution horizons, auxiliary-model data, and annotation procedure (Table 3; Apps. C.4, D.3).

We summarize the responses to each reviewer below:


Reviewer tVFx (Initial Score: 3)

  • Concern: Component contributions and the continuity confound.
    • The component ablations and continuity-aware baseline show benefits from completion-time evaluation and structured search beyond smoother execution alone (Tables 2, 5).
  • Concern: Window labels and the effect of searching at every replan.
    • We specified the video-based enter–commit annotation criteria and report outcomes inside and outside PDWs (App. C.1; Table 25). PREEMPT achieves 93.6% versus 80.8% for Every replan on LIBERO-10 (Table 6).
  • Concern: Real-robot setup.
    • We specified the execution horizons and the shared auxiliary training data, including the trajectory-level training, validation, and test split (App. C.4).
  • Concern: LIBERO gains explanation.
    • We complemented the original LIBERO results with paired outcomes and the 3.4-point gain on LIBERO-Plus (Tables 3, 13).

Reviewer sTqo (Initial Score: 2)

  • Concern: Reliability of candidate scoring.
    • From identical RoboCasa states and the same 257 candidates, the completion-time evaluator achieves 68.0% success, versus 61.4% for the binary success classifier and 60.1% for the nominal action (Table 23).
  • Concern: Value of physical decision windows and scheduling.
    • PREEMPT achieves 93.6% versus 86.2% for fixed-interval scheduling at comparable search cost (Table 6). Same-state interventions yield 22 rescues and 16 harms inside PDWs, versus 11 and 16 outside, supporting the choice of intervention times (Table 25).
  • Concern: Improvement beyond continuity.
    • We provided the continuity-aware baseline and structural-cost ablations, separating the benefits of predictive selection and motion regularization (Tables 2, 5).
  • Concern: LIBERO gains explanation.
    • We reported paired outcomes and confidence intervals for the original results, alongside the larger gain on LIBERO-Plus (Tables 3, 13).
  • Concern: Real-robot protocol.
    • We specified training and calibration data. Both methods use a nominal execution horizon of 10 steps; PREEMPT applies corrections over up to five steps (App. C.4).

Reviewer DPUU (Initial Score: 2)

  • Concern: Component ablations and stronger baselines.
    • Replacing the completion-time evaluator with either simpler scorer lowers success by 11.0 points. We also tested structural costs, policy sampling, continuity correction, and fixed/random scheduling. PREEMPT uses 74.9% fewer candidate evaluations than Every replan while achieving higher success (Tables 2, 5–6).
  • Concern: LIBERO uncertainty and evaluator calibration.
    • We reported paired confidence intervals for the original LIBERO comparisons and quantify calibration against the evaluator's distinct completion-probability and discounted-completion targets (Apps. D.3, F.3).
  • Concern: Frequency and benefit of corrections.
    • We reported a 36.3% correction frequency in the additional LIBERO study and 24 rescues versus 14 harms across 500 state pairs (App. F.4).
  • Concern: Data and annotation cost.
    • The auxiliary models share 565 real-robot demonstrations; annotation and review require approximately 0.783 person-hours per task (App. C.4).

We sincerely appreciate everyone's time and dedication invested in evaluating our work, especially the AC. We appreciate all the reviewers' suggestions and your consideration of the additional results and clarifications.

AuthorsOpenReview ↗

Global Response: Paper Revisions

We thank all reviewers for their thoughtful feedback. The revised manuscript incorporates the following experiments, analyses, and clarifications:

  • In Sec. 5.2 (Table 3), we added an evaluation on LIBERO-Plus across seven perturbation categories and all four LIBERO suites.
  • In Sec. 5.2 (Table 2), we added a continuity-corrected RoboCasa baseline to examine the improvement beyond smoother action execution.
  • In Sec. 5.4 (Tables 5–6), we added component and search-scheduling ablation studies.
  • In Appendix F.3 (Tables 23–24), we added a comparison of action selection from identical states using shared candidates, together with calibration analysis of the completion-time evaluator.
  • In Appendix F.4 (Table 25), we added paired intervention results inside and outside physical decision windows and reported how often corrections are executed.
  • In Appendix D.3 (Table 13), we added analysis of paired rescues and harms and confidence intervals for the LIBERO and RoboCasa performance gains.
  • In Sec. 5.3 and Appendix C.4, we further clarified the real-robot training data, execution horizons, annotation procedure, and calibration protocol.
  • We moved the window-localization and latent-prediction analyses from the original Sec. 5.4 to Appendices F.1–F.2, and the model-size and inference-cost analyses from the original Sec. 5.5 to Appendices E.2–E.4.
  • We refined the visual presentation of Figure 2 for a more polished presentation.
  • We also made minor language edits and standardized table captions and formatting.

To help the AC and the reviewers navigate the revisions. We highlighted headings and captions for new content in blue, and those for content moved to the appendix in green.

Across benchmarks, PREEMPT outperformed state-of-the-art baselines on real-world evaluation with +10%, +3.4% on LIBERO-Plus simulation tasks and +3.7% on RoboCasa simulation tasks.

Our individual responses detailed how these addition evaluations address each reviewer's concerns. We sincerrely appreciate the reviewers' feedback and their consideration of the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.