acceptodds
Under review as a conference paper at ICLR 2027

LocoWM: High-Precision Locomotion through World-Model Guided Residual Adaptation

Abstract

High-precision locomotion combines motion-command tracking with precise regulation of task-relevant physical states, enabling robots to interact reliably with their surroundings during motion. Joint end-to-end optimization can leave precision objectives insufficiently optimized, while reactive residual control adjusts actions only after deviations become observable. We present LocoWM, a world-model-guided preactive residual adaptation framework for high-precision locomotion. A base policy provides command-following locomotion, while an action-conditioned world model predicts a sequence of future physical states from proprioceptive history and the proposed base action. A residual adapter conditions on this predicted sequence to generate additive action corrections that compensate for anticipated deviations. Two-stage training first learns locomotion and action-conditioned dynamics, then freezes both modules while training the adapter, separating locomotion acquisition from precision adaptation. Experiments spanning terrain leveling, acceleration compensation, and push recovery demonstrate improved control precision and disturbance robustness over end-to-end and reactive residual baselines. Demos are available at: https://sites.google.com/view/locowm

Primary area:
applications to robotics, autonomy, planning
Keywords:
World models, Residual control, High-precision locomotion

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_r4VDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LocoWM studies stable transportation of unsecured payloads by a wheeled-legged robot while tracking velocity commands. The method first learns a base locomotion policy and a short-term task-state predictor, then freezes both and trains a residual adapter to correct actions using the predictions. Go2-W simulation experiments report improvements in payload retention and tray stability, complemented by Go2-W real-robot demonstrations and a simulation extension to the G1 humanoid.

Critical strengths
  1. A coherent framework addressing a practically meaningful problem. LocoWM addresses payload stabilization during locomotion through a clear separation between locomotion acquisition and precision adaptation. Using physically interpretable task substates to connect the precision objective, prediction target, and residual-controller input provides a well-motivated design.
  2. Substantial experimental coverage within the target application. The evaluation covers four terrain types, acceleration compensation, push recovery, physical Go2-W demonstrations, and an additional humanoid simulation study. Comparisons with end-to-end and reactive residual baselines show meaningful improvements: relative to Recon, payload-retention success increases from 61.5% to 91.1% on slopes and from 67.7% to 87.5% on bumps.
Critical weaknesses
  1. The methodological novelty over predictive residual control is insufficiently established. Conditioning residual corrections on predicted consequences of base actions already appears in KORR (Gong et al., 2025), including an MLP-based dynamics variant. Frozen-base residual adaptation is also established in ReST-RL. LocoWM introduces meaningful differences through task-specific physical substates, multi-step prediction, and staged training, but neither discusses KORR nor sufficiently isolates the benefits of these differences. Consequently, the methodological advance over existing designs remains unclear.
  2. Task breadth and hardware generalization remain limited. The three main tasks are useful but primarily examine different disturbances within the same payload-stabilization objective. Go2-W hardware results are primarily qualitative, while the G1 extension remains simulation-only. Its success-rate improvements over ReST-RL are modest—1.9–3.1 percentage points—and lack uncertainty estimates. These results provide preliminary evidence, but do not yet establish reliable generalization to humanoid hardware or broader precision-control tasks.
Questions for authors
  1. Novelty clarification: How does LocoWM differ technically from KORR and its MLP-based predictive residual variant? Which results establish the additional benefits of the task-substate interface, multi-step prediction, and staged training?
  2. Mechanism analysis: Can the authors report prediction accuracy under both the base and adapted policies, together with horizon and action-conditioning ablations, to clarify whether accurate future forecasts drive the gains?
  3. Evaluation clarification: How many independent training seeds and evaluation trials support the results? Could confidence intervals be provided, particularly for the relatively small G1 improvements over ReST-RL?
  4. Additional hardware validation: Could the authors provide repeated quantitative hardware comparisons against the residual baselines? A physical G1 evaluation would be particularly valuable for assessing whether the simulated humanoid gains transfer to real bipedal locomotion.
1 comment

Responses


Q1: How does LocoWM differ from KORR and its MLP variant, and which results isolate its design choices? (W1 & Q1)

A1: Thank you for pointing out KORR. We have added it to (Sec. 2, L112–118, P3). In our view, LocoWM differs mainly in what the residual is conditioned on and how that conditioning is trained; we do not claim frozen-base residual adaptation as a contribution (as in ReST-RL, React, and Recon).

  1. Design Differences: KORR is evaluated on simulated manipulation, whereas LocoWM targets legged locomotion with unsecured payloads and is validated on hardware. Transferring KORR to this setting exposes limitations that the staged training, task substates, and multi-step prediction you mention address (Appendix C.3–C.4, Tables 9 & 11, P26–27):

    • Staged Training (Train–Query Consistency): KORR trains its dynamics on executed actions (ab+ara_b+a_r) but queries it with aba_b, a mismatch under which R2R^2 drops from 0.88 to 0.70; LocoWM trains and queries the WM on base-policy rollouts.
    • Staged Training (Frozen WM): KORR updates its dynamics with the RL objective; doing so on our WM lowers R2R^2 to 0.27 and SR to 77.30%.
    • Task Substates: KORR predicts a latent state, and its MLP variant the full next state; predicting the full observation instead of task substates lowers SR to 89.43%.
    • Multi-Step Prediction: Forecasting 5 steps lets the adapter correct in advance; one-step prediction lowers SR to 89.70%.
  2. Matched Comparison: From the same base checkpoint and budget, LocoWM reaches 93.63% SR vs. 71.16% (KORR-MLP) and 76.48% (KORR-Koopman), both below Recon (80.23%). KORR-Koopman is close to the jointly trained variant (77.30%), consistent with the analysis above. (Appendix C.3, Table 10, L1374, P26)


Q2: Prediction accuracy under both policies, and do accurate forecasts drive the gains? (Q2)

A2:

  1. Accuracy under Both Policies: By design, the WM predicts the future under the base policy, which the adapter then corrects. The R2R^2 drop from 0.88 to 0.70 under the adapted policy (A1.1) is thus expected: the executed future already contains the residual's correction. (Appendix C.3, Table 9a, L1356, P26)

  2. Horizon & Action Conditioning: (Appendix C.4, Table 11, L1404, P27)

    • Horizon: Multi-step prediction improves SR (H=1: 89.70% → H=5: 93.63%), saturating at H=9 (93.76%).
    • Action Conditioning: Without aba_b, MSE rises 2.3× and SR drops to 84.22%.
  3. Accurate Forecasts Drive the Gains:

    • Accuracy Tracks SR: Across WM checkpoints, SR rises with R2R^2 (0.78 → 0.88; 87.42% → 93.63%) and saturates with it. (Appendix C.4, Table 13, L1433, P27)
    • Task-Relevant Accuracy Matters: Full-observation prediction has better aggregate R2R^2 (0.90) but lower SR, as it is less accurate on task-relevant components (R2R^2: angular velocity 0.58 vs. 0.89; projected gravity 0.30 vs. 0.91). (Appendix C.4, Table 12, L1416, P27)
    • No Predictive Supervision, No Gain: RL-only (LocoWM without predictive supervision) reaches 69.68% SR, on par with React (69.75%).

Q3: How many seeds and trials, and could CIs be provided, especially for G1? (Q3)

A3:

  1. Seeds & Trials: In Go2-W simulation, each method uses 5 independent training seeds, each evaluated per terrain with 4096 parallel environments (one trial each). Standard deviations are computed across trials and 95% CIs across seeds (Appendix C.1.1, Tables 6–7, L1080–1107, P21); G1 CIs are in (Appendix D.2, Table 14, L1470, P28).

  2. G1 vs. ReST-RL: 95% CIs of the SR difference:

    TaskDifference (pp)95% CI (pp)
    Command Track+3.1[+0.80, +5.40]
    Push Robot+1.9[−0.02, +3.82]
    Push Object+2.4[+0.77, +4.03]

    The CI excludes zero on Command Track and Push Object. On Push Robot, its lower bound touches zero, so we do not claim a significant improvement there. The G1 study tests transfer to another embodiment; the main evidence remains on Go2-W.


Q4: Repeated quantitative hardware comparisons and a physical G1 evaluation? (W2 & Q4)

A4:

  1. Go2-W Hardware Comparison: Each of five methods was run 10 times on three hardware terrains (150 trials). LocoWM retains the payload in 28/30 trials vs. 23/30 for Recon, the strongest baseline, with the lowest pitch (5.06° vs. 7.92°) and roll (3.69° vs. 8.57°) errors. (Appendix C.2, Table 8, L1252, P24)

  2. Physical G1 & Task Breadth: We agree that the G1 study is simulation-only and all tasks concern payload stabilization. Our Limitations now note that other precision objectives (e.g., end-effector stabilization) remain to be validated (Sec. 5, L470–475, P9).


Summary

Thank you for recognizing our coherent framework and substantial experimental coverage. If our responses address your concerns, we would sincerely appreciate your consideration for raising the score.

Reviewer_z42WOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper deals with the task of legged-wheeled locomotion while maintaining the balance of the tray carried on the back of the robot. The paper proposes a multi-stage learning framework, where a base policy for locomotion and a world model conditioned on the action of the base policy are firstly trained, and then use the two models’ output to train a residual adapter to output corrected actions that can complete the balancing task. Experimental results show that the trained models can complete the transport over four different terrains with at least 87% success rate.

Critical strengths

Overall, I think it is a strong submission. I think the biggest contribution of this paper is that it provides a solid solution for a useful task that implements a simple idea/methodology, i.e., predicting the future of actions from a base policy and conditioning a residual policy on the prediction. Such a solution could inspire future work to further fulfill the potential of this idea.

Besides, the presentation is somewhat clear with nice figures and cool demos. The target problem is well-motivated. The baselines are chosen reasonably and experiments include four different terrains and push recovery.

Critical weaknesses

This is a solid paper and I don’t see a major weakness in it. I don’t give a higher rating because I think this paper’s contributions and novelty are limited. The core idea of the proposed framework, i.e., predicting the future of actions from a base policy and conditioning a residual policy on the prediction, is not novel. For example, in KORR (Gong et al. Robust Online Residual Refinement via Koopman-Guided Dynamics Modeling. 2025), the framework is very similar but with different dynamics models and application tasks. With KORR, the argument for the uniqueness of this paper, such as “LocoWM instead uses a learned short-horizon dynamics predictor to condition preactive residual corrections on action-conditioned future task substates in a single forward pass.” (L108-110), is largely weakened.

The lack of quantitative results in real-world experiments is also a problem. The demos could be cherry-picked and it’s hard to evaluate the robustness of the proposed method in the real world compared to the baselines.

Besides, I think the writing of the main contributions (L083-093) can be improved. For example, the first sentence says the authors “introduce a [good] interface for LocoWM”, without justifying the contribution of LocoWM. If LocoWM is not something useful, why would we care about introducing something good for it? In short, I expect the authors to state the real “value” and “contributions” in this part of the paper, instead of just summarize the methodology in two bullet points and the results in one. For now, it’s hard for the readers to understand and evaluate the contributions of this paper from reading this part.

I also have concerns about the branding. The name, “LocoWM”, makes me expect that the framework is about a world model that helps locomotion. To some extent, it is indeed. But the world model is only trained to predict future states of the base policy, whose output is not the real action to control the robot beyond training stage 1. A more accurate way to describe the key of the framework would be a WM-guided correction (as the title does), rather than a WM for locomotion. Also, for the “loco” part, I feel the key of the application is obviously more about “high-precision carrying” instead of a more general “locomotion”.

Questions for authors

What are the success rates of LocoWM and the baselines in the real-world experiments? Why are they not reported?

How accurate is the WM?

Why do you just condition the WM once on the base policy actions? Have you considered about conditioning the WM again on the corrected actions and see if it can push the SR further?

Have you tried to distill the base policy and the adapter into a single end-to-end policy?

3 comments

Responses


Q1: What are the real-world success rates of LocoWM and the baselines? (W2 & Q1)

A1: We have added a repeated quantitative comparison: each of five methods was run 10 times on each of three hardware terrains under a fixed protocol, and all 150 trials are reported. Overall SR: Base 23.33%, E2E 36.67%, React 60.00%, Recon 76.67%, and LocoWM 93.33%, also with the lowest errors. (Appendix C.2, Table 8, L1252, P24)


Q2: How accurate is the WM, and why query it only once? (Q2 & Q3)

A2: The WM answers "where would the base policy drive the task substate without correction?", and the adapter corrects accordingly. (Appendix C.3–C.4, P26–27)

  1. Accuracy: Over H=5H=5 steps, R2=0.88R^2=0.88 (angular velocity 0.89, projected gravity 0.91; linear acceleration, unobserved, 0.700.70).
  2. Accuracy Drives Control: Across WM checkpoints, SR rises from 87.42% to 93.63% as R2R^2 improves from 0.78 to 0.88.
  3. Single Query: The correction substantially changes the future: against the executed future, R2R^2 drops to 0.70. Corrected actions therefore lie outside the WM's training distribution, and the adapter is trained on z^(ab)\hat z(a_b), so a second query would be off-distribution for both; training the WM on corrected actions instead would follow KORR (A4).

Q3: Have you tried distilling LocoWM into a single policy? (Q4)

A3: Thank you for the suggestion. We distilled the full LocoWM into one student policy:

MethodPitch RMS / peak (°↓)Roll RMS / peak (°↓)aza_z mean / peak (m/s²↓)SR (%↑)ms/step (↓)
LocoWM1.36 / 3.440.63 / 2.830.71 / 3.9293.6315.89
Student1.55 / 4.061.71 / 6.590.84 / 4.5187.3911.49

The student exceeds React (69.75%) and Recon (80.23%) without explicit prediction at inference, so a single network can represent the behavior, but RL alone struggles to learn it. The student is slightly worse on all six error metrics, and although it is ~28% faster, the modular LocoWM already runs at 50 Hz on hardware.


Q4: How does LocoWM differ from KORR? (W1)

A4: Thank you for pointing out KORR. We now discuss it as the closest prior work, replacing the cited L108–110 claim (Sec. 2, L112–118, P3). KORR is evaluated on simulated manipulation, whereas LocoWM targets legged locomotion with unsecured payloads and is validated on hardware. Transferring KORR to this setting exposes limitations that LocoWM's design addresses (Appendix C.3–C.4, Tables 9 & 11, P26–27):

  1. Train–Query Consistency: KORR fits its dynamics on executed transitions (ab+ara_b+a_r) but queries it with aba_b, under which R2R^2 drops from 0.88 to 0.70; LocoWM trains and queries the WM on base-policy rollouts.
  2. Frozen WM: KORR updates its dynamics with the RL objective; doing so on our WM lowers R2R^2 to 0.27 and SR to 77.30%.
  3. Task Substates & Multi-Step Prediction: KORR predicts the next latent state; LocoWM predicts 5 steps of the quantities governing payload stability. Full-observation or one-step prediction lowers SR to 89.43% and 89.70%.

Matched Comparison: From the same base checkpoint and budget, LocoWM reaches 93.63% SR vs. 76.48% (KORR-Koopman) and 71.16% (KORR-MLP), both below Recon (80.23%). As KORR-MLP shares LocoWM's MLP model class, the gap is not due to the dynamics model alone. (Appendix C.3, Table 10, L1374, P26)


Q5: Contribution paragraph and branding. (W3 & W4)

A5: Thank you for these concrete suggestions.

  1. Contributions: We rewrote them as value → design → evidence (Sec. 1, L085–097, P2):

    • Value: A frozen locomotion policy gains a precision skill without retraining, at the cost of two small forward passes per step.
    • Design: The forecast targets physical task substates; the predictor is trained under the base policy and frozen during adaptation (A4).
    • Evidence: The highest SR on all terrains and 28/30 vs. 23/30 successful hardware trials.
  2. Branding: We agree that the WM guides the correction rather than locomotion itself, and that our experiments focus on payload transport; we now state both explicitly (Sec. 1, L078–080, P2; Abstract, L040–041, P1). To avoid confusion across reviews, we keep the name during the rebuttal, but are open to a more fitting one in the final version.


Summary

Thank you for the positive assessment and for recognizing our solid solution to a useful task, clear presentation, and reasonable baselines. If our responses address your concerns, we would sincerely appreciate your consideration for raising the score.

I appreciate the additional experiments. But I don't think the paper becomes substantially stronger now.

Transferring KORR to this setting exposes limitations that LocoWM's design addresses

I don't see how KORR's methodological design necessarily limits its applicability to LocoWM's setting. Moreover, the core idea is still very similar and these differences cannot be considered as novel methodological contribution in my view.

Contributions: We rewrote them as value → design → evidence

To be honest, I think the rewritten contribution paragraphs are even worse now. It's more like bad AI writing with vague bolded paragraph-opening summaries and make the paper's main contributions less clear. Writing contributions as "value → design" doesn't really make sense to me.

Therefore, I would like to keep my rating as weak accept.

Thank you for the further feedback.

We appreciate the reviewer's assessment of novelty and understand the similarity between KORR and LocoWM in using prediction to guide correction. We would like to briefly summarize the core differences between the two:

  • in KORR, the base and the execution policy serve the same goal, and prediction is used to enhance the robustness;
  • while in LocoWM, the base is only responsible for instruction tracking, the world model predicts the base actions (potential biases), and the residual then uses this prediction to achieve a new goal.

Regarding the contributions, we have revised the text again using simpler and more straightforward language, as follows:

  • We present LocoWM, which lets a frozen locomotion policy perform a new precision task through a residual guided by a world-model forecast.

  • It is shown that anticipating the effect of the base action is more effective for the residual than reacting to the current state. With a forecast, the residual can correct the action before a deviation appears. The forecast comes from a learned world model, so no analytical model or online optimization is needed. Using this forecast as the residual's input improves payload retention.

  • Ablations show that the forecast helps most when it is action-conditioned, targets task-relevant physical quantities, and stays frozen during residual training. Regression experiments confirm that its accuracy on these quantities correlates positively with control performance.

Thank you again for your time and constructive feedback.

Reviewer_LzZ8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work introduces LocoWM: a method for high-precision locomotion through residual adaptation. First, a locomotion policy is trained to follow a motion command, and alongside it an action-conditioned model learns to predict a sequence of future, task-relevant states (task substate). Both modules are then frozen, and a residual adaptation module is trained to generate additive action corrections. It uses the predicted future substates to bring the task-relevant quantities closer to their desired values. Simulation experiments on a Go2-W robot show improved terrain leveling and payload retention over end-to-end and observation-conditioned residual baselines. Additional simulation experiments evaluate humanoid tray transport on a Unitree G1, reporting higher payload-retention success than the evaluated baselines. Lastly, the authors demonstrate zero-shot transfer to hardware and present terrain traversal, acceleration compensation, and push recovery.

Critical strengths
  1. Improved performance in simulation. LocoWM improves terrain leveling and payload retention over the end-to-end and residual baselines across several terrains. Those results demonstrate the practical value of the approach even though the contribution of explicit forecasting needs further isolation.

  2. Successful zero-shot hardware transfer. The Go2-W experiments demonstrate terrain traversal, acceleration compensation, and push recovery using proprioceptive observations. These demonstrations support the feasibility of deploying the approach beyond simulation.

Critical weaknesses
  1. The distinction between "reactive" and "preactive" control is overstated. In the sections 2 and 3.2 the authors argue that observation-conditioned residual adaptation modules can only react after a deviation becomes observable. However, such "reactive" methods can learn anticipatory corrections from observation history and the base action, without explicitly predicting future states. The proposed method itself, predicts future states from observation history and base action. With sufficient capacity, a residual policy receiving the same inputs could therefore, in principle, represent the same mapping to corrective actions. Explicit forecasting may improve sample efficiency, generalization or other aspects of the method, but it does not introduce a fundamentally new ability to anticipate. The React and Recon ablations are useful, but do not fully isolate the benefit of forecasting. Section 4.1 defines React as receiving the deployable observation history, with Recon additionally receiving a reconstructed current task substate. Neither description specifies the current base-policy action as an explicit input, whereas LocoWM’s adapter receives it. Appendix B.6 provides LocoWM’s network architectures and optimization settings, but does not clearly establish matched capacity and training budgets across the baselines. A residual baseline receiving the same observation history and current base-policy action, with comparable capacity and training budget, would help isolate the benefit of explicit forecasting.
  2. The incremental contribution over closely related work is unclear. KORR (Gong et al., ICRA 2026) predicts the next latent state using learned Koopman dynamics, given the current state and a proposed action from a frozen base policy. It then conditions a learned residual adaptation module on that prediction and executes the sum of the two actions. This closely related work is neither discussed nor compared against. LocoWM does differ in multiple aspects: multi-step prediction of substates, freezing the dynamics model during residual training, and the locomotion application. These differences might be valuable, but the experiments do not isolate their effects or establish their advantages over the closely related approach. Clearer positioning relative to KORR, together with a matched comparison against a KORR-style method adapted to the same locomotion tasks, would help establish the contribution.
Questions for authors
  1. What is the methodological contribution beyond KORR? Can a matched comparison against a KORR-style method on the same locomotion tasks establish the benefits of the proposed method's design choices?

  2. How accurate is the state prediction model from states visited with the residual controller in the loop? Comparisons of one-step versus multi-step prediction and action-conditioned versus action-unconditioned prediction would help identify which design choices contribute to performance.

  3. Are React and Recon matched to LocoWM in network capacity and training budget? Their descriptions indicate access to observation history. Do they also receive the current base-policy action explicitly? If not, how does an otherwise matched baseline with this additional input perform?

3 comments

Responses


Q1: What is the methodological contribution beyond KORR? (W2 & Q1)

A1: Thank you for pointing out KORR. We have added it to (Sec. 2, L112–118, P3). KORR is evaluated on simulated manipulation, whereas LocoWM targets legged locomotion with unsecured payloads and is validated on hardware. Transferring KORR to this setting exposes two limitations that LocoWM avoids by construction (multi-step & substates: A2.2):

  1. Train–Query Mismatch: KORR fits its dynamics on executed transitions (ab+ara_b+a_r) but queries it with aba_b. Scored against these two futures, the same predictor's R2R^2 drops from 0.88 to 0.70. Instead, LocoWM trains and queries the WM on base-policy rollouts. (Appendix C.3, Table 9a, L1356, P26)

  2. Joint Training with RL: KORR updates its dynamics with the residual's RL loss, eroding predictive semantics and making the residual's input non-stationary. Doing so lowers R2R^2 from 0.88 to 0.27 and SR from 93.63% to 77.30%. (Appendix C.3, Table 9b, L1356, P26)

We implemented KORR-Koopman and KORR-MLP on Go2-W from the same base checkpoint and training budget. LocoWM reaches 93.63% SR vs. 76.48% and 71.16%, both below Recon (80.23%). KORR-Koopman is close to the jointly trained variant (77.30%), consistent with limitation 2. (Appendix C.3, Table 10, L1374, P26)


Q2: How accurate is the residual-in-loop state predictor, and what drives its performance? (Q2)

A2:

  1. Residual-in-the-Loop Accuracy: By design, the WM predicts the future under base-policy execution, which the adapter then corrects. Against this target, R2=0.88R^2=0.88; against the executed future with the residual in the loop, R2=0.70R^2=0.70 (A1.1). The gap reflects the applied correction rather than prediction error.

  2. Design Ablations: (Appendix C.4, Tables 11–12, L1404–1416, P27)

    • Horizon: Multi-step prediction improves SR (H=1: 89.70% → H=5: 93.63%), saturating at H=9 (93.76%).
    • Action Conditioning: Without aba_b, MSE rises from 0.050 to 0.117 and SR drops to 84.22%.
    • Prediction Target: Full-observation prediction has lower aggregate MSE (0.023) but lower SR (89.43%): its metrics are dominated by 51 task-irrelevant dimensions, and on task-relevant components it is less accurate (R2R^2: angular velocity 0.58 vs. 0.89; projected gravity 0.30 vs. 0.91).
  3. Accuracy vs. Control: Across WM checkpoints, SR rises with R2R^2 (0.78 → 0.88; 87.42% → 93.63%), demonstrating that improved prediction accuracy translates into better performance. (Appendix C.4, Table 13, L1433, P27)


Q3: Are React and Recon matched to LocoWM in capacity and training budget? Do they receive the current base-policy action? (Q3)

A3: Yes. We have added the omitted aba_b to the baseline descriptions (Sec. 4.1, L307–310, P6; Appendix B.6, L967, P18). All adapters receive the observation history and aba_b, differing only in extra conditioning: none (React), reconstructed current substate (Recon), or predicted future substates (LocoWM). All adapters share the same architecture, base checkpoint, and Stage-2 iterations. The WM uses the same Stage-1 rollouts, requiring no extra interaction.


Q4: The distinction between "reactive" and "preactive" control is overstated. (W1)

A4: Thank you for this insightful point. We agree that a reactive residual with the same inputs could in principle represent the same mapping, and that our wording overstated the distinction. As you suggest, the benefit of forecasting lies in what is learned in practice, not a new ability to anticipate:

  1. Equal Capacity Does Not Imply Equal Learning: Model-based and model-free RL, or predictive inverse dynamics and end-to-end behavior cloning, do not differ in representational capacity, yet the former learn behaviors that the latter fail to learn. Analogously, "preactive" means the correction is explicitly conditioned on predicted future substates, which enables it to learn better actions.
  2. The Extra Information Lies in the Training Signal: With identical deployment inputs, the WM receives dense supervision on future substates during training, whereas a reactive adapter can learn how its current action shapes the future only indirectly from reward.

With identical inputs, capacity, and base checkpoint:

MethodStage-2 iter.SR (%↑)
React10k69.75
React20k72.23
RL-only10k69.68
LocoWM10k93.63

Doubling React's budget yields a modest gain (+2.48), and RL-only (LocoWM without predictive supervision) performs similarly to React, suggesting that the gain stems mainly from explicit forecasting. We have revised the wording accordingly (Sec. 2, L108–109, P3; Sec. 3.2, L262–267, P5).


Summary

Thank you for recognizing our performance and zero-shot hardware transfer. We hope the new experiments address your concerns. If so, we would sincerely appreciate your consideration for raising the score.

Thank you for the additional experiments, clarifications, and the added positioning relative to KORR. My concerns regarding the baseline fairness and the reactive/preactive overstatement are addressed.

However, LocoWM and KORR share a substantial methodological overlap. The contribution seems to be primarily in the refinement and application to locomotion, supported by simulation and hardware results.

In my view, the incremental changes from KORR are not sufficiently analyzed:

  • Train-Query Mismatch: Table 9a compares the same predictor on two different targets: a base policy target and a residual in the loop target. Of course a predictor that was trained on a base policy target is going to be performing better on that original target. However, this does not show that training a predictor on executed state-action transitions prevents accurate prediction of a base policy target.
  • Full observation vs task-substate: The full observation versus task-substate ablation is confounded: only the task-substate target includes linear acceleration. The full observation target does not. I don't understand why the full observation target cannot include linear acceleration.

The advantage of the methodological changes from KORR are therefore not sufficiently ablated in my opinion. Given the big overlap with KORR I remain unconvinced that the scientific contribution is sufficient. I therefore maintain my weak-rejection recommendation.

Thank you for the follow-up and for confirming that the baseline-fairness and reactive/preactive concerns are resolved.

Contribution. We appreciate your assessment of novelty and understand the similarity between KORR and LocoWM in using prediction to guide correction. To our knowledge, KORR is a promising recent work that has so far been evaluated in simulated manipulation, with hardware validation and follow-up studies yet to come. Our contribution is the design that makes this structure work for legged locomotion on physical hardware, supported by ablations: multi-step prediction (93.63% vs. 89.70%), a frozen WM (vs. 77.30%), the task-substate target (vs. 89.43%), and the full method against KORR (vs. 76.48%).

Regarding your two points on how these differences are analyzed:

  • Train–query mismatch. The base and executed policies are trained for different rewards: the base policy for command tracking, the executed policy additionally for the precision reward rprecr^{\mathrm{prec}}. They therefore induce fundamentally different trajectory distributions. This mismatch is symmetric: Table 9a measures it in one direction (MSE 0.050→0.1290.050\rightarrow0.129), and a predictor trained on executed transitions would face the same mismatch on the base-policy target.

  • Full observation vs. task substate. Linear acceleration is absent from the full observation simply because, as is standard in legged locomotion, onboard accelerometer readings are too noisy to be used as policy input, whereas the task substate can include it as a privileged training label in simulation.

We hope these clarifications address your remaining concerns. Thank you again for your careful and constructive engagement throughout the discussion. We are especially grateful that you recognized our additional experiments and found the baseline-fairness and reactive/preactive concerns resolved. Your comments have clearly improved the paper.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

Revision Summary

We sincerely thank all reviewers for their careful reading and constructive feedback. We have made every effort to address each concern through new experiments, clarifications, and revisions, all highlighted in blue in the revised PDF:

  1. New Experiments

    • Added a matched comparison with KORR (Koopman and MLP dynamics) on Go2-W, with analyses of train–query consistency and joint RL training (Appendix C.3; R1, R2, R3).
    • Added world-model ablations on prediction horizon, action conditioning, and prediction target, plus per-component accuracy and accuracy vs. control across checkpoints (Appendix C.4; R1, R2, R3).
    • Added a repeated quantitative hardware comparison: 5 methods × 3 terrains × 10 trials = 150 real-world trials (Appendix C.2; R2, R3).
    • Added matched-input reactive baselines with doubled training budget and an RL-only variant (R1), and end-to-end distillation of LocoWM (R2).
    • Added standard deviations and 95% CIs for Go2-W and G1 results (Appendix C.1.1, D.2; R3).
  2. Methodological Clarifications

    • Positioned LocoWM relative to KORR in Related Work (Sec. 2; R1, R2, R3).
    • Clarified that the reactive/preactive distinction concerns what is learned via dense predictive supervision, not representational capacity (Sec. 2, 3.2; R1).
    • Clarified that all residual baselines receive the base action and share the same architecture and training budget (Sec. 4.1, Appendix B.6; R1).
    • Clarified that the WM forecasts the uncorrected base-policy future and guides the correction rather than controlling locomotion (Sec. 1; R2).
  3. Presentation Improvements

    • Rewrote the contribution paragraph as value → design → evidence (Sec. 1; R2).
    • Specified unsecured-payload transport in the abstract and extended the Limitations to other precision objectives (Abstract, Sec. 5; R2, R3).

Overall

We are grateful for the encouraging affirmations from R1 (LzZ8), R2 (z42W), and R3 (r4VD) on:

  • Overall Quality: (R2: overall, I think it is a strong submission, this is a solid paper and I don't see a major weakness in it, no substantive concerns about the paper's soundness).

  • Problem Significance: (R2: the target problem is well-motivated, a useful task), (R3: a practically meaningful problem).

  • Method Design and Contribution: (R2: a solid solution, could inspire future work to further fulfill the potential of this idea), (R3: a coherent framework, a clear separation between locomotion acquisition and precision adaptation, physically interpretable task substates ... provides a well-motivated design, meaningful differences through task-specific physical substates, multi-step prediction, and staged training), (R1: these differences might be valuable).

  • Performance and Effectiveness: (R1: improves terrain leveling and payload retention over the end-to-end and residual baselines across several terrains, demonstrate the practical value of the approach), (R2: complete the transport over four different terrains with at least 87% success rate), (R3: show meaningful improvements).

  • Real-World Validation: (R1: successful zero-shot hardware transfer, support the feasibility of deploying the approach beyond simulation), (R2: cool demos), (R3: physical Go2-W demonstrations).

  • Experimental Coverage and Presentation: (R3: substantial experimental coverage within the target application), (R2: the baselines are chosen reasonably, nice figures), (R1, R2: clear and easy to follow).

We have worked hard to address every concern, and believe the new KORR comparison, world-model analyses, and hardware evaluation substantially strengthen the paper. We hope our responses have fully addressed your concerns, and we welcome any further suggestions. If so, we would sincerely appreciate your consideration in raising the score.

Dear Reviewers,

As the discussion period is coming to a close, please take a moment to review the authors’ responses and revised manuscript. Kindly assess whether your concerns have been adequately addressed or if any questions remain, and update your scores accordingly, if necessary.

Thank you for your time and thoughtful evaluations.

Best regards, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.