acceptodds
Withdrawn from ICLR 2027

LaRA-WM: Test-Time Latent Action Refinement for Robotic Manipulation in Dynamic Environments

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Robotic manipulation policies trained from visual observations often achieve strong in-distribution performance, yet remain brittle under out-of-distribution changes such as novel objects, altered lighting, and unseen scene layouts. A key limitation is that most existing policies directly map high-dimensional observations to low-level motor commands, leaving little room for deliberation or correction before action execution. In this paper, we propose LaRA-WM (Latent Robot Action World Model), a framework that shifts robotic control from direct action regression to reward-guided optimization in a compact, task-centric latent action space. During offline training, LaRA-WM learns visual and language-conditioned latent action representations from expert demonstrations, together with an action decoder and a reward-feature scorer that evaluates candidate latent intents. At test time, instead of producing a one-shot action, LaRA-WM iteratively refines latent actions using the Cross-Entropy Method guided by the learned scorer, and only then decodes the optimized latent intent into an executable motor command. This design enables the policy to “think before acting” in a structured semantic space, avoiding the instability and local optima associated with raw action-space search. Experiments in the RoboTwin manipulation benchmark show that LaRA-WM consistently improves task success rates, returns, and out-of-distribution generalization over direct policies and non-refinement latent baselines. These results suggest that test-time latent action refinement provides an effective and efficient mechanism for robust robotic manipulation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robotic Manipulation, Test-Time Adaptation, Out-of-Distribution Generalization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Bc4bOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LaRA-WM, a robotic manipulation framework that performs reward-guided test-time search in a learned latent action space. The method combines frozen DINOv2 and CLIP encoders, a conditional variational latent action model, an action decoder, an auxiliary feature-transition model, and a learned reward scorer. At deployment, the Cross-Entropy Method (CEM) refines candidate latents before decoding a motor command. Experiments on 12 RoboTwin tasks report improved in-distribution and out-of-distribution success relative to behavior-cloning, action-chunking, diffusion, and VLA baselines, with ablations examining latent modeling, reward supervision, and refinement.

Critical strengths
  1. Coherent integration of representation learning and test-time action selection: The framework provides a clear mechanism for revising an action before execution, while avoiding pixel-level future generation. Separating expensive visual encoding from lightweight candidate scoring is a sensible design for batched test-time search (Sections 3.2–3.4).

  2. Consistent gains across the evaluated manipulation tasks: Tables 1–2 report an average OOD success rate of 0.37 versus 0.27 for OpenVLA-oft. Importantly, Table 3 also includes a stronger latent/world-model variant without refinement, which reaches 0.24; the improvement to 0.37 provides more informative evidence for refinement than comparison with the minimal latent bottleneck alone.

  3. Useful component diagnostics and measured inference cost: The ablations distinguish feature-prediction accuracy from closed-loop task performance rather than assuming they are equivalent. Appendix E reports five CEM iterations with 64 candidates each and approximately 52 ms policy latency on H200, within the evaluated 10 Hz control budget.

Critical weaknesses
  1. The supervision and reliability of the reward scorer are insufficiently established: Equations (5)–(6) regress immediate rewards from expert demonstrations, but the manuscript does not specify the actual reward construction, whether dense shaping is used, or whether failed/suboptimal transitions provide coverage beyond expert behavior. CEM nevertheless evaluates hypothetical latent actions outside the observed demonstrations. The aggregate reward–environment correlation in Table 3 does not establish that the scorer correctly ranks alternative actions at the same state. Moreover, Algorithm 1 uses neither predictive rollouts nor a return/value target. Consequently, the link between immediate-reward optimization, reliable candidate ranking, and completion of multi-stage tasks remains under-explained.

  2. The claimed advantage of latent-space search over raw-action search is not isolated: The introduction contrasts LaRA-WM with raw-action verifiers, but the experimental tables contain no budget-matched raw-action search baseline. Table 4 specifies a 32-dimensional latent and a 7-dimensional motor action, so the advantage is not dimensional compression of the motor command itself. A learned representation could still improve search geometry, but KL regularization alone does not establish that CEM avoids poor local optima or remains within physically plausible actions. The current results also do not distinguish iterative CEM refinement from simply scoring more independently sampled candidates.

  3. Important implementation and evaluation details remain incomplete: Although Appendix E supplies the iteration count, sample count, latency, and three-seed rollout protocol, the PDF does not specify the elite count, the value of the KL coefficient, or the complete feature-transition objective and its weighting. Appendix B describes the types of OOD shifts but not their numerical ranges or exact held-out asset/configuration lists. Tables 1–3 provide no uncertainty estimates despite the reported repeated runs. These omissions make the magnitude and reproducibility of the robustness gains difficult to assess independently.

Questions for authors
  1. What exact reward labels supervise the scorer, and what coverage of unsuccessful or suboptimal behavior is available? On a small set of held-out states, can the authors check whether the scorer's ranking of sampled candidate actions agrees with their realized rewards or downstream outcomes?

  2. On two representative tasks, how does latent CEM compare with raw-action CEM using comparably trained scorers and the same evaluation budget, and with a single-round best-of-320 latent sampler? Reporting success and policy latency would help distinguish representation benefits from search-budget effects.

  3. Can the authors provide the complete training objective, missing CEM/regularization settings, and precise OOD configuration, together with per-seed dispersion or confidence intervals from the already completed evaluation runs?

Reviewer_uxF7OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes the LaRA-WM method for robotic manipulation. The method employs a Conditional Variational Autoencoder to learn a latent action space, leveraging frozen DINOv2 and CLIP features; it simultaneously trains an action decoder, a feature-space world model, and a reward scorer. During the testing phase, the method utilizes the reward scorer to perform a search within the latent action space via the Cross-Entropy Method, subsequently decoding the optimal latent action into a concrete robot action. Experimental results demonstrate that LaRA-WM achieves a higher success rate in OOD scenarios compared to ACT, DP, and OpenVLA-oft.

Critical strengths

The paper studies an important problem: how to use test-time computation to make manipulation policies more robust under distribution shift.

Critical weaknesses

W1. Writing and presentation need major improvement. By default, RoboTwin is a bimanual setting with a camera in the middle that sees the whole workspace. The views in Figure 2 are very unusual. In many frames, the robot arm almost fully blocks the camera, so it is hard to tell what the task is. Figure 2 also does not show a baseline or a no-refinement rollout for comparison, so it does not show what the refinement changes.

W2. The evaluation setting is unclear. RoboTwin uses the bimanual AgileX robot by default. The robot in Figure 2 appears to be a Franka arm. Table 4 in the appendix lists an action dimension of 7, which corresponds to a single arm. If the authors modified RoboTwin, they should state this clearly. They should also explain how bimanual tasks are done with one arm. In addition, the examples do not look like the official RoboTwin OOD setting, which randomly changes the background. it should clearly show how OOD is build and what is the OOD range of different factors.

W3. Strong recent baselines are missing. ACT, Diffusion Policy, and OpenVLA-oft are relatively old baselines. Outperforming them does not clearly show the benefit of the method. The paper should compare with recent VLA models such as π0.5 and GR00T, or with world-action models. The paper already cites some of these works but does not compare with them.

W4. The ablation results seem to contradict the paper’s own claim. In Table 3, the OOD success rate is 0.12 without the world model and 0.10 with it. This suggests the world model does not help and may hurt. Based on Eq. 5, Eq. 8, and Algorithm 1, the reward scorer and CEM do not use the world model, and the world model is not used at test time. So a variant with the reward scorer and CEM but without the world model can be built. This ablation is missing. Without it, the role of the world model, which the method is named after, is unclear.

Questions for authors

Q1. If RoboTwin was modified, what was changed, and how are bimanual tasks handled with a 7-D action?

Q2. How exactly is the OOD setting built, and how does it differ from the official RoboTwin setting?

Q3. How does LaRA-WM compare with π0.5, GR00T, or a SOTA world-action model?

Q4. What is the result of the reward scorer plus CEM without the world model? If performance does not drop, what is the role of the world model?

Reviewer_76yqOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes LaRA-WM, a framework for improving out-of-distribution (OOD) generalization in robotic manipulation through reward-guided test-time latent action refinement. The method learns a compact latent action representation from expert demonstrations using a conditional variational autoencoder, together with an auxiliary future-feature world model, an action decoder, and a learned reward scorer. During inference, the framework employs the Cross-Entropy Method (CEM) to optimize candidate latent actions according to the predicted rewards before decoding the selected latent into executable robot actions. Experiments on the RoboTwin benchmark report improvements over several imitation-learning and vision-language-action baselines. In particular, LaRA-WM achieves an average OOD success rate of 0.37, compared with 0.27 for OpenVLA-OFT. The authors also provide ablation studies and qualitative case studies to investigate the contributions of the proposed components.

Critical strengths
  1. The paper addresses the important challenge of OOD generalization in robotic manipulation by introducing reward-guided optimization in a task-conditioned latent action space. Instead of directly predicting motor commands, the framework enables action evaluation and refinement before execution, offering a promising approach to improving decision robustness under visual and spatial distribution shifts.

  2. The proposed framework combines learned latent action representations, a reward scorer, and CEM-based test-time refinement. By evaluating and refining multiple latent action candidates before execution, the method provides a practical alternative to one-shot action prediction. The ablation results further suggest that iterative refinement contributes to improved task success rates.

Critical weaknesses

1.The proposed framework mainly combines existing techniques, with limited methodological novelty. Unlike prior world-model planning approaches such as LeWorldModel, the proposed world model does not directly participate in test-time decision-making, and its independent contribution is not adequately demonstrated. 2.The reward scorer is trained on offline expert demonstrations but is used to evaluate unseen latent actions during test-time optimization. Its reliability under OOD conditions remains insufficiently validated, raising concerns about reward exploitation and the effectiveness of CEM-based refinement. 3.The experimental evaluation lacks critical comparisons with compute-matched Best-of-N selection and raw action-space CEM, making it difficult to isolate the benefits of latent optimization. Moreover, the absence of real-world robotic experiments limits the evidence supporting the method's practical robustness and generalizability.

Questions for authors
  1. How does the reward scorer reliably rank unseen latent actions under OOD conditions when trained primarily on expert demonstrations?
  2. Does latent-space CEM outperform compute-matched Best-of-N selection and raw action-space CEM?
  3. Can the authors demonstrate the effectiveness and real-time feasibility of LaRA-WM on physical robots?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.