acceptodds
Withdrawn from ICLR 2027

PearlVLA: Progressive Embodied Action-Plan Refinement in Latent Space

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Current Vision-Language-Action (VLA) models face a trade-off between efficient action generation and explicit deliberation. Directly decoding actions from vision-language backbone representations enables low-latency control, whereas textual reasoning, pixel-level subgoals, or world-model evaluation of decoded actions can improve planning but incur substantial latency and computational cost. We propose PearlVLA, a VLA framework that progressively refines a VLM-derived latent plan using feedback from the predicted consequence of each intermediate plan. PearlVLA uses a frozen latent world model (LaWM) pretrained on action-free video. At each refinement round, the current plan produces a continuous latent action code, and the LaWM predicts the corresponding latent visual subgoal. A future-guided plan refiner uses this subgoal to update the plan, so each revision reshapes the next LaWM query. After rounds, the refined plan is passed once to the host policy's action interface to produce an action chunk. We further introduce Causal Refinement-Grouped Process-Reward RL to optimize latent refinement by comparing rewards from the longer-horizon imagined futures of plan edits made at the same refinement state. Experiments on the LIBERO and RoboCasa benchmarks show that PearlVLA performs competitively against strong existing methods.

Primary area:
applications to robotics, autonomy, planning
Keywords:
VLA, latent world model

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_SQUMOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PearlVLA, a VLA framework that progressively refines latent action plans using feedback from a pretrained latent world model (LaWM). The method iteratively predicts future visual representations and uses them to update the latent plan before generating actions. The authors also introduce CRG-PRL to further optimize the refinement process through reward-based learning. Experiments are conducted on LIBERO, LIBERO-Plus, and RoboCasa using OpenVLA and GR00T N1.5 backbones.

Critical strengths

The paper is generally well written and easy to follow. The idea of using predicted future states to guide latent plan refinement is interesting, and the framework supports both regression-based and flow-matching action policies. The improvements on LIBERO-Plus, particularly under camera and robot-state perturbations, are also encouraging.

Critical weaknesses

My main concern is that the proposed framework introduces substantial complexity and computational overhead without demonstrating sufficiently significant improvements. The baseline comparisons are not convincing, as LIBERO is already a highly saturated benchmark, several competitive lightweight VLA models are missing, and the RoboCasa experiments mainly compare against older models rather than more recent alternatives. The method also heavily relies on a separately pretrained, benchmark-specific latent world model, raising serious concerns about its scalability and practicality in real-world settings. Furthermore, the improvements from progressive refinement itself are relatively marginal, and the additional RL stage provides almost no improvement despite introducing a complicated training pipeline. Overall, I do not find the current experimental evidence sufficient to justify the complexity or practical significance of the proposed approach.

Questions for authors
  1. How do the authors justify the substantial additional complexity and inference overhead? The proposed method requires multiple world model queries before action generation, which significantly increases inference latency. Given the relatively small improvements, it is unclear whether this trade-off is practical for real-world VLA deployment.

  2. Why are stronger and more recent baselines not included? LIBERO is already highly saturated, and competitive lightweight VLA methods such as VLA-Adapter are missing from the comparison. Similarly, the RoboCasa evaluation relies on GR00T N1.5 rather than more recent models such as N1.7. A stronger and more up-to-date comparison would be important to establish the significance of the proposed method.

  3. How well does the proposed method generalize beyond the pretrained world model's data distribution? The approach relies on benchmark-specific LaWM checkpoints, but it is unclear whether such a relatively small latent world model can accurately predict action consequences in more diverse and challenging environments. How would the refinement process behave when the world model produces inaccurate predictions?

  4. What is the actual benefit of iterative refinement and the additional RL stage? According to Table 4, progressive refinement only improves performance by 0.6 percentage points over the cached-LaWM baseline, while CRG-PRL further improves LIBERO and RoboCasa by only 0.2 and 0.3 points, respectively. These gains seem insufficient to justify the additional complexity, especially without stronger evidence on more challenging benchmarks or real-world manipulation tasks.

Reviewer_Fh18OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PearlVLA, which uses a frozen latent world model to iteratively refine a latent plan before action generation. Each round produces a latent action code from the current plan, predicts future visual features, and uses this feedback to update the plan tokens. The final plan conditions the action head or expert. The authors further introduce CRG-PRL, which optimizes RefineNet by comparing the imagined consequences of alternative plan edits from the same refinement state. The framework is instantiated on OpenVLA and GR00T and reports improvements on LIBERO, LIBERO-Plus, and RoboCasa.

Critical strengths
  1. Future-guided plan refinement is well motivated. The method feeds predicted consequences back into plan updates, allowing each revised plan to change the next prediction. This goes beyond one-shot future conditioning or repeated processing of a fixed auxiliary signal.

  2. The experiments demonstrate useful performance gains across different policy interfaces. The method is applied to both action-regression and flow-matching policies and improves performance on multiple benchmarks. The robustness gains on LIBERO-Plus are notable, while the OpenVLA mechanism ablations and inference-cost measurements help characterize the approach.

Critical weaknesses
  1. Consistency between the two conditioning pathways in GR00T is insufficiently examined. The original VLM hidden states remain the action expert’s cross-attention condition, while refinement updates only the additional plan prefix. When refinement changes the initial plan, these pathways may convey inconsistent information. VLM-KV dropout encourages the expert to use the prefix, but the paper does not establish how potential conflicts are resolved or whether the original conditioning weakens the intended correction.

  2. The independent contribution of RL is small, and its reliability remains unclear. CRG-PRL improves LIBERO and RoboCasa by only 0.2 and 0.3 percentage points, respectively, without repeated-training results or confidence intervals. Given its multi-branch prediction, rendering, and reward-scoring pipeline, the evidence is insufficient to assess whether these gains are stable and justify the additional training complexity.

  3. A key GR00T ablation separating additional tokens from future-guided refinement is missing. The implementation adds plan queries, an action-expert prefix, and auxiliary code supervision, but is primarily compared with vanilla GR00T. A baseline retaining the same plan-query/prefix interface and trained only with the action loss, followed by separate additions of code supervision and LaWM refinement, is needed to identify the source of the gains.

Questions for authors
  1. When the refined plan and the original VLM conditioning convey inconsistent information, how does the action expert use the revised plan? Could you provide targeted analysis or experiments assessing whether the original conditioning interferes with or counteracts the intended action correction?

  2. Could you add a GR00T baseline with the same plan-query/prefix interface trained only with action supervision, and then separately introduce code supervision and LaWM refinement to isolate their contributions?

  3. Are the small RL improvements stable? Please provide a comparison against continued SFT under the same compute budget, together with actual training time or GPU hours.

Reviewer_LUdDOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces PearlVLA, a method to iteratively refine latent plan by a world model (LaWM). LaWM predicts latent subgoal given observations and latent codes and refines the latent plan. A Causal Refinement-Grouped Process-Reward RL further optimizes latent refinement by comparing rewards from the longer-horizon imagined futures. Experiments are performed on LIBERO, LIBERO-Plus and RoBoCasa.

Critical strengths
  • PearlVLA improves OpenVLA-OFT by large margin on LIBERO-Plus under zero shot setting.
Critical weaknesses
  • The performance of PearlVLA is on par with LaWAM on LIBERO, PearlVLA also uses LaWM as the world model. Furthermore, the latency of PearlVLA is also slightly higher than LaWAM. These make the method contribution of refinement framework weaker.

  • Given the comparable performance of LaWAM on LIBERO, LaWAM also needs to be compared on LIBERO-Plus to see the zero shot performance. The improvement over OpenVLA-OFT may come from the use of LaWM.

  • On RoBoCasa, stronger baselines are missing, e.g. Xiaomi-Robotics-1, RLDX-1 and World2Act, all of them exceed 70% averaged success rate. Same as in previous comment, LaWAM needs to be compared.

  • CRG-PRL has trivial improvement, with 0.2 and 0.3 on LIBERO and RoBoCasa.

Questions for authors

See weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.