- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper argues that the intermediate reasoning state of a Vision-Language-Action (VLA) policy should be represented not as a single committed vector but as a conditional distribution over task-level plans. The authors introduce the Generative Plan Latent (GPL): a small rectified-flow head that samples a low-dimensional plan from a frozen VLA's conditioning representation, projects it into soft tokens, and conditions the action expert on it, while the backbone stays frozen and only LoRA plus a projector are trained. This makes the reasoning state addressable along three axes: coverage, steering, and outcome adaptation. The paper further proposes a redundancy principle: an explicit plan helps only when it carries decision information the observation does not already determine, measured by a redundancy score. Experiments span SimplerEnv, RoboCasa, LIBERO, and a maze domain, with both hand-designed and behavior-learned plan coordinates.
Critical strengths
- The paper targets a real and under-examined blind spot: existing VLAs may already be generative at the action level, but their upstream decision is still collapsed into one point per observation. The authors turns the intermediate decision into a distribution that can be sampled, steered, and reweighted, rather than merely a decided output, which is appealing.
- The related-work positioning is complete. Beyond the conceptual comparison table of latent-reasoning VLAs, the paper implements matched baselines that isolate the effect of where multimodality lives.
- The proposed method is lightweight and easy to graft onto a pretrained VLA, and the evaluation is broad.
Critical weaknesses
- The paper’s contribution is closer to an incremental repositioning. Each component, i.e., rectified flow, classifier-guided sampling, Boltzmann reranking, latent skills, soft-token prefix conditioning, LoRA, is well established. The novelty lies mainly in re-situating a known idea at the VLA decision bottleneck and packaging it into a unified interface. That is a legitimate contribution, but it reads as a recombination of existing ideas rather than a fundamentally new method.
- The evaluation is not fully convincing for the paper's broader claim. Most multimodal settings are constructed by the authors precisely. The headline metrics are command-mode rate and steer range rather than task success rate, and the reported absolute success is low (GPL grasps at roughly 0.33 versus 0.50 for the language baseline), with the plan-conditioning interface itself incurring a non-trivial competence penalty. A policy that follows commands more often but grasps less often may look favorable on steer range without being better in practice.
Questions for authors
- Given the reported competence penalty of the plan-conditioning interface and the low grasp rate of the plan arm, how do you weigh the added controllability against the loss in raw execution?
- Could you report the end-to-end task success rate on a setting that is not purpose-built?
2 comments
Thanks for the response. My concern about the incremental contribution has been resolved. However, in Table B, GPL leads to a slight performance drop. While I accept the explanation of "no gain" based on the "redundancy principle", the slight performance drop may need more clarification.
Overall, the rebuttal has significantly strengthened my view of the paper. I am raising my score accordingly.
Response to Reviewer yQDm
We thank the reviewer for their constructive evaluation and positive remarks regarding our framing and evaluation.
W1: Core contribution and component isolation. While individual building blocks like rectified flows and LoRA adapters are established, their synthesis at the VLA decision bottleneck is novel. These parts do not automatically yield our observed properties when combined; our experiments identify exactly where they fail to do so:
Our core contribution is an understanding of where a distribution over decisions must live and when a reasoning plan is genuinely needed. Prior mechanisms (e.g., value-based selection of latent skills, classifier-steered latent diffusion reasoning) do not test these specific questions.
Q1: Steer range vs. practical success. We tested this with an end-to-end metric, commanded success: the requested object is grasped and lifted.
Table A. Commanded success in our two-cube task. Grasping the other cube or failing to grasp counts as failure; 95% bootstrapped CIs over 8 held-out layouts.
The 0.50 vs. 0.33 grasp rate cited by the reviewer counts grasps of either cube: the language arm grasps more often, but only 51% of its grasps take the requested cube (chance level), against 100% for the plan arm. Net following (commanded success minus wrong-cube completions) is +33 points for the plan code vs. +1 for language. On π0, the plan arm outperforms the action chunk at every tested guidance strength (w = 1, 2, 4, 8) and also beats the action arm's best setting (no guidance) by +10 to +14 points.
Weighing the trade-off. The 0.274 grasp-rate cost in LIBERO follows the adapter seed, not the plan: it ranges from 0.00 to 0.61 across six seeds and does not track steering strength (r=0.18). On the real robot (below), the added control costs no detectable raw success and raises it under deployment shifts.
Q2: SimplerEnv tasks and the redundancy principle. On four standard SimplerEnv tasks, the plan interface shows no detectable difference in task success (400 paired episodes).
Table B. Task success (100 paired episodes per task).
In these fully observed tasks the plan carries no information the image lacks, so the redundancy principle predicts no gain, and we detect none. When an occlusion removes the decision from the observation, the plan helps: in an occluded "spoon on towel" task (spoon hidden from step 5), a carried plan from GPL's flow head raises task success from 0.12 (stock π0) to 0.38 (+0.26 [+0.12, +0.40]; n=50 per arm, two-sided Fisher p=0.005).
Real robot: task success, not only steer range. On a real SO-101 arm (general response; video in the supplementary material), the plan-free π0.5 completes the task in 17/17 runs but chooses the strategy itself, so a request is met only by chance (50%). With GPL, the robot executes the requested strategy and completes the task in 43/52 runs (83%; exact binomial p<10−5), with raw success of 45/52 (not detectably lower; one-sided Fisher p=0.12). When a deployment shift disables one strategy, reweighting GPL's plans from each run's outcome, with no gradient step, raises task success from 45% to 81% (raised target) and from 45% to 64% (roll on its side). Following commands translates into higher success in practice.