acceptodds
Withdrawn from ICLR 2027

FV-PMP: Accelerating Vision-Language-Action Models with Feedback-Verified Plan-Memory Propagation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-Language-Action (VLA) models achieve strong performance in robot ma- nipulation, but frequent policy refreshes impose substantial inference costs during closed-loop execution. Under receding-horizon control, an action-chunk VLA predicts multiple future actions, while only a limited portion of the predicted sequence is executed before the next policy refresh. Executing longer prefixes reduces inference frequency, but the predicted actions may become less compatible with the evolving state as the underlying observation becomes stale. This raises a question: can unexecuted predictions support subsequent control without be- ing directly replayed? We propose FV-PMP, a Feedback-Verified Plan-Memory Propagation framework for efficient closed-loop VLA execution. Between visual policy refreshes, a lightweight propagator regenerates actions from current propri- oceptive feedback and recent execution history. To support deeper propagation, FV-PMP converts the unexecuted action suffix into Remaining-Plan Memory, re- taining compact motion and task-progress cues as conditioning information rather than executable commands. A Candidate-Conditioned Continuation Verifier esti- mates whether a propagated candidate remains suitable for continued propagation under the current execution context, while hard execution constraints further regu- late propagation. Together, they determine whether execution continues through lightweight propagation or restores current visual conditioning through corrective or full replanning. Both replanning modes generate a fresh action chunk and renew the plan context. Experiments on SimplerEnv using the CogACT backbone show that FV-PMP reduces VLA invocation frequency to 32.38% of full-policy execu- tion, achieving 3.27× average inference acceleration without sacrificing aggregate closed-loop task performance

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action models, robot manipulation, efficient inference, action propagation, action chunking

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_2qjDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a lightweight Feedback-Verified Plan-Memory Propagation (FV-PMP) framework for efficient closed-loop VLA execution. The proposed framework aims to let the previously unexecuted actions support subsequent control without being simply replayed. More specifically, FV-PMP contains several modules, including a Remaining-Plan Memory, and a Candidate-Conditioned Continuation Verifier. Through the design, whether action execution continues through propagation or restores current visual observation by corrective or full replanning would be determined, resulting in a new action chunk while renewing the plan context. Experiments are conducted on the SimplerEnv benchmark with CogACT as the VLA backbone.

Critical strengths
  1. Efficient closed-loop VLA execution is important and practical for robot deployment. It allows a robot to execute the precise action chunk while preserving the inference efficiency.
  2. The designed modules are generally reasonable to reflect the functionality of the paper’s claims.
Critical weaknesses
  1. The evidence from the experiments is generally too weak. This paper solely conducts its experiments on one simulation benchmark, SimplerEnv, without containing any real-robot experiments to support the claims. Especially for action execution efficiency, the simulator already bypasses many factors (e.g., simulator can pause during model inference) that would occur in real-robot execution. Therefore, the results will be meaningful only when they are measured from the real-world execution.
  2. In addition, this paper also doesn’t provide the correlation between the adopted simulation benchmark and the real robot in this paper’s latency-aware execution cases, so the sim-to-real gap is hard to be measured. The success on the simulation benchmarks does not have guarantee to be transferred into the real robot, making the experiments hard to truly support the efficient closed-loop action execution claims in real-world scenarios.
  3. Also, this paper solely uses a single simulation benchmark but does not contain other benchmarks in different simulation frameworks (e.g., LIBERO/LIBERO-PRO which are based on MuJoCo) to verify at least the proposed method can be generalized across different simulator types.
Questions for authors
  1. Is the proposed method model-agnostic? The current method is based on the CogACT backbone. Whether the proposed approach can be applied to different VLAs (e.g., GR00T N1.7, MolmoAct2) or different model sizes?
Reviewer_tGGSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FV-PMP, a closed-loop execution framework that reduces expensive calls to a Vision-Language-Action (VLA) policy by interleaving lightweight action propagation with selective visually conditioned replanning. The key ideas are to (i) generate short-range actions from recent execution history and proprioceptive feedback, (ii) summarize the unexecuted suffix of the last action chunk into a compact Remaining-Plan Memory (RPM) to guide deeper propagation, and (iii) use a candidate-conditioned continuation verifier plus simple execution rules to decide when to continue propagation versus trigger corrective (cached) or full replanning. On SIMPLER with a frozen CogACT backbone, FV-PMP reduces VLA invocations to about one-third and achieves 3.49×/3.05× average speedups on Visual Matching/Variant Aggregations with competitive task success.

Critical strengths
  1. The Remaining-Plan Memory is a simple but effective representation that converts the unexecuted action suffix from executable commands into compact conditioning cues (motion tendency, first gripper event, relative horizon), enabling deeper lightweight propagation without direct replay.

  2. The candidate-conditioned continuation verifier operates on proprioceptive context and the proposed action, providing an adaptive, state-dependent mechanism for deciding how far to propagate, unlike fixed-horizon or naive chunk-replay strategies.

  3. Comprehensive comparisons on SIMPLER (VM and VA) with multiple acceleration baselines, including recent caching/pruning approaches, show consistent speedups with comparable or better success.

Critical weaknesses
  1. The verifier’s supervision is based on teacher-agreement in action space with a fixed ε tolerance and gripper match; this proxy may not fully reflect task-level feasibility or visual consistency, and it ignores visual input at verification time.
  2. The verifier is trained on H2 local candidates but deployed on deeper, plan-conditioned candidates; while the paper audits transfer, supervision remains somewhat mismatched to the deployed regime.
  3. The first propagated step (H1) is executed without verification, which can introduce avoidable errors in sensitive states; this design choice is only briefly justified.
  4. Evaluation is limited to SIMPLER in simulation with a single frozen backbone (CogACT); no real-robot results or tests on other VLAs (e.g., π0/π0.5, OpenVLA variants) are provided to stress generality.
  5. Table 1 mixes re-measured results (FV-PMP vs Full CogACT) with “reported” results for other baselines, potentially confounding hardware/runtime protocol differences; the fairness of speedup comparisons may thus be imperfect.
  6. Comparisons omit several closely related adaptive commitment/verification methods (e.g., DV AC, PAV, A3, SV-VLA) that also aim to lengthen prefixes or adaptively verify continuation, which limits positioning against the broader state of the art.
  7. The action-agreement threshold (ε = 0.1 on mixed translation/rotation units) is not normalized or justified; sensitivity analyses to ε or gripper-thresholding could strengthen confidence in verifier targets.
  8. The exact computational overhead of the verifier and propagators (though presumably small) is not quantified; an explicit breakdown would clarify net gains.
Questions for authors
  1. How sensitive are the verifier’s performance and deployed VLA rate to the teacher-agreement tolerance ε and the gripper binarization threshold (0.5)? Could you report a small sensitivity study or justify ε via unit-normalized metrics?
  2. Have you tried augmenting the verifier with a very lightweight visual signal (e.g., a tiny CNN feature or optical-flow magnitude) to capture visual changes that proprioception alone may miss? What is the overhead vs. gain?
  3. In Table 1, speedups for competing methods are “reported.” Can you provide apples-to-apples measurements for at least one or two strong baselines under your hardware/protocol, or alternatively report your speedups only against your measured full CogACT and relegate cross-paper comparisons to success rates?
  4. How robust are results to different chunk lengths K and to different VLA backbones (e.g., π0/π0.5)? Does RPM generalize when K changes, or does the progress encoding need retuning?
  5. What fraction of failures are attributable to executing the unverified H1 step? Could a fast H1 verification (even with a laxer threshold) recover some failures without materially affecting speedups?
  6. Would combining FV-PMP with internal sparsification (e.g., VLA-Cache/AC2-VLA) further improve throughput? If you have preliminary numbers, please share.
  7. Can you quantify the compute overhead of the two propagators and the verifier (FLOPs or latency per step), to make the net budget clearer?
Reviewer_Aw3BOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses the high inference cost of closed-loop VLA execution caused by frequent calls to the visual policy. FV-PMP summarizes the unexecuted portion of a VLA-predicted action chunk into averaged motion directions. It combines these averaged motion directions with recent actions and robot states and uses an MLP to generate actions between consecutive visual-policy calls. A verifier then decides whether to continue execution or trigger cheaper or full visual replanning. Experiments on SIMPLER demonstrate its inference acceleration.

Critical strengths

The method is reasonably motivated, simple, and plug-and-play. It does not require retraining the VLA, since only lightweight modules are introduced and trained. The paper also provides relatively comprehensive ablation studies for its individual components.

Critical weaknesses

My main concern is that the reported speedup depends heavily on a weak CogACT execution protocol: the model predicts a 16-step action chunk but executes only one action before replanning. The method is also evaluated only on simple pick-and-place simulation tasks.

  1. The advantage over Direct Chunk Replay is unclear. Fig. 3(a) shows that Direct Chunk Replay achieves a strong success-invocation-rate trade-off across most operating points, substantially outperforms Fixed Propagation, and remains close to FV-PMP. For example, executing the first four actions of each predicted chunk requires roughly 25% VLA invocation while achieving about 61% success, whereas FV-PMP uses 33.75% VLA invocation for 64.86% success. Thus, Direct Chunk Replay uses fewer VLA calls with only a modest drop in success. The reported 3x speedup may therefore mainly result from comparing against a baseline that invokes CogACT at every control step and executes only one action at a time, rather than demonstrating a clear advantage of the proposed propagator.

  2. The evaluation scope is limited. The method is evaluated only on simple pick-and-place simulation tasks in SIMPLER, without real-robot experiments and more complex simulation benchmarks. In more challenging tasks, recursively regenerating actions with a small MLP may fail to preserve the temporal consistency of the original VLA action chunk and maintain task success.

Questions for authors
  1. Fig. 3(a) shows that Direct Chunk Replay achieves a reasonable efficiency-success trade-off and remains competitive with FV-PMP. What is FV-PMP’s explicit advantage? Please respond Weakness 1.

  2. More complex simulation benchmarks and real-robot experiments are needed. The current experimental evaluation is incomplete.

Reviewer_zRyFOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes FV-PMP (Feedback-Verified Plan-Memory Propagation), a framework that uses two lightweight MLPs to generate subsequent actions based on action history, robot proprioceptive feedback, and Remaining-Plan Memory. A verifier adaptively determines whether to continue propagating actions or to invoke the VLA for replanning.

Experiments using CogACT on the SIMPLER benchmark demonstrate an average inference speedup of 3.27×, reducing the VLA invocation rate to approximately 32.38% while maintaining competitive task success rates.

Critical strengths

1. Feedback-Conditioned Action Propagation

The method uses real-time proprioceptive feedback and Remaining-Plan Memory to regenerate actions rather than directly replaying the original action chunk. This provides a meaningful approach to reducing VLA inference costs while maintaining closed-loop control.

2. Adaptive Verification and Replanning

The verifier adaptively determines whether to continue action propagation or trigger visually conditioned replanning, achieving a favorable balance between efficiency and task success. The ablation studies also support the effectiveness of the key components.

Critical weaknesses

W1. Unfair Baseline and Unclear Acceleration Attribution

The Full CogACT baseline performs inference at every control step (execution horizon = 1), whereas FV-PMP invokes the VLA approximately once every three steps on average. Therefore, the reported acceleration may largely result from increasing the execution horizon. Although Figure 3 includes comparisons with fixed execution strategies, the paper lacks sufficiently comprehensive horizon-matched, invocation-matched, and success-matched evaluations of inference latency and task success.

W2. Method Complexity and Insufficient Generalization

The method introduces three additional trainable MLPs, Plan Memory, and multiple control rules, but the necessity of this complexity compared with simpler alternatives is not sufficiently justified. Furthermore, the experiments are limited to CogACT and SIMPLER, without validation across different VLA backbones, dynamic/OOD environments, or real-world robotic systems.

Questions for authors

Q1. Verifier Reliability and Failure Detection

The verifier relies only on action history, proprioceptive states, and candidate actions, and is supervised using agreement with the frozen VLA. How can the verifier reliably detect errors made by the teacher VLA itself or action failures caused by changes in the external environment? Could the authors provide an analysis of false acceptances, false rejections, and representative failure cases?

Q2. Necessity of Two-Stage Propagation

Why is the Local Propagator used for h=1,2h=1,2, while the Plan-Conditioned Propagator is introduced at h≥3h\geq3? Have the authors evaluated alternatives such as a single MLP, different stage-switching thresholds, or consistently incorporating Plan Memory throughout propagation?

Q3. Task-Dependent Performance and Failure Modes

Why does FV-PMP perform well on tasks such as PickCan but substantially underperform SP-VLA on certain Drawer tasks? Are these performance gaps primarily caused by MLP prediction errors, ineffective Plan Memory, or inaccurate verification and replanning decisions?

Q4. Remaining-Plan Memory and Propagation Horizon

Remaining-Plan Memory depends on the original action chunk generated by the VLA, and the maximum propagation depth is limited to $H_{\max}=15$. Is this design primarily suited to fixed-length action chunks? How would changes in action chunk length or the use of different VLA action heads affect the effectiveness of the proposed method?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.