- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a lightweight Feedback-Verified Plan-Memory Propagation (FV-PMP) framework for efficient closed-loop VLA execution. The proposed framework aims to let the previously unexecuted actions support subsequent control without being simply replayed. More specifically, FV-PMP contains several modules, including a Remaining-Plan Memory, and a Candidate-Conditioned Continuation Verifier. Through the design, whether action execution continues through propagation or restores current visual observation by corrective or full replanning would be determined, resulting in a new action chunk while renewing the plan context. Experiments are conducted on the SimplerEnv benchmark with CogACT as the VLA backbone.
Critical strengths
- Efficient closed-loop VLA execution is important and practical for robot deployment. It allows a robot to execute the precise action chunk while preserving the inference efficiency.
- The designed modules are generally reasonable to reflect the functionality of the paper’s claims.
Critical weaknesses
- The evidence from the experiments is generally too weak. This paper solely conducts its experiments on one simulation benchmark, SimplerEnv, without containing any real-robot experiments to support the claims. Especially for action execution efficiency, the simulator already bypasses many factors (e.g., simulator can pause during model inference) that would occur in real-robot execution. Therefore, the results will be meaningful only when they are measured from the real-world execution.
- In addition, this paper also doesn’t provide the correlation between the adopted simulation benchmark and the real robot in this paper’s latency-aware execution cases, so the sim-to-real gap is hard to be measured. The success on the simulation benchmarks does not have guarantee to be transferred into the real robot, making the experiments hard to truly support the efficient closed-loop action execution claims in real-world scenarios.
- Also, this paper solely uses a single simulation benchmark but does not contain other benchmarks in different simulation frameworks (e.g., LIBERO/LIBERO-PRO which are based on MuJoCo) to verify at least the proposed method can be generalized across different simulator types.
Questions for authors
- Is the proposed method model-agnostic? The current method is based on the CogACT backbone. Whether the proposed approach can be applied to different VLAs (e.g., GR00T N1.7, MolmoAct2) or different model sizes?