- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
ForkPRM introduces executable counterfactual supervision for embodied vision-and-language navigation. By branching the environment in a simulator from shared histories under a frozen continuation policy, it generates baseline-relative process labels without human annotation. These train a verifier that ranks candidate actions. While shared-prefix supervision improves pairwise ranking, its branch-selection gains remain small compared to an oracle, showing that ranking accuracy does not yet translate to full policy utility.
Critical strengths
The proposed method assumes the availability of a reliable simulator. If the assumption holds, the verifier can be trained effectively, and the result seems to validate that. But the paper does not address the fundamental challenge of policy learning, as explained below.
Critical weaknesses
This paper presents a PRM, implemented as a verifier, trained on process labels generated by a simulator. So, the quality and fidelity of the simulator directly bound the reliability of the training labels. If the simulator's physics, rendering, or navigation graph dynamics diverge too much from reality (or even from ideal reasoning), the verifier learns from noisy or flawed counterfactual outcomes.
Therefore, while the tech seems useful in some settings with a reliable simulator, as validated in the paper, this study does not address the fundamental challenges of policy learning for physical AI -- the lack of reliable simulators in many settings. Notice that if such a reliable simulator is available for a setting, it is easy to generate unlimited amounts of training data for policy learning, and whether to use PRM is not that important because the original goal of PRM is to improve the sample efficiency of policy learning.
Questions for authors
it would be useful to evaluate the effectiveness of ForkPRM on different settings where simulators are of different quality. It would give us a sense what ForkPRM can achieve beyond the simulator.