- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
IP-GRPO uses a world model to predict future observations of candidate actions, and uses the difference between the imagined progress and the current progress as an auxiliary reward topost-train existing navigation policies using GRPO. The average success rate across two strategies and two benchmarks improved by 4.83 percentage points; the average success rate across three real-world robotic tasks was 76.7%. The auxiliary module is only used during training.
Critical strengths
- IP-GRPO retains the original deployment process, eliminating the need for online future prediction or candidate search, thus possessing practical value.
- Improvements are consistent across both policies and benchmarks. The full method also outperforms task-reward-only GRPO, and the real-robot experiments provide preliminary evidence of zero-shot transfer.
Critical weaknesses
- Lack of substantial methodological novelty. This paper primarily combines existing navigation strategies, world models, semantic progress reasoning modules, and GRPO. Its main contribution lies in the imagined-progress reward. However, the ablations simultaneously change future observations and reward formulation, failing to isolate the benefit of imagined futures over simple progress differencing.
- The mechanism and reliability of rewards remain unclear. Auxiliary candidates and actions are sampled independently, and immediate rewards do not directly evaluate the consequences of actions. The correlation of the scorer on failure trajectories drops to 0.3831; candidate ranking is based on the same scorer. This evaluation does not independently establish that the reward accurately reflects true task progress.
Questions for authors
- Could you add two controls: progress differences computed from actual observations before and after execution, and imagined progress scores without subtracting current progress? Please match the number of environment interactions and policy updates, and report results across multiple training seeds.
- How does the reward from independently sampled candidates improve policy learning, given the different sampling temperatures (1.0 for candidates and 0.2 for execution)? Please provide the training configuration and the core reward, advantage, and loss implementations.
- How are progress labels assigned when the agent leaves the reference route? Please report the evaluation sample sizes and check whether imagined progress differences agree with ground-truth progress changes from actual rollouts.
Response to Authors
Thank you for the additional experiments and clarifications. While they address some of my concerns, I still find the methodological contribution rather limited.
Prior VLN works have already explored GRPO-based post-training and progress-based supervision (e.g., VLN-R1, Progress-Think, and SACA). The main novelty here lies in constructing rewards from imagined candidate futures, but the benefit of independently sampled candidates remains insufficiently justified, as the reward does not directly evaluate the executed action.
Overall, I do not find the incremental contribution sufficiently convincing to change my original rating.