- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
MARVL improves vision-language rewards for robotic reinforcement learning through scene–view disentanglement, stage-specific task-direction projection, and threshold-based reward shaping. It achieves strong results on eight Meta-World tasks, with additional, limited evidence of robustness across environments, camera settings, and RL algorithms.
Critical strengths
-
Important problem and clear motivation. The framework systematically addresses viewpoint sensitivity, unreliable progress estimation, and semantic misalignment in VLM rewards.
-
Strong main empirical results. MARVL achieves high final success rates across all eight Meta-World tasks and substantially outperforms the selected baselines on several tasks.
-
Multiple forms of validation. Ablations and experiments involving camera changes, Panda-Gym, TD3, and generated subgoals provide initial support for the framework’s effectiveness.
Critical weaknesses
-
Monotonicity claims exceed the evidence. The analysis relies on assumed progress monotonicity, additional alignment conditions, and a vanishing-noise limit, without adequately covering the practical setting of . Figure 17 also shows clear reward decreases.
-
Additional supervision confounds attribution. MARVL uses paired multi-view data and manually specified intermediate goal images, but lacks simple goal-image reward and standard fine-tuning baselines under matched supervision.
-
Novelty and related-work positioning are insufficiently established. The components closely relate to existing disentanglement, projection, and thresholding methods, while directly relevant work such as DrS [1], REDS [2], and T²-VLM [3] is omitted.
[1] Mu et al. DrS: Learning Reusable Dense Rewards for Multi-Stage Tasks. ICLR 2024.
[2] Kim et al. Subtask-Aware Visual Reward Learning from Segmented Demonstrations. ICLR 2025.
[3] Zhao et al. Training-free Generation of Temporally Consistent Rewards from VLMs. arXiv:2507.04789.
Questions for authors
-
Can the authors establish monotonicity conditions for and evaluate them using ranking correlations and monotonicity violation rates across multiple trajectories?
-
Do all baselines receive the same pretrained models, adaptation data, and intermediate-goal information? How do simple image-distance rewards and existing projection methods perform under matched supervision?
-
What is MARVL’s distinct contribution beyond existing view disentanglement, directional projection, and stage-aware rewards? Can the authors add the relevant discussion and comparisons?
-
Table 2 lists ViT-B/32 together with a patch size of 14. Which configuration was actually used?
-
Please specify the number of evaluation episodes, the meaning of shaded regions, and the number of seeds used in all supplementary experiments.
Thank you for the clarifications. Your response addresses most of my concerns. I've increase my rating to weak acceptance.