- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces RoboCritic, a vision-language model that jointly predicts task progress, describes the observed execution state, and provides actionable guidance for robot policies. The model maintains bounded visual and feedback history to support ongoing assessment. The authors construct RPF-1M by adding aligned process annotations to existing robot trajectories. The same trained critic supplies rewards for policy learning and language guidance during execution. Evaluation includes reward prediction, feedback quality, ablations, and two real-world manipulation tasks, with improvements over the reported baselines.
Critical strengths
The paper addresses a meaningful problem through a coherent formulation. Robot policies benefit from feedback that explains the current physical situation and suggests an appropriate continuation. Combining progress assessment, state descriptions, and corrective guidance is well motivated, particularly when execution deviates from the intended plan. Supporting both policy learning and execution guidance with the same critic makes the formulation practically useful.
The dataset and experiments provide positive evidence for the approach. The aligned process-feedback annotations are a potentially useful contribution, and the evaluation covers several aspects of the proposed system. The ablations support the usefulness of richer feedback and the real-robot experiments demonstrate downstream applicability. Overall, these results make the approach promising.
Critical weaknesses
The distinctive contribution of joint modeling remains insufficiently established. Progress modeling, semantic state understanding, language-based correction, and historical context all have substantial precedent. Their integration through aligned supervision and a shared feedback interface is reasonable, but the paper provides limited insight into what is gained specifically by coupling these capabilities. The schema ablation supports the complete feedback configuration, but changes both supervision and historical context. Consequently, the connection between the proposed joint design and the downstream benefits remains unclear, limiting the significance of the contribution.
The empirical evidence is stronger for progress prediction than for reliable assessment and guidance during failures and recovery. The failure-state evaluation provides useful evidence about prediction quality, but does not directly establish improved failure detection or recovery during execution. The real-robot results are encouraging, although the limited task coverage and lack of reported variability across training runs make their robustness difficult to assess. These limitations leave uncertainty about whether the approach consistently delivers the behavioral advantages that motivate the work.
Questions for authors
-
What do the authors consider the main new insight established by jointly modeling progress, state, and guidance? The schema ablations demonstrate a benefit from the overall configuration; how should that benefit be interpreted in relation to additional supervision, richer history, and improved downstream behavior?
-
Do the downstream gains primarily arise from better guidance during ordinary task progression, or from detecting and correcting failures? An analysis of the existing evaluation trials separating these situations would materially strengthen my understanding of whether the results support the central motivation.
-
Could the authors provide actual failure and recovery demonstrations beyond the schematic example in Appendix C.7, showing when RoboCritic’s feedback successfully guides recovery and when it fails?