- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes PROACT, a progress-guided adaptive contextualization framework for one-shot in-context learning with vision-language-action models. The method estimates execution progress relative to a reference demonstration and dynamically adjusts visual token allocation and attention routing accordingly. An auxiliary future visual feature prediction objective is further introduced to encourage behavior grounding. Experiments on LIBERO-Gen, IsaacSim-Gen, and RoboMME report improvements over several baselines, with additional evaluations using human video demonstrations.
Critical strengths
- The paper addresses a practically relevant problem: dynamically utilizing video demonstrations during robot execution, rather than treating them as static context.
- The reported results show consistent improvements across multiple simulation benchmarks, and the ablation studies provide some evidence for the effectiveness of individual components.
Critical weaknesses
-
Confusing experimental design and insufficiently controlled comparisons. The evaluation mixes compositional generalization LIBERO-Gen, procedural memory RoboMME, and demonstration-conditioned manipulation IsaacSim-Gen, without clearly distinguishing which aspects of ICL are being evaluated. More importantly, the baselines differ substantially in their architectures, training configurations, and access to contextual information. For example, Table 2 compares PROACT with pi0.5, OpenVLA, and MME-VLA, although the former two are not inherently designed for demonstration-conditioned ICL, while MME-VLA focuses on historical memory. Such comparisons make it difficult to attribute the reported improvements specifically to the proposed ICL mechanism. Furthermore, the RoboMME training protocol is insufficiently specified, and its difficulty descriptions do not fully align with the original benchmark. Overall, the experimental evidence is less convincing than the reported success rates might suggest.
-
Lack of real-world robot experiments. All closed-loop evaluations are conducted in simulation. Given that the proposed method targets practical demonstration-conditioned robot manipulation, real-world validation is important, particularly for assessing robustness to visual discrepancies, execution deviations, and imperfect demonstrations. The absence of real-robot experiments substantially limits the practical significance of the reported results.
-
Limited methodological novelty. The proposed framework primarily combines existing techniques, including temporal alignment, importance-based token allocation, attention reweighting, and auxiliary future representation prediction. While their integration is reasonable, the individual components are conceptually straightforward, and the paper does not clearly establish a fundamentally new learning mechanism. The improvements could potentially be achieved through simpler adaptive demonstration retrieval or attention strategies.
Questions for authors
- Can the authors provide more controlled comparisons using the same backbone, training data, demonstration inputs, and computational budget? In particular, how does PROACT compare with a standard demonstration-conditioned policy equipped with simple progress-based retrieval?
- Could the authors clarify the training and evaluation protocols for RoboMME, including the demonstration information available to each baseline and the discrepancies between the reported difficulty settings and the original benchmark?
- Are there plans to evaluate PROACT on real robots, particularly under viewpoint changes, execution errors, and demonstrations collected from different embodiments?