acceptodds
Under review as a conference paper at ICLR 2027

CRAVE: Recovering Progress from Repeated Demonstrations for Contact-Rich Robot Post-Training

Abstract

Progress-conditioned robot post-training can emphasize the brief transitions that determine contact-rich task completion, but its supervision typically requires manual semantic stages, a separately trained value estimator, or elapsed-time proxies that confuse time with task advancement. We introduce Cross-Episode Recurrent Abstraction via Multimodal Evidence (CRAVE), an offline relabeler that recovers an episode-consistent progress coordinate from visual–state configurations that recur across demonstrations. Unlike framewise temporal proxies, CRAVE couples assignments over each complete episode and converts future progress into ordinal training conditions for π0.5 while leaving its objective and deployed architecture unchanged. Mechanistically, on 300 duration-balanced held-out garment-folding episodes, trajectory coupling localized human-referenced phase boundaries 12.72 s more accurately than framewise assignment (95% paired-bootstrap confidence interval, 11.38–14.09 s). Downstream, across 180 real-robot rollouts from nine matched policy checkpoints, Direct CRAVE completed 11/20, 15/20 and 16/20 trials at 150, 300 and 450 demonstrations, versus 10/20, 14/20 and 16/20 for the strongest matched comparator (plain behavior cloning or an adapted historical χ0 Stage Advantage labeler). Across tasks, the same relabeling interface transferred to nail painting and low-data writing: observed mean four-stage completion was 35% for CRAVE versus 30% for SFT over 20 nail-painting episodes per route, and the CRAVE-labeled writing policy completed 16/19 unique rollouts after training on 55 demonstrations. Operationally, a common eight-A100 engineering projection placed CRAVE label construction at 3.7–5.6 min versus 66–137 min for inference and export from an existing Stage Advantage checkpoint. Repeated demonstrations can therefore provide progress supervision for contact-rich VLA post-training without human phase labels, on-policy collection, or a separately trained reward, value or advantage estimator.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, vision-language-action models, imitation learning, progress estimation, contact-rich manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_4jMKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies Learning from Demonstrations (LfD) and proposes CRAVE, a training-free offline method for recovering task progress from repeated robot demonstrations. CRAVE combines frozen DINOv3 visual features with robot-state information, applies dimensionality reduction and clustering to identify recurrent configurations across demonstrations, and performs whole-trajectory decoding to obtain a coherent progress estimate. Based on the recovered progress, the method estimates the future progress increment of each state and converts it into ordinal progress conditions (LOW/MIDDLE/HIGH), which are then used to post-train a π0.5 policy.

The key contribution is to use recurrence across repeated demonstrations as an implicit source of progress supervision, avoiding manually annotated task stages or a separately trained reward/value model. The authors validate the recovered progress representation and further evaluate its utility for downstream robot policy learning on several manipulation tasks. On garment folding, CRAVE improves the success rate over plain π0.5 SFT from 50% to 55% with 150 demonstrations and from 55% to 75% with 300 demonstrations, while both methods achieve 80% with 450 demonstrations. The paper also evaluates a low-data writing task with 55 demonstrations, where plain SFT achieves 90% success and CRAVE achieves 84.2%, as well as a nail-painting task where CRAVE provides a modest improvement in stage completion. Overall, the results suggest that CRAVE can recover meaningful task-progress signals from repeated demonstrations and can improve downstream policy performance in some settings, although the improvements over standard SFT are not consistent across all evaluated data regimes and tasks.

Critical strengths
  1. The paper proposes a simple and training-free offline progress estimation/relabeling method based on repeated demonstrations. The method does not require training an additional reward, value, or advantage model, making it relatively easy to integrate into existing Learning from Demonstrations pipelines.
  2. The idea of exploiting recurrent configurations across demonstrations to construct a shared progress coordinate is interesting. In particular, the whole-trajectory decoding results provide evidence that the recovered progress signal is more temporally coherent than frame-wise assignment.
Critical weaknesses
  1. The downstream benefit of CRAVE appears limited and inconsistent. While CRAVE produces a meaningful progress signal, it is unclear whether this signal translates into a practically meaningful improvement in the learned policy. For garment folding, the success rates of BC and CRAVE are 50% vs. 55%, 55% vs. 75%, and 80% vs. 80% at 150, 300, and 450 demonstrations, respectively. Moreover, the reported confidence intervals for the CRAVE-vs.-BC differences all include zero. Therefore, the current evaluation does not provide sufficient statistical evidence to distinguish a consistent CRAVE improvement from rollout-level variation.
  2. The experiments do not clearly establish the data regime in which CRAVE is beneficial. If the recovered progress signal provides additional useful supervision from an existing demonstration corpus, one might expect its benefit to be particularly visible in some data regimes. However, the low-data writing experiment does not show an advantage: plain π0.5 SFT achieves 18/20 success (90%), whereas Direct CRAVE achieves 16/19 (84.2%). Similarly, the advantage on garment folding disappears at 450 demonstrations. These results make it difficult to determine when the additional CRAVE conditioning is actually useful for downstream policy learning.
  3. The downstream evaluation is relatively limited for establishing generality. The method is motivated as a generic approach for recovering progress from repeated demonstrations, but the robot experiments cover only a small number of tasks and relatively constrained settings. The nail-painting experiment also shows only a modest improvement (35% vs. 30% mean stage completion). More diverse tasks, particularly tasks with different temporal structures, substantial state aliasing, or multiple valid solution trajectories, would be useful for understanding when the recurrence assumption underlying CRAVE holds and when it fails.
  4. The robot evaluation has substantial statistical limitations. The experiments use only one trained checkpoint per method and relatively few real-robot rollouts. Therefore, the reported uncertainty mainly captures rollout-level variation rather than variation across training runs. This makes it difficult to distinguish improvements caused by CRAVE from stochasticity introduced during policy training and evaluation.
Questions for authors
  1. What empirical evidence demonstrates that CRAVE conditioning actually improves downstream policy learning rather than merely producing a reasonable progress representation? The progress-recovery evaluation appears convincing, but the downstream policy improvements are considerably less clear.
  2. Under what data regime do the authors expect CRAVE to provide the largest benefit? The garment-folding improvement is largest at 300 demonstrations but disappears at 450 demonstrations, while the 55-demonstration writing experiment does not outperform plain SFT. Is there an explanation for this behavior?
  3. How sensitive is CRAVE to demonstrations containing multiple valid strategies or substantially different trajectories? Since the method relies on recurrent cross-episode configurations to establish a shared progress coordinate, it is unclear how the approach behaves when demonstrations do not share a consistent temporal or state-space structure.
  4. Could the authors provide results across multiple independently trained policy checkpoints? This would help separate the effect of CRAVE from training variance and make the downstream performance comparison substantially more convincing.
Reviewer_PYUyOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes a progress annotation method for VLA single task SFT by grouping similar frames among the fine tuning corpus. The groups that have high occurrence across multiple runs get labeled as ordered prototypes (basically keyframes), which are used as progress labels to SFT Pi0.5 with low, middle, and high prompt labels to indicate relative advantage. The idea is appealing, but many pieces of the method seem drawn from other recent papers, and, most importantly, the effects on policy performance seems small, and is validated with too few rollouts to have confidence in the statistics.

Critical strengths

The paper addresses an important problem of labeling SFT data for VLAs with advantage or progress weights to improve fine tuned policy performance on a specific task. VLA fine tuning can be very sensitive to the quality and volume of the fine tuning data. With mixed quality data, the policy may learn poor behaviors as often as good behaviors without some externally derived applied quality metric. Many such metrics have been proposed in the literature, but most require training a reward model, heavy auxiliary human or AI labeling of the data, or architectural changes and training changes for the base model. All of these can be difficult to do in practice. Thus there is a substantial need for a lightweight advantage auto-labelling model, as proposed in this paper.

Critical weaknesses

My main concerns are listed below is order of decreasing importance:

  1. The experimental validation draws conclusions from 20 policy rollouts. It should difference with vanilla BC of 1-4 successes, mainly on one task (folding pants). This too small a sample size to draw conclusions about the real effectiveness of the method. After collecting 450 training demos, why not at least roll out the policy 100 times? Policy rollouts are cheap and easy.

  2. Despite the low volume of evaluation data, the performance gap is small compared to the simplest alternative (vanilla BC).

  3. The method seems very similar to UR-VC. The motivation and high-level technique of these two methods seem identical. Table 8 and 9 seems to show UR-VC out performing CRAVE, but puts UR-VC in a different category "UR-VC is a separate source-native D450 diagnostic, not a tenth matched arm" (which I do not understand).It would be important to compare against UR-VC as a baseline and rigorously document a gap methodologically and empirically between the two methods. At line 121 the text "The distinction is therefore framewise correction of a time proxy versus whole-episode induction of a latent progress coordinate. Because UR-VC also changes the encoder, retrieval rule and binary interface, we treat it as a source-native diagnostic rather than a label-source-matched comparator." is used to distinguish between the two methods. Unfortunately I do not understand this phrase, as it seems obscured by jargon.

  4. The biggest potential impact of a method like this is a step in a self-improvement data flywheel, where the method labels advantage of a policies own rollouts, and retrains the policy to get better. This potential avenue for impact was not explored or, I think, mentioned in the paper. Small data single task fine tuning on its own seem to be a mostly solved problem for VLAs.

Questions for authors
  1. Can you perform 100 policy rollouts rather than 20 to give greater confidence in the results? I'm not asking for training a new policy, just roll out the existing ones more.

  2. Can you do this on more that one task (pants folding)?

  3. Can you show statistically significant improvement over vanilla BC?

  4. Can you rigorously explain the difference with UR-VC, in clear jargon free terms, and can you document performance relative to the method?

  5. Can you address the potential for using CRAVE within a self-improvement fly wheel? Many of the latest reward model papers are focusing on this bigger application rather than single stage, single task SFT for VLAs.

Reviewer_ppYTOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

CRAVE derives a progress signal from repeated demonstrations without manual stage annotations or an additional neural value model. It combines visual and robot-state features, identifies recurrent phase prototypes, and decodes a globally consistent phase path using dynamic programming. Progress differences are converted into ordinal conditioning labels for policy post-training.

Critical strengths
  1. In tasks that are costly to label manually and often experience temporary setbacks during remote manipulation, restoring progress based on successful demonstrations is of practical value.
  2. Instead of using a value-reward model, restoring a more reasonable task progress based on similar states that recur in repeated robot demonstrations seems reasonable and is also of practical value.
Critical weaknesses
  1. The use of visual features and state prototypes to structure demonstration data shares similarities with prior work [1].
  2. The progress proxy still relies on temporal ordering and successful endpoints. Similar recurrent states may correspond to different stages of task completion, while necessary holding actions or repeated contacts may produce little apparent progress. The paper should discuss how these ambiguities affect the inferred progress and the resulting training labels.
  3. In the garment-folding task (Table 1), CRAVE achieves 11/20, 15/20, and 16/20 successes with 150, 300, and 450 demonstrations, respectively, compared with 10/20, 14/20, and 16/20 for the strongest baseline at each scale. All reported pairwise difference confidence intervals include zero (Appendix A.7, Table 10). In the writing task (Appendix A.10, Table 12), SFT achieves 18/20 successes, compared with CRAVE’s 16/19. These results provide limited evidence of a reliable improvement in downstream task success.
  4. The D450-CRAVE result is sensitive to the inclusion of a small set of trials. Appendix A.5 transparently reports that replacing five original green-cloth trials changes the result from 14/20 to 16/20 successes and reduces the mean observed rollout duration from approximately 100.5 to 75.4 seconds. Although the paper reports this sensitivity, the reason for replacement and whether the inclusion criteria were established independently of the observed outcomes remain unclear.

[1] SIEVE: Structure-Aware Data Selection for Imitation Learning with VLA Models, arXiv:2607.06442.

Questions for authors
  1. Why were the five green-cloth trials replaced, and were the replacement criteria established before observing the outcomes? Do the conclusions hold when all valid executed trials are included or analyzed separately by collection batch?
  2. What might explain CRAVE’s lower observed success rate than SFT on the writing task? Under what conditions do repeated or stationary contact phases make progress-difference conditioning unreliable?
  3. Under what conditions does the proposed method produce noisy or misleading progress labels, and how do these errors affect policy behavior? Characterizing its failure modes and applicability limits would strengthen its practical value.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.