- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces GRIP, a model that predicts dense rewards for robotic manipulation from a small set of labeled visual examples. Its main configuration combines a frozen visual encoder with a frozen pretrained regressor, allowing tasks to be specified through context without additional parameter updates. The paper also investigates optional feature adaptation and uses predicted rewards together with visual diversity to select demonstrations for behavior cloning. The evaluations demonstrated improved reward prediction and more effective demonstration selection.
Critical strengths
-
The paper provides a clear and intuitive way to specify what progress means for a task. The connection between task specification and reward prediction is interesting.
-
The framework is simple and makes useful use of existing pretrained models. Keeping the main encoder and regressor frozen allows the model to switch tasks without additional optimization.
Critical weaknesses
- The first and biggest concern is that the method assumes that a task’s dense reward can be inferred from a single image, but from my experience, successful manipulation often depends on small geometric differences and physical conditions that are quite difficult to observe visually. This concern is particularly relevant to the main configuration, which uses a global visual embedding without action history or contact measurements, since additional labeled examples cannot resolve distinctions that the input does not capture.
-Also the connection between dense reward prediction and the downstream application needs stronger justification. Demonstration selection summarizes predicted rewards near the end of a trajectory, so its usefulness may largely come from recognizing proximity to completion.
Questions for authors
In the shared context setting, do CLIP text features enable useful transfer between related tasks, or would replacing them with arbitrary task identifiers produce similar behavior?
What motivated representing trajectory diversity using only the first observed frame, and how would the selection rule distinguish demonstrations that start similarly but follow different manipulation strategies?
Why using only the first frame overlooks differences in manipulation strategies, many demonstration have similar or even identical first frames. They might be treated as redundant despite containing useful info.
5 comments
Q3. Can similar or identical starting frames cause useful demonstrations to be treated as redundant?
The selector combines reward quality with diversity, so similar starting frames do not automatically exclude a demonstration. We also tested whether adding later observations to the diversity representation improves downstream selection. We compared the first frame with two alternatives: a mean over all frame features, and a concatenation of four segment means that retains coarse temporal order. Only the diversity representation changes; GRIP scores, the selection objective, and BC training remain fixed. The table reports BC success (%) at K=20 over three paired seeds.
| Diversity features | MetaWorld MT10 | ManiSkill |
|---|---|---|
| First observed frame | 93.00 ± 5.19 | 34.42 ± 2.65 |
| Whole-trajectory mean | 88.07 ± 3.49 | 32.29 ± 3.82 |
| Four-phase features | 84.47 ± 11.32 | 36.08 ± 1.94 |
Using the whole-trajectory mean lowers success on both benchmarks. Four-phase features raise the ManiSkill mean from 34.42% to 36.08%, while the first frame remains best on MetaWorld. The first frame therefore provides a useful, simple coverage measure, and phase-aware diversity is a promising extension for retaining differences later in the trajectory.
Conclusion
GRIP turns a few labeled observations into a dense visual reward for a new task, without fitting a task-specific model. The same predictor can then score unlabeled demonstrations for policy training.
[1] Ma, Yecheng Jason, et al. "Vision language models are in-context value learners." ICLR. 2025.
Thank the authors for the detailed clarification!
Aopologies, this is a bit late but could author provide real-robot expirement results? Does the model maintain it's capability in real lifte setting? does the real life expierment show the generalization aspect of the model?
Response to Reviewer jpBH
Thank you for your careful review and questions about visual reward prediction and demonstration selection. We respond to each point below.
W1. Can a single image capture the geometry and physical conditions needed to infer dense rewards?
A single frame often shows how far a manipulation task has progressed, and labeled examples tell GRIP how to score those visible changes. Estimating progress from visual observations is also the basis of GVL [1], which uses Gemini-1.5-Pro to predict per-frame completion from a set of shuffled frames.
In Table 2, with the same frozen CLIP features and 100 labeled frames, GRIP achieves Spearman correlations of 0.6811 on MetaWorld and 0.8276 on ManiSkill. The strongest fitted regressors reach 0.6187 (kNN) and 0.8135 (Ridge), respectively. These results establish the usefulness of single-frame reward prediction on these benchmarks.
We also tested whether a short history adds useful information. Averaging the current and two preceding observation features improves Spearman from 0.7507 to 0.7582 on MetaWorld and from 0.8455 to 0.8574 on ManiSkill, while keeping the feature dimension unchanged. These paired evaluations use the same 100 context labels, three seeds, and up to 10 uniformly sampled query frames per trajectory. Single-frame inputs already provide useful progress estimates, and recent observations improve them further. Contact conditions that are absent from the visual input remain a limitation.
W2. Does demonstration selection validate dense rewards, or mainly recognition of proximity to completion?
GRIP is designed to predict reward at each observation, giving downstream users a progress curve from which to define selection criteria. Our current BC application is an initial use of these predictions. We score the final portion of each trajectory with a low reward quantile to favor sustained completion and reduce the influence of isolated high predictions. At K=20 on MetaWorld MT10, adding GRIP scores to visual diversity raises BC success from 36.8% to 94.6% (Table 4).
Dense predictions also make other selection criteria possible. For example, future selectors could require sufficient completion and then favor smoother progress, penalize substantial reward drops, or assess progress across different phases of the task. These are directions for further exploration. The same predicted reward curve lets users choose criteria suited to their downstream objective without retraining the reward model.
Q1. Do CLIP text features enable transfer between related tasks, or would arbitrary task IDs work similarly?
Arbitrary task IDs can support shared-context prediction. TabPFN receives numerical task features, and an ID identifies which task each example belongs to. We replaced CLIP text features with one-hot IDs for all 50 MetaWorld tasks, using the same 512-dimensional input width, 100 labeled context frames per task, and three seeds.
We also tested random normalized 512-dimensional IDs and shuffled the assignment of text embeddings to tasks. In this additional evaluation using 10 query frames per trajectory, Spearman is 0.7455 for CLIP text, 0.7422 for random dense IDs, and 0.7394 for shuffled text. Arbitrary dense IDs thus recover nearly the same prediction accuracy. Task identity is sufficient for effective shared-context prediction in this setting, with CLIP text providing a convenient encoding.
Q2. Why use the first observed frame for diversity, and how does selection distinguish different strategies from similar starts?
The first observed frame helps a fixed demonstration budget cover different initial conditions. Selecting by reward alone can retain many high-quality demonstrations from similar starting scenes. First-frame diversity reduces that redundancy, while the GRIP score favors demonstrations that reach higher completion.
Demonstrations with similar starts can still receive different GRIP quality scores, but the first-frame diversity term does not distinguish their manipulation strategies. We will describe its role as initial-condition coverage.