acceptodds
Under review as a conference paper at ICLR 2027

Vision-Language Reward Models Confuse Near-Misses with Successes in Robot Manipulation

Abstract

Vision–language models (VLMs) are increasingly used as reward models for robot manipulation, where they filter and rerank a policy's rollouts. They are typically evaluated on how well they separate successful rollouts from failed ones. Most failures in these evaluations are obvious, such as a dropped object, yet reward models are now often applied to policies that are already competent, where nearly every rollout ends close to success. To measure this setting, we find for each task in simulation how far the object can be moved before the task no longer counts as complete, and we create failures just past that limit (near-misses) and far beyond it (gross failures). We test nine judges on four LIBERO suites. Every judge is worse at telling successes from near-misses than from gross failures. This also holds for failures that a robot policy (OpenVLA) makes on its own, not only for the failures we create. When a judge picks the best of three rollouts, near-misses erase on average 24–62% of its advantage over picking at random, depending on the suite. We also create rollouts in which the object is moved but the task still succeeds. Judges score these lower than untouched successes, which suggests that they track how far the object moved rather than whether the task was done. Across the judges we test, near-miss accuracy varies more from task to task than from judge to judge, so a benchmark's score depends heavily on which tasks it includes. Fine-tuning on our near-misses helps on new tasks from the suites it was trained on but not on an unseen suite. We recommend that reward-model benchmarks report near-miss accuracy separately and show how it varies with task difficulty.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language models, reward models, robot manipulation, near-miss failures, evaluation methodology, best-of-N selection

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_CQ6yOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors study the question of whether VLM reward models can tell robot maniulation successes from failures in near-miss situations. To this end, they introduce a boundary bisection, which uses LIBERO's success predicates to identify the smallest "push" that makes each demonstration fail. Replaying at multiple different pushes produces a simulator label of a near-miss, bad failures, and perturbed successes. The authors use 9 different judge configurations and four suites, finding that every judge does worse on near-misses. The paper also reports that judges similarly penalise perturbed successes, which suggests they track displacement rather than task completion. An important finding is that near-miss accuracy depends largely on the task than on the judge. These findings are validated on OpenVLA-7B's own failures in MetaWorld.

Critical strengths
  1. I think the basic premise and the idea are well motivated. Connecting the near-misses to the

  2. The paper is well-written and easy to follow.

  3. The experimental design is overall sound. I find the perturbed-success control a particularly good design choice.

  4. The result on the pre-trained encoders matching purpose-trained reward models on the harder tasks is quite interesting.

Critical weaknesses
  1. After some thought, I am not sure the displacement finding in Section 6 could have been any different, since the success condition is latent. So unless the prompt explicitly states this threshold, the judge has no way of figuring out where it lies. This is particularly relevant to the graded prompts, where ranking 0.4cm above 2.3cm seems reasonable. Could the authors share the exact prompts used?

  2. With respect to the visual characteristics of the near-misses: I think without a perceptual reference, it may be difficult to judge whether the near-miss gap is larger than the images allow us to see. For example, any scorer that grows with visual change will find near-misses harder. Could the authors clarify on whether the cm offsets are perceptible at the resolution used?

  3. The authors claim that measuring failures in units of each task's boundary makes a near-miss "mean the same thing on every task". However, being equally fast past the limit does not make a failure equally visible. I think Section 6.5 seems to indicate this too: task identity explains 41% of near-miss variance vs. 6% for judge identity. Could the authors please reconcile this with their original framing?

  4. The authors may also wish to consider citing / discussing relevant work on papers attempting to improve the known inaccuracies that come with using VLMs for success / progress evaluation. Some pointers: (1) Fu et al. (2024), "FuRL"; (2) Zakharov et al. (2025) "GoalLadder"; or (3) Huang et al. (2024), “The Dark Side of Rich Rewards”.

Questions for authors

See "Weaknesses".

4 comments
AuthorsOpenReview ↗

Response to reviewer CQ6y (Part 1/2)

Thank you for the thoughtful review. We looked into your first three points and have revised the paper accordingly. Judges miss failures that people can plainly see in the same frames, with it being the most obvious on Object. On Spatial, the near-misses sit close to the limit of what these images show, for both people and judges. The general response covers two label corrections and the human-study setup (all numbers below use the corrected labels).

W1. Could the displacement finding have come out any other way? The threshold is latent, so the judge has no way to know where it is. Please share the exact prompts.

We agree with the argument. If a judge can't know where the threshold is, the sensible thing for it to do is give a lower score the further the object ends up from the goal. So seeing it mark down a pushed success doesn't tell us that it is ignoring completion. We've dropped the "not completion" claim and retitled Section 6 (see the general response). The exact prompts are in Appendix K of the revised PDF.

To test your point further, we did the following:

The first was whether telling the judge the threshold fixes the problem. We added one sentence to the prompt giving the tolerance in centimeters (3 to 7 cm depending on the task) and re-scored four local VLM configurations, GPT-5.5, and Claude. We found that it makes almost no difference since over the 23 judge-suite cells, near-miss AUROC moved by between -.11 and +.04. It got significantly better in 4 cells and worse in 4. Our theory is that the judges can't make use of the number because a 3 cm difference is too small to see in these images.

The second was whether a judge's score drops sharply right where the task goes from success to failure, which is what we'd expect if it did know the threshold. On Spatial only 1 of 10 functional judges shows a significant drop at 3 cm, and we can rule out drops bigger than roughly 25-40% of the judge's total fall from success to gross failure. On Object, 6 of 12 judges do show a drop at the boundary, but the object also ends up much further away at that same point, so we can't say which of the two the judge is reacting to.

We also hoped the human raters would tell us what a reasonable score looks like here, but they don't. Given the same frames, both people and core judges can barely tell untouched successes from pushed ones (general response). So we think this is still an open question.

W2. Without a perceptual reference it is hard to know if the near-miss gap is bigger than the images allow. Are the cm offsets perceptible at this resolution?

It depends a lot on the suite. On Object the offsets are easy to see, on Goal you can see most of them, and on Spatial they are right at the edge of what the image shows. We also found that how visible a failure is explains about half or more of the gap on Spatial and Goal.

We didn't have a perceptual reference, so we ran a blinded human study to get one (the setup and the comparison with judges are in the general response).

At 256 pxSpatialObjectGoal
Near failure against the same instance's 0.75x success2.9 px (1.2 cm)22.2 px (13.0 cm)4.7 px (2.2 cm)
Near failures under 5 px from that success81%2%53%
Near-misses both raters called failures16%96%71%
Human near AUROC.85.99.98
Core judges, same clips.66.64.66

To put the Spatial numbers in context, one Qwen3-VL vision token covers 32 px at this input size, so a typical Spatial near-miss differs from its success by about a tenth of a token. People have trouble here too since they call only 16% of the Spatial near-misses failures, although they do still rank them well below the successes.

AuthorsOpenReview ↗

Response to reviewer CQ6y (Part 2/2)

W2, continued.

You also pointed out that any scorer whose output grows with visual change will find near-misses harder. We checked this and it does explain a lot of the gap. To test it, we only compared near and gross failures from the same task whose images changed by a similar amount.

Mean near/gross gapSpatialObjectGoal
All failures.194.198.112
Same task, similar object pixel shift.079 [.048, .109].137 [.117, .168].025 [-.032, .094]
Same task, similar image change.087 [.052, .113].116 [.057, .159].050 [.010, .088]

We're less sure about the Object number because it moves between .030 and .144 depending on where we put the bins. On Goal, once pixel shift is matched, the gap turns negative for 5 of 11 judges, including all five core VLMs.

We also looked at whether people have a near/gross gap themselves. They do, and how it compares with the judges' gap changes from suite to suite.

Near/gross gapSpatialObjectGoal
People.15.01.02
Core judges.16.21.08
People minus judges.00
−.09,+.08-.09, +.08

| -.20 [-.27, -.14] | -.06

−.12,+.01-.12, +.01

|

On Object people have almost no gap while the judges have a big one, so it is clearly a judge problem there. On Goal it probably is as well, but the interval just reaches zero. On Spatial people struggle about as much as the judges, so the gap by itself doesn't show that the judges are doing anything wrong. Because of this, the revision now leads with the direct comparison of people and judges on near-misses.

Finally, we tried giving the judges a better view. For the VLM judges on identical rollouts, re-rendering at 512 px actually makes the gap a bit wider (by .03, .02 and .05). A wrist camera narrows it by .01, .09 and .05, only really helping on Object. For human raters, with 512-pixel video and the tolerance stated, their near AUROC is .86, .99 and .94, so it doesn't really help people either.

W3. The paper says measuring failures in units of each task's boundary makes a near-miss "mean the same thing on every task". But equally far past the limit isn't equally visible, and Section 6.5 seems to show that. Can you reconcile these?

The multiple puts every failure the same distance past its own task's limit, measured in units of the push. We agree that this isn't the same as making failures equally easy to see, or equally close to the goal, and we revise our wording for this.

SpatialObjectGoal
Median largest object shift at 1.0x, range over tasks5.2-13.1 px17.2-25.9 px6.9-43.6 px
Near failures, median distance from the target3.7 cm10.9 cm4.1 cm (two tasks with a target)

So the way we'd reconcile the two is that the multiple only tells you how far past the limit a failure is. How hard that failure is to see still depends on the task, which is what Section 6.5 picks up. Over 27 tasks, the task explains 43% of the variance in near AUROC and the judge explains 4%. Although inside a single suite the two can be much closer.

We think visibility is the most likely reason the task matters so much, but we weren't able to show it. A task’s mean near AUROC correlates with how many pixels separate its near failures from its 0.75x success, but not significantly after Holm correction.

W4. Related work on FuRL, GoalLadder and "The Dark Side of Rich Rewards".

Thank you for the suggested works, and we have included all three in the related work. We found the third work on false-positive rewards to be the most relevant as a near-miss scored like a success is exactly the kind of false positive mentioned.

We thank you again for the review. The points on the latent threshold and on visibility made the paper clearer, and we’re happy to answer anything else during the discussion.

AuthorsOpenReview ↗

Revised manuscript uploaded

Thanks again for the time and effort you've put into reviewing our manuscript.

We'd like to let you know that the manuscript has been updated, with all revisions marked in blue.

If you have any concerns or further suggestions, we'd be glad to discuss them.

Authors of 52295

AuthorsOpenReview ↗

Looking forward to your reply

Dear Reviewer CQ6y,

We sincerely thank you for your efforts in reviewing our work and for your great support! Please let us know if you need any further information or if there are additional points you would like to discuss with us. We would be glad to engage further discussion with you.

Thank you very much!

Best regards,

Authors of #52295

Reviewer_gzbzOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether vision–language reward models can distinguish successful robot manipulation trajectories from near-misses, rather than merely from obvious failures. It introduces an instance-specific boundary-bisection procedure: demonstration actions are perturbed, and simulator success predicates identify a perturbation magnitude at which the task fails. Multiples of this estimated boundary generate geometric near-misses, gross failures, and perturbed trajectories that remain successful. Structural failures are additionally constructed by stopping multi-stage tasks before all goal conditions are satisfied.

The evaluation covers nine judge configurations spanning published reward models, general-purpose VLMs, and pretrained image encoders, with different coverage across datasets. Every judge passing the gross-failure functionality screen performs worse on near-misses than on gross failures at the reported suite level. Best-of-3 experiments translate this gap into selection performance: near-misses remove 24–62% of the suite-level average advantage over random selection. Experiments with OpenVLA-generated failures provide supporting evidence beyond constructed perturbations, while a Meta-World replication primarily supports the encoder results because the tested VLM judges barely discriminate gross failures there.

The paper further examines score penalties on perturbed successes, task-versus-judge variation, and LoRA fine-tuning on generated failures. Its main contribution is a controlled evaluation methodology that exposes weaknesses hidden by aggregate success-versus-failure discrimination.

Critical strengths
  1. The boundary-based intervention produces failures of different severities while retaining simulator-verified labels. Keeping perturbed trajectories that still succeed is particularly valuable: it allows the study to test whether scores track visual changes even when completion remains unchanged. Structural failures and policy-generated failures further reduce reliance on a single synthetic failure family. This design makes the paper more consequential than another aggregate success-detection benchmark.

  2. The best-of-3 experiments make the discrimination gap operationally meaningful rather than leaving it as an AUROC observation. Exact evaluation over distractor subsets, instance-level bootstrapping, pretrained-encoder baselines, and explicit reporting of failed hypotheses strengthen the empirical contribution. The appendix’s documentation of measurement errors and corrections is also a substantive reproducibility strength, although I have not independently verified the claimed repository artifacts.

Critical weaknesses
  1. How often does success become non-monotonic as perturbation magnitude increases? What initial search bracket and perturbation directions are used? On a representative subset, does a denser sweep confirm that six-step bisection identifies the first failure transition and preserves near/gross assignments?

  2. How strongly do boundary multiples correlate with task-relevant state-space violation after accounting for task identity? Does the main near-versus-gross result persist across alternative perturbation directions or timings?

  3. Can you separate displacement-sensitive confidence from failure to recognize completion? In particular, what completion-sensitive effect sizes remain compatible with the matched-distance confidence intervals? Would you revise the Section 6 heading to reflect the limits of the evidence?

  4. Does the suite-level gap remain when all evaluated judge configurations are included, or when the gross-failure screen is determined on separate demonstration instances? How sensitive are conclusions to the 0.55 threshold?

  5. Can you report both instance-paired ranking accuracy and conventional pooled AUROC for all judges using the same scored trajectories? Do the main comparisons and model rankings change under a common estimator?

  6. How variable are the fine-tuning gains across training seeds and alternative held-out-suite choices? Also, does the best-of-3 degradation persist when candidate pools are sampled directly from policy rollouts rather than constructed with exactly one success and two failures?

Questions for authors

See the weakness.

7 comments
AuthorsOpenReview ↗

Response to reviewer gzbz (Part 1/3)

Thank you for these questions and the thorough review. We address each question below:

W1. How often is success non-monotonic in the push? What bracket and directions? Does a denser sweep confirm the bisection?

It is monotonic for most instances. When the scene is held fixed, 79% of the instances we swept never succeed again after their first failure. The near and gross bands mostly hold up on Spatial and Object, with Goal being weaker.

Our original submission described the push and the six-step search but left out the details needed here. The push adds a fixed +x offset to the controller's normalized position command. It ramps up linearly from zero at the middle of the episode to full size at the end. We search for the boundary inside the bracket [0.05, 1.2] in those units, and we throw out an instance if the largest push in the bracket doesn't make it fail. After six halvings, the bracket is 0.018 wide, and we use the smallest failing push as the boundary.

For the dense sweep, we took the first five demonstrations of every task on the three single-stage suites. This gives us 111 instances that can be swept out of 150 total. We stepped the push by 0.005 up to 1.5x the boundary while keeping the scene fixed.

SpatialObjectGoal
Instances swept424425
No success after the first failure79%89%64%
First failure within 10% of the pool boundary49%63%32%
Rendered near failures still in the near band77%97%51%
Rendered gross failures still in the gross band99%85%100%
Mean gap, all swept instances.226.201.088
Mean gap, monotone instances only.236.210.090

We can see that Goal is clearly the weakest suite, which we mention in the revised PDF. 19 of the instances still succeed at the largest push in the bracket, with only 32% of its first failures landing within 10% of the boundary we used for the pools. The non-monotone instances are not what creates the gap, since the gap is about the same when we keep only the monotone ones. The appendix has the versions with the scene redrawn and with the any-step label.

W2. Do boundary multiples track task-relevant state violation once task identity is accounted for? Does the result hold for other directions and timings?

Within a task, yes. A larger multiple goes with a larger distance from the goal. For other directions and timings, the gap stays positive but its size changes, and with the late ramp on Goal it almost disappears.

Within-task rank correlation of multiple with object-to-goal distanceSpatial (10 tasks)Object (10 tasks)Goal (2 tasks with a target)
All pushed rollouts.889 [.819, .935].864 [.807, .907].923 [.794, .983]
Failures only.762 [.629, .854].665 [.526, .787].782 [.536, .957]
Mean near/gross gap (baseline on the same instances)SpatialObjectGoal
Push +y.269 (.189).153 (.198).119 (.093)
Push -x.311 (.190).150 (.198).118 (.117)
Ramp from 25% of the episode.209 (.191).211 (.198).070 (.088)
Ramp from 75% of the episode.193 (.227).120 (.197).012 (.019)

For each variant we re-ran the bisection on up to five demonstrations per task. In 9 of the 12 cells every functional judge has a positive gap. The other three are the -x push on Object (7 of 8 judges), the early ramp on Goal (7 of 8) and the late ramp on Goal (3 of 8). In that last cell the gap almost disappears, though it has only 8 instances.

AuthorsOpenReview ↗

Response to reviewer gzbz (Part 2/3)

W3. Can you separate displacement-sensitive confidence from failure to recognize completion? What effect sizes do the matched-distance intervals allow? Would you revise the Section 6 heading?

We can't separate them in general, more on this below. Furthermore, based on our experiments, we have revised the Section 6 heading to "Judge Scores Fall with Displacement", with the section saying that a step at task completion is either small or can't be identified.

We could no longer run the matched-distance test after correcting the labels. With the corrected labels, whether the object ends within 3 cm decides the Spatial label in 98.6-100% of rollouts per task, so successes and failures almost never sit at the same distance. No single task has enough of both in the distance range they share. Most of the overlap we had before came from the any-step label.

So a different question we ask is whether a judge's score jumps at the threshold. On Spatial we fit each judge's score within an instance, with one distance slope below 3 cm and another above it, using only rollouts that end close to 3 cm.

Bandwidth around 3 cmInstancesJudges with a significant step (of 10)Median upper bound, share of the judge's success-to-gross dropLargest upper bound
1 cm124024%40%
1.5 cm140117%31%
2 cm148110%25%

This means that on Spatial we can rule out a completion step bigger than about 25-40% of a judge's gross drop, but a smaller one is still possible.

On Object we can't separate the two, because distance and completion change at the same point (95% of successes end within 4.1 cm and 95% of failures end beyond 6.9 cm). Six of the 12 functional judges step at the boundary there and on Goal no judge has a significant step.

The penalty on pushed successes is still there after relabelling, but narrower than what we claimed. For the five core VLMs it is significant in 8 of 13 judge-suite pairs and in none on Object. For GPT-5.5 and Claude it is significant in 1 of 12. Human raters also separate untouched from pushed successes barely better than chance (general response), so they can't tell us whether a penalty is reasonable.

W4. Does the gap remain with all judge configurations included, or with the screen set on separate instances? How sensitive is it to the 0.55 threshold?

Yes and yes. The gap stays positive at every threshold we tried.

Mean near/gross gapAll configurationsScreen at .55Screen set on the other halfThresholds .50-.80
Spatial.141 [.125, .158].181.175.154-.212
Object.182 [.160, .206].196.196.182-.196
Goal.098 [.073, .123].108.103.098-.158
Long, geometric.106 [.091, .123].106.106.106-.219
Long, structural.081 [.062, .100].111.109.101–.223

We also split the instances in half at random 200 times, letting each half set the screen for the other, with mean gap being positive every time. Of the 60 configuration-suite cells, 57 have a positive gap and 52 have an interval above zero.

AuthorsOpenReview ↗

Response to reviewer gzbz (Part 3/3)

W5. Report instance-paired ranking accuracy and conventional pooled AUROC on the same trajectories. Do comparisons and rankings change?

The main comparisons hold under all three estimators. The ranking of judges holds on Spatial and Object, but not on Goal or Long geometric.

Functional judgesSpatialObjectGoalLong, geometric*Long, structural*
Mean gap, submission's pooled AUROC.181.196.108.106.111
Mean gap, conventional pooled AUROC.178.197.111.102.113
Mean gap, instance-paired ranking accuracy.182.252.108.161.140
Paired interval above zero11/1113/138/128/97/7
Near-miss rank correlation, submission vs paired.99.99.58.43.82

The difference between the first two is that our estimator puts each failure's own success into the pool, so a success can be counted more than once, while the conventional one counts each success once. Values move by no more than .03 between them, and every functional judge keeps a positive gap under all three. What does change is which judge is best on near-misses on Goal and on Long geometric, so the revision no longer ranks judges there from a single estimator. Per-judge values for all three in the appendix.

W6, first part. How variable are the fine-tuning gains across seeds and held-out suites?

The gains on the trained suites are consistent across seeds. Transfer to a suite the model never saw is rather small and not reliable.

Unseen suiteTraining labelsNear gain, held-out tasks of trained suitesNear gain, unseen suiteSeeds with an unseen-suite interval above zero
Object (submission)Binary+.174 ± .023+.106 ± .0273 of 3
Object (submission)Calibrated+.189 ± .021+.082 ± .0312 of 3
Goal (alternative)Binary+.286 ± .015−.054 ± .0300 of 3
Goal (alternative)Calibrated+.277 ± .007−.047 ± .0250 of 3

Values are the mean ± SD over three training seeds. Every seed has a held-out gain with an interval above zero. However on the unseen suite, a single seed's gain can be anywhere from -.099 to +.148, which is why we now say fine-tuning "does not transfer reliably".

W6, second part. Does the best-of-3 degradation persist when pools come straight from policy rollouts?

Yes on average, but the intervals are wide. We took every triple of OpenVLA's own rollouts within a task (360 rollouts, with 226 successes, 25 near failures and 63 gross failures) and compared pools from the same task with the same number of successes.

Pools from policy rollouts, eight functional judges
Gain over random when all failures are gross+.249
+.186,+.339+.186, +.339

| | Gain over random when all failures are near | +.130

+.022,+.227+.022, +.227

| | Drop | .119 [.021, .260], 48% of the advantage | | Judges individually significant | 2 of 8 |

During this process, we found and corrected two errors in our pipeline (see the general response).

The near/gross gap stays positive under what was asked for, though the size varies and individual judges sometimes cross zero. We take back the claim "displacement, not completion". Numbers use all tasks and corrected labels, except the LIBERO-Long rows, which use the submission’s labels.

Thanks again for the review. We were able to spot both corrections from your questions which made the paper more precise, and we will revise the PDF accordingly.

Reviewer_6T2iOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors talk about an interesting problem with VLM reward models and explain their findings about successful robot manipulation vs. near-misses. They introduce a failure boundary and provide performance measurements on how much worse judges are on near misses. Experiments are done on LIBERO suites, OpenVLA, and Meta-World.

Critical strengths
  1. Problem definition and intro are clear, and the motivation is easy to understand.
  2. Authors provide a very detailed supplementary with the result logs.
  3. Boundary bisection makes close to success a controlled variable.
  4. Authors also provide recommendations for reward-model benchmarks, which I think is a great touch.
Critical weaknesses
  1. Task coverage is pretty narrow. For example:
  • Libero-Spatial only uses 3 of 10 tasks.
  • Libero-object also uses just 5 of 10 tasks. Selection criteria aren’t explained. Few instances in LIBERO-Goal. Everything is in simulation, and there is only one 256x256 third-person camera. Also, the fine-tuning transfer claim is based on one suite.
  1. Near-miss labels and the displacement, not completion are not validated against what the task means. For example: No human labels. In Figure 1, 1.15b failure shows a bowl resting on the plate, like the 0.75b success. The gripper is still above the bowl in both frames. Without human labels, it is not clear which near-misses are visible failures or what a human ceiling could be.

Also, the prompts ask how completely the task was accomplished. on a 1-5 scale. But it doesn’t give numeric tolerance, i.e., 3 cm, etc. Without tolerance, judges should lower their success probability as the object nears the boundary.

Questions for authors
  1. How many seeds were each experiment run on? Kindly provide the seed numbers for all the experiments.
  2. Weakness 1
  3. Weakness 2.
  4. Four of the five VLM configurations are built on Qwen VL 8B. Is the low judge variance (6%) partly because most judges share the same base model? Add at least two judges from other model families.
  5. Why only 3 Spatial and 5 Object tasks?
6 comments
AuthorsOpenReview ↗

Response to reviewer 6T2i (Part 1/2)

We sincerely thank you for the time and effort spent reviewing our paper. Both of your main concerns are fair, and we ran new experiments for each. In short, people can see most of these near-misses where the judges can't, and the gap holds on all ten tasks of every suite. The human-study setup is in the general response, and the numbers below use the updated labels described there.

W2. No human labels. In Figure 1 the 1.15b failure resembles the 0.75b success. What is the human ceiling?

We ran a blinded human study. On Object, raters looking at the same eight frames as the judges get a near-miss AUROC of .99, and the core judges get .64. A near-miss there usually means the object missed the basket. That's obvious to a person, but the judges still score it like a success. Goal looks the same (.98 against .66). Raters beat every judge on both suites.

Spatial is different, and the Figure 1 example you pointed to comes from that suite. In the 1.15b frame the bowl is still on the plate, a little over 3 cm from the center. The simulator counts that as a failure, but most of our raters called clips like it a success.

Spatial8 frames, 256 px512 px video, tolerance stated
Failures at 1.15x that raters called successes75%38%
Near-misses both raters accepted, of 25177

Raters do notice the difference. They're less sure about Spatial near-misses than about clean successes (AUROC .85, against .66 for the core judges), but most of the time they still answer yes. For these clips the simulator's 3 cm rule is stricter than what a person means by "on the plate". In the updated PDF we've replaced the Figure 1 example with an Object instance that people do call a failure. We now describe Spatial near-misses as threshold cases, and we've added the clips people accepted to the appendix.

On the human ceiling, with 512-pixel video and the tolerance stated, raters' near-miss AUROC is .86, .99 and .94. That's about the same as at the judges' input. On Spatial they call more near-misses failures (40%), though the video view always came second, so some of that may be practice.

W2, continued. The prompt gives no numeric tolerance, so a judge should lower its score as the object nears the boundary.

We agree, and we've updated Section 6 to reflect this (details in the general response).

We also tried stating each task's tolerance in the prompt. We ran this on four local VLM configurations plus GPT-5.5 and Claude Opus 5.5, and it hardly helps. Near-miss AUROC changes by −.11 to +.04, and as many judge–suite cells get worse as get better.

A judge that knows the threshold should also drop its score right at 3 cm. On Spatial, only one of the 10 judges we tested does (Qwen3-VL-8B), and its step is small. People show no clear step at the judges' input either, so we don't hold its absence against the judges.

AuthorsOpenReview ↗

Response to reviewer 6T2i (Part 2/2)

W1 and Q5. Why only 3 Spatial and 5 Object tasks?

It came down to compute. Each subset was chosen before we scored any judge on its suite. We've now run all ten tasks on both suites and doubled Goal to 12 demonstrations per task, and the gap holds on every suite. On Spatial, near-miss distractors now erase 46% of a judge's best-of-3 advantage, up from 24% in the submission (the general response has the breakdown).

Over the 27 tasks, the task explains 43% of the variance in near-miss AUROC and the judge 4%. Within Spatial or Object alone the judge matters about as much as the task, and we've updated the text to say that.

W1, continued. One 256x256 third-person camera.

Resolution doesn't explain the gap. We re-rendered the same rollouts at 512 px and from the wrist camera.

Mean near/gross gap (256 px baseline on the same instances)SpatialObjectGoal
512 px third-person.22 (.19).21 (.18).12 (.09)
Wrist camera.13 (.19).08 (.19).05 (.09)

The gap does get smaller from the wrist camera. On Spatial and Goal, though, that's almost entirely down to two judges, the goal-image encoders. They compare each frame with a picture of the goal, and that picture changes when the camera does. For the VLM judges, the wrist camera only clearly narrows the gap on Object.

W1, continued. Everything is in simulation.

All our findings are for simulated scenes. We've updated the paper to state that and to name a real-robot test as the main open question.

W1, continued. The fine-tuning transfer claim rests on one suite.

To check this we added a second split and three seeds per condition. On new tasks from suites the model was trained on, the gain is stable (+.17 to +.29 in near-miss AUROC). On a suite it never saw, the result is mixed, with +.11 and +.08 on Object and −.05 on Goal. The paper now says the gains transfer unreliably, where before it said they don't transfer.

Q4. Four of the five VLM configurations share a Qwen base. Is the low judge variance an artifact of that?

No, we see the same thing outside the Qwen family. We added Claude Opus 5.5 and Gemini 4 Argon and extended GPT-5.5 to Goal and all the new tasks. All three show the gap on every suite. With them included, the task explains 40% of the variance in near-miss AUROC and the judge 5%. Among those three alone it's 72% and 5%.

Q1. How many seeds? Please list them.

The submission used seed 0 for fine-tuning, and we've now run seeds 0, 1 and 2 for each condition. It's the only step where the seed changes the result. The rollouts replay demonstrations with a fixed push (simulator seed 0), and the local judges don't sample. The API judges do. Repeat calls on the binary prompt gave the same answer on 88 of 100 rollouts for Claude, 36 of 39 for GPT-5.5 and 49 of 50 for Gemini. The appendix lists every seed and where it's set in the code.

We hope the human study, the full task coverage and the new model families settle your two main concerns. If they do, we'd be grateful if you'd consider raising your score. If something is still missing, tell us and we'll try to run it during the discussion period.

AuthorsOpenReview ↗

Revised manuscript uploaded

Thanks again for the time and effort you've put into reviewing our manuscript.

We'd like to let you know that the manuscript has been updated, with all revisions marked in blue.

If you have any concerns or further suggestions, we'd be glad to discuss them.

Authors of 52295

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We thank the reviewers and the area chair for their time and careful reviews. A few points came up in more than one review, so we cover them here.

The gap holds on all ten tasks

VLM reward models are much better at telling successes from gross failures than from near-misses, and when a judge picks the best of three rollouts, near-miss distractors erase 30–61% of its advantage. We ran the extra tasks, model families, human labels, perturbations and estimators requested, and the gap is positive on every suite for every functional judge (gross-failure AUROC of at least .55).

All ten tasks per suite, corrected labelsSpatialObjectGoal
Median near/gross gap.168.186.060
Judges with a gap interval above zero6/68/86/8
Best-of-3 advantage erased by near-miss distractors46%61%30%

The near/gross gap is a judge's AUROC against gross failures minus its AUROC against near-misses.

The gap also shows up outside the Qwen family. GPT-5.5, Claude Opus 5.5 and Gemini 4 Argon have it, with an interval above zero in all 18 judge–prompt–suite cells.

Human labels

Reviewers 6T2i and CQ6y asked whether a person can see these near-misses. They can, most clearly on Object and Goal. We ran a blinded study on 71 instances, each with an untouched success, a pushed success (about 0.75x the boundary), a near failure (1.0–1.35x) and a gross failure (2.5x or more). Four outside raters each saw half the instances exactly as the judges do (eight frames at 256 px, the judges' question) and the other half as 512-pixel video with the tolerance stated. Every clip got two ratings in each view.

At the judges' input (8 frames, 256 px)SpatialObjectGoal
Human near-miss AUROC.85.99.98
Core judges, same clips.66.64.66
Humans minus core judges+.19 [.09, .28]+.36 [.29, .43]+.32 [.23, .40]
GPT-5.5 and Claude Opus 5.5, same clips.55.69.68
Gemini 4 Argon, same clips.50.76.65
Near-misses both raters called failures16%96%71%

Core judges are the paper’s seven local judges. The intervals don’t account for differences between raters, so the true uncertainty is wider.

Revised manuscript

Section 6 is now titled "Judge Scores Fall with Displacement". The old title said judges score displacement and ignore completion, and all three reviewers questioned the second half. This is a fair point. A judge that doesn't know the threshold should lower its score as the object moves away from the success boundary, so a penalty on pushed successes can't tell us whether it tracks completion. We checked with people too. From the same frames they can barely tell a pushed success from an untouched one (AUROC .53 to .61, intervals include .5), and the core judges do no better (.53 to .57). The smaller wording changes are in our individual replies.

Running gzbz's checks, we also found two problems on our end and fixed both. We were counting a success if the goal was met at any step, so an object pushed through the goal and left elsewhere still counted. We now read success at the end of the rollout, which moves 53, 3 and 17 rollouts to failure on Spatial, Object and Goal. Section 3 also said each demonstration "replays exactly", but LIBERO's reset re-samples fixture positions on Spatial and Goal, so fixtures shift by up to 2.6 cm. The text now reflects this.

Here's how best-of-3 moved as a result, in case you're comparing against the submission.

Best-of-3 advantage erasedSpatialObjectGoal
As submitted24%62%26%
All ten tasks41%62%24%
All ten tasks, corrected labels46%61%30%

The corrected labels move each suite by a few points, but the paper's main finding holds. Spatial's big jump comes from the extra tasks, since the three we submitted are easier than the other seven.

Our replies use all tasks and corrected labels unless marked. All new results are in the updated PDF.

Best regards, Authors of #52295

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.