acceptodds
Withdrawn from ICLR 2027

GroundingWAM: Learning Where to Focus in World Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

World Action Models (WAMs) achieve high success rates on manipulation benchmarks, yet can struggle when execution departs from familiar demonstration trajectories. Following an unexpected object drop, a policy may fail to relocalize the target and resume manipulation. We probe this grounding weakness through auxiliary localization supervision: the predicted target object location can remain near the gripper even after the object has moved elsewhere. We interpret this behavior as a consequence of the strong correlations between object positions, robot states, and task progress in successful demonstrations, which allow models trained with end-to-end regression objectives to exploit predictable patterns without consistently grounding their predictions in current visual evidence. Motivated by this observation, we introduce GroundingWAM, which uses task-relevant spatial and language annotations to directly supervise internal attention during action generation, guiding action queries toward relevant image regions and instruction tokens. Trained on successful demonstrations without additional recovery demonstrations, GroundingWAM achieves 99.3% success on standard LIBERO and 87.1% on LIBERO-Plus benchmark. On our object-release setting, it improves success from 47.79% to 64.71%, with a gain of 41.75 percentage points on LIBERO-Spatial. Component comparisons and rollout visualizations support internal grounding supervision as a practical way to strengthen manipulation beyond familiar execution patterns.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Model, Manipulation, Robotics, Visual Grounding

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DUcTOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

WAM with high LIBERO success often fail to recover after a grasped object is forcibly dropped, and that an auxiliary box-regression head keeps predicting the object near the gripper after the drop. It proposes Grounding Attention Distillation, a thresholded loss that forces action queries in selected heads to put a minimum attention mass on simulator-derived target patches and instruction tokens, and reports gains on LIBERO, LIBERO-Plus, and a custom object-release benchmark.

Critical strengths

Asking whether high nominal success hides poor recovery is a useful question. The object-release probe cleanly separates the object–gripper relationship from task identity.

The release protocol is specified in unusual detail, including trigger geometry, drop and regrasp criteria, denominators, and paired initial states and seeds.

The method adds no inference-time cost or dependencies. Annotations are used only during training, and the deployed policy keeps the standard observation-to-action interface.

Critical weaknesses

Author need to test on a real-world machine to test the performance.

It have shortcut learning problem. Both signals come from the same simulator boxes on the same correlated demonstration distribution, so nothing principled prevents learned attention from collapsing toward the gripper after a drop, just as the box head does. The most direct test of the hypothesis is missing: the full model has a box head, yet its post-drop IoU is never reported, and attention maps are shown for only two hand-picked episodes.

The evidence that GAD improves “grounding” in general is weak. The large gain appears only on Spatial (48.5 to 90.3), while Object improves just 39.5 to 44.0, which points to a suite-specific effect rather than better relocalization. On LIBERO-Plus the full-benchmark gain over GAM is 1.6 points, Camera is worse than ImageWAM, and the ablation differences (87.45 vs 88.80) come from single-seed, single-rollout runs with no confidence intervals. Baseline numbers also mix published results with reproduced ones under each model’s own inference settings.

Supervising attention toward annotated evidence is well established (“Right for the Right Reasons”, GuidedVLA, OREO), and the paper adds a hinge threshold applied to arbitrarily chosen layers and heads (5×6, τ = 0.4/0.1) with no sensitivity analysis.

Questions for authors

See Weakness

2 comments
AuthorsOpenReview ↗

Response to Reviewer DUcT (1)

A1 (Weakness 1): Real-Robot Experiments

Thank you for suggesting real-robot experiments. We conducted experiments using a Franka R3 arm with two cameras and demonstrations of a two-object pick-and-place setting. Grounding DINO and SAM 2.1 provided front-camera boxes and masks. We screened the target-object and basket annotations and trained both policies on the same retained demonstrations. The deployed policies required no detector.

ImageWAM and GroundingWAM were trained for 5,000 steps from the same pretrained FLUX weights. After the initial grasp and a further 48 control steps (2.4 seconds), we paused inference, removed and relocated the target object, and resumed the policy. Success required regrasping the relocated target and placing it in the basket.

Evaluation protocolImageWAMGroundingWAM
Randomized initial object positions27/50 (54.0%)38/50 (76.0%)
Nine shared relocation positions, three trials per position15/27 (55.6%)20/27 (74.1%)
Combined42/77 (54.5%)58/77 (75.3%)

The second protocol used a fixed starting arrangement from which both policies could grasp the target. Both policies were evaluated at the same nine relocation positions, with three trials per position. In four of its 23 failed randomized-placement trials, ImageWAM attempted to grasp the distractor. These experiments demonstrate recovery improvements on physical hardware with externally generated grounding annotations.

A2 (Weakness 2): Post-Drop Localization of the Full Model

Thank you for pointing out the missing localization comparison. We evaluated the built-in box heads of Box-only and full GroundingWAM on the same post-drop observations across all ten LIBERO-Spatial tasks. Both models received identical camera views, robot states, and task instructions, controlling for differences in their rollout trajectories.

GroundingWAM improved task-averaged target-box IoU from 0.566 to 0.724, with improvements on nine of ten tasks. In the 29 frames where the projected gripper-to-object distance was at least one ground-truth box diagonal, frame-averaged IoU improved from 0.071 to 0.569. These results support improved localization when the gripper's position becomes an unreliable cue to the object's location.

We also quantified attention across 37 perturbation tasks. In the external-camera view, the dropped object occupied an average of 1.06% of the image while receiving 16.20% of the attention in the supervised heads. The median frame-level attention enrichment was 17.28× relative to uniform allocation. These measurements extend the evidence beyond the selected visualizations and support increased attention to the dropped target.

A3 (Weakness 2): Interpretation of Grounding Attention and Shortcut Learning

Thank you for raising this point. We agree that supervised attention maps and improved task success alone do not establish that a policy has stopped exploiting object–gripper correlations. We view GAD as a representation-learning objective that shapes the intermediate representations used for action prediction through grounding supervision.

We additionally evaluated both policies from identical post-drop states. For each state, we replayed the same control prefix and verified that the simulator state, proprioception, and both camera images matched before handing control to each policy. Without attention intervention, ImageWAM completed the task in 31/109 cases (28.4%), compared with 60/109 (55.0%) for GroundingWAM, a gain of 26.6 percentage points. This comparison controls for differences in the states reached before recovery.

We also transferred 90% of the attention mass assigned to the dropped object to background patches in the six supervised heads of each supervised layer. The modified attention probabilities were used to weight the value vectors during action computation. GroundingWAM's task success changed from 55.0% to 56.0%, showing little sensitivity to this intervention.

Training-time gradients can improve the representations used for action prediction even when perturbing the supervised attention heads at inference produces little change in task success. The experiments support improved post-drop localization and recovery, while the precise mechanism through which the learned representations improve recovery remains open.

AuthorsOpenReview ↗

Response to Reviewer DUcT (2)

A4 (Weakness 3): Recovery Gains across Suites

Thank you for highlighting the differences across suites. We further decomposed recovery into regrasp and subsequent task completion. The following rates condition on an eligible confirmed drop, excluding episodes that completed the task during the forced-open interval.

SuiteImageWAM regrasp rateGroundingWAM regrasp rateGainImageWAM regrasp-and-completion rateGroundingWAM regrasp-and-completion rateGain
Spatial61.00%88.77%+27.76 pp40.39%83.29%+42.90 pp
Object39.89%43.67%+3.77 pp27.13%32.08%+4.95 pp
Goal24.22%38.13%+13.90 pp20.50%33.75%+13.25 pp
Long50.18%56.60%+6.42 pp31.79%47.92%+16.13 pp
Overall47.32%59.35%+12.03 pp31.46%51.21%+19.75 pp

Regrasp improved in all four suites, and Spatial exhibits the largest gain.

A drop changes both the object's location and its grasping pose. Grounding supervision may help preserve the use of target-object information when its spatial relationship to the gripper and background becomes unfamiliar. All Spatial tasks involve a black bowl, and the demonstrations include grasps from multiple sides. When the bowl lands upright, its approximately rotationally symmetric rim supports similar grasps across different yaw angles. For a rectangular object, falling onto a different face can require a wrist orientation and finger placement that are poorly represented in training. Accurate localization may therefore be insufficient to recover a successful grasp. This provides a possible explanation for the uneven gains across suites.

A5 (Weakness 3): LIBERO-Plus Results and Comparison Protocols

Thank you for raising these evaluation questions. On all 10,030 LIBERO-Plus scenarios, GroundingWAM achieved 87.08%, compared with published results of 85.5% for GAM and 83.1% for ImageWAM. We acknowledge that Camera performance was lower than ImageWAM's published result, at 76.99% versus 80.8%.

On the shared 2,000-scenario component-evaluation subset, ImageWAM and GroundingWAM achieved 83.80% and 88.80%, respectively. The largest improvements occurred under changes to the robot's initial state, from 52.10% to 86.01%, and object layout, from 77.89% to 86.67%. Sensor-noise performance decreased from 96.49% to 87.72%, and background performance decreased from 91.96% to 90.56%. These results identify the conditions under which the method provides its strongest benefits. The gains under robot-state and object-layout changes are consistent with the motivation behind our action-to-vision attention guidance, which encourages action queries to attend to task-relevant visual evidence when familiar spatial relationships change.

Regarding result provenance, all external baseline scores in Table 1 come from the respective publications. For the release benchmark in Tables 2–3, we evaluated the policies under the same release and scoring protocol while retaining their released inference settings.

A6 (Weakness 4): Relationship to Prior Attention-Guidance Methods

Thank you for highlighting the related work. Although prior methods employ similar attention-guidance designs, our work has a distinct motivation and evaluation focus. Our central question is whether grounding supervision improves recovery when a WAM departs from the demonstrated trajectory.

Our investigation began with post-drop recovery failures, which suggested that the representations used for action prediction might inadequately encode task-relevant evidence. We first introduced auxiliary box supervision to encourage these representations to encode object location. This improved recovery, but visualizations showed that box predictions could still remain near the gripper after the object had fallen. This observation motivated GAD, which directly supervises intermediate attention to task-relevant image regions and instruction tokens.

The resulting supervision provides benefits complementary to box prediction without adding recovery demonstrations. On the Spatial release benchmark, task success was 48.50% for ImageWAM, 73.00% with box supervision, 81.00% with guidance alone, and 90.25% with both.

Our contribution centers on the recovery problem, the localization diagnostics motivating the design, and the demonstrated benefits of grounding supervision in WAMs. The thresholded attention-mass objective is the implementation used to encourage sufficient attention to annotated evidence; we do not claim the mass objective itself as a standalone contribution.

Reviewer_yw6rOpenReview ↗
Rating
1
Confidence
4
Informativeness
3
Presentation
3
Soundness
1
Read review
Paper summary
  1. The paper investigates why World Action Models fail to recover after object drops despite strong nominal manipulation performance, attributing this partly to reliance on object–gripper correlations.

  2. GroundingWAM combines auxiliary box prediction with direct supervision of action-to-vision and action-to-text attention using task-relevant region and phrase annotations.

  3. Without recovery demonstrations, the method improves overall object-release success from 47.79% to 64.71%, while achieving 99.3% on LIBERO and 87.1% on LIBERO-Plus.

Critical strengths
  1. A meaningful evaluation beyond nominal success. The object-release protocol tests whether policies respond to disrupted execution, covering 37 tasks, eight transport thresholds, and 1,720 episodes per model rather than selected failure cases.

  2. Strong recovery gains with informative component comparisons. On Spatial, success increases from 48.50% to 73.00% with box supervision, 81.00% with GAD alone, and 90.25% with both, supporting their complementary value (Table 5).

  3. No recovery data or inference-time grounding system is required. Grounding annotations are training-only, and the full LIBERO-Plus evaluation provides additional evidence beyond the custom perturbation setting.

Critical weaknesses
  1. The causal interpretation exceeds the evidence. Supervised attention maps and improved success do not establish that the policy has stopped exploiting object–gripper shortcuts. Visualizations use selected supervised heads on different policy trajectories; neither causal interventions nor quantitative post-drop localization comparisons substantiate the proposed mechanism.

  2. The central design choices are insufficiently isolated. AV and AT supervision are not ablated separately, and sensitivity to supervised heads, layers, and attention thresholds is unreported. The experiments therefore do not establish which component is necessary or whether the specific attention-mass objective outperforms simpler grounding regularizers.

  3. Deployment relevance and recovery generality remain limited. All experiments are simulated and rely on simulator-derived training annotations, without testing noisy grounding labels or physical robots. Recovery gains are also uneven: Object improves only from 39.50% to 44.00%, versus the highlighted 41.75-point Spatial gain (Table 2).

Questions for authors
  1. Can you evaluate policies from identical post-drop simulator states and intervene on target, gripper, or background evidence to test whether GAD changes the information actually used for action generation?

  2. What are the separate contributions of AV and AT? Can you provide layer/head/threshold sensitivity, simpler attention- supervision controls, and results across multiple training seeds?

  3. Do the gains survive annotations from an external grounding model rather than simulator geometry? Can you demonstrate real- robot recovery and explain the substantially weaker improvement on Object tasks?

2 comments
AuthorsOpenReview ↗

Response to Reviewer yw6r (1)

A1 (Weakness 1): Interpretation of Grounding Attention

Thank you for raising this point. We agree that supervised attention maps and improved task success alone do not establish that a policy has stopped exploiting object–gripper correlations. We view GAD as a representation-learning objective that shapes the intermediate representations used for action prediction through grounding supervision. Its training-time gradients can improve these representations even when perturbing the supervised attention heads at inference produces little change in task success.

To quantify post-drop localization, we evaluated the built-in box heads of Box-only and full GroundingWAM on the same post-drop observations across all ten LIBERO-Spatial tasks. Both models received identical camera views, robot states, and task instructions. GroundingWAM improved task-averaged target-box IoU from 0.566 to 0.724, with improvements on nine of ten tasks. In the frames where the projected gripper-to-object distance exceeded one ground-truth box diagonal, frame-averaged IoU improved from 0.071 to 0.569.

We also quantified attention across 37 perturbation tasks. The dropped object occupied an average of 1.06% of the image while receiving 16.20% of the attention in the supervised heads. The median frame-level attention enrichment was 17.28× relative to uniform allocation. These results support improved post-drop localization and increased attention to the target. The precise mechanism through which the learned representations improve recovery remains open.

A2 (Weakness 2): Motivation and Scope of the Attention-Guidance Design

Thank you for the suggestion. We treat A2V and A2T supervision as an integrated grounding objective and do not claim them as separate contributions. Our central question is whether grounding supervision improves recovery when a WAM departs from the demonstrated trajectory. We therefore prioritized evaluating the integrated objective and its complementary benefits to box supervision; separating A2V and A2T was a lower experimental priority.

Although prior methods employ similar attention-guidance designs, our work has a distinct motivation and evaluation focus. Our investigation began with post-drop recovery failures, which suggested that the representations used for action prediction might inadequately encode task-relevant evidence. We first introduced auxiliary box supervision to encourage these representations to encode object location. This improved recovery, but visualizations showed that box predictions could still remain near the gripper after the object had fallen. This observation motivated GAD, which directly supervises intermediate attention to task-relevant image regions and instruction tokens.

The resulting supervision provides benefits complementary to box prediction without adding recovery demonstrations. On the Spatial release benchmark, task success was 48.50% for ImageWAM, 73.00% with box supervision, 81.00% with guidance alone, and 90.25% with both. Our contribution centers on the recovery problem, the localization diagnostics motivating the design, and the demonstrated benefits of grounding supervision in WAMs. The thresholded attention-mass objective is the implementation used to encourage sufficient attention to annotated evidence; we do not claim the mass objective itself as a standalone contribution.

A3 (Weakness 3, Question 3): Real-Robot Recovery with External Grounding Labels

Thank you for suggesting this evaluation. We conducted real-robot experiments using a Franka R3 arm with two cameras and demonstrations of a two-object pick-and-place setting. Grounding DINO and SAM 2.1 provided front-camera boxes and masks. We screened the target-object and basket annotations and trained both policies on the same retained demonstrations. The deployed policies required no detector.

ImageWAM and GroundingWAM were trained for 5,000 steps from the same pretrained FLUX weights. After the initial grasp and a further 48 control steps (2.4 seconds), we paused inference, removed and relocated the target object, and resumed the policy. Success required regrasping the relocated target and placing it in the basket.

Evaluation protocolImageWAMGroundingWAM
Randomized initial object positions27/50 (54.0%)38/50 (76.0%)
Nine shared relocation positions, three trials per position15/27 (55.6%)20/27 (74.1%)
Combined42/77 (54.5%)58/77 (75.3%)

In four of its 23 failed randomized-placement trials, ImageWAM attempted to grasp the distractor. These experiments demonstrate recovery improvements on physical hardware with annotations produced by external grounding models.

AuthorsOpenReview ↗

Response to Reviewer yw6r (2)

A4 (Weakness 3, Question 3): Recovery Gains across Suites

Thank you for highlighting the differences across suites. We further decomposed recovery into regrasp and subsequent task completion. The following rates condition on an eligible confirmed drop, excluding episodes that completed the task during the forced-open interval.

SuiteImageWAM regrasp rateGroundingWAM regrasp rateGainImageWAM regrasp-and-completion rateGroundingWAM regrasp-and-completion rateGain
Spatial61.00%88.77%+27.76 pp40.39%83.29%+42.90 pp
Object39.89%43.67%+3.77 pp27.13%32.08%+4.95 pp
Goal24.22%38.13%+13.90 pp20.50%33.75%+13.25 pp
Long50.18%56.60%+6.42 pp31.79%47.92%+16.13 pp
Overall47.32%59.35%+12.03 pp31.46%51.21%+19.75 pp

Regrasp improved in all four suites, and Spatial exhibits the largest gain.

A drop changes both the object's location and its grasping pose. Grounding supervision may help preserve the use of target-object information when its spatial relationship to the gripper and background becomes unfamiliar. All Spatial tasks involve a black bowl, and the demonstrations include grasps from multiple sides. When the bowl lands upright, its approximately rotationally symmetric rim supports similar grasps across different yaw angles. For a rectangular object, falling onto a different face can require a wrist orientation and finger placement that are poorly represented in training. Accurate localization may therefore be insufficient to recover a successful grasp. This provides a possible explanation for the uneven gains across suites.

A5 (Question 1): Identical Post-Drop States and Attention Intervention

Thank you for suggestion. We evaluated both policies from identical post-drop states. For each state, we replayed the same control prefix and verified that the simulator state, proprioception, and both camera images matched before handing control to each policy.

Without attention intervention, ImageWAM completed the task in 31/109 cases (28.4%), compared with 60/109 (55.0%) for GroundingWAM, a gain of 26.6 percentage points. This comparison controls for differences in the states reached before recovery.

We also transferred 90% of the attention mass assigned to the dropped object to background patches in the six supervised heads of each supervised layer. The modified attention probabilities were used to weight the value vectors during action computation. GroundingWAM's task success changed from 55.0% to 56.0%. Thus, this intervention did not produce a substantial reduction in recovery success. Together with the localization results, this supports interpreting GAD as training-time supervision that improves learned representations.

A6 (Weakness 3, Question 3): Sensitivity to Grounding-Label Noise

Thank you. Our real-robot experiments use boxes and masks generated by Grounding DINO and SAM 2.1. We screened the target-object and basket annotations before training both policies on the same retained demonstrations. GroundingWAM consistently outperformed ImageWAM under both evaluation protocols, achieving 76.0% versus 54.0% success with randomized initial positions and 74.1% versus 55.6% at predefined relocation positions. These results show that the recovery gains extend to externally generated grounding annotations in a real-world setting.

Reviewer_Sh3zOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper essentially tries to fix one specific instance of causal confusion/overfitting to spurious correlations in the imitation learning of World Action Models. They point out that while popular WAMs like Fast-WAM/ImageWAM show high scores on manipulation benchmarks like LIBERO and even on benchmarks designed to explicitly test for robustness like LIBERO-Plus, they still leverage spurious visual correlations to solve the tasks instead of utilizing general and task relevant features.

They test this by dropping the objects picked up by policies in simulation and asking them to do the task again from this dropped state. They show that previous WAMs are not able to retry and pickup the objects again. They argue that this is because they have overfitted to the specific visual positions and features of the objects.

They further show that if they try to predict the bounding boxes of the objects to pickup using a WAM, then the prediction works well until the object is dropped. However once the object is dropped, the bounding box of the object is wrongly predicted to be still around the gripper. This demonstrates that the model has overfitted to the positions of the gripper and the objects during its training. They also validate the same using the attention maps between the action head of the WAMs and the visual observations, showing that they too are wrong following a drop.

Their fix grounding attention distillation is quite straightforward. They obtain ground truth masks for the objects and the target entity in the prompt and regularize the action-image and action-text attention maps to assign weights to the object and the target entity. This forces the WAM to focus on the objects and reduces spurious correlations and causal confusion.

Through experiments they show that this regularization increases recovery in the dropped object case, even though they did not add any new recovery training data which is quite cool in my opinion. They also don't show any degradation in performance in the baseline performance because of this regularization.

Through ablations, they show that while just adding bounding box prediction or attention regularization helps, its their combination which works the best.

The authors point out as a limitation that their approach induces some very specific recovery patterns by fixing one specific case of causal confusion and doesn't solve the problem of recovery at large, which is fair.

Critical strengths
  1. The problem they address is very real, well scoped and of practical consequence. They don't argue to solve spurious correlation at large and focus on fixing one specific instance of it. Their solution to that problem is also neat and quite straightforward and seems to work well.
  2. I was pleasantly surprised that their method introduced recovery behavior without needing any additional recovery demonstrations.
Critical weaknesses
  1. It would have been great to see some real world hardware experiments or experiments on other benchmarks. Also a lot of this seems to be a overfitting issue due to small number of demos on LIBERO. Especially the bounding box prediction going off-rails seems to be just low-data overfitting. Exploring how these things disappear with scaling the data or adding diversity would have been interesting.
  2. Adding the bbox prediction in the action head instead of the vision head seems like an odd choice and might be the reason why the bbox is overfitting to the actions.
Questions for authors
  1. Did you try best-of-N sampling while testing for recovery from dropped points? Did the WAMs just not work at all when they failed or did they work if you just sampled enough from the same state.
1 comment
AuthorsOpenReview ↗

Response to Reviewer Sh3z

A1 (Weakness 1): Real-Robot Experiments

Thank you for suggestion. We use a Franka R3 arm with two cameras to collect demonstrations of a simple two-object pick-and-place setting. We train ImageWAM and GroundingWAM for 5,000 steps on the collected demonstrations, starting from the same pretrained FLUX weights. At test time, we instruct the robot to place one object in the basket, with the other present as a distractor. After the robot grasps the target, we manually introduce a disturbance and evaluate whether the policy can regrasp it and complete the task. We measure overall success with randomized initial object positions and assess the spatial range of recovery with predefined relocation positions shared by both policies.

A trial counted as successful only if the robot regrasped the relocated target and placed it in the basket. With randomized initial object positions, GroundingWAM succeeded in 38/50 trials (76.0%), compared with 27/50 (54.0%) for ImageWAM. In four of its 23 failed trials, ImageWAM attempted to grasp the distractor.

We also tested a fixed starting arrangement from which both policies could grasp the target. We used the same nine relocation positions for both policies: eight around the target’s initial position and one directly beneath the gripper. Each policy was tested three times per position. GroundingWAM succeeded in 20/27 trials (74.1%), compared with 15/27 (55.6%) for ImageWAM. Across both protocols, GroundingWAM succeeded in 58/77 trials (75.3%), compared with 42/77 (54.5%) for ImageWAM, a gain of 20.8 percentage points.

A2 (Weakness 2): Adding Bounding-Box Prediction to the Action Head

Thank you for raising this question. The box queries and action queries share transformer parameters, but the attention mask prevents them from attending to one another. Box predictions are therefore not decoded from predicted actions. We place the box queries in the shared expert to provide a parallel localization readout and to encourage the representations used for action generation to encode object location through auxiliary supervision.

A3 (Question 1): Best-of-N Sampling

Thank you for suggestion. We tested whether a second sample could rescue a failed recovery by running each policy twice from the same 53 post-drop states in LIBERO-Spatial. We used inference seed 42 for the first attempt and seed 43 for the second. An attempt counted as successful only if the policy regrasped the dropped object and completed the task.

PolicyFirst-attempt successesFirst-attempt failures rescued by the second attemptPass@2
ImageWAM30/532/23 (8.7%)32/53 (60.4%)
GroundingWAM46/534/7 (57.1%)50/53 (94.3%)

A second sample rescued some failures for both policies. With two attempts per state, GroundingWAM recovered in 18 more states than ImageWAM. Here, pass@2 counts a state as solved if either rollout succeeded.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.