acceptodds
Under review as a conference paper at ICLR 2027

Correcting Strategy Selection in Generalist Robot Policies

Abstract

Large-scale robot datasets record multiple strategies, or ways of completing the same task: a microwave can be opened by pulling its handle or by pressing its release button, and a lid can be taken off by lifting its knob or by pushing its rim. A given object, however, often affords only one of them. We find that generalist policies trained on such data can carry out an unsupported strategy, one the object does not afford, such as reaching for a handle on a microwave that does not have one. The training data demonstrate the requested strategy about as often as or more often than the strategy the policy carries out, and stating the requested strategy in the instruction does not fix the failure. We therefore ask what drives the selection, and identify two features of the input that matter: the color of the object and the object nouns in the instruction. Recoloring the object to the most common color of the objects on which the training data demonstrate the requested strategy, and deleting object nouns that draw the policy's attention away from the part this strategy acts on, both shift the policy toward the requested strategy. We turn these findings into a method that decides for each input whether and how to edit it, without retraining the policy. Across three simulated tasks, the edits raise the average success of -DROID from 8% to 57%. Same edits also improve two other DROID-trained policies.

Primary area:
applications to robotics, autonomy, planning
Keywords:
generalist robot policies, robot manipulation, strategy selection, test-time steering, input editing, attention analysis

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_mhKbOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates unsupported strategy selection in generalist robot policies: a policy attempts an interaction that the test object does not afford, despite receiving an instruction specifying the feasible strategy. The proposed method recolors objects according to strategy-conditioned training-data statistics and deletes object nouns when doing so increases an attention-based score on the requested interaction part. Experiments cover three simulated tasks, with additional transfer evaluations on two other DROID-trained policies.

Critical strengths

The failure mode is practically relevant, and the proposed interventions are straightforward to interpret.

Critical weaknesses
  1. Method described in main paper is unclear. The method combines demonstration retrieval, color selection, noun deletion, and attention scoring, but provides neither equations nor pseudocode describing the complete algorithm. This makes decision sequence difficult to verify. For example, the attention score lacks a self-contained mathematical definition.
  2. The method’s central selection rules are insufficiently justified. The observation edit selects the most frequent color among demonstrations of the requested strategy and defines “typical” colors through a greater-than-50% coverage rule. Neither the particular threshold nor the superiority of this selection rule is convincingly validated. And what if the objects in the training data are not a single color?
  3. Noun deletion may be too limited in specific scenes. For example, changing “take the lid off the box by lifting the knob” to “take the lid off by lifting the knob” removes information that distinguishes the box from other containers. In a scene containing both a box and another container with a knobbed lid, the original instruction can uniquely identify the target, whereas the edited instruction may become ambiguous. Consequently, the experiments demonstrate the benefit of noun deletion under favorable referential conditions. This limits the practical scope of the instruction-editing component and calls for an explicit semantic-preservation condition.
  4. The experimental scope is too narrow to establish the generality of the proposed procedure. The main evaluation uses three tasks, one principal object per task, and ten object poses with three rollouts each. Held-out poses test variation around the same objects, rather than generalization to new instances or environments. Consequently, the paper does not establish whether its edit-selection rules work beyond the specific objects used to develop and analyze the method or solve problem mentioned in W3.
  5. The cross-policy results provide limited evidence of practical transfer. Table 6 transfers edits selected for π0.5-DROID and uses asset recoloring rather than the online pipeline. Some improvements are small, and no uncertainty estimates are reported. Since the thirty rollouts are grouped within ten poses, pose-level variability also matters.
Questions for authors
  1. Tables 9 and 10 appear to report the same pooled attention-score changes over the ten held-out poses (layers-11-to-13), but give different values. Why?
  2. On the existing tasks, how does the attention-based rule compare with simply deleting eligible object nouns, and how does the color-selection rule compare with a fixed recoloring baseline?
Reviewer_qEQnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies a failure mode of VLA policies trained on the DROID dataset, identifying two biases that affect their performance: object color and object nouns in language instructions. Based on these findings, the authors propose a test-time steering method that modifies visual observations and language instructions to better match the training distribution. In their experiments, the proposed approach improves the success rate of pi0.5 on three simulated manipulation tasks.

Critical strengths

The paper is overall well-written and studies interesting failure modes and biases of DROID-trained policies. The proposed method is simple and does not require additional policy training. The experimental results show a clear improvement over the original pi0.5-droid.

Critical weaknesses

W1. The main concern is that the problem is studied only in policies trained on DROID. The proposed method therefore relies heavily on DROID-specific biases, which somewhat limits its general applicability. I would like to see evaluations on VLAs trained on other datasets to understand whether the proposed approach addresses a general strategy-selection problem.

W2. The experimental evaluation is limited to only three simulated manipulation tasks. The selected tasks seem to be specifically designed around strategy-selection failures. I would recommend expanding the evaluation to a larger set of tasks, including some common benchmarks.

W3. The baseline comparisons are limited as the proposed method is mainly evaluated against the original policies without input editing. There are no comparisons against existing test-time steering or instruction optimization methods.

Questions for authors

Q1. How are the target objects for SAM 3 automatically identified at test time? I am also curious about the accuracy of SAM 3 segmentation. It would be helpful to include an analysis of segmentation and tracking errors and how they affect downstream task performance.

Reviewer_TVtSOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies strategy-selection failures in generalist robot policies, where a policy attempts a manipulation strategy that the test object does not afford, even when the instruction explicitly specifies the supported strategy. The authors propose a training-free input-editing procedure that recolors objects according to demonstrations of the requested strategy and deletes object nouns when their removal increases attention to the relevant object part. Across three simulated tasks, the online procedure improves π0.5-DROID’s aggregate success from 7/90 to 51/90. Additional experiments examine color and instruction ablations and transfer the selected edits to two other DROID-trained policies using asset-level recoloring.

Critical strengths

The paper identifies a concrete failure mode and demonstrates substantial improvements on its main simulated examples without modifying policy weights. The comparisons across colors, noun deletions, and part-only instructions provide useful evidence that input edits have selective effects on behavior. Using separate poses for attention-based decisions and rollout evaluation is also a positive aspect of the experimental design.

Critical weaknesses

The main weakness is that the experiments do not sufficiently establish the reliability of the proposed edit-selection procedure beyond the studied examples. The evaluation covers three main object instances and four nouns, with held-out scenes varying pose rather than object identity. The geometric variants in Table 13 are tested only without the edits, while the additional-policy experiments reuse edits selected for π0.5 and employ asset-level recoloring, leaving the complete selection-and-online-editing procedure insufficiently validated across objects and policies. The evaluated implementation also relies on manually annotated part and object masks rendered from each object’s 3D model, and the supervision and calibration required for new objects remain insufficiently characterized. Furthermore, the edited-policy results primarily report task success without the comparable strategy labels provided for the baseline in Table 1, leaving unclear how frequently the edits correct strategy selection and which failures persist after selecting the supported strategy. Tables 9 and 10 also appear to report the same pooled attention-score changes but give different values, such as +0.032 versus +0.134 for “box.” Although the signs agree, the unexplained discrepancy makes the decision criterion difficult to verify. The substantial improvements on the main examples are meaningful, but these gaps concern the central contribution and require substantive additional validation. I therefore recommend clear rejection.

Questions for authors
  1. With the edit-selection rule fixed, how does the method perform on previously unused object instances, including the geometric variants in Table 13? Please report both task success and strategy-selection outcomes.

  2. Could you provide post-edit strategy labels using the same protocol as Table 1, distinguishing unsupported-strategy selection from execution failures after selecting the supported strategy?

  3. Could you reconcile the attention-score changes reported in Tables 9 and 10, particularly +0.032 versus +0.134 for “box”? Please specify whether the observations, pose sets, or noise samples differ, and identify which statistics determined the editing decisions.

  4. What object-specific supervision and calibration are required for a new object? In particular, how would the target-part and object masks be obtained without the manually annotated 3D models used in the current evaluation?

  5. How does the complete procedure perform on the additional policies when recoloring is applied online and instruction edits are selected using each policy’s own attention scores? The current experiments reuse π0.5’s edits and apply recoloring directly to simulator assets.

  6. How many distinct object instances and scenes underlie the color statistics for each requested strategy? Do the typical-color labels and prescribed edits remain unchanged when statistics are weighted by distinct objects rather than demonstration episodes?

  7. Could you provide paired results by object pose and an uncertainty analysis for the reported improvements, particularly the smaller gains in Table 6? How consistent are these gains across poses and repeated rollouts?

Reviewer_h14LOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors study affordance strategy mismatch, where specific instructions and images do not lead to the correct manipulation strategy for the target task. The authors analyze the data distribution to build a query database that finds the most aligned task descriptions for a desired motion. They then use this index to align the proper text description and target-object color, increasing the chances of model success for the given task. They run ablations on 3 separate tasks, with clearly defined strategies to isolate the benefit of their steering approach. Their results show promise on Pi_{0.5} and mixed results on other models.

Critical strengths

This work is interesting because the authors conduct an in-depth analysis of \pi_{0.5} to examine the training distribution and identify relevant motions for task completion.

We found the results interesting: aligning the color & task descriptions could make a substantial difference in performance (a 50% improvement in some cases).

We appreciate the effort to isolate the variables in specific tasks (i.e. a narrow distribution of motions to complete the testing task).

Critical weaknesses

Our biggest concern with the paper is summarized in the introduction's contribution wording: “By amounts that differ across policies and tasks,” which captures our criticism: these results are strongly tied to the pi_{0.5} checkpoint used. We see the paper as a case study showing that targeted strategies can steer a model, but this may vary across models. The most compelling transfer result in Table 6 is that the recoloring strategy performed substantially better on MolmoAct2-DROID for the Lid task, but otherwise does not generalize well to other models and, for all we know, other tasks, given the limited distribution of tasks considered in most of the experimental analysis.

Furthermore, because the authors use a single model, it’s difficult to assess how robust this approach is across different training runs. This is a significant limitation because the same \pi_{0.5} architecture with different weights may not respond the same way under the proposed interventions. This might be mitigated by validating the pipeline on more than 3 tasks, depending on the efficacy of the proposed manual pipeline.

From a robotics perspective, even at best, this approach is not usable for the real world as is. A 50% success rate is a dismal overall result, and we would strongly encourage them to re-evaluate how to frame their findings. We believe the strongest point of the analysis is that the consequences data distribution has steered the model. We would not consider this practically usable as presented without additional development. Real-world experiments would strengthen the paper by revealing any practical issues with the approach.

Minor Concerns:

Table 1: It is not clear what each label means, and Section 5.2 does not explain the labels meaningfully

The text as a whole would benefit from listing only the % success rates & the total trials performed. We did not enjoy the current report structure.

We recommend expanding your literature search to include, at a minimum, “adversarial reprogramming,” at least as a predecessor idea to prompt steering, among other ideas.

Questions for authors

The reviewers don’t need to answer this in the rebuttal, but the reviewer is interested in the cost of using the proprietary LLM for this research. This seems like a significant aspect of the work; could another research group reasonably cover these costs as part of replication?

Some work suggests that discrete prompts can also help steer models.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.