acceptodds
Under review as a conference paper at ICLR 2027

DISCERN: Language-Guided Behavioral Search with World Models

Abstract

World models offer a way to evaluate policy behaviors by predicting action consequences before real-world execution. For test-time policy steering, however, their effectiveness depends not only on predicting outcomes but also on proposing sufficiently diverse behaviors to search over. Existing steering methods based on world models primarily sample actions from a policy under a fixed instruction, restricting the search to the behaviors induced by that language conditioning. We introduce DISCERN, which steers a frozen Vision-Language-Action policy at test time by searching over task-preserving rephrasings of the original task instruction. For each rephrasing, an action-conditioned world model predicts the policy's behavior over several action chunks in closed loop, and a vision-language verifier selects the instruction whose predicted outcome best satisfies the task. On LIBERO-PRO, DISCERN improves frozen-policy success by percentage points on tasks with lock-in failures, where the policy ignores the instruction and repeats a motion from training, significantly outperforming all baselines. Our analyses further show that varying the instruction reaches successful behaviors that action sampling with a fixed instruction misses, and that closed-loop previews improve language selection over a single action chunk prediction. These results identify language as an effective way to sample behaviors for policy steering with world models.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World models, test-time steering, language-conditioned policies, behavioral reachability, robot manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jWH8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The proposed framework DISCERN (Language-Guided Behavioural Search with World Models) is presented in submission 51116; it is a test-time policy-steering approach that improves the instruction-following reliability of previously trained Vision-Language-Action (VLA) robot policies without updating policy weights or running trial rollouts in the real environment.

The main conceptual contribution of DISCERN is to show that rephrasing language expands the behavioural search space much more effectively than sampling from action space. However, in practice, misclassifications during VLM verification, predictive drift in the world model over longer time horizons, and high inference latency currently limit its effectiveness.

Critical strengths
  1. Conceptual Formulation of Language as a Search Interface for Frozen Policies: The authors identify a fundamental bottleneck in existing world-model policy steering: sampling actions under a fixed prompt restricts search to a narrow, potentially failing region of the policy's behavioral distribution6. By formalizing language rephrasing as a deployment-time control variable u_t, DISCERN provides a principled mechanism to unlock latent behavioural modes in frozen VLAs without parameter updates or reward model fine-tuning. The spatial density analysis in Figure 3 and Table 3 provides clear empirical proof that language proposals shift action distributions globally toward desired targets (37.4% mode reachability) where action-space sampling remains trapped in fixated modes.

  2. Rigorous Analysis of Predictive Limits and Temporal Intervention Windows: The paper delivers a high-quality empirical dissection of when world-model previews are effective in Section 5.4; see Figure 4.

  3. Training-Free, Modular System Execution with Strong Baseline Standardization: Unlike baseline approaches that require retraining latent verifiers (CoVer), latent world models with text-alignment narrators (FOREWARN), or Q-value models (RL2-VLA) on task demonstrations30more_horiz, DISCERN operates completely training-free using off-the-shelf components3334. Furthermore, the authors standardize all baselines on the same frozen π0.5\pi_{0.5} checkpoint, initial states, and candidate instructions (Tables 5–6).

Critical weaknesses

In Section 5.1, the authors added: "Since A2World was trained on demonstrations from the LIBERO scenes used to derive LIBERO-PRO tasks, we do not claim to test for world-model generalization." In my view, this is a limitation since a) the action-conditioned world model (A2World) was evaluated on the exact same tabletop environments and visual assets it was trained on, and b) in a true zero-shot deployment (e.g., novel room lighting, unseen objects, busy scenes), world-model rollouts inevitably suffer from visual hallucinations and physical artifacts. Because A2World was in-distribution, the paper does not quantify how rapidly DISCERN's performance collapses when preview images degrade under domain shift.

Unless I misunderstood, there is high fragility across open-weight and smaller VLMs. It appears that DISCERN's performance drops significantly when moving away from top-tier commercial frontier models:

  • Gemini Robotics ER 2 / Gemini Flash: 19.9% success
  • Qwen-35B: 16.3% success
  • Qwen-27B: 16.6% success

For ICLR participants, this point matters. The results suggest that the verification step depends substantially on the fine-grained visuospatial reasoning capabilities of proprietary frontier models, such as interpreting wrist-camera depth cues, gripper orientations, and multi-column visual grids. Open-weight models currently lack the visual accuracy needed to reliably rank candidate futures, complicating local or edge deployment.

Questions for authors

Latency & Computational Overhead: What is the average time required to complete a single decision step (generating 8 candidate rollouts for 30 steps with A2World and querying the Gemini ER 2 verifier API)?

VLM Verifier Error Attribution: In cases where DISCERN failed on a lock-in task but "Best fixed instruction" succeeded, what proportion of failures was caused by the VLM verifier selecting the wrong column despite an accurate world-model rollout, versus the world model generating an inaccurate visual rollout?

Out-of-Distribution World Model Behaviour: Please comment on the method’s performance when synthetic noise or visual blur is introduced into A2World predictions to simulate world-model errors on previously unseen scenes? Of particular interest could be “motion blur,” or out-of-focus blur. Does the verifier's default-stay rule effectively prevent erratic instruction switching under degraded preview conditions?

Generalization Across VLA Architectures: Have the authors evaluated DISCERN on alternative open-source VLA backbones, such as OpenVLA, to determine whether instruction rephrasing consistently resolves lock-in failures across various policy architectures?

2 comments

We thank the reviewer for acknowledging our method’s strong motivation and analysis and for raising insightful questions. We address these questions below:

W1 / Q3. In Section 5.1, the authors added: "Since A2World was trained on demonstrations from the LIBERO scenes used to derive LIBERO-PRO tasks, we do not claim to test for world-model generalization." In my view, this is a limitation since a) the action-conditioned world model (A2World) was evaluated on the exact same tabletop environments and visual assets it was trained on, and b) in a true zero-shot deployment (e.g., novel room lighting, unseen objects, busy scenes), world-model rollouts inevitably suffer from visual hallucinations and physical artifacts. Because A2World was in-distribution, the paper does not quantify how rapidly DISCERN's performance collapses when preview images degrade under domain shift.

Thank you for raising this. We ran new real-world experiments and successfully deployed a world model trained on cross-embodiment data zero-shot to unseen objects and scenes. Our evaluation results show improved real-world task success over the original pi0.5 policy and VLS baseline, addressing the reviewer’s concern about the performance degradation under domain shift:

MethodNarwhalPomegranateBell PepperAvg.
pi0.53/200/200/200.050
VLS6/201/203/200.167
DISCERN8/209/205/200.367

W2. Unless I misunderstood, there is high fragility across open-weight and smaller VLMs. It appears that DISCERN's performance drops significantly when moving away from top-tier commercial frontier models: For ICLR participants, this point matters. The results suggest that the verification step depends substantially on the fine-grained visuospatial reasoning capabilities of proprietary frontier models, such as interpreting wrist-camera depth cues, gripper orientations, and multi-column visual grids. Open-weight models currently lack the visual accuracy needed to reliably rank candidate futures, complicating local or edge deployment.

We agree that verifier quality matters, and our results show that open-weight verifiers already retain much of the performance. With the same prompt and previews, Qwen (16.3–16.6%) and UnifoLM-ER-1 (17.0%; new result) all improve over the canonical instruction (13.2%). Because the verifier is a modular, training-free component, DISCERN improves with embodied VLMs, and such models are increasingly deployable on-device (e.g., Gemini Robotics On-Device).

Q4. Generalization Across VLA Architectures: Have the authors evaluated DISCERN on alternative open-source VLA backbones, such as OpenVLA, to determine whether instruction rephrasing consistently resolves lock-in failures across various policy architectures?

Thank you for this question. We ran new experiments to test language-steerability of OpenVLA and found that averaged across the 80 simulation tasks, grid search over a set of rephrased instructions validated at the best one improved task success rates from 7.75% to 14.25%.

Q2. VLM Verifier Error Attribution: In cases where DISCERN failed on a lock-in task but "Best fixed instruction" succeeded, what proportion of failures was caused by the VLM verifier selecting the wrong column despite an accurate world-model rollout, versus the world model generating an inaccurate visual rollout?

We did not calculate per-decision accuracy labels for world-model previews, but our ablations provide some insight into the limiting components of our method. On lock-in tasks, a perfect world model recovers only 16% of the gap to the best fixed instruction. This gap persists even with accurate previews, suggesting that verifier selection leads to most of the failures. World-model and ground-truth candidate rankings are also well-correlated (Spearman ρ = 0.64, Fig. 4a), consistent with the verifier being the main bottleneck.

Q1. Latency & Computational Overhead: What is the average time required to complete a single decision step (generating 8 candidate rollouts for 30 steps with A2World and querying the Gemini ER 2 verifier API)?

On our real robot, a full decision (eight previews plus the Gemini ER2 query) takes a median of 87 s (Appendix D): 75 s for previews and 12 s for the verifier. In simulation, A2World's unoptimized 16-step video diffusion takes about 6 min per decision. DISCERN is agnostic to the world model, and recent few-step and single-pass world models report significantly faster rollouts. Swapping in such a model, running all eight previews in one batch, or distilling the sampler would reduce this cost substantially without changing the method. We acknowledge this latency in our limitations section, but we expect this to be less of an issue over time considering the growing effort of the community to implement faster world models.

Reviewer_MZqdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Test-time steering for a frozen VLA. A VLM rephrases the instruction into 8 candidates, each gets rolled out in a world model, and a VLM verifier looks at the predicted end frames and picks which instruction to run. Evaluated on LIBERO-PRO with pi0.5 with a significant improvement on a subset of tasks, and modest improvements on all 80.

Critical strengths
  1. Rephrasing the instruction reaches behavioral modes that action sampling just can't (Table 3, Fig 3). Pretty nice result.
  2. Baselines are sensible, same frozen policy everywhere, and the simulator-preview and oracle rows make it obvious where the losses are coming from.
Critical weaknesses
  1. Rephrase with a VLM, preview in a world model, pick with a VLM. All three pieces already exist and has been combined in a variety of ways in past works [1,2,3]. The paper also mentions that prior work train "an additional component", but DISCERN itself relies on a world model trained on the exact LIBERO scenes it's evaluated on (5.1 says so), which is the same thing. "off-the-shelf" verifier is running on a long hand-tuned prompt (App C) explaining the Franka, the wrist cam finger tips, LIBERO conventions, which is not off-the-shelf: it's prompt engineering for this benchmark. That being said, the language -> performance results were insightful, and better established "laws" for language alignment would greatly improve the paper.
  2. Another concern is that this method only works at short horizons, which is exactly where you need it least. Usable preview is 30 steps, so ~1.5 s, because the world model drifts after that (Fig 4b, 5a). So the verifier is judging 1.5 s of progress. On anything with actual longer-horizon reasoning, where short-term progress doesn't mean much, this will just reward greedy behavior. The paper admits the gap in 4.3 but never tests on a longer horizon task.
  3. There are 3 components that can potentially fail, policy, world model, verifier, and the gap to the oracle (29.6 vs 79.6) says the verifier already is the bottleneck. It would be insightful to run experiemnts to stress test the different components which is a understudied area in these types of algorithms.
  4. World model data requirements aren't discussed at all. It is free in sim. However, in the real world, where you'd actually want this, world model data is a lot harder to get than policy data

[1] https://arxiv.org/abs/2310.10625 [2] https://arxiv.org/abs/2506.00613 [3] https://arxiv.org/abs/2606.00267

Questions for authors
  1. Can you run longer-horizon / multi-stage tasks where early progress isn't predictive of success? Does it still help, or does it go greedy?
  2. Have you tried distilling the selected rollouts back into the policy? That would address the inference latency you list as a limitation and make this a much stronger paper.
  3. Any real-robot results? Even a small study, since world model data is the limiting factor there.
  4. How sensitive is everything to the verifier prompt in App C? What happens with a short generic one?
2 comments

We thank the reviewer for their careful review and thoughtful questions. We are glad you found the language to performance results insightful. Please find our answers to these questions below:

W1a / Q3. Rephrase with a VLM, preview in a world model, pick with a VLM. All three pieces already exist and has been combined in a variety of ways in past works [1,2,3]. The paper also mentions that prior work train "an additional component", but DISCERN itself relies on a world model trained on the exact LIBERO scenes it's evaluated on (5.1 says so), which is the same thing.

Thank you for this suggestion. We agree that each component exists in prior work. Our distinction is that DISCERN improves steerability of a frozen policy through language optimization without real-world rollouts on the target task. Based on the reviewer’s feedback, we ran new real-world experiments where we deployed a world model pretrained on cross-embodiment data zero-shot, with unseen objects and scenes. Our evaluation results show improved real-world task success over the original pi0.5 policy and VLS baseline, without any need to fine-tune the world model:

MethodNarwhalPomegranateBell PepperAvg.
pi0.53/200/200/200.050
VLS6/201/203/200.167
DISCERN8/209/205/200.367

W1b / Q4. "off-the-shelf" verifier is running on a long hand-tuned prompt (App C) explaining the Franka, the wrist cam finger tips, LIBERO conventions, which is not off-the-shelf: it's prompt engineering for this benchmark. That being said, the language -> performance results were insightful, and better established "laws" for language alignment would greatly improve the paper.

We also agree that the prompt in App. C is detailed. We evaluated the sensitivity to the verifier prompt by using a simple prompt (shown below). We found that despite the simpler prompt, our method maintains high performance compared to baselines, achieving 17.8 ± 3.3% task success rate compared to the original 19.0 ± 3.5%. This suggests that the gains in our method do not depend on including guided reasoning or LIBERO and Franka conventions. In practice, a useful prompt for judging can also be generated zero-shot with a VLM.

Prompt:

“Task: {task}

Eight columns show possible futures: third-person view on top, wrist view below. Column 1 is the current plan. Keep 1 unless it is heading to the wrong object and another column is clearly heading to the right one.

Reply with the column number, then one sentence.”

W3. There are 3 components that can potentially fail, policy, world model, verifier, and the gap to the oracle (29.6 vs 79.6) says the verifier already is the bottleneck. It would be insightful to run experiments to stress test the different components which is a understudied area in these types of algorithms.

The paper ablates each component: world-model vs. simulator previews (Table 2), closed- vs. open-loop previews (Table 4), preview horizon and world-model error (Sec. 5.4), and verifier models (Fig. 5c). We also added an additional ablation that removes the world model: the verifier selects among the same candidate instructions using only the current observation, with no previews. Across all 80 tasks, success falls from 19.0% to 13.9% (Table 4, "w/o previews"), close to the canonical instruction's 13.2%. This shows that the gains in our method come from the previews rather than from instruction selection alone.

W4. World model data requirements aren't discussed at all. It is free in sim. However, in the real-world, where you'd actually want this, world model data is a lot harder to get than policy data

We agree that collecting real-world data for world-model training is more challenging than obtaining data in simulation. In our new real-world experiments, we use a world model pretrained on only existing public robot datasets (DROID, Bridge, Fractal, FMB, BC-Z, TACO Play, and FurnitureBench) and deploy it zero-shot, with no data collection or fine-tuning on our setup. We report qualitative results of world model previews in Appendix D of the revised paper.

W2 / Q1. Another concern is that this method only works at short horizons, which is exactly where you need it least. Usable preview is 30 steps, so ~1.5 s, because the world model drifts after that (Fig 4b, 5a). So the verifier is judging 1.5 s of progress. On anything with actual longer-horizon reasoning, where short-term progress doesn't mean much, this will just reward greedy behavior. The paper admits the gap in 4.3 but never tests on a longer horizon task.

Thank you for this comment. We agree that each preview is short, and Sec. 5.4 analyzes exactly this trade-off between world-model error and candidate spread. The failures we target are early commitments to the wrong object or location, which short previews can detect. This pick-and-place language-alignment setting is standard in most steering work (VLS, FOREWARN, CoVer, etc.) and tasks with misleading early progress remain open. As world models improve, DISCERN's previews, and the tasks it can address, extend directly without changes to the method.

Q2. Have you tried distilling the selected rollouts back into the policy? That would address the inference latency you list as a limitation and make this a much stronger paper.

Thank you for this suggestion. While training steered behaviors back into the policy is an exciting extension, it has been shown in prior work (Prompt-Driven Exploration; Jiang et al., 2026) and is not the focus of our paper. We focus on steering a policy at deployment, without training. Distillation is a complementary and interesting direction that could amortize inference latency. We have included this note in Section 6 of the revised paper.

Reviewer_2rxmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Paper summary: This paper proposes DISCERN, a framework that searches for language instructions for the robot to complete a task. The idea behind DISCERN is: given the original task description and original image, indicating the original state; a VLM is used to generate different language instructions along with the original description to guide the robot to complete the task. For each language instruction, a closed loop is generated, where the policy model predicts the next action, and the next observation/state will be predicted by another world model, given the current observation and predicted action. Finally, a verifier will verify which prompt is most successful at completing the task.

Contribution:

  1. A test-time, training-free approach to steer the frozen robot policy by searching for the best rephrased language instruction.
  2. Experimental results show that on LIBERO-PRO, DISCERN outperforms other instruction-generating baselines.
Critical strengths

Paper strength:

  1. Clear motivation design: This paper introduces a clear motivation: at inference time, the trained policy picks up objects at the training object’s position or places them at the training placement location instead of following the instruction, as illustrated by the lock-in failure cases.
  2. Training-free approach: This training-free approach can be easily plugged into any pretrained policy model.
Critical weaknesses

There are several weaknesses in the paper below:

  1. Method novelty: It seems this paper is a combination of existing model/frameworks and the issues it tries to address: steering the policy from language are pointed out by previous research [1], which limits the novelty of this paper
  2. Limitation of language steering: Section 5.1 states that only 23/80 lock-in tasks make the policy steerable, while there is no improvement in the other tasks, which limits the application of language steering.
  3. RL formulation: The formulation of this task is similar to an RL formulation for optimizing the prompt, so my question is: can we apply RL training to fine-tune the VLM (only one prompt at test time), instead of generating several prompts and testing with each prompt at test time as this paper has done?
  4. Experimental setting: What is the ratio of instructions per sample reported in Table 2? The mean is 29.6, which is slightly lower than 31.7 for the 8x1 setting in Figure 10.
  5. Computational cost: What is the computational cost, on average, for generating 8 instructions per task?
  6. Diverse range of policy model: Can this mechanism run on different policy model? What is the performance if we change the world model?

[1] Jiang, Sunshine, et al. "Prompt-Driven Exploration." arXiv preprint arXiv:2607.08837 (2026).

Questions for authors

See weaknesses section.

1 comment

We thank the reviewer for acknowledging our method’s strong motivations and contributions and raising insightful questions. We address the reviewer’s questions below.

W1. Method novelty: It seems this paper is a combination of existing model/frameworks and the issues it tries to address: steering the policy from language are pointed out by previous research [1], which limits the novelty of this paper.

We agree that steering through language is not a novel contribution. Our contribution is different from the related work as we explore a way to do training-free language optimization without real-world rollouts. As laid out in Table 1, ours is the first method that successfully shows steering via language without using real-world policy rollouts. To strengthen our contribution, we added new real-world results (Section 5.6) that show the zero-shot applicability of our method to new robot deployments.

MethodNarwhalPomegranateBell PepperAvg.
pi0.53/200/200/200.050
VLS6/201/203/200.167
DISCERN8/209/205/200.367

W6. Diverse range of policy model: Can this mechanism run on different policy model? What is the performance if we change the world model?

Thank you for this question. We tested language-steerability of OpenVLA and found that averaged across the 80 simulation tasks, grid search over a set of rephrased instructions validated at the best one improves task success rates from 7.75% to 14.25%. We ask that the reviewer refer to our additional real-world results for evidence of world model robustness.

W4. Experimental setting: What is the ratio of instructions per sample reported in Table 2? The mean is 29.6, which is slightly lower than 31.7 for the 8x1 setting in Figure 10.

Table 2 uses eight instructions per task (the original plus seven rephrases). All results outside the ablation in Figure 11 (Figure 10 in the original submission) use this setting. The gap comes from the preview model and thinking levels. Figure 11's 31.7% uses simulator previews at medium thinking, while Table 2's 29.6% uses world-model previews at high thinking. We revised Figure 11’s caption to make this clear.

W5. Computational cost: What is the computational cost, on average, for generating 8 instructions per task?

We use Gemini Robotics ER 2 to propose candidates. It is called once per episode and returns all seven rephrases given the original instruction and scene observation in a single query, costing under one cent. DISCERN does not depend on a particular VLM, so an open-weight model (e.g., the Qwen models we evaluate as verifiers in Fig. 5c) can replace it with no other changes. The main cost is instead previews and verification at each decision.

W2. Limitation of language steering: Section 5.1 states that only 23/80 lock-in tasks make the policy steerable, while there is no improvement in the other tasks, which limits the application of language steering.

Thank you for raising this. We note in the limitations section that our approach “can only recover failures for which its candidate instructions can elicit a successful behavior.” We also found that the baselines in Table 2 show similarly small gains outside the lock-in tasks. While outside the scope of our work, recent work has also shown that policies can be trained to become more responsive to language specification [1], increasing the potential of language steering for recovering desired behavior.

[1] Chen, William, et al. "Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Control." arXiv preprint arXiv:2602.13193 (2026).

W3. RL formulation: The formulation of this task is similar to an RL formulation for optimizing the prompt, so my question is: can we apply RL training to fine-tune the VLM (only one prompt at test time), instead of generating several prompts and testing with each prompt at test time as this paper has done?

Thank you for the insightful question. While we can apply RL training methods as an additional stage to fine-tune the VLM [2] or the policy to make testing more efficient, such approaches fall outside of the scope of our paper. We focus on using language optimization to steer policies without needing additional policy rollouts or training. We have included this note in Section 6 of the revised paper.

[2] Bhatia, Jagdeep, et al. “Adapting Generalist Robot Policies with Semantic Reinforcement Learning.” arXiv preprint arXiv:2606.31958 (2026)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.