acceptodds
Withdrawn from ICLR 2027

From Failures to Learnable Worlds: Policy-Conditioned Environment Generation for Navigation Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Simulated environments enable scalable training data generation for embodied agents, but more data does not necessarily provide what a particular agent needs to learn. Existing environment-generation pipelines largely optimize generic properties such as realism, diversity, or task coverage, while different pretrained policies can fail for different environmental reasons, with some struggling when targets are hard to see and others when navigation requires long blind paths or multiple room transitions. We therefore introduce WEDGE, a framework for weakness-guided navigation learning in which the failure modes of a pretrained agent guide the generation of training environments at each adaptation round. Rather than assuming a predefined set of tasks or skills to improve, WEDGE derives its generation targets from the policy's own success and failure patterns within the same task. Given a pretrained policy, WEDGE uses an LLM to propose executable weakness features from contrasting successful and failed rollouts, automatically validates them on the diagnostic set, and selects the strongest validated weakness based on its success-rate gap. The validated weaknesses are then mapped to the environmental factors that control them, such as target placement, start configurations, and scene layouts. WEDGE varies these factors to construct environments in the difficulty regimes where the policy struggles, producing targeted training experience for adaptation. Across three navigation agents with different adaptation mechanisms, we find that policies exhibit distinct weakness profiles and benefit from different training environments. Weakness-conditioned adaptation consistently outperforms matched random and hard-example alternatives on ProcTHOR, while the resulting gains for Uni-NaVid and Gemini also extend to HM3D after ProcTHOR adaptation. Iterative re-diagnosis further shows that dominant weaknesses can persist or shift as the policy improves, providing updated guidance for subsequent environment construction. Together, these results show that policy failures are not only diagnostic signals, but actionable guidance for constructing training environments that target a policy's current weaknesses.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
environment generation, weakness-driven, navigation agent

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_eivWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes WEDGE, a framework that uses policy-specific navigation failures to guide training environment generation. An LLM identifies weakness features from successful and failed episodes, which are statistically validated and used to generate targeted environments. The generated environments are filtered based on rollout success rates before policy adaptation. Experiments on DD-PPO, Uni-NaVid, and Gemini show consistent improvements on ProcTHOR and cross-domain transfer to HM3D. The problem is interesting, and the results are promising. However, the methodological novelty and experimental validation have some limitations.

Critical strengths
  1. The paper addresses an interesting problem of generating training environments based on policy-specific failures. 2. The identified weakness features are interpretable and provide measurable descriptions of navigation failures. 3. The method shows consistent improvements across three agents with different adaptation mechanisms. 4. The cross-domain improvements on HM3D demonstrate promising generalization beyond the training simulator.
Critical weaknesses
  1. The methodological novelty is limited. Weakness discovery largely uses predefined simulator primitives, while environment selection and generation rely on statistical filtering, fixed heuristics, and existing procedural operators. The paper also lacks a comparison with simpler feature-selection methods without an LLM. 2. The claim of policy-specific environment generation is not sufficiently validated. The paper does not examine whether training environments targeting one agent's weakness are less effective for other agents. 3. The benefit of weakness-conditioned generation is not fully isolated from difficulty-based selection. A baseline using randomly generated environments with the same rollout-based calibration is missing. 4. The statistical reliability of the results is unclear, as confidence intervals and multiple-seed evaluations are not reported. Some improvements are relatively small, particularly for Gemini on ProcTHOR.
Questions for authors
  1. How does LLM-based weakness discovery compare with directly ranking predefined simulator features? 2. How does WEDGE compare with random environment generation using the same rollout-based calibration? 3. Can the authors evaluate cross-policy weakness conditioning to better support the claim of policy-specific training? 4. How sensitive are the results to the fixed rollout selection threshold of 2–3 successes out of five trials?
Reviewer_AXCFOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes WEDGE, a method for generating training environments around the failures of pretrained navigation agents. It first compares the agent’s successful and failed episodes, then an LLM suggests measurable features that attempt to explain the failures, such as how visible the target is. Each feature is then checked against fixed rules, and the one with the largest gap in success rate (biggest weakness) is chosen. WEDGE then builds new ProcTHOR environments that aim to fall in the range where the agent struggles, keeping only those the agent solves 2-3 / 5 times. Finally, the agent is trained on these environments using its usual learning method, and the process can be repeated. The authors test WEDGE on three agents (DD-PPO, Uni-NaVid and Gemini). WEDGE beats random and hard-example baselines on ProcTHOR for all three. The authors find that the improvements are largest where each agent was weakest and that a second round of diagnosis can find a new weakness.

Critical strengths

S1. WEDGE works with very different agents. DD-PPO uses RL, Uni-NaVid is fine-tuned on its own successes, and Gemini only gets examples in its prompt. The method changes only which environments the agent trains on, not how it learns. For Uni-NaVid and Gemini, the gains also carry over to HM3D.

S2. The weaknesses are well-defined, interpretable, and easy to act on. Each is a measurable quantity, such as how visible the target is or how much of the path is walked before the target comes into view. Each is also checked against clear rules before it is used. Since these quantities have a direct relation to things the simulator can change, the "diagnosis" can lead straight to new training environments.

Critical weaknesses

W1. I am not convinced that the experiments show that targeting a policy’s own weaknesses is what, in fact, helps.

The core idea is that each agent should get training environments built around its own failures. But I don't see this directly tested. The simplest test I could think of would be to swap weaknesses, for example, training Uni-NaVid on environments built for DD-PPO’s weakness and then checking whether it gains less. Without this, the reported gains could come from any reasonably hard, well-structured ProcTHOR data.

The core idea is that each agent should be trained on environments built around its own failures. The paper shows that this kind of targeted training works better than simply using hard examples. Figure 4b also shows that the agents improve most in the situations they were weakest at. But I never see it checked whether a weakness has to be the agent’s own. A simple such test would be to give Uni-NaVid the environments built for DD-PPO’s weakness and see whether it improves less. Without this, the results show that targeted training helps, but not that it necessarily needs to be tailored to each agent.

Selecting training data based on learnability is nothing new (2nd step of WEDGE), so the question is how much the first step matters. For DD-PPO, Table 2 shows that weakness matching alone matters the most. For Uni-NaVid and Gemini, there is no such comparison. Also, the random and hard-example baselines do not appear to use the same difficulty filter. Applying the filter to random environments would show how much of the improvement for these two agents actually comes from the weakness diagnosis.

I am also not very convinced about the claim that different agents have different weaknesses. Uni-NaVid and Gemini end up with the same top weakness. The only agent that differs is DD-PPO, which is also the only one trained in ProcTHOR, so that difference may simply reflect where it was trained.

Last, the weaknesses the LLM finds are almost the same quantities that the authors already listed in their hand-made vocabulary. A simple search over those quantities would likely find the same ones. There is also no comparison showing what the LLM adds.

W2. The performance improvements of WEDGE are very small and could just be noise.

Each result is reported based on a single run, no seeds, no confidence intervals, no significance tests. The gains might just originate from noise. For instance, Gemini improves on ProcTHOR by 2.7 on 300 test episodes. This is about the same size as the sampling error of one success-rate estimate at that scale. Then, the second round of adaptation adds only another 0.2-1.2, which barely supports the claim that re-diagnosis brings further gains.

This Gemini result is especially fragile. One example trajectory gets added to the prompt, hand-picked from 64 successes with no clear rule. We don’t know how much results change with a different example, for WEDGE or for the baselines. The explanation that follows is also generic navigation advice rather than anything specific to the diagnosed weakness.

Despite this, the authors claim that WEDGE "consistently" beats the baselines. With such small variance and no variance, this might as well just be luck.

Questions for authors

Q1. How was the single worked example for Gemini chosen from the 64 successful trajectories?
Q2. How exactly is the adaptation budget matched across methods?
Q3. If we were to simply rank every primitive and simple ratios of primitives by success-rate gap, would we find the same weaknesses as the LLM?
Q4. Do the random and hard-example baselines use the same 2-3 / 5 difficulty filter as WEDGE?

Reviewer_aqZ2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes the method WEDGE using a navigation agent's failures to guide the creation of training environments. A chosen LLM proposes measurements associated with failure, which are checked against diagnostic data and used to modify object placements, scene layouts or starting positions. The method selects environments that are challenging but still solvable and uses the resulting experience to adapt the agent through training or in-context examples. DD-PPO, Gemini and Uni-Navid report improvements over random and hard example baselines on ProcTHOR.

Critical strengths

The evaluation of WEDGE is done across three agents using different adaptation methods which are RL, fine tunning and in-contex examples. The paper reports improvements over random and hard example controls for all three method. It is clearly shown that benefits on HM3D and ProcTHOR adaptation for Uni-Navid and Gemini supporting practical value of the approach beyond a single agent or adaptation method.

Critical weaknesses

1 - LLMs' benefit is not clearly established as several reported weakness are already available in supplied simulator measurements. Target visibility and room transitions are direct inputs, while blind path ratio combines two existing measurements. Also, the paper doesn't compare its discovery procedure with a simpler method that ranks these measurements by their association failure and then uses the same generation and calibarion steps. This is a unclear point for me as I don't conclude that how much the proposed diagnosis contributes beyond a simple feature base curriculum.

2- Table 1 reports improvements over hard example controls 1.4 percentage points for Uni-Navid and 1.9 points for Gemini on ProcTHOR where variation across independent runs is lacking. Also, Gemini's adaptation uses one selected worked example per round without examining sensitivity of that choice. Therefore, I am not certain about reported improvements are reproducible.

Lastly, PROWL: Prioritized Regret-Driven Optimization for World Model Learning (Guzel et al, 2026) is missing from references as it uses actively discovered failures to guide the curriculum for world model improvement.

Questions for authors

1 - Could you compare the LLM-based diagnosis with simply ranking the supplied measurements by their association with failure while keeping envrionment generation and rollout calibration unchanged? I believe this will help the isolation of contribution of proposed discovery method.

2- Can you report uncertainty estimates for the improvements over the hard example baselines ?

3-Are the episodes for final evolution excluded from weakness discovery, worked example selection and adaptation? It's important to clarify how these data are separated.

4- I wonder how was Gemini's worked example selected from the 64 successful trajectories?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.