acceptodds
Under review as a conference paper at ICLR 2027

Task-Conditioned Decision Geometry for World-Model Planning

Abstract

World-model planning turns predicted consequences into actions by comparing candidates for the current task. Yet standard world models learn to predict what each candidate will cause independently of the task, whereas planning depends on task-conditioned differences between candidates. Under a finite representation budget, predictive training can therefore discard a small but decision-critical difference while retaining low prediction error. We address this mismatch with Predictive Co-formation of Task-Induced Geometry (PC-TIG), which jointly shapes a shared representation through task-independent factual prediction and task-conditioned candidate comparison. PC-TIG combines a factual world model with a query-conditioned scorer and supports Joint and prediction-priority updates for both feedback actions and complete action segments. Across five-seed stochastic navigation and multi-objective HalfCheetah experiments, PC-TIG improves task-conditioned closed-loop control and segment selection over matched predictive, value-aware, and geometry-learning baselines while maintaining accurate dynamics. Our results show that effective world models must preserve not only predicted consequences, but also the task-dependent distinctions needed to choose among them.

Primary area:
foundation or frontier models, including LLMs
Keywords:
world model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_6wvgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the mismatch between task-independent prediction and task-conditioned candidate comparison in world-model planning. PC-TIG trains a shared representation with a factual prediction branch and a query-conditioned scorer, using either joint updates or prediction-priority updates. The feedback realization combines risk-sensitive Bellman targets with exact dynamic-programming anchors, while the pathwise realization combines prefix utility regression with a maximum candidate-error term. Evaluation covers five-seed stochastic navigation and fixed-candidate multi-objective HalfCheetah segment selection, with fresh-head representation attribution and an additional single-development-seed online compatibility study.

Critical strengths
  1. The evaluation makes useful efforts to isolate representation effects. Methods compare the same candidates, navigation policies are evaluated under the exact true transition kernel, and fresh identically trained heads are fitted to frozen representations. The latter retains a PC-TIG advantage, providing evidence beyond a better original scorer (Table 4 and Appendix E). This is consequential because representation co-formation is the central claim.
  2. The fixed-candidate decision improvements are supported by multiple metrics and five training seeds. On HalfCheetah, Joint reduces preference regret from 0.560 for Latent-Space VA to 0.216, while improving top-1 selection and ranking metrics (Table 3). Table 17 reports a paired regret advantage of 0.3443 with a positive interval and all five runs favoring Joint. These are meaningful gains within the evaluated setting.
Critical weaknesses
  1. The distinctive contribution is not sufficiently isolated from ordinary task-supervised joint representation learning. Equation 28 combines prefix utility regression and a maximum final-candidate error term, but Table 3 lacks the direct control that preserves the residual scorer, factual supervision, training data, encoder task gradients and update budget while removing only the maximum-error term. Appendix G detaches the scorer's task loss at the representation input for the adapted baselines. The existing results therefore support the value of task gradients shaping representations more directly than they establish the additional benefit of the proposed comparison objective. Navigation has a useful Value-aware Joint control, but its Bellman and optimization treatment differs. Furthermore, the navigation prediction-only comparator scores latent Euclidean distance (Appendix C), rather than applying the actual risk/energy/horizon objective to the learned physical kernel. A task-specific DP planner on the learned kernel would be a directly relevant comparator. The theory does not resolve these issues. Proposition 1 concerns coordinate-Euclidean scores, Proposition 2 concerns query-independent scorers, and Theorem 1 is a general cost-estimation perturbation bound. None establishes a specific necessity for learning PC-TIG's geometry rather than evaluating predicted consequences with the known task functional.

  2. The evidence for preserving factual prediction on unseen trajectories needs clarification. Appendix H defines BF as 32 training instances repeatedly used for factual optimization and feasibility checks, and the final paragraph of p. 21 states that these reference instances define the prediction measure. It is unclear whether Table 3's prediction loss is also evaluated on BF. If so, Feasible's lowest loss establishes protection on the optimized reference set rather than held-out factual prediction. Please identify the evaluation set and provide observation, reward and cumulative-return errors on the 1,024 test instances. Separately, the feedback feasibility proof concerns raw Euclidean gradients, while Appendix C specifies AdamW. The raw-gradient inner product need not remain compatible after adaptive preconditioning, momentum and weight decay. Table 12's projected-gradient dot products do not establish compatibility of the actual parameter displacement. This is a clarification of the claimed guarantee, not a rejection of the measured transition accuracy.

Questions for authors
  1. Which examples define the prediction-loss rows in Tables 3 and 17? Are they the repeatedly optimized BF instances? Can you report the same metric and its observation/reward/return components on the held-out test instances?
  2. What is the HalfCheetah result when only the maximum-candidate error term is removed, with the scorer, factual objective, encoder task gradients, samples and budget otherwise unchanged?
  3. How does navigation perform when the same task-specific DP procedure operates on the learned physical transition kernel? Also, which MAD planner produced Table 2, given the different scorer descriptions in Appendices C and G?
  4. Does feedback Feasible project the actual AdamW displacement or only the raw gradient? Please clarify the guarantee and provide actual-step factual directional derivatives or loss-change diagnostics.
3 comments
AuthorsOpenReview ↗

Follow-up on Additional Evidence and Further Discussion

Dear Reviewer 6wvg,

Thank you again for your detailed and constructive feedback. Following your suggestions, we conducted several targeted experiments, including the matched maximum-error ablation, a common task-specific DP planner comparison, held-out factual prediction evaluation, and actual AdamW-step diagnostics. These additions directly address your concerns about the distinctive contribution of PC-TIG, evaluation fairness, and factual preservation.

As the ICLR rebuttal discussion period is approaching its end, we would sincerely appreciate the opportunity to hear your thoughts on whether these new results and clarifications have addressed your main concerns. We would be very happy to discuss any remaining questions.

Thank you very much for your time and consideration.

Dear Reviewer 6wvg, we would be grateful if you could let us know whether our responses have addressed your concerns, and we are happy to provide any further clarification before the discussion period ends.

Dear Reviewer 6wvg,

With only a few hours remaining in the discussion period, we sincerely hope you could take a moment to review our rebuttal and additional experiments. We would be extremely grateful for your feedback and the opportunity to address any remaining concerns before the discussion closes.

Thank you very much for your time and consideration.

Reviewer_p7XUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper addresses a mismatch between predictive accuracy and decision quality in world-model planning. A world model predicts the consequences of candidate actions, but planning depends on how those consequences compare under a particular task. The authors formalize and leverage these task-dependent candidate rankings and pairwise cost gaps as task-conditioned decision geometry.

The proposed PC-TIG framework co-forms factual prediction and decision geometry within a shared representation: a task-independent predictive branch preserves candidate consequences, while a query-conditioned scoring branch shapes the representation toward distinctions relevant for planning. Joint and Feasible offer different ways to reconcile these objectives: either optimizing them together or prioritizing factual prediction. The principle is instantiated for both feedback planning through Bellman recursion and complete-segment comparison through trajectory utilities. Experiments on stochastic navigation and multi-objective HalfCheetah show improved task-conditioned decisions compared with matched baselines.

Critical strengths
  1. A coherent connection between prediction and decision-making. The central idea is clearly motivated and carried through into the optimization framework. In particular, the Feasible updates provide a principled way of allowing decision supervision to shape the representation while restricting changes that conflict with factual learning. The distinction between first-order gradient compatibility and finite-step feasibility is technically meaningful.

  2. A unified principle across different planning interfaces. Developing both feedback and pathwise realizations strengthens the contribution. The distinction between single-action candidates requiring future replanning and complete segments with explicit consequences is well motivated.

Critical weaknesses
  1. Dependence on privileged decision supervision. In the feedback realization, the decision objective relies partly on exact dynamic-programming cost-to-go targets computed from the known transition kernel. Such supervision is generally unavailable in settings where learning a world model is most valuable. Without testing weaker or approximate targets, it is difficult to establish how much of the improvement comes from co-formation itself rather than access to privileged value information.

  2. The navigation environment is too simple. World models should target environments that are hard to model, and although navigation provides a visual perceptual stream, its discrete state and action spaces remove most of that difficulty: the representation never has to resolve continuous-state ambiguity. A continuous-state variant would be a far stronger test of co-formation.

Questions for authors
  1. Does co-formation remain effective without exact-DP supervision? In particular, do the improvements persist when you remove the exact-value anchor or replace it with approximate or bootstrapped targets?

  2. How complex is the perceptual stream for the navigation experiments? The task may not be sufficiently hard to leverage this approach; consider moving to continuous navigation rather than a grid world.

Minor clarifications that won't affect my assessment: the relationship between the general candidate-cost notation and the feedback value targets could be clearer. It would also help to define explicitly how trajectory prefixes are used for scorer supervision, and to distinguish the realization-specific objectives in Algorithm 1.

4 comments
AuthorsOpenReview ↗

Follow-up on Exact-DP Independence and Continuous-Control Evidence

Dear Reviewer p7XU,

Thank you again for your valuable feedback and recognition of our work. Following your suggestions, we evaluated PC-TIG with reduced and zero exact-DP anchor weights, confirming that the learned-successor Bellman objective can support feedback planning without privileged exact-value supervision. We also clarified the complementary evidence from continuous-state/action HalfCheetah experiments and TD-MPC2 online control, together with their respective evaluation scopes.

As the ICLR rebuttal discussion period is approaching its end, we would sincerely appreciate any further thoughts on whether these additions address your main concerns. We would welcome the opportunity to discuss any remaining questions or suggestions.

Thank you very much for your time and consideration.

I thank the authors for addressing my concerns conscientiously and in detail.

The additional experiments in Table P1 sufficiently address my first concern (W1) and the corresponding question (Q1). This increases my confidence in the method's applicability beyond settings with privileged supervision.

However, my second concern (W2) remains. While I appreciate the separate evaluations on continuous control and discrete navigation, I believe the real challenge for world models lies in combining visual or semantic complexity with continuous control. I recognize that such an evaluation is difficult, but consider it important for demonstrating practical relevance. If the authors believe their existing experiments sufficiently address this concern or my concern is unrealistic, I would welcome further discussion.

Overall, I appreciate the additional evidence, which would otherwise motivate a higher rating. However, I will maintain my current evaluation, as I believe stronger acceptance recommendations should be reserved for work that also demonstrates effectiveness in the before mentioned "realistic" scenarios.

AuthorsOpenReview ↗

Replying to Follow-up on Continuous Visual Control Evidence

Thank you for clarifying W2. We understand the remaining concern: the agent should handle visual observations and continuous control together, rather than in two separate experiments. We therefore added a new closed-loop evaluation that directly combines them.

We use PointMaze-Teleport, a visual continuous-control environment based on the PointMaze family in OGBench [1]. The agent receives only 64×64 RGB images and must infer its location and the maze structure from pixels. It then navigates using continuous 2D actions while accounting for the environment's stochastic teleport transition. At every decision point, the learned world model predicts future outcomes and iCEM replans from the latest observation. All methods use the same planner and are evaluated on the same 60 episodes across three seeds.

MethodSuccessesSuccessGain over E0
Prediction-only world model (E0)29/6048.3%—
MAD-style objective33/6055.0%+6.7 pp
Frozen encoder + task head34/6056.7%+8.4 pp
PC-TIG37/6061.7%+13.4 pp

PC-TIG raises closed-loop success from 48.3% to 61.7% over the prediction-only world model. Since the test episodes are paired, we can compare the two methods episode by episode: PC-TIG succeeds on eight episodes where E0 fails, while E0 has no unique success. This improvement is statistically significant (two-sided exact paired test, p=0.0078p=0.0078).

This experiment directly evaluates the setting described in W2 within one task: the model must extract task-relevant structure from RGB observations, predict the effects of continuous actions, handle a stochastic transition, and replan online. The controls also help locate the source of the improvement. Using a MAD-style objective reaches 55.0%, and adding a task head to a frozen visual encoder reaches 56.7%; jointly adapting the representation with PC-TIG reaches 61.7%. Together, these results show that PC-TIG remains effective when visual perception and continuous model-based control are required at the same time.

Thank you for pointing out this missing evaluation. We hope this direct evidence resolves W2 and would appreciate your reconsideration of the rating.

[1] Park S, Frans K, Eysenbach B, et al. Ogbench: Benchmarking offline goal-conditioned rl[C]//International Conference on Learning Representations. 2025, 2025: 94937-94982.

AuthorsOpenReview ↗

PC-TIG (#17407): Request for Follow-up Response from Reviewer p7XU

Dear Reviewer p7XU,

Thank you again for your thoughtful follow-up and for clarifying your remaining concern.

We have since added a new PointMaze-Teleport experiment that directly combines visual observations and continuous control, as you suggested. As the discussion period is nearing its end, we sincerely hope you could take a moment to review this new evidence and share your thoughts on whether it addresses W2.

We greatly appreciate your continued engagement and consideration.

Reviewer_MnmiOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how world-model representations can preserve distinctions that matter for decisions under different tasks. PC-TIG combines factual prediction with a query-conditioned scorer, using either joint optimization or prediction-priority updates. Experiments on stochastic navigation and multi-objective HalfCheetah show improved candidate selection while retaining predictive accuracy. A fresh-head evaluation provides additional evidence that the learned representation contributes to the gains.

Critical strengths
  1. The separation between factual prediction and task-dependent comparison is well motivated. The method provides a concrete way to train a reusable predictive representation for changing preferences, and the action-gap analysis helps explain when scoring errors affect the selected action.

  2. The controlled experiments are a strong part of the paper. In HalfCheetah, Joint reduces preference regret from 0.560 for Latent-Space VA to 0.216, with improvements in ranking metrics as well. I also found the fresh-head experiment useful: retaining an advantage after retraining identical scorers makes the representation-level explanation more convincing.

Critical weaknesses
  1. The specific contribution of the comparison objective is not fully isolated. Equation (9) combines prefix utility regression with a maximum-candidate error penalty, but the HalfCheetah results do not include an otherwise identical Joint model without that penalty. This leaves some uncertainty about what the proposed objective adds beyond joint prediction and query-conditioned utility regression.

  2. The evidence for broader planning applications remains limited. Navigation uses exact-DP supervision, while the main HalfCheetah experiment ranks fixed candidate segments. Appendix A provides useful online-control results, but only for one development seed. These limitations do not invalidate the controlled findings, though the broader claims should be stated more cautiously.

Questions for authors
  1. How does HalfCheetah performance change when only the maximum-error term in Eq. (9) is removed, keeping the remaining setup fixed?

  2. How sensitive are the navigation improvements to the exact-DP anchor? Any existing results with a reduced anchor weight would help clarify this dependence.

2 comments
AuthorsOpenReview ↗

Follow-up on Maximum-Error Ablation and Planning Evaluation

Dear Reviewer Mnmi,

Thank you again for your thoughtful review and encouraging assessment of our work. In response to your specific suggestions, we conducted a matched ablation showing that the maximum-error term reduces HalfCheetah preference regret from 0.339 to 0.215, and additional experiments demonstrating that PC-TIG remains effective with reduced or zero exact-DP supervision. We also clarified the scope of our planning evaluations.

As the ICLR rebuttal discussion period is approaching its end, we would be very grateful for any further thoughts you might have on these additional results. We sincerely welcome the opportunity for further discussion.

Thank you very much for your time and consideration.

AuthorsOpenReview ↗

PC-TIG (#17407): Request for Response from Reviewer Mnmi

Dear Reviewer Mnmi,

As the discussion period is nearing its end, we sincerely hope to have the opportunity to discuss our rebuttal with you, particularly the new maximum-error ablation and experiments without exact-DP supervision.

We would greatly appreciate it if you could review these results and let us know whether they address your concerns. We look forward to hearing your thoughts.

Thank you very much for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary: Key Revisions and Additional Evidence

We sincerely thank all reviewers for their thoughtful and constructive feedback. In response, we have conducted targeted experiments and clarified the theoretical motivation, optimization guarantees, and evaluation protocols. The main improvements are summarized below.

  1. Isolating the contribution of the maximum-error objective. We conducted a matched HalfCheetah ablation that removes only the maximum-candidate error term while keeping all other training components identical. The complete PC-TIG objective reduces preference regret from 0.339 to 0.215 (36.6% improvement) and increases top-1 accuracy from 0.696 to 0.771 across five seeds. This directly demonstrates the additional benefit of candidate-wise error control beyond ordinary task-supervised joint learning.

  2. Establishing improvements under identical planning procedures. We evaluated all methods using the same task-specific dynamic-programming planner on their learned physical transition kernels. PC-TIG Joint reduces closed-loop regret from 8.28 to 5.47 in Risk (33.9%) and from 7.92 to 5.87 in Bottleneck (25.9%) relative to Prediction-only. These results confirm that the decision advantage persists when the planner is held fixed.

  3. Evaluating dependence on exact-DP supervision. We reduced and removed the exact-DP anchor while keeping the learned-successor Bellman objective and training configuration unchanged. PC-TIG continues to learn useful feedback policies without exact-DP supervision. In Risk, reducing the anchor weight from 1.5 to 0.25 improves regret from 15.79 to 13.53. The HalfCheetah experiments provide complementary evidence using observed trajectories and rewards without a known transition kernel or exact-DP targets.

  4. Verifying factual prediction on unseen trajectories. We clarified that the reported prediction losses are evaluated on 1,024 held-out test instances rather than the 32 training reference instances. Additional evaluations across observation, reward, and cumulative-return prediction show that PC-TIG Feasible achieves the lowest aggregate held-out error (0.162), compared with 0.210 for Latent-Space VA. PC-TIG Joint also improves all three factual prediction components.

  5. Clarifying optimization guarantees under AdamW. We distinguished first-order projected-gradient compatibility from the behavior of actual optimizer updates. An additional AdamW-step audit identifies the finite-step limitations of the submitted Feasible update. We further evaluated a transactional extension that verifies proposed parameter displacements before acceptance, observing zero factual-loss increases across 8,000 audited steps while maintaining comparable planning regret.

  6. Clarifying the scope of planning evaluations and theoretical contributions. We made the connection between the decision-error bound, the maximum-error objective, and task-conditioned candidate selection more explicit. We also distinguished the evidence from closed-loop stochastic navigation, continuous-state/action HalfCheetah segment selection, and the development-seed TD-MPC2 online-control experiments, clarifying the scope and limitations of each evaluation.

Together, these additional results strengthen the central claim that jointly learning factual prediction and task-conditioned decision geometry improves decision quality beyond conventional predictive or task-supervised representations, while preserving factual prediction performance in the evaluated settings.

We sincerely appreciate the reviewers' time and consideration. We hope these additional experiments and clarifications help address the concerns raised during the review process, and we would be very grateful for any further comments or discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.