- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies the mismatch between task-independent prediction and task-conditioned candidate comparison in world-model planning. PC-TIG trains a shared representation with a factual prediction branch and a query-conditioned scorer, using either joint updates or prediction-priority updates. The feedback realization combines risk-sensitive Bellman targets with exact dynamic-programming anchors, while the pathwise realization combines prefix utility regression with a maximum candidate-error term. Evaluation covers five-seed stochastic navigation and fixed-candidate multi-objective HalfCheetah segment selection, with fresh-head representation attribution and an additional single-development-seed online compatibility study.
Critical strengths
- The evaluation makes useful efforts to isolate representation effects. Methods compare the same candidates, navigation policies are evaluated under the exact true transition kernel, and fresh identically trained heads are fitted to frozen representations. The latter retains a PC-TIG advantage, providing evidence beyond a better original scorer (Table 4 and Appendix E). This is consequential because representation co-formation is the central claim.
- The fixed-candidate decision improvements are supported by multiple metrics and five training seeds. On HalfCheetah, Joint reduces preference regret from 0.560 for Latent-Space VA to 0.216, while improving top-1 selection and ranking metrics (Table 3). Table 17 reports a paired regret advantage of 0.3443 with a positive interval and all five runs favoring Joint. These are meaningful gains within the evaluated setting.
Critical weaknesses
-
The distinctive contribution is not sufficiently isolated from ordinary task-supervised joint representation learning. Equation 28 combines prefix utility regression and a maximum final-candidate error term, but Table 3 lacks the direct control that preserves the residual scorer, factual supervision, training data, encoder task gradients and update budget while removing only the maximum-error term. Appendix G detaches the scorer's task loss at the representation input for the adapted baselines. The existing results therefore support the value of task gradients shaping representations more directly than they establish the additional benefit of the proposed comparison objective. Navigation has a useful Value-aware Joint control, but its Bellman and optimization treatment differs. Furthermore, the navigation prediction-only comparator scores latent Euclidean distance (Appendix C), rather than applying the actual risk/energy/horizon objective to the learned physical kernel. A task-specific DP planner on the learned kernel would be a directly relevant comparator. The theory does not resolve these issues. Proposition 1 concerns coordinate-Euclidean scores, Proposition 2 concerns query-independent scorers, and Theorem 1 is a general cost-estimation perturbation bound. None establishes a specific necessity for learning PC-TIG's geometry rather than evaluating predicted consequences with the known task functional.
-
The evidence for preserving factual prediction on unseen trajectories needs clarification. Appendix H defines BF as 32 training instances repeatedly used for factual optimization and feasibility checks, and the final paragraph of p. 21 states that these reference instances define the prediction measure. It is unclear whether Table 3's prediction loss is also evaluated on BF. If so, Feasible's lowest loss establishes protection on the optimized reference set rather than held-out factual prediction. Please identify the evaluation set and provide observation, reward and cumulative-return errors on the 1,024 test instances. Separately, the feedback feasibility proof concerns raw Euclidean gradients, while Appendix C specifies AdamW. The raw-gradient inner product need not remain compatible after adaptive preconditioning, momentum and weight decay. Table 12's projected-gradient dot products do not establish compatibility of the actual parameter displacement. This is a clarification of the claimed guarantee, not a rejection of the measured transition accuracy.
Questions for authors
- Which examples define the prediction-loss rows in Tables 3 and 17? Are they the repeatedly optimized BF instances? Can you report the same metric and its observation/reward/return components on the held-out test instances?
- What is the HalfCheetah result when only the maximum-candidate error term is removed, with the scorer, factual objective, encoder task gradients, samples and budget otherwise unchanged?
- How does navigation perform when the same task-specific DP procedure operates on the learned physical transition kernel? Also, which MAD planner produced Table 2, given the different scorer descriptions in Appendices C and G?
- Does feedback Feasible project the actual AdamW displacement or only the raw gradient? Please clarify the guarantee and provide actual-step factual directional derivatives or loss-change diagnostics.
3 comments
Dear Reviewer 6wvg, we would be grateful if you could let us know whether our responses have addressed your concerns, and we are happy to provide any further clarification before the discussion period ends.
Dear Reviewer 6wvg,
With only a few hours remaining in the discussion period, we sincerely hope you could take a moment to review our rebuttal and additional experiments. We would be extremely grateful for your feedback and the opportunity to address any remaining concerns before the discussion closes.
Thank you very much for your time and consideration.
Follow-up on Additional Evidence and Further Discussion
Dear Reviewer 6wvg,
Thank you again for your detailed and constructive feedback. Following your suggestions, we conducted several targeted experiments, including the matched maximum-error ablation, a common task-specific DP planner comparison, held-out factual prediction evaluation, and actual AdamW-step diagnostics. These additions directly address your concerns about the distinctive contribution of PC-TIG, evaluation fairness, and factual preservation.
As the ICLR rebuttal discussion period is approaching its end, we would sincerely appreciate the opportunity to hear your thoughts on whether these new results and clarifications have addressed your main concerns. We would be very happy to discuss any remaining questions.
Thank you very much for your time and consideration.