- Weak contribution arising from backbone selection and benchmarking results.
LCPP builds upon many prior approaches: expectile regression and advantage-weighted policy extraction (IQL, AWR); world model architecture (LeWM). Hence, the authors provide a formal investigation of extraction reliability, where the critic encoder is decoupled from the actor's encoder by reusing the frozen world model encoder (LeWM) for the critic while training the actor's encoder from scratch. However, this investigation assumes that the evaluation goals lie 50 transition steps from the initial state, which establishes the choice of hyperparameter (e.g., Equation 3, Remarks 2 and 3) used in the analyses, limiting the impact of the formal results on the specialized evaluation suite.
Moreover, the benchmarking results from the stable-worldmodel benchmark are unreliable. By referring to Section 4.1 and the anonymous GitHub from the authors, I found that LCPP is trained and evaluated with the stable-worldmodel benchmark. LeWM evaluated performance with the offline dataset used for training to initialize the reachable pairs of the initial state and future goal. While it is common practice in offline GCRL, the stable-worldmodel benchmark does not filter out pairs that have already succeeded. This means that the agent will succeed at whatever it does since the environment is initialized in the successful states. To empirically demonstrate that, I compare the scores of the official LeWM paper, reproduced with the same evaluation setup, and reproduced with a success-filtered evaluation setup below:
Table 1: success rates w/ eval_goals setup (eval_budget=50, goal_offset=50, same episode and start indices)
| task | official LeWM score | reproduced | already-succeeded pairs | reproduced w/ filtering (n: num_envs=num_total - num_filtered) |
|---|
| PushT | 96 | 32.0 | 0 | 32.0 (n=50) |
| TwoRoom | 87 | 56.0 | 2 | 54.2 (n=48) |
| Cube | 74 | 60.0 | 14 | 44.4 (n=36) |
| Reacher | 86 | 84.0 | 1 | 83.7 (n=49) |
Table 2: success rates w/ LeWM's setup (eval_budget=50, goal_offset=25, random episodes and starts, seeds 42/43/44)
| task | official LeWM score | reproduced | already-succeeded pairs (across seeds) | reproduced w/ filtering over seeds |
|---|
| PushT | 96 | 90.0 | 0/0/0 | 90.0 ± 3.3 |
| TwoRoom | 87 | 85.3 | 3/6/4 | 83.9 ± 3.1 |
| Cube | 74 | 70.0 | 17/20/19 | 51.9 ± 12.0 |
| Reacher | 86 | 86.7 | 0/0/0 | 86.7 ± 4.1 |
where I followed the same evaluation setup reported by the authors and LeWM's setup, and I computed averaged success rates over episodes by removing filtered, already-succeeded episodes. As a result, PushT and Reacher tasks do not involve already-succeeded pairs at all, while Cube and TwoRoom tasks include a few pairs. One can find that reproduction of LeWM is not reliable for either the stable-worldmodel protocol or the authors' protocol. Based on the above reproduced results, I feel the experimental results of LCPP are scientifically unreliable and overclaimed, inheriting the limitations from the stable-worldmodel benchmark. Regarding baseline selection, I agree that all references need to be published in peer-reviewed conference/journals in world model research since the research evolves too fast. However, selecting world model baselines that are not peer-reviewed for primary experimental investigations based on unreliable experimental setup (no filtering) can mislead not only future readers but also AI researchers.
- Unclear problem formulation and presentation.
Throughout the manuscript, it requires substantial time to understand the problem formulation the authors have tackled and the motivation for decoupling encoders. LCPP enables offline GCRL with zero inference cost of a pretrained world model during test-time by decoupling encoders under an actor-critic framework, which is the most critical benefit of LCPP, I think. There are extensive papers on independently parameterizing the actor and critic encoders in model-free RL [1-4] or removing inference-time cost of world models [5-7]. However, the current manuscript lacks the connection to that literature. Additionally, the manuscript simply lists the prior approaches with their own explanations without logical connections to LCPP, which does not explain the motivation for decoupling the actor and critic encoders for world models. Furthermore, the readability of the manuscript should be improved significantly. I summarized the remaining questions regarding presentation in the question section below.
Reference
[1] Garcin, Samuel, et al. "Studying the interplay between the actor and critic representations in reinforcement learning." ICLR 2025
[2] Ni, Tianwei, Benjamin Eysenbach, and Ruslan Salakhutdinov. "Recurrent model-free RL can be a strong baseline for many POMDPs." ICML 2022
[3] Raileanu, Roberta, and Rob Fergus. "Decoupling value and policy for generalization in reinforcement learning." ICML 2021
[4] Cobbe, Karl W., et al. "Phasic policy gradient." ICML 2021
[5] Nguyen, Hoang, Xiaohao Xu, and Xiaonan Huang. "Latent Geometry Beyond Search: Amortizing Planning in World Models." arXiv 2026
[6] Georgiev, Ignat, et al. "PWM: Policy learning with multi-task world models." ICLR 2025
[7] Zhao, Yi, et al. "Simplified temporal consistency reinforcement learning." ICML 2023