- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates whether the place-cell model of Zhao et al. (2025), non-negative embeddings of multi-scale random-walk transition kernels, can serve as an effective planner when fitted to offline trajectories rather than an analytical ground-truth map. The proposed method discretizes visited positions into a spatial grid, trains a coordinate network on the multi-scale kernel over this grid, and plans via sampled ascent on the learned similarity representation following Lin et al. (2026), termed Preplay.
Critical strengths
- The paper explains why the greedy read-out of Zhao et al. (2025) fails when the kernel is fitted to offline data.
Critical weaknesses
Overall, as the authors disclosed in the AI use statement, the paper appears to be mostly written by AI, which is allowed at this conference. However, the authors should have revised the paper for better readability and chosen proper experiments.
-
First of all, the paper is very hard to read, and the writing must be improved.
1.1 Almost every sentence contains a colon or a semicolon (157 colons and 103 semicolons, including the semicolons in citations), e.g.,
The pair weight is not: removing ...in L184 andThe map is a network: a shared trunk ...in L170.1.2 The symbol in Figure 1 is not defined. I recommend making each figure and its caption self-contained.
1.3 Several sentences are not written in an academic tone, e.g.,
Table 2 varies nothing but where the waypoints come from, ...in L373.1.4 In L99, it is hard to understand what
14–7–3means.1.5 In L107,
Because the analysis needed it, and the analysis is what we defend.is not grammatically correct.1.6 The first sentences of many paragraphs are hard to understand. For example, the authors begin Section 3.1 with
Only the planning is search-free, not the pipeline.(L160). However, it is unclear what the authors want to argue.1.7 Some terms are not properly explained, such as
cov. Although readers can infer that it is related tocoverage, it is better to give the full name when a term first appears.1.8 The free set is used in L161 before it is defined in L166.
-
The paper heavily relies on Zhao et al. (2025), but there is no visualization or analysis of place cells.
-
The novelty of the paper compared with Zhao et al. (2025) and Lin et al. (2026) is not well addressed, which could be due to the writing. If the authors clearly explain the novelty, it would be easier to understand the contribution of the paper.
-
The authors omitted the count-based exploration literature. Equation 6 is exactly the same as the standard count-based exploration bonus in reinforcement learning [R1, R2], but the authors did not acknowledge it. Also, regarding the choice of , the authors state
We report $\alpha = 1/2$ because it was the first value we tried(L432). This is not a valid reason for choosing a hyperparameter. -
Multiple experiments in the Appendix are not well justified, and most of them might be unnecessary.
5.1 The main argument of the paper is related to neither hardware nor inference time. Hence, I do not understand the contribution of Table A3, which compares the evaluation on CPU and GPU.
5.2 The authors show the results of each seed in Table A4, which is appreciated. However, it would be better to explain why seeds 7, 14, and 1234 were chosen. Otherwise, I recommend simply labeling them Seed 1, Seed 2, and Seed 3.
-
The comparison with the OGBench baselines in Table 1 is not fair. Preplay discretizes the x-y positions of the state (L165-167), which the baselines (e.g., HIQL) do not use.
Minor In L354, 'like for like' -> 'like-for-like'.
[R1] Bellemare, Marc, et al. "Unifying count-based exploration and intrinsic motivation." NeurIPS. 2016.
[R2] Tang, Haoran, et al. "#Exploration: A study of count-based exploration for deep reinforcement learning." NeurIPS. 2017.
Questions for authors
N/A