- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Building on the closed-form semi-relaxed OT Bellman backup of Dam (2026), the paper studies Laplacian OT (LOT) regularization, whose backup is a mixture of cost-shifted local softmaxes indexed by an anchor-normalized action kernel. It then catalogues which kernel properties yield which guarantees: concentration gives a pruning bound and a localized planner (LocalLOTPlan) whose complexity depends on an effective action count rather than ; a Poisson-truncated heat kernel gives local computation; uniformly stable costs give reward-free geometry learning and tabular Q-learning convergence; and strictly positive couplings let inverse policy curvature be read as effective resistance on a value-dependent graph. The submission spans 151 pages with 23 theorems. Validation is limited to synthetic numerical illustrations on path, cycle, and ring graphs with up to actions and small tabular MDPs, all reported in the appendix.
Critical strengths
None. The closed-form analysis is internally consistent, but I could not identify a strength that bears on acceptance at ICLR given the concerns below.
Critical weaknesses
Format and verifiability: The paper is a 151-page monograph with 23 theorems and roughly 70 supporting results, whose main text largely states results and defers definitions, algorithms, and all figures to the appendix. Its correctness cannot be reasonably verified within a conference review, and the 9-page body is not self-contained.
No evidence of relevance to RL practice: The only experiments are controlled illustrations on synthetic graphs and toy tabular MDPs with known transition kernels. No learning agent, standard benchmark, or comparison with existing RL methods is reported, so it remains unknown whether action geometry "buys" anything in practice.
Limited scope of the guarantees: By the authors' own statements, planning bounds target a scalar regularized root value rather than policy return, learning results are tabular, and the effective-resistance identity yields neither an algorithm nor a rate. Combined with the reliance on Dam (2026) for the core closed form, the contribution reduces to a catalogue of properties.
Questions for authors
-
Beyond Lemma 1 and Theorem 1, which results are new relative to Dam (2026)? Please identify the one or two theorems you consider the core contribution.
-
Can you show on any standard discrete-action RL benchmark (e.g., discretized control or large-action recommendation tasks) that an LOT-based agent improves return or sample efficiency over MaxEnt baselines?
-
All experiments assume a known kernel and exact expected- evaluations. How do the planning guarantees behave with function approximation or sampled transitions?
-
Assumption 2 requires local kernel access in time. In which practical action spaces is such access available without scanning all actions?
-
Given its length, the work may be better suited to a journal (e.g., JMLR), where the full proofs can be properly refereed.
We have decided to withdraw our submission from ICLR 2027 to revise the presentation, clarify the scope of our guarantees, and strengthen the empirical evaluation. We thank the reviewers and area chair for their time and feedback.