- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes Q-Rep by learning an geometrically optimal quasimetric representation function of the states for GCRL. By parametrizing the quasimetric function as asymmetric functional, the method is simple in implementation and requires only modification to learning value function. Experiments on the OGBench standard benchmark showcases that method obtains comparable performance to existing methods.
Critical strengths
It is easy to follow the text in the paper. The math notation is the same as in the literature, making it easy to read the paper and theorems. The results on the OGBench showcases that method could outperform existing algorithms.
Critical weaknesses
-
The "Conclusion" section completely misses the limitations of the approach and feels like it is not finished. Basically, it is just 2 sentence summary of the abstract, which completely ignores obtained results, insights made by the paper and, importantly, not discusses limitations.
-
The paper completely ignores vast literature in the same spirit of learning optimal representations in GCRL (see e.g [1, 2, 3]). Those papers claim learning theoretically optimal GCRL representations, so it is hard to understand whether claims on optimality of proposed Q-Rep are indeed novel, since there are no comparisons with existing literature.
-
It is not clear what is the novelty in the Section "3.1 MAIN IDEA". From my understanding, all of the introduced theorems are just reiteration of Park. et al "Dual Goal Representations" with more math rigor. It is not clear for me what is the exact contribution?
-
There are no ablation studies.
[1] Bill Chunyuan Zheng et al. : "Multistep Quasimetric Learning for Scalable Goal-conditioned Reinforcement Learning"
[2] H.Viswanath et al.: "Mollified Value Learning"
[3] V. Giammarino et al.: "Physics-informed Value Learner for Offline Goal-Conditioned Reinforcement Learning"
Questions for authors
Unfortunately, I could not download and inspect code since the page showcases "This content can’t be shown. The repository is expired.". It would be better if author's could fix this.
Questions:
-
Is it correct that the main contribution is in mathematical foundation of quasimetric embeddings? Inspecting Algorithm 1, the only modification is parametrization of , where F is a learned assymetric functional from Section 3.1 by equation 4?
-
What are the limitations of the Q-Rep? What are the examples of the e.g environments, where quasimetric representations are not optimal?
-
Is OGBench is the only established benchmark in the GCRL literature? From my experience, OGBench is very synthetic and does not generally showcases weaknesses of the algorithms.
-
Could the authors clarify what the Exogenous Block CMP (Ex-BCMP) framework contributes to the theory of Q-Rep specifically? Ex-BCMP is borrowed from Efroni et al. (2021) and was already used by Park et al. (2025) to establish noise invariance for Dual Goal Representations.
-
Is there any experiment that tests the Ex-BCMP condition directly? Sec. 5.4 notes that Gaussian goal noise does not, so one example would be goals rendered with different distractors that share the same latent state.
Overall, my main concern is that it is not clear what is the exact novelty and insight the paper introduces. I would consider increasing the score if the authors can answer the questions.