- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper presents ReasAtlas, which organizes mathematical statements from multiple textbooks into a shared hierarchy. The hierarchy is then used to provide retrieval context and supervise embedding training through IC-MCE.
The paper reports some positive results, but I remain unconvinced by its motivation and the necessity of the proposed representation. The experiments support the usefulness of hierarchical information, rather than the broader claim that a shared exposition provides a meaningfully better representation of mathematical knowledge. I do not think the current contribution is sufficiently established for ICLR.
Critical strengths
-
The construction of a large-scale mathematical knowledge hierarchy across 63 textbooks involves considerable effort. Preserving source formulations and provenance is potentially useful.
-
The paper includes reasonably controlled retrieval experiments, particularly the comparisons between flat cross-entropy and IC-MCE, with encouraging improvements across two embedding models.
Critical weaknesses
-
The motivation is not convincing. It is unclear why mathematical knowledge needs to be reorganized into a shared textbook-based hierarchy. The paper does not identify concrete limitations of existing retrieval methods that would justify this additional construction process.
-
The constructed hierarchy lacks convincing validation. The paper claims to align equivalent mathematical knowledge across sources, but provides limited direct evidence that these alignments are accurate. The evaluation largely focuses on topic-level quality rather than the correctness of cross-source statement consolidation.
-
The experiments do not fully support the central claims. Most retrieval queries are generated from statements in the constructed corpus, raising concerns about generalization. While the results show that hierarchical information can improve retrieval, they do not establish why the proposed shared exposition is preferable to simpler alternatives.
Questions for authors
-
The paper evaluates topic-level quality, but how accurate is the cross-source statement consolidation itself? Is there any statement-level human evaluation, especially for formulations with different assumptions?
-
Do the improvements from IC-MCE generalize to independently authored mathematical queries? The Stacks experiments evaluate contextual representations rather than IC-MCE.
-
How does IC-MCE compare with existing hierarchy-aware contrastive learning objectives under the same training and sampling settings?
Thank you for the additional experiments and clarifications. The new evaluation on independently authored mathematical queries and the comparison with HiMulCon address some of my previous concerns. However, I still have some reservations about the overall contribution. While the additional experiments provide useful evidence, I am not yet fully convinced that they sufficiently establish the necessity and broader value of the proposed representation.
Overall, the rebuttal improves the empirical evidence, but does not fully resolve my concerns regarding the motivation and broader significance of the work. I therefore raise my score from 1 to 2.