- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes HyperGuide, which embeds reasoning prefixes in hyperbolic space and guides LLM generation through continuous directional signals. The first stage constructs prefix trees from sampled trajectories and learns state representations. The second stage trains a direction prediction head and finetunes the LLM through LoRA. The predicted direction is converted into a virtual token to guide generation. The authors evaluate accuracy and generated token counts on mathematics and coding tasks. They also investigate the mechanism through experiments.
Critical strengths
-
The paper is well organized, and the two-stage training framework is easy to follow. Representing reasoning branches in hyperbolic space for generation is an interesting idea.
-
Despite its relatively complex training procedure, the method has practical value. It derives step-level preferences from final-answer verification without manual annotations of intermediate steps.
Critical weaknesses
-
The motivation for introducing geometric guidance is underdeveloped. The paper explains why hyperbolic space is suitable for embedding trees. However, it does not sufficiently explain why reasoning should benefit from this additional embedding stage. See Question 4.
-
Prefix-tree distance may not reflect semantics. Tree distance only depends on generation history and may be affected by wording and step segmentation. §5.4 verifies structural preservation but does not examine semantic consistency. See Question 2, 3.
-
Several design choices lack sufficient motivation. In §4.2, uses a margin proportional to the depth gap, while uses a fixed margin. The paper does not explain why these objectives require different treatments or how this choice affects the learned geometry. See Question 5.
I think the relationship between semantics and geometry is the most important. If authors can show the alignment between tree structure, geometric distance, and semantic similarity, I would consider to increase the score.
Questions for authors
-
The Poincaré ball is invariant under global rotations. Could changes in random seeds across generation rounds cause the model to switch between different but equivalent embedding configurations, leading to inconsistent guidance?
-
How should the relationship between semantics and geometry be understood? Consider 4 sampled trajectories , where express the same solution method in different wording. If they diverge early, could the objectives place the two trajectories far apart in hyperbolic space, while placing some closer together? like:
b a o a bShared prefixes could even encourage this layout if share an opening unrelated to the solution. Could some experiments test whether the geometry reflects semantic similarity? Besides, how should the “unreliability” of long-range relationships mentioned in Appendix B.5 be interpreted?
-
The experiments in §5.5 show that the predicted direction is more effective than its negation. However, the objectives do not require higher-cost candidates to lie in the opposite direction from lower-cost candidates. Thus, need not correspond to a worse candidate; it could just represent another solution method. Could the authors clarify the meaning of this intervention?
-
The method mainly estimates costs through rollouts and increases the model’s preference for lower-cost steps. Is it therefore better suited to tasks where successful trajectories are easy to sample? Could it face limitations similar to conventional RL on unfamiliar problems (such as open problems in mathematics) even when some expert demonstrations data are available? If so, some baseline comparisons on RL methods (e.g. DPO) would be helpful.
-
What motivates the different margin designs in Eqs. (1) and (2)? Equation (1) uses , whereas Eq. (2) uses a fixed regardless of the gap . Could the authors clarify the rationale and compare fixed versus gap-dependent margins?