- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Veri-GPS to train LLMs from symbolic solver feedback to organize subgoals across geometric proof and calculation within a bounded interaction budget. The authors extend the Newclid symbolic solver and introduce Goal-related Diagnostics to identify missing premises. They combine Teacher–Solver Interaction trajectories with Goal Augmentation for supervised fine-tuning (SFT), followed by Dynamic-Difficulty Reinforcement Learning (DDRL) to further optimize the subgoal-selection policy. Experiments on 3,082 problems across five geometry benchmarks achieve an overall pass@1 of 82.12%.
Critical strengths
- A relatively complete neural-symbolic collaborative solving framework. This paper expands the Newclid backend by combining geometric proofs with exact calculations and introducing goal-related diagnostics to provide feedback for subgoal selection and revision in the model. This design allows the system to make use of verified intermediate results over multiple interactions, offering a useful framework for geometric reasoning.
- Strong experimental performance across multiple benchmarks. This paper achieved an overall pass@1 of 82.12% on 3,082 problems from five geometry datasets, which is an improvement compared to the standalone symbolic solver.
Critical weaknesses
There's a lack of a simple baseline for diagnostic-guided planning. While the paper shows improvements over a standalone symbolic solver, it does not compare the learned subgoal-selection policy with a simple heuristic planner using the same Newclid backend and Goal-related Diagnostics. This makes it hard to judge how much extra benefit the learned subgoal-selection policy actually provides compared to just using diagnostic guidance directly.
Questions for authors
What's the difference between Veri-GPS and RAVS (Agentic Geometry Problem Solving via Human-like Parallel Bidirectional Reasoning), especially in terms of subgoal planning, solver feedback, and proof search strategies? Figuring out these differences helps evaluate the novelty of the proposed method.
2 comments
Thank you for the rebuttal. The authors have addressed my concerns and clarified the issues raised in my review. I therefore have no further major concerns and will maintain my original score of 3.
Dear Reviewer S53T,
Thank you again for your careful review and helpful suggestions. We have added further experiments and clarifications in our responses and revised manuscript.
As the discussion period is nearing its end, we hope to hear your thoughts. If our responses have addressed your concerns, we would be grateful for your confirmation. If any concerns remain, we would welcome the opportunity to discuss them with you before the discussion closes.
Regards from the authors.