- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
Reap implements MCTS inside Lean, sharing proof-state transitions, reconstruction, replay, and logging between inference and training. Reaper-1 is a 1.7B backbone trained through continued pre-training, supervised learning, and search-based expert iteration. With a separate premise selector, it solves 195/244 miniF2F-test problems at pass@32.
Critical strengths
S1. The integrated infrastructure combines native proof states, replay verification, and step-level logging in a useful platform for studying proof search.
S2. The reported miniF2F-test improvement is substantial, demonstrating effectiveness of the MCTS-based reinforcement learning
S3. The behavioral analysis is careful and very detailed
Critical weaknesses
W1. What does MCTS add beyond a simpler search? Table 2 retains 193 versus 195 solves without neural value guidance, although guidance reduces mean proof actions by roughly 6% on shared problems; Table 4 reaches 188 solves at 512 simulations. These useful controls do not compare MCTS with simpler search using the same trained model, leaving the practical inference-time benefit of this central design choice unresolved.
Questions for authors
- Using the same RL5 model and retriever, how does best-first search compare with MCTS under matched generation and Lean-execution budgets? This would test the inference-time search choice, not whether MCTS was useful during training.
2 comments
I thank the authors for their responses. I maintain my score.
Thank you for suggesting this comparison. To examine the inference-time search choice, we implemented best-first search (BFS) in Reap and compared it with MCTS on miniF2F-test using the same RL5 checkpoint and PS-2 retriever, without retraining. Both configurations use 32 L40 GPUs for the prover, pass@8, and a 7,200-second timeout.
MCTS solves 55 more problems (+22.5 percentage points), while BFS uses substantially less time and fewer tokens per successful attempt. Time is measured end to end through final verification, and completion tokens include all tactics generated during each successful attempt. This comparison uses a common timeout, but does not separately match generation and Lean-execution budgets.
Our current BFS implementation is a preliminary baseline developed for this ablation. Unlike MCTS, it has not been adapted to the AND–OR tree structure used during training rollouts. This mismatch may partly explain the large performance gap, so these results should not be interpreted as a definitive comparison between fully optimized search algorithms.
For inference-time search recommendation, we are continuing to improve the BFS implementation to make it stable and easy to use in the final open-source release.