- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces CDRL, which combines neural-guided MCTS with symbolic reasoning and reusable failure certificates for neutrino flavor-model search. Failed candidates produce constraints that are propagated to future search through SAT/BCP. The method is evaluated on three theory spaces and compared with AMBer and several controlled ablations.
Critical strengths
The mechanism is well motivated. Instead of reducing an expensive scientific evaluation to a scalar reward, the method reuses structured failure information to prune future search.
The component ablations are convincing. Removing certificates, MCTS, or the neural component produces substantially different degradation patterns, and these experiments are reported across multiple random seeds.
Critical weaknesses
The main limitation for me is novelty positioning. MCTS, neural guidance, SAT propagation, and conflict-style reusable constraints are individually established ideas, so I see the contribution primarily in their integration for this scientific-search setting rather than as a fundamentally new RL paradigm.
I would also interpret the AMBer comparison more cautiously. CDRL uses substantially fewer terminal physics evaluations, which is a meaningful result, but the two systems do not use matched execution configurations, and the paper explicitly does not treat its wall-clock comparison as matched compute.
Questions for authors
Can the authors provide a smaller matched-compute comparison with AMBer or an equivalent scalar-reward baseline using the same hardware, caching, parallelism, and physics-evaluation stack? This would make the efficiency claim substantially stronger.
3 comments
Q2. Can the authors provide a matched-compute comparison with AMBer using the same hardware, caching, parallelism, etc.?
A: We thank the reviewer for this suggestion and have run both comparisons. First, we reran AMBer's PPO agent with four seeds on the same hardware, with the same evaluation cache and number of parallel workers, with the same evaluation budget per run (1M, 1M and 6M). Second, our No symbolic ablation is an equivalent scalar-reward baseline that matches CDRL on every part of the setup, i.e., hardware, caching, parallelism, physics stack, planner, network, reward and budget. It differs only in that it learns from the scalar reward alone, without certificates.
| Search space | Method | Evaluations per run | Valid models (%) | Neutrino models (%) | Min. parameters |
|---|---|---|---|---|---|
| A4 x Z4 | Full CDRL | 1M | 28.1 +/- 3.3 | 0.10 +/- 0.01 | 5 |
| No symbolic (scalar reward only) | 1M | 14.9 +/- 3.1 | 0.01 +/- 0.02 | 7 | |
| AMBer PPO | 1M | 12.5 +/- 5.0 | 0.01 +/- 0.01 | 5 | |
| PPO + SAT & certificates | 1M | 13.9 +/- 4.1 | 0.02 +/- 0.00 | 5 | |
| A4 x ZN | Full CDRL | 1M | 26.9 +/- 3.7 | 0.19 +/- 0.05 | 5 |
| No symbolic (scalar reward only) | 1M | 16.2 +/- 2.1 | 0.03 +/- 0.01 | 7 | |
| AMBer PPO | 1M | 9.7 +/- 3.5 | 0.01 +/- 0.01 | 5 | |
| PPO + SAT & certificates | 1M | 11.0 +/- 3.1 | 0.01 +/- 0.01 | 5 | |
| T19 x Z4 | Full CDRL | 6M | 18.2 +/- 2.1 | 0.09 +/- 0.02 | 4 |
| No symbolic (scalar reward only) | 6M | 5.4 +/- 4.9 | 0.04 +/- 0.02 | 5 | |
| AMBer PPO | 6M | 8.8 +/- 3.6 | 0.02 +/- 0.01 | 5 | |
| PPO + SAT & certificates | 6M | 11.3 +/- 2.6 | 0.03 +/- 0.01 | 4 |
Under these matched conditions, CDRL's valid model rate is 2.2x, 2.8x and 2.1x that of AMBer, and its neutrino model rate is 10x, 19x and 4.5x. The fully matched scalar-reward baseline shows where this gain comes from. With the same stack and budget but no certificates, the valid rate falls to 14.9%, 16.2% and 5.4%, close to AMBer's level. Certificates therefore account for most of the improvement, not differences in execution configuration. Adding the SAT layer and certificates to PPO also helps (12.5% to 13.9%, 9.7% to 11.0%, 8.8% to 11.3%), but the gain is much larger with MCTS, where each clause prunes the lookahead at every node it reaches. We have added these matched comparisons to Section 5.2, Figure 2(a,b) and Table 6 (Appendix G.4).
Summary of additional results
Beyond the points above, the revision adds several new experiments and analyses.
- Certificate-type ablations and feasibility-check baselines. Learning rank clauses clearly beats checking rank on every model without learning from failures (e.g., 17.9% to 26.9% valid on A4 x ZN), learned symmetry clauses add to this, and the full system is best in every space. Even without the network and MCTS, adding learned clauses to a SAT search finds the first neutrino models and lowers the minimum parameter count from 12 to 7. See Section 5.3, Figure 2(c,d) and Table 7.
- Table 1 gives four-run means and Table 2 the best run, following AMBer's convention, and both captions now state this explicitly.
- We state the invariant that keeps one representative of every equivalence class available for any number of workers and any synchronization order, and verify it exhaustively for the moduli in our theory spaces (Appendix B.3).
- We discuss why the analyzer carries over unchanged across the three theory spaces and which properties a new domain needs, such as molecule design and crystal structure search (Section 6).
- A discussion of why many distinct viable models matter for upcoming tests of flavor models (DUNE, T2HK, neutrinoless double beta decay). See Appendix D.
We thank the reviewer for the positive assessment and the helpful suggestions. We hope the matched-compute comparison, the clarified novelty positioning and the additional experiments above resolve the remaining concerns, and we would be grateful if the reviewer would consider raising their score to further support the paper.
Dear reviewer, thank you again for your positive review. As the discussion period is ending soon, we wanted to check whether our response resolved your questions. We clarified the novelty positioning and added the requested matched-compute comparison: an AMBer rerun and our No symbolic scalar-reward baseline, both on the same hardware, cache, parallelism and evaluation budget as CDRL. If any concerns remain, we would be glad to address them. Otherwise, we would be grateful if you would consider raising your score.
Q1. Novelty
A: We uploaded a revised PDF with all changes marked in green. Section, figure and table numbers below refer to the revised version. We agree that MCTS, neural guidance, SAT propagation and conflict-style constraints are each established. What CDRL introduces is the mechanism that connects them. To the best of our knowledge, hard constraints learned during search have not been used inside the DNN+MCTS setting of AlphaZero, which is widely used for discovery because it learns and plans at the same time [1, 2, 3]. Earlier work used MCTS to improve SAT solvers [4, 5], but CDRL works in the other direction and uses SAT reasoning to prune a learned MCTS search.
For example, Neo[6] looks for one program that meets a given specification and finds conflicts by reasoning over logical specifications of the program’s components. It uses them to prune a backtracking search guided by a fixed, pretrained model, with no planning and no learning during search. CDRL looks for many inequivalent models under an expensive evaluator. A rank failure only becomes visible after the physics pipeline has evaluated a complete model, so CDRL learns its clauses from these evaluations. It removes both invalid and redundant models, and its clauses act inside a self-play loop where MCTS plans with a policy-value network that keeps training. Deduction-guided RL [7] uses deductions to update the policy, which only lowers the probability of a bad choice, while CDRL clause removes the choice. Oracle-guided synthesis [8] passes constraints from an oracle to a solver that looks for one solution, without a learned policy or planning. Symbolic advice for MCTS [9] uses a formula fixed by the user before search, which only biases selection within a bounded horizon and never changes. In CDRL, the clause set grows during search from what the evaluator reveals, every clause is shared across workers, and a canonical-representative rule keeps them sound under parallel sharing. The setting is also new in physics. RL has been used for model building [10, 11, 12], and SAT solvers have searched string vacua with fixed constraints [13], but we are not aware of work that learns constraints from a physics evaluator during search. Bringing conflict-driven learning to flavor model building is itself new, and the mapping is non-trivial. AMBer [10] uses the same physics pipeline but learns from failures only through a scalar reward. CDRL adds the missing layer which is a certificate analyzer that extracts the choices responsible for each failure or redundancy, and a SAT encoding that turns them into clauses the search enforces.
This mechanism, not the individual components, accounts for the gains. Under a matched budget and hardware, CDRL’s valid model rate is 2.2x, 2.8x and 2.1x that of AMBer, and its neutrino model rate is 10x, 19x and 4.5x. Most of this gain comes from the certificates. Removing them while keeping the same planner, network and reward drops the valid rate from 28.1% to 14.9%, 26.9% to 16.2%, and 18.2% to 5.4%, close to AMBer’s level. Adding the same certificates to PPO helps only modestly, so the gains come from combining learned constraints with learning and planning, which is what CDRL introduces.
References
[1] Silver et al., Mastering the Game of Go without Human Knowledge, Nature 2017.
[2] Fawzi et al., Discovering Faster Matrix Multiplication Algorithms with Reinforcement Learning, Nature 2022.
[3] Mankowitz et al., Faster Sorting Algorithms Discovered using Deep Reinforcement Learning, Nature 2023.
[4] Schloeter, A Monte Carlo Tree Search Based Conflict-Driven Clause Learning SAT Solver, INFORMATIK 2017.
[5] Keszocze et al., Improving SAT Solving Using Monte Carlo Tree Search-based Clause Learning, Advanced Boolean Techniques 2020.
[6] Feng et al., Program Synthesis using Conflict-Driven Learning, PLDI 2018.
[7] Chen et al., Program Synthesis Using Deduction-Guided Reinforcement Learning, CAV 2020.
[8] Polgreen et al., Satisfiability and Synthesis Modulo Oracles, VMCAI 2022.
[9] Busatto-Gaston et al., Monte Carlo Tree Search Guided by Symbolic Advice for MDPs, CONCUR 2020.
[10] Baretz et al., Towards AI-assisted Neutrino Flavor Theory Design, Communications Physics 2026.
[11] Halverson et al., Branes with Brains: Exploring String Vacua with Deep Reinforcement Learning, JHEP 2019.
[12] Harvey and Lukas, Quark Mass Models and Reinforcement Learning, JHEP 2021.
[13] Faraggi et al., Satisfiability Modulo Theories and Chiral Heterotic String Vacua with Positive Cosmological Constant, Phys. Lett. B 2021.