- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
After having the LLM explain to me the paper's core concept (e.g. percolation), the main intuitive idea, the claims and results (as can be seen by my prompt above). Here is my attempt to a summary:
The authors are looking in the ability of LLMs to generalize from shallow problems training (e.g. problems with 2 reasonning steps) to deeper problems (e.g. with 8 reasoning steps). Their claim is that this ability depends on data connectivity, and not on the volume of the problem set or the choice of optimizer. To demonstrate this, they created COVERWEAVE, a synthetic data engine that creates a set of shallow problems and control their connectivity (how the different reasoning steps appear together). They show that a trained policy can solve a d-step problem if all the d required reasoning steps (atoms) belong to a single connected graph. Hence they derive a probability of successfully solving a problem based on the connectivity. They show that optimizing this connectivity threshold allow models to perform deep reasoning with 4x less data. Further, they show that their results transfers to real-world multi-step chemical retrosynthesis.
Critical strengths
The result is amazing. It allowed me to rethink what is important for training LLM-based policies.
The link to percolation is amazing. At the time where LLM based agentic tools can prove results, conduct experiments and write papers from a single prompt, these papers are extremely valuables. Linking different areas of science (here bringing percolation to improve the training efficiency of reasoning agents) is crucial. The paper does it very well, the formalization is good, and once understood, the results are striking.
Finally, this results is not only interesting for this specific RL area.
Critical weaknesses
However, the paper clearly lacks accessibility. Without the LLM to pre-digest the paper for me, it would have extremely difficult for me to understand the problem at hand. I asked an RLVR expert colleague if he understood it after 5 mins spent on it: no. I believe that accessibility is key. The authors are facing the risk that a large audience that would be interested in these results simply do not get it, because of the clarity issue. I think the paper is written by/for percolation experts that also have advance knowledge in RLVR reasoning agent training.
I really invite the authors to rethink how they can improve the clarity of their paper (as Hindsight Experience Replay does). Formalization is important, but it can go in the appendix. Their main result on reasoning atom connectivity is crucial to our community, and it would be a shame that most researcher miss it because of accessibility. Take my prompt and my summary, try to bring this intuition in the paper, while keeping your formalization.
Convince me that you do so, and I'll be more than glad to go from weak reject to strong accept.
Questions for authors
None.