- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper investigates why the sequential sampling fails to improve the reasoning performance in long-horizon task. The work explains that the sequential sampling will get struck in a local minimum in the semantic space. The authors proposed using mixture models to deal with this case. They carried out experiments across 27 combinations of methods models and benchmarks to prove their claims empirically.
Critical strengths
The works provides a way to define the attractor in the semantic space. The empirical results show that at step 4, a large proportion of sample will stay in a small valley and could not get out.
The author proposed a simple approach to get out of the local minimum by calling for help from other model, using weaker model as perturbation.
Critical weaknesses
In Table 3, the mixed models only help to improve the overall performance slightly, the increases are mostly round 2-3%.
The idea of using mixed model is quite simple, like looking at different direction to get out of the loop you are stuck in. It lacks a new vision/angle to view the problem or bring a new insight to help solving the problem. It is not at the level of top-tier acceptance.
The concept of attraction is well defined in mathematics, but it is not clear how it will be applied to real situations, because it could differs from case to case.
Questions for authors
No
2 comments
Reply to reviewer Vni5's response
Thank you for your reply! You raise many of the same concerns as reviewer XBsf initially made, so we have repeated some of our replies to them here, as reviewer XBsf was satisfied with these responses.
First, there is no explicit algorithm provided in the main text, making it unclear how the proposed method is actually executed in practice.
Our intervention is described on line 320. We now have added pseudocode to Appendix M.
Secondly, relying solely on Definition 4.1 leaves the terminologies very vague, i.e. the choice of parameters, how readers imagine the states.
All parameters for identifying attractors are given in Appendix B.4 (lines 889–932). States are defined concretely for each method in Table 1. For example, in Self-Refine a state is the current answer. Following XBsf’s suggestions, we have also added:
- a full worked example that follows a real IMO-AnswerBench problem from answers to embeddings, clusters, transitions and attractors, including one optimal and one suboptimal attractor (Appendix B.4.4)
- an additional experiment to confirm that identified attractors satisfy Definition 4.1 (Appendix B.4.3).
The intuition about escaping local minima is very standard
Our intuition relates to escaping local minima in optimisation, but it applies to reasoning in natural language rather than model training. Prior work has not used the same framing to show that sequential test-time scaling fails in this way, as is also noted by reviewer aAfj.
I looked for a mathematical explanation or theoretical proof for why mixing models help to escape these local minima and improves overall performance, but there is none.
Our goal is to provide empirical evidence for our hypotheses rather than to prove them mathematically. It is standard for test-time scaling research to be published with empirical contributions rather than theoretical proofs [1,2,3].
[1] Wang, Xuezhi, et al. “Self-Consistency Improves Chain of Thought Reasoning in Language Models.” The Eleventh International Conference on Learning Representations (2023).
[2] Madaan, Aman, et al. "Self-refine: Iterative refinement with self-feedback." Advances in neural information processing systems 36 (2023): 46534-46594.
[3] Wang, Junlin, et al. "Mixture-of-agents enhances large language model capabilities." International Conference on Learning Representations (2025).
Response to reviewer Vni5
Thank you reviewer Vni5 for your feedback.
Question for reviewer Vni5. You indicate that you have no questions for us to answer; is there any further information we can provide that you believe would improve our work? We have put considerable effort into this work and are eager to improve it.
[W1] Effect size
The main concern about effect size seems to come from a misreading of Table 3, which we clarify first.
Table 3 reports attractor hit rates, not task performance, and our intervention lowers the rate of hitting an attractor within four states by 21.2 percentage points on average.
[W2] Novelty
We believe this is actually a strength of our intervention! Its simple nature helps isolate where the improvements we obtain come from.
We strongly disagree with this statement. Our novel contributions are as follows (also listed in the introduction):
We have not found any evidence in published literature of a similar investigation into this problem, and we believe that our work provides a useful new perspective on why sequential test-time scaling often falls short of parallel scaling.
[W3] Real situations
We investigate benchmarks for mathematics, games (chess, Sudoku, ...), and ARC-AGI-1, as these tasks (1) require minimal external knowledge and (2) are easily verifiable. We have tried to present a variety of benchmarks, which we believe measure reasoning skills that are often relevant to real situations. It is of course true of any method that the results may differ from case to case.