- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Test-time scaling has proven to be very effective, but exploration of multiple solutions can be very computationally expensive. In order to reduce this cost, we can balance exploration vs exploitation of reasoning trajectories, but prior work has not optimized directly for this trade-off. To bridge this gap, the authors propose to optimize pass@k over the solution candidate pool. In the proposed framework, they model candidate similarity across reasoning traces through hidden states correlation and evaluate predicted success through a linear probe connected to the hidden states.
Critical strengths
- The exploration vs exploitation trade-off for test-time scaling is well-explained
Critical weaknesses
- Contribution seems weak. As the authors themselves report, many parts of the method were already previously explored in literature: pass@k has been used for post-training to improve exploration during reasoning, and RepExp uses hidden states to measure solution candidates similarity. Modeling the joint distribution of rollouts through a latent Gaussian model seems the main contribution.
- The analysis of the results is very shallow. The authors present interesting failure cases of RepExp and Linear Probe, but unfortunately do not investigate further, and the method is not ablated.
Questions for authors
- Have you studied qualitatively whether PKTP preserves more diverse solution candidates compared to other baselines?
- Have you validated the pipeline? An oracle could provide the future correctness of the rollouts or the diversity of the final proposed solution, this would help to check whether each part of the pipeline pulls its weight.
8 comments
W2.
We thank the reviewer for this helpful comment. We have added an ablation of the method and investigated both failure cases across all eight models
Ablation. PKTP has two stages, modeling the rollouts jointly and selecting the set that maximizes Pass@. The following table replaces each stage with its independent counterpart and reports Pass@ averaged over the eight models. Each stage helps on its own, and the full method is the best at every budget.
| Modeling | Selection | @2 | @4 | @8 | @16 |
|---|---|---|---|---|---|
| Independent | Top- (Linear Probe) | 50.38 | 60.13 | 69.21 | 77.69 |
| Independent | Pass@ | 50.78 | 60.82 | 70.24 | 78.54 |
| Joint | Top- | 51.31 | 61.20 | 70.14 | 78.39 |
| Joint | Pass@ (PKTP) | 51.63 | 62.00 | 71.13 | 79.35 |
Greedy optimization. We also compare our greedy selection against the exact optimum of the original model (estimated with samples), solved via mixed-integer programming with Gurobi. Averaged over the eight models, greedy selection with our approximation matches the exact optimum at every budget.
| Selection | @2 | @4 | @8 | @16 |
|---|---|---|---|---|
| Exact optimum (Gurobi) | 51.32 | 61.89 | 71.12 | 79.16 |
| Greedy with our approximation | 51.36 | 61.95 | 71.14 | 79.30 |
Failure cases. The following table reports, for the rollouts each method keeps, their accuracy and the number of distinct final answers they reach, averaged over the eight models. The Linear Probe keeps accurate rollouts, but with fewer distinct answers than random selection, so they fail together (Figure 4). In contrast, RepExp keeps many distinct answers, but they come from inaccurate rollouts (Figure 5). PKTP balances the two and achieves the highest Pass@.
| Method | Accuracy of kept rollouts | Distinct final answers | Pass@4 |
|---|---|---|---|
| Random | 38.6 | 2.50 | 60.2 |
| Linear Probe | 40.2 | 2.40 | 60.1 |
| RepExp | 37.3 | 2.58 | 60.6 |
| PKTP | 39.4 | 2.50 | 62.0 |
We also verify the failure of the Linear Probe directly on the representations. The following table reports how much higher the mean pairwise cosine similarity of the rollouts it keeps is than that of random rollouts of the same problem. It is higher on every model and at every budget, so the probe concentrates its picks on a single region of similar reasoning.
| Model | ||||
|---|---|---|---|---|
| OpenMath-Nemotron-7B | +0.046 | +0.042 | +0.039 | +0.035 |
| DeepSeek-R1-Distill-Qwen-7B | +0.041 | +0.041 | +0.038 | +0.033 |
| DeepSeek-R1-Distill-Qwen-1.5B | +0.038 | +0.034 | +0.031 | +0.028 |
| Qwen3-1.7B | +0.022 | +0.023 | +0.024 | +0.025 |
We will add these analyses to the paper.
Q1.
As a qualitative example, we take the following AIME 2026 problem, on which only of the rollouts of DeepSeek-R1-Distill-Qwen-7B are correct, and look at the rollouts each method keeps.
A standard fair six-sided die is rolled repeatedly. Each time the die reads 1 or 2, Alice gets a coin; each time it reads 3 or 4, Bob gets a coin; and each time it reads 5 or 6, Carol gets a coin. The probability that Alice and Bob each receive at least two coins before Carol receives any coins can be written as , where and are relatively prime positive integers. Find .
The correct answer is . The following table shows, for every kept rollout, how it starts and its final output.
| Method | Start of the rollout | Final output |
|---|---|---|
| Linear Probe | "Okay, so I have this problem about rolling a die repeatedly …" | … |
| "Okay, so I have this problem where a six-sided die is rolled repeatedly …" | … | |
| "Okay, let's try to figure out this probability problem …" | … | |
| "Alright, so I need to figure out the probability …" | … | |
| ——— | ——— | ——— |
| RepExp | "Okay, so I have this problem where a six-sided die is rolled repeatedly …" | the probability … is … |
| "Okay, so here's this problem about rolling a die and giving coins …" | … | |
| "Okay, so I need to figure out the probability …" | finally … | |
| "Alright, so I have this problem where Alice, Bob, and Carol are getting coins …" | … | |
| ——— | ——— | ——— |
| PKTP | "Okay, so we have this problem where a fair six-sided die is rolled repeatedly …" | summing the probabilities of these sequences … the probability is … |
| "Okay, so I have this problem where a fair six-sided die is rolled repeatedly …" | … | |
| "Alright, so I have this probability problem here …" | The final probability is found to be … | |
| "Alright, let's tackle this probability problem step by step …" | … |
The Linear Probe keeps four rollouts that make the same mistake and reach the same wrong answer, and RepExp keeps four different but wrong ones, while PKTP keeps different solutions that include the correct one. We will add more such examples to the appendix.
We thank the reviewer for raising this point, and we would like to clarify how our work relates to these prior works. We respectfully note that Pass@k is an evaluation metric rather than a method, so sharing it does not by itself make two methods overlap. It is the standard test-time measure for code generation [1] and for scaling inference by repeated sampling [2], and, as the reviewer mentions, some works also use it as a reward in post-training so that the model's samples explore more, but to our knowledge it has not been explored as the objective for selecting rollouts in inference-time scaling. PKTP fills this gap by using Pass@k to decide which k of N partial rollouts to continue, with the model left unchanged, which also makes it complementary to these post-training methods.
Our main contribution is to view pruning as the optimization of Pass@k itself, rather than as a diversity selection problem, as in RepExp, or as a set of independent decisions on each rollout, as in the Linear Probe. When a test-time scaling method keeps k rollouts, what matters in the end is whether at least one of them is correct, which is exactly Pass@k. For example, when Best-of-k is applied to the k kept rollouts, the selected answer can only be correct if at least one of them is. Neither diversity nor the quality of each rollout on its own measures this. In other words, Pass@k is not a method we borrow but the actual metric that the pruning step has to optimize. RepExp and the probe both use hidden states, which are a standard representation, but each optimizes a proxy for this metric rather than the metric itself. We will make this explicit in the contributions section and highlight it more in the introduction.
Optimizing Pass@k directly is not straightforward, and making it possible required two further contributions. First, the Pass@k of a set depends on whether its rollouts fail together, so we model the joint distribution of the rollouts with a latent Gaussian model. Second, evaluating Pass@k under this model has no closed form, so we derive a closed-form approximation that is submodular and can therefore be optimized greedily with a guarantee, with a single model serving every budget k without retraining. This approximation also recovers the probe and RepExp as its two limiting cases, and PKTP learns how to move between them as k grows.
To show that this is not trivial, and as suggested by Reviewer p45q, we add a new baseline, Cluster + Probe, which partitions the N partial rollouts of a problem into k clusters of their representations and keeps the rollout of each cluster that the probe scores highest. It improves on the Linear Probe as k grows, since it adds diversity, but since it does not optimize Pass@k it still stays below PKTP on every model. Clustering uses the distance between representations as a stand-in for rollouts failing independently, and one pick per cluster fixes the trade-off in advance, while the trade-off that maximizes Pass@k depends on the problem and on the budget. The following tables report Pass@k for four of the models, as in our response to Reviewer p45q.
Qwen3-0.6B
DeepSeek-R1-Distill-Qwen-1.5B
DeepSeek-R1-Distill-Qwen-7B
Skywork-OR1-Math-7B
References
[1] M. Chen et al. Evaluating Large Language Models Trained on Code. arXiv:2107.03374, 2021.
[2] B. Brown et al. Large Language Monkeys, Scaling Inference Compute with Repeated Sampling. arXiv:2407.21787, 2024.