acceptodds
Under review as a conference paper at ICLR 2027

Direct Pass@k Optimization for Thought Pruning: Balancing Quality and Diversity in Test-Time Reasoning

Abstract

Test-time scaling has emerged as a key inference strategy for reasoning models, where a model generates a large number of candidate solutions in the hope of finding at least one correct solution. Running all candidate solutions to completion can be computationally demanding, so recent work prunes the candidate pool after generating only a short prefix, committing further computation only to the surviving candidates. Existing pruning strategies divide broadly into two categories: Quality-based methods assign a score to each partial solution and discard those predicted to ultimately lead to an incorrect final solution, while diversity-based methods select partial solutions that follow different reasoning trajectories and are therefore likely to cover different final solutions. The tension between these two objectives is an instance of exploration versus exploitation, yet little to no prior work on pruning exposes this trade-off directly, let alone optimizes it. In this paper, we introduce a pruning procedure that addresses this trade-off directly by optimizing *Pass@k* over the selected solution pool. In particular, it chooses the pool to maximize the probability that at least one of its partial solutions, when continued to completion, yields a correct answer. Because this objective simultaneously rewards high-quality partial solutions and penalizes redundancy among them, quality and diversity are naturally balanced by the criterion itself rather than through separate ad-hoc design choices. On eight open-weight reasoning models, our method outperforms existing quality-based and diversity-based pruning strategies. The gains are particularly pronounced when the pruning procedure keeps a moderate number of partial solutions for continued generation, highlighting the value of directly balancing quality and diversity during pruning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
reasoning, pruning, pass@k, optimization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_12nZOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

Test-time scaling has proven to be very effective, but exploration of multiple solutions can be very computationally expensive. In order to reduce this cost, we can balance exploration vs exploitation of reasoning trajectories, but prior work has not optimized directly for this trade-off. To bridge this gap, the authors propose to optimize pass@k over the solution candidate pool. In the proposed framework, they model candidate similarity across reasoning traces through hidden states correlation and evaluate predicted success through a linear probe connected to the hidden states.

Critical strengths
  • The exploration vs exploitation trade-off for test-time scaling is well-explained
Critical weaknesses
  1. Contribution seems weak. As the authors themselves report, many parts of the method were already previously explored in literature: pass@k has been used for post-training to improve exploration during reasoning, and RepExp uses hidden states to measure solution candidates similarity. Modeling the joint distribution of rollouts through a latent Gaussian model seems the main contribution.
  2. The analysis of the results is very shallow. The authors present interesting failure cases of RepExp and Linear Probe, but unfortunately do not investigate further, and the method is not ablated.
Questions for authors
  1. Have you studied qualitatively whether PKTP preserves more diverse solution candidates compared to other baselines?
  2. Have you validated the pipeline? An oracle could provide the future correctness of the rollouts or the diversity of the final proposed solution, this would help to check whether each part of the pipeline pulls its weight.
8 comments

W1.

We thank the reviewer for raising this point, and we would like to clarify how our work relates to these prior works. We respectfully note that Pass@kk is an evaluation metric rather than a method, so sharing it does not by itself make two methods overlap. It is the standard test-time measure for code generation [1] and for scaling inference by repeated sampling [2], and, as the reviewer mentions, some works also use it as a reward in post-training so that the model's samples explore more, but to our knowledge it has not been explored as the objective for selecting rollouts in inference-time scaling. PKTP fills this gap by using Pass@kk to decide which kk of NN partial rollouts to continue, with the model left unchanged, which also makes it complementary to these post-training methods.

Our main contribution is to view pruning as the optimization of Pass@kk itself, rather than as a diversity selection problem, as in RepExp, or as a set of independent decisions on each rollout, as in the Linear Probe. When a test-time scaling method keeps kk rollouts, what matters in the end is whether at least one of them is correct, which is exactly Pass@kk. For example, when Best-of-kk is applied to the kk kept rollouts, the selected answer can only be correct if at least one of them is. Neither diversity nor the quality of each rollout on its own measures this. In other words, Pass@kk is not a method we borrow but the actual metric that the pruning step has to optimize. RepExp and the probe both use hidden states, which are a standard representation, but each optimizes a proxy for this metric rather than the metric itself. We will make this explicit in the contributions section and highlight it more in the introduction.

Optimizing Pass@kk directly is not straightforward, and making it possible required two further contributions. First, the Pass@kk of a set depends on whether its rollouts fail together, so we model the joint distribution of the rollouts with a latent Gaussian model. Second, evaluating Pass@kk under this model has no closed form, so we derive a closed-form approximation that is submodular and can therefore be optimized greedily with a guarantee, with a single model serving every budget kk without retraining. This approximation also recovers the probe and RepExp as its two limiting cases, and PKTP learns how to move between them as kk grows.

To show that this is not trivial, and as suggested by Reviewer p45q, we add a new baseline, Cluster + Probe, which partitions the NN partial rollouts of a problem into kk clusters of their representations and keeps the rollout of each cluster that the probe scores highest. It improves on the Linear Probe as kk grows, since it adds diversity, but since it does not optimize Pass@kk it still stays below PKTP on every model. Clustering uses the distance between representations as a stand-in for rollouts failing independently, and one pick per cluster fixes the trade-off in advance, while the trade-off that maximizes Pass@kk depends on the problem and on the budget. The following tables report Pass@kk for four of the models, as in our response to Reviewer p45q.

Qwen3-0.6B

Method@1@2@4@8@16
Random (Pass@kk)24.034.445.456.567.6
Linear Probe24.933.844.356.267.1
RepExp23.933.545.055.967.0
Cluster + Probe24.935.046.758.168.4
PKTP25.636.347.258.469.3

DeepSeek-R1-Distill-Qwen-1.5B

Method@1@2@4@8@16
Random (Pass@kk)27.438.449.560.270.3
Linear Probe30.139.849.959.469.7
RepExp27.338.549.860.672.6
Cluster + Probe30.139.749.861.171.0
PKTP30.341.452.962.571.7

DeepSeek-R1-Distill-Qwen-7B

Method@1@2@4@8@16
Random (Pass@kk)37.548.358.568.277.5
Linear Probe37.446.756.466.076.6
RepExp37.447.258.670.079.7
Cluster + Probe37.447.758.668.178.7
PKTP38.649.559.770.079.3

Skywork-OR1-Math-7B

Method@1@2@4@8@16
Random (Pass@kk)42.553.162.771.579.8
Linear Probe44.454.262.971.379.4
RepExp42.554.564.572.179.4
Cluster + Probe44.454.263.272.280.3
PKTP44.755.765.673.280.3

References

[1] M. Chen et al. Evaluating Large Language Models Trained on Code. arXiv:2107.03374, 2021.

[2] B. Brown et al. Large Language Monkeys, Scaling Inference Compute with Repeated Sampling. arXiv:2407.21787, 2024.

W2.

We thank the reviewer for this helpful comment. We have added an ablation of the method and investigated both failure cases across all eight models

Ablation. PKTP has two stages, modeling the rollouts jointly and selecting the set that maximizes Pass@kk. The following table replaces each stage with its independent counterpart and reports Pass@kk averaged over the eight models. Each stage helps on its own, and the full method is the best at every budget.

ModelingSelection@2@4@8@16
IndependentTop-kk (Linear Probe)50.3860.1369.2177.69
IndependentPass@kk50.7860.8270.2478.54
JointTop-kk51.3161.2070.1478.39
JointPass@kk (PKTP)51.6362.0071.1379.35

Greedy optimization. We also compare our greedy selection against the exact optimum of the original model (estimated with 256256 samples), solved via mixed-integer programming with Gurobi. Averaged over the eight models, greedy selection with our approximation matches the exact optimum at every budget.

Selection@2@4@8@16
Exact optimum (Gurobi)51.3261.8971.1279.16
Greedy with our approximation51.3661.9571.1479.30

Failure cases. The following table reports, for the 44 rollouts each method keeps, their accuracy and the number of distinct final answers they reach, averaged over the eight models. The Linear Probe keeps accurate rollouts, but with fewer distinct answers than random selection, so they fail together (Figure 4). In contrast, RepExp keeps many distinct answers, but they come from inaccurate rollouts (Figure 5). PKTP balances the two and achieves the highest Pass@44.

MethodAccuracy of kept rolloutsDistinct final answersPass@4
Random38.62.5060.2
Linear Probe40.22.4060.1
RepExp37.32.5860.6
PKTP39.42.5062.0

We also verify the failure of the Linear Probe directly on the representations. The following table reports how much higher the mean pairwise cosine similarity of the rollouts it keeps is than that of kk random rollouts of the same problem. It is higher on every model and at every budget, so the probe concentrates its picks on a single region of similar reasoning.

Modelk=2k=2k=4k=4k=8k=8k=16k=16
OpenMath-Nemotron-7B+0.046+0.042+0.039+0.035
DeepSeek-R1-Distill-Qwen-7B+0.041+0.041+0.038+0.033
DeepSeek-R1-Distill-Qwen-1.5B+0.038+0.034+0.031+0.028
Qwen3-1.7B+0.022+0.023+0.024+0.025

We will add these analyses to the paper.

Q1.

As a qualitative example, we take the following AIME 2026 problem, on which only 22%22\% of the rollouts of DeepSeek-R1-Distill-Qwen-7B are correct, and look at the 44 rollouts each method keeps.

A standard fair six-sided die is rolled repeatedly. Each time the die reads 1 or 2, Alice gets a coin; each time it reads 3 or 4, Bob gets a coin; and each time it reads 5 or 6, Carol gets a coin. The probability that Alice and Bob each receive at least two coins before Carol receives any coins can be written as mn\tfrac{m}{n}, where mm and nn are relatively prime positive integers. Find 100m+n100m+n.

The correct answer is 754754. The following table shows, for every kept rollout, how it starts and its final output.

MethodStart of the rolloutFinal output
Linear Probe"Okay, so I have this problem about rolling a die repeatedly …"S00=13⋅59+13⋅59=1027S_{00} = \frac{1}{3} \cdot \frac{5}{9} + \frac{1}{3} \cdot \frac{5}{9} = \frac{10}{27} … 1027\boxed{1027}
"Okay, so I have this problem where a six-sided die is rolled repeatedly …"P(0,0)=13⋅P(1,0)+13⋅P(0,1)=…=1027P(0, 0) = \frac{1}{3} \cdot P(1, 0) + \frac{1}{3} \cdot P(0, 1) = \ldots = \frac{10}{27} … 1027\boxed{1027}
"Okay, let's try to figure out this probability problem …"f(0,0)=…=527+527=1027f(0, 0) = \ldots = \frac{5}{27} + \frac{5}{27} = \frac{10}{27} … 1027\boxed{1027}
"Alright, so I need to figure out the probability …"P(0,0)=…+13⋅0=…=1027P(0, 0) = \ldots + \frac{1}{3} \cdot 0 = \ldots = \frac{10}{27} … 1027\boxed{1027}
—————————
RepExp"Okay, so I have this problem where a six-sided die is rolled repeatedly …"the probability PP … is 4225\frac{4}{225} … 625\boxed{625}
"Okay, so here's this problem about rolling a die and giving coins …"P(0,0)=…=7108+554=17108P(0, 0) = \ldots = \frac{7}{108} + \frac{5}{54} = \frac{17}{108} … 1808\boxed{1808}
"Okay, so I need to figure out the probability …"finally P(0,0)=29P(0,0) = \frac{2}{9} … 209\boxed{209}
"Alright, so I have this problem where Alice, Bob, and Carol are getting coins …"f(0,0)=…=1027f(0, 0) = \ldots = \frac{10}{27} … 1027\boxed{1027}
—————————
PKTP"Okay, so we have this problem where a fair six-sided die is rolled repeatedly …"summing the probabilities of these sequences … the probability is 754\frac{7}{54} … 754\boxed{754}
"Okay, so I have this problem where a fair six-sided die is rolled repeatedly …"P(0,0)=14/81P(0, 0) = 14/81 … 1481\boxed{1481}
"Alright, so I have this probability problem here …"The final probability is found to be 38\frac{3}{8} … 11\boxed{11}
"Alright, let's tackle this probability problem step by step …"f00=…=1027f_{00} = \ldots = \frac{10}{27} … 1027\boxed{1027}

The Linear Probe keeps four rollouts that make the same mistake and reach the same wrong answer, and RepExp keeps four different but wrong ones, while PKTP keeps different solutions that include the correct one. We will add more such examples to the appendix.

Reviewer_p45qOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to select partial reasoning rollouts for continued generation under a limited inference budget. Existing methods prioritize either predicted correctness or representation diversity. The authors propose PKTP, which selects a subset by approximately maximizing the probability that at least one retained rollout produces a correct answer. The method uses a latent Gaussian model to capture rollout quality and correlations, derives a tractable approximation, and applies greedy selection. Experiments across eight reasoning models compare PKTP with random selection, Linear Probe, and RepExp.

Critical strengths
  • The motivation is reasonable. Selecting individually promising rollouts may retain similar reasoning paths, while selecting only for diversity may spend compute on low-quality rollouts. Considering both factors is useful for test-time pruning.

  • The experiments cover eight models and examine different numbers of retained rollouts, candidate pool sizes, and pruning cutoffs. The results generally support the benefit of the proposed method.

Critical weaknesses
  • My main concern is that the challenge is not clearly explained. After reading the motivation, a natural approach is to cluster the prefixes using their representations and use a probe to select the highest-scoring prefix from each cluster. This would consider both quality and diversity. Section 3 proposes a more involved formulation, but I find it difficult to understand what limitation of this simple approach it addresses. Comparing against this baseline would help clarify the contribution.
Questions for authors
  • Around line 95, the authors argue that probe-selected rollouts tend to be similar and fail together. Figure 4 gives an example, but it would be helpful to validate this across problems by measuring representation similarity and shared failures.

  • How does the method perform when training and testing on different datasets? The current cross-validation includes problems from each benchmark in both training and evaluation. Testing on a held-out benchmark would help show whether the learned selection policy generalizes.

5 comments

W1.

We thank the reviewer for this insightful suggestion, which is a natural and strong baseline that helps us sharpen the contribution of the paper. Our main motivation is to optimize Pass@kk directly, and the approach the reviewer suggests is an interesting heuristic that combines the exploration and exploitation trade-off implicitly but is not necessarily optimal for Pass@kk. We explain below what we mean by that, and we nonetheless implement it as a new baseline and compare against it.

Clustering uses representation distance as a stand-in for independent failure. Taking one pick per cluster fixes the exchange rate between quality and diversity in advance, whereas the optimal rate for Pass@kk depends on both the problem and the budget. Our formulation reads dependence from the same representations, but learns how strongly it translates into joint failure. It then maximizes the probability that a set is not entirely wrong at the given kk. Thus, the balance between quality and diversity emerges directly from this estimate rather than from a fixed rule.

We implement that baseline as Cluster + Probe, which partitions the 128128 prefixes of a problem into kk clusters and keeps the prefix of each cluster that the probe scores highest. The baseline is a real improvement on the Linear Probe once kk grows, which confirms that the representations carry a diversity signal, and at k=1k = 1 it coincides with the Linear Probe, since a single cluster leaves only the highest-scoring prefix.

PKTP is nonetheless above it on every model, and we attribute this to optimizing Pass@kk directly rather than through a proxy for it. We read this as evidence that the hard part is how a small budget is allocated and not whether diverse prefixes can be found at all, and allocating it is what a fixed one pick per cluster cannot do.

Qwen3-0.6B

Method@1@2@4@8@16
Random (Pass@kk)24.034.445.456.567.6
Linear Probe24.933.844.356.267.1
RepExp23.933.545.055.967.0
Cluster + Probe24.935.046.758.168.4
PKTP25.636.347.258.469.3

DeepSeek-R1-Distill-Qwen-1.5B

Method@1@2@4@8@16
Random (Pass@kk)27.438.449.560.270.3
Linear Probe30.139.849.959.469.7
RepExp27.338.549.860.672.6
Cluster + Probe30.139.749.861.171.0
PKTP30.341.452.962.571.7

DeepSeek-R1-Distill-Qwen-7B

Method@1@2@4@8@16
Random (Pass@kk)37.548.358.568.277.5
Linear Probe37.446.756.466.076.6
RepExp37.447.258.670.079.7
Cluster + Probe37.447.758.668.178.7
PKTP38.649.559.770.079.3

Skywork-OR1-Math-7B

Method@1@2@4@8@16
Random (Pass@kk)42.553.162.771.579.8
Linear Probe44.454.262.971.379.4
RepExp42.554.564.572.179.4
Cluster + Probe44.454.263.272.280.3
PKTP44.755.765.673.280.3

Q1.

We thank the reviewer for this helpful suggestion, and we agree that a single example is not enough. We therefore measure both parts of the claim over every test problem, using the probe and the splits of Table 1.

The first part is that the rollouts the probe ranks highest are similar to one another. The following table reports how much higher the mean pairwise cosine similarity of their representations is than that of kk random rollouts of the same problem. It is higher on every model and at every budget, so the probe behaves like a mode collapse, concentrating its picks on a single region of similar reasoning instead of spreading them across different approaches.

Modelk=2k=2k=4k=4k=8k=8k=16k=16
OpenMath-Nemotron-7B+0.046+0.042+0.039+0.035
DeepSeek-R1-Distill-Qwen-7B+0.041+0.041+0.038+0.033
DeepSeek-R1-Distill-Qwen-1.5B+0.038+0.034+0.031+0.028
Qwen3-1.7B+0.022+0.023+0.024+0.025

The second part is that similar rollouts fail together. For every problem we take all pairs of its rollouts and compute the cosine similarity of their representations, and we then split the pairs into ten equally sized groups, from least to most similar. For each group, the following table reports how much more often the two rollouts of a pair are both correct or both wrong than they would be if they were independent. Agreement grows with similarity on every model, as Eq. 1 assumes.

Model12345678910
OpenMath-Nemotron-7B-2.72-1.44-1.11-0.93-0.48+0.12+0.58+1.05+1.38+1.75
DeepSeek-R1-Distill-Qwen-7B-1.50-1.01-0.61-0.43-0.30-0.20+0.04+0.20+0.62+1.50
DeepSeek-R1-Distill-Qwen-1.5B-1.41-0.97-0.72-0.50-0.44-0.08+0.10+0.37+0.48+1.45
Qwen3-1.7B-1.53-1.36-1.06-0.80-0.35+0.01+0.36+0.72+1.02+1.10

Q2.

To measure how our method performs under transfer to unseen benchmarks, we rerun the whole pipeline holding out one benchmark at a time and training on the rest.

The following table averages Pass@kk over the eight models under the held-out benchmark split. PKTP remains the best method, showing that the learned selection policy transfers to benchmarks it has never seen.

Method@2@4@8
Random (Pass@kk)49.7860.1269.56
Linear Probe50.2960.1969.30
RepExp49.6260.5970.28
PKTP51.4261.9770.97

As an example of a single held-out benchmark, the following table reports HMMT for four of the models, showing that PKTP consistently outperforms the baselines.

ModelMethod@2@4@8
Qwen3-0.6BRandom (Pass@kk)29.739.850.9
Linear Probe35.944.651.7
RepExp31.238.947.9
PKTP38.851.054.6
OpenMath-Nemotron-1.5BRandom (Pass@kk)51.260.669.0
Linear Probe55.064.772.9
RepExp51.363.672.3
PKTP55.864.073.4
DeepSeek-R1-Distill-Qwen-7BRandom (Pass@kk)47.055.665.5
Linear Probe46.153.566.0
RepExp46.354.968.7
PKTP48.160.268.7
OpenMath-Nemotron-7BRandom (Pass@kk)52.562.873.1
Linear Probe54.865.474.4
RepExp54.362.673.4
PKTP57.070.879.8
Reviewer_LiEPOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies which partial reasoning rollouts to keep when only a limited number can be continued. It proposes PKTP, which models the joint correctness outcomes using a latent Gaussian model with learned quality scores and representation-dependent correlations. A second-order approximation gives a closed-form surrogate for the joint likelihood during training and for the all-failure probability during selection. The resulting surrogate pass probability is shown to be monotone and submodular, allowing greedy selection with a standard approximation guarantee. Experiments on eight open-weight reasoning models and several mathematical benchmarks show gains over quality-only and diversity-only selection, particularly at moderate selection budgets.

Critical strengths
  1. The paper gives a coherent way to connect quality-based and diversity-based selection. The overall idea of estimating set-level success and selecting a subset is intuitive. The rank-one updates and the ability to use one trained predictor across different kk also make the construction useful.

  2. The experiments provide encouraging evidence across eight models, with problem-level cross-validation and results for different prefix lengths and pool sizes. The gains at moderate kk suggest that combining quality and diversity can be useful, even though the method does not dominate every baseline at every budget.

Critical weaknesses
  1. Lack of validation of whether the approximation is good enough. Thm 1 and proposition 1 provide an approximation to Pr⁡θ\Pr_\theta and prove that the approximation is valid, but the paper does not validate the benefits of this approximation under the actual experimental settings: whether it is indeed relatively fast to compute and relatively accurate, especially whether it preserves the ranking of candidate subsets. Since the greedy guarantee applies to the surrogate objective, its relationship with the original probability objective remains insufficiently clear.
  2. The contributions of training and the selection rule are not separated. PKTP changes both how the quality predictor is trained and how subsets are selected, and the current comparisons cannot explain where the gains come from. Comparisons with top-k and simple quality–diversity combinations using the same fixed predictor are needed to more clearly assess the value of the proposed selection rule.
Questions for authors
  1. Approximation quality: Under the actual experimental settings, how accurate is the approximation in thm 1 compared with Monte Carlo estimates? Does the approximation error change the ranking of candidate subsets and the final selection?
  2. Modeling motivation: In the definition of Z\mathbf{Z}, what is the motivation for using the shared Gaussian variable f\mathbf{f} and independent noise m\mathbf{m} in H⊤f+fH^\top\mathbf{f}+\mathbf{f}? What are they intended to capture, respectively? Is the role of the rollout correlation term only to ensure that Cov⁡(Zi,Zj∣H)=hi⊤hj\operatorname{Cov}(\mathbf{Z}_i,\mathbf{Z}_j\mid H)=h_i^\top h_j? Why is H⊤HH^\top H, obtained from hidden representations, suitable for describing the correlations between the latent correctness scores of different rollouts?
5 comments

Q1 and W1.

We appreciate the reviewer's feedback on validating our approximation. We compare our approximation with an accurate Monte Carlo estimate with 4×1054 \times 10^5 samples. We use all test problems of the eight models. The following table reports Pass@kk for both. They are essentially the same. This holds even at large budgets, where our probability estimate is less accurate. Our approximation is also deterministic and submodular. The greedy guarantee therefore applies.

@2@4@8@16
Monte Carlo, 4×1054 \times 10^5 samples51.2162.0371.2579.82
Our approximation51.1261.8871.2579.86
Error of our probability estimate, relative to Monte Carlo0.3%1.1%5.0%30.6%

Our approximation also preserves the ranking of the rollouts. The following table reports how many of the rollouts it keeps are also kept by Monte Carlo.

@2@4@8@16
Rollouts kept by both our approximation and Monte Carlo99.3%98.3%96.0%92.5%

Monte Carlo is also much slower. We run it with fewer samples and compare the rollouts it selects with ours. We measure how likely each selection is to be all wrong, using a separate Monte Carlo estimate with 2×1052 \times 10^5 samples. The following table reports how much more likely this is for Monte Carlo, in relative terms. With 500500 samples, Monte Carlo is already about 2525 times slower than our approximation. It also selects worse rollouts.

MethodTime per problem@2@4@8@16
Our approximation0.1 ms0000
Monte Carlo, 500500 samples2.7 ms+0.46%+1.95%+5.74%+11.14%
Monte Carlo, 2,0002{,}000 samples8.3 ms+0.13%+0.55%+1.87%+4.16%
Monte Carlo, 8,0008{,}000 samples29.2 ms+0.04%+0.20%+0.66%+1.21%
Monte Carlo, 32,00032{,}000 samples113 ms+0.01%+0.04%+0.15%-0.25%

W2.

We agree that separating the two is important, and we now do so. PKTP has two stages, modeling the rollouts jointly and selecting the set that maximizes Pass@kk. The following table replaces each stage with its independent counterpart and reports Pass@kk averaged over the eight models. Each stage helps on its own, and the full method is the best at every budget. In particular, with the same fixed predictor, our selection rule improves on top-kk for both predictors.

ModelingSelection@2@4@8@16
IndependentTop-kk (Linear Probe)50.3860.1369.2177.69
IndependentPass@kk50.7860.8270.2478.54
JointTop-kk51.3161.2070.1478.39
JointPass@kk (PKTP)51.6362.0071.1379.35

Q2.

Intuitively, f\mathbf{f} is a Gaussian prior on the correct direction in representation space for a given problem, which cannot be known in advance. The learned quality term gives our best estimate of how promising each rollout is on its own. The latent vector f\mathbf{f} is drawn once per problem and shared across all rollouts, adding hi⊤fh_i^\top \mathbf{f} to reflect how closely each rollout's representation aligns with that direction. Rollouts with similar representations align with it in the same way, so they succeed or fail together, which is where the correlation comes from. The noise mi\mathbf{m}_i captures the randomness specific to each rollout after the cutoff.

The correlation term is therefore not only a technical device, since whether the kept rollouts fail together is exactly what Pass@kk depends on. Without it, PKTP would reduce to keeping the kk highest-scoring rollouts, as the Linear Probe does. Using representations for it is also natural, since they encode the reasoning a rollout has followed so far, and it is supported by the data. For every problem we take all pairs of its rollouts and compute the cosine similarity of their representations, and we then split the pairs into ten equally sized groups, from least to most similar. For each group, the following table reports how much more often the two rollouts of a pair are both correct or both wrong than they would be if they were independent, in points. Agreement grows with similarity on every model, as Eq. 3 assumes by modeling the correlation of two rollouts through the cosine similarity of their representations.

Model12345678910
OpenMath-Nemotron-7B-2.72-1.44-1.11-0.93-0.48+0.12+0.58+1.05+1.38+1.75
DeepSeek-R1-Distill-Qwen-7B-1.50-1.01-0.61-0.43-0.30-0.20+0.04+0.20+0.62+1.50
DeepSeek-R1-Distill-Qwen-1.5B-1.41-0.97-0.72-0.50-0.44-0.08+0.10+0.37+0.48+1.45
Qwen3-1.7B-1.53-1.36-1.06-0.80-0.35+0.01+0.36+0.72+1.02+1.10

Comments on the paper

1 comment

Hi Reviewers,

The authors have provided responses. Please check if they have addressed your concerns.

Thanks, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.