acceptodds
Under review as a conference paper at ICLR 2027

When Does Sharing a Sampled Reasoning Prefix Pay Off?

Abstract

Generating several reasoning candidates and keeping any correct one raises coverage, the fraction of problems that at least one candidate solves. Continuing one sampled prefix into several candidates makes this cheaper, but every candidate then inherits the prefix's early choices. We ask when sharing a sampled prefix pays off at an equal budget of generated tokens. When continuations keep the decoding settings and generation allowance of independent sampling, sharing the first tokens saves a fraction of a second candidate's generation and removes a fraction of its coverage gain, the share of success variance these tokens explain. On each problem, independent sampling at the same expected budget covers it at least as often as every preset allocation of candidates to sampled prefixes of length if and only if . Measuring both fractions from continuations of stored prefixes, under plans fixed before sampling, on grade-school mathematics of intermediate difficulty with four base models and a thinking model, we find that early tokens resolve success variance faster than they consume generation: the shortest tested prefixes explain 1.8 to 3.5 times their share of generation. At the same expected budget, eight candidates sharing one prefix covered significantly fewer problems than independent sampling for every model and branch position tested. By the time candidates branch, they already share part of their chances of success.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.