Let The Model Choose: Selecting Samplers For Diverse Answers

Best-of-N selection, pass@k evaluation, and offering users alternatives all draw several answers from one language model. Independent sampling spends part of this budget on repeats. Stratified arithmetic sampling coordinates the answers instead. Each answer decodes one random number drawn from its own slice of the unit interval, so the set spreads over the model’s options while every answer keeps the model’s distribution. Kept for the whole answer, this number is rescaled at every token. Unfortunately, that rescaling operation leads to accumulated precision errors, and in float64 it runs out of precision after a median of 44–79 tokens of a real response. The rest of the text is then chosen by rounding error. This, and similar issues, have prevented more widespread adoption of arithmetic sampling and similar methods. To overcome this flaw, we introduce early release: an answer switches back to ordinary sampling as soon as its prefix interval lies inside its slice. We prove that early release produces exactly the joint distribution of answer sets of infinite-precision stratified arithmetic sampling, and that with eight answers it needs more than float64’s 53 bits with probability below 2−50. In practice it releases after a median of 6–12 tokens and runs 14% faster than always-on arithmetic sampling. It composes with top-p, top-k, min-p, and p-less. Across models from 360M to 70B parameters, early release finds 11.4% more distinct correct answers on CoverageQA over a grid of four truncation rules and four temperatures, raises pass@8 on all 11,313 TriviaQA validation questions from 34.72% to 35.93%, and yields 3.6% more quality-approved meanings on NoveltyBench, with no detectable change in the quality of individual answers.