Coupled Autoregressive Decoding: Marginal Contracts and Success Geometry

Generating several answers gives a language model multiple chances to solve a problem. The benefit depends on whether those answers succeed together or cover one another's failures. Coupled decoding changes this dependence while preserving each answer's distribution. We study when coordinating answers in this way increases the chance of obtaining at least one correct answer. We compare two requirements for matching the original model: preserving each complete answer's distribution, and additionally preserving its next-token probabilities after observing the past tokens of every answer in the batch. The additional restriction limits coordination and can reduce the highest achievable success probability. Neither requirement alone guarantees improvement. To understand which sampling methods help, we use an arithmetic decoder that generates a complete answer from a single number between zero and one. Each answer corresponds to an interval of numbers that generate it, so correct answers occupy a collection of intervals. Dividing the original interval into equal pieces and drawing one number uniformly and independently from each piece provides a guarantee: with a common decoder and correctness rule, the chance of obtaining a correct answer is at least as high as with independent uniform draws from the whole interval. This guarantee does not generally hold for equally spaced points moved together by a random amount, or for sampling anew within equal pieces at each token. Both alternatives can help or hurt. Reordering token intervals can even reverse the effect of equally spaced sampling without changing individual answer probabilities. Experiments find no supported gain for the selected couplings on open-ended math, but a gain from equally spaced sampling on multiple-choice Countdown. A separate Countdown study finds a moderate association between estimated success geometry and independently measured gains.