Shared-Prefix Self-Consistency
Abstract
Self-consistency improves reasoning by sampling complete reasoning traces in- dependently and aggregating their answers by majority vote, but each addi- tional trace requires another full generation. We introduce Shared-Prefix Self- Consistency (SP-SC), a simple variant that generates complete reasoning trunks, retains a prefix from each, and samples additional continuations. Original trunks and continuations contribute equally to the final vote. Using Qwen3-14B on AIME 2025 and HMMT 2025, we examine how prefix length, trunk count, and continuation budget affect accuracy and generated-token cost. Offline evaluation over fixed generation pools shows that selected configurations improve the ob- served trade-off relative to standard, adaptive, and early-stopping self-consistency. For example, on AIME 2025, an eight-candidate configuration achieves 81.72% accuracy using 24.5% fewer generated tokens than eight-sample self-consistency, which achieves 79.44%. Longer prefixes can reduce generation cost while low- ering accuracy; multiple trunks mitigate this sensitivity in several evaluated con- figurations. A separate rescue analysis shows that erroneous traces can contain prefixes from which correct continuations emerge.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.