Learning When to Sample: Confidence-Aware Selective Sampling for Efficient Chain-of-Thought Reasoning

Large language models (LLMs) can achieve strong reasoning performance through chain-of-thought (CoT) reasoning, yet they often generate unnecessarily long reasoning paths that incur high inference cost. Self-consistency-based approaches push accuracy higher, but they require sampling and aggregating multiple reasoning paths, leading to substantial computational overhead. In this paper, we introduce a confidence-aware selective sampling framework that, at inference time, analyzes a reasoning trajectory to adaptively determine whether to trigger multi-path sampling to potentially obtain better performance. The framework uses trajectory-level and sentence-level linguistic features extracted from reasoning states to guide selective multi-path reasoning. We train it on MedQA and evaluate it on MedQA and under mild calibration-only transfer on MathQA, MedMCQA, and MMLU, without further fine-tuning. Experimental results show that the proposed framework maintains comparable performance to full and efficient multi-path reasoning baselines, with insignificant accuracy changes of $-0.40\pm 0.57$% and $-0.30\pm 0.58$%, respectively, while dramatically reducing token usage by $72.0\pm 4.8$% and $37.3\pm 8.8$%. These findings demonstrate that reasoning trajectories contain rich signals for uncertainty estimation, enabling a simple, transferable mechanism to balance accuracy and efficiency in LLM reasoning.