Beyond Embedding Mixing: Value-Level Candidate Fusion for LLM Reasoning

Chain-of-thought reasoning selects a single token at each decoding step, limiting the direct use of alternative-candidate information. Soft Thinking mixes candidate embeddings to preserve multiple reasoning directions, but its trajectory often favors the top-1 candidate. Subsequent work introduces reweighting or regularization to mitigate this bias. However, our layerwise analysis shows that these adjustments can change the dominant candidate without preventing predictions from progressively approaching a single candidate's distribution across layers. We hypothesize that, when candidates are merged into a shared input representation before entering the network, layerwise processing amplifies higher-weighted candidates' influence while attenuating alternatives. We therefore examine the role of aggregation depth and find that delayed aggregation alleviates single-candidate dominance in model predictions. Building on this finding, we propose DIVER, a training-free decoding method that independently contextualizes candidate tokens under the same reasoning prefix before fusing their attention Values at selected internal layers. DIVER maintains a single discrete generation trajectory and, at these layers, replaces the selected token’s Values with a probability-weighted mixture of the independently computed candidate Values. The fused Values are stored in the KV cache, making information from unselected candidates accessible to subsequent tokens through attention. We conduct Value-path patching experiments to identify fusion layers where candidate information can effectively influence subsequent reasoning. Entropy gating further concentrates fusion on uncertain steps, retaining standard CoT elsewhere to avoid unnecessary perturbations and computation. Experiments across reasoning benchmarks demonstrate average accuracy gains of up to **2.90%** and **4.14%** over CoT and Soft Thinking, respectively.