acceptodds
Under review as a conference paper at ICLR 2027

Beyond Embedding Mixing: Value-Level Candidate Fusion for LLM Reasoning

Abstract

Chain-of-thought reasoning selects a single token at each decoding step, limiting the direct use of alternative-candidate information. Soft Thinking mixes candidate embeddings to preserve multiple reasoning directions, but its trajectory often favors the top-1 candidate. Subsequent work introduces reweighting or regularization to mitigate this bias. However, our layerwise analysis shows that these adjustments can change the dominant candidate without preventing predictions from progressively approaching a single candidate's distribution across layers. We hypothesize that, when candidates are merged into a shared input representation before entering the network, layerwise processing amplifies higher-weighted candidates' influence while attenuating alternatives. We therefore examine the role of aggregation depth and find that delayed aggregation alleviates single-candidate dominance in model predictions. Building on this finding, we propose DIVER, a training-free decoding method that independently contextualizes candidate tokens under the same reasoning prefix before fusing their attention Values at selected internal layers. DIVER maintains a single discrete generation trajectory and, at these layers, replaces the selected token’s Values with a probability-weighted mixture of the independently computed candidate Values. The fused Values are stored in the KV cache, making information from unselected candidates accessible to subsequent tokens through attention. We conduct Value-path patching experiments to identify fusion layers where candidate information can effectively influence subsequent reasoning. Entropy gating further concentrates fusion on uncertain steps, retaining standard CoT elsewhere to avoid unnecessary perturbations and computation. Experiments across reasoning benchmarks demonstrate average accuracy gains of up to **2.90%** and **4.14%** over CoT and Soft Thinking, respectively.

Primary area:
generative models
Keywords:
Large Language Models, Continuous Reasoning, Training-Free Decoding

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_PHJ9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper argues that mixing candidate embeddings at the input can lead to single-candidate dominance during layerwise processing. It proposes DIVER, which independently processes candidate tokens before fusing their attention Values at selected internal layers. The experiments show improvements across multiple models and benchmarks. However, it remains unclear whether the suppressed candidates contain useful information and whether the performance gains justify the additional computation. Stronger accuracy improvements or lower computational costs would strengthen the contribution.

Critical strengths
  1. The paper is well motivated by a concrete layerwise analysis rather than intuition alone.

  2. The experimental coverage is broad enough to make the gains credible. DIVER is evaluated on seven backbones from 1.7B to 32B, on math, science, and code benchmarks, and achieves the highest average accuracy on every backbone.

Critical weaknesses
  1. The fusion layers are selected by a per-model calibration procedure on 1,000 synthetic arithmetic tasks and then fixed for all domains. Performance is sensitive to this choice (Table 4: fusing at layers 1–4 or all layers is yields lower average accuracy than 21–24), but the paper does not examine whether the selection is robust: it does not report what layers would be chosen when calibrating on science or code examples, nor how accuracy changes when the window is shifted by a few layers.

  2. The accuracy gains come with additional computational costs. The current comparisons do not sufficiently establish whether DIVER offers a better accuracy–runtime trade-off than alternatives under a comparable budget.

Questions for authors
  1. Are candidates whose influence weakens under embedding mixing more likely to lead to correct or efficient solutions? Can the authors analyze their downstream outcomes?

  2. Under the same runtime budget, does DIVER outperform CoT with multiple samples and voting, or other inference-time methods?

  3. Fusion layers are selected using simple arithmetic tasks. Although these layers yield gains across several benchmarks, how sensitive are the selected layers and final performance to the calibration samples or task domain?

5 comments
AuthorsOpenReview ↗

Response to Reviewer PHJ9 [1/2]

We thank the reviewer for the positive comments on our research motivation, layerwise analysis, and experimental coverage. Below, we provide responses to the reviewer’s questions about fusion layer selection, computational cost, and the downstream effects of alternative candidates.

W1&Q3: Fusion layer selection and sensitivity to window shifts

To examine whether the Value-path patching results depend on the task domain, we repeat the analysis on a controlled code dataset, keeping all other settings the same as in the arithmetic experiments. The table below reports recovery toward the second candidate. For both models, recovery at L21–24 is notably higher than at either adjacent window, providing support for retaining this window when moving from the controlled arithmetic setting to the controlled code setting.

WindowQwen3-4B-Thinking-2507Qwen3-8B
17–204.73%6.24%
21–24(We choose)72.14%73.91%
25–2814.75%7.94%

Moreover, to evaluate the robustness of fusion layer selection, we shift the original L21–24 window four layers earlier and later. We compare L17–20, L21–24, and L25–28 on mathematical reasoning, scientific question answering, and code generation tasks using Qwen3-4B-Thinking-2507 and Qwen3-8B. Since AIME 2024 is a small dataset, we perform eight evaluation runs on this dataset and report the mean accuracy. The results are as follows:

QWEN 4B Thinking 2507

WindowAIME2024GPQA DiamondLiveCodeBenchAverage
21–24 (Our choice)84.58%68.69%68.46%73.91%
17–2082.50%65.15%67.03%71.56%
25–2883.33%66.67%67.74%72.58%

QWEN 8B

WindowAIME2024GPQA DiamondLCBAverage
21–24 (Our choice)78.75%60.10%62.01%66.95%
17–2076.67%61.62%59.50%65.93%
25–2877.92%59.60%60.57%66.03%

On both models, the original L21–24 configuration achieves the highest average accuracy across tasks. This indicates that our selected fusion layers exhibit good robustness across different tasks. We have added the relevant experiments and discussion to the paper and highlighted them in blue.

AuthorsOpenReview ↗

Response to Reviewer PHJ9 [2/2]

W2&Q2: DIVER's performance and efficiency advantages under comparable computational budgets

To better evaluate the trade-off between accuracy and computation time, we have added experiments under comparable computational budgets, comparing DIVER with the CoT Sampling and Soft Thinking baselines combined with Self-Consistency.

First, we compare DIVER, CoT Sampling+SC@3, and Soft Thinking+SC@3 on AIME 2024 and AIME 2025. We use SC@3 as a common low-budget setting for both baselines. At this setting, both baselines have runtimes close to that of DIVER. Each method is evaluated eight times, and we report the average accuracy and average runtime across the two datasets.

MethodModelAverage TimeAverage Accuracy
CoT Sampling + SC@3Qwen3-4B-Thinking-250741.26 min79.79%
Soft Thinking + SC@3Qwen3-4B-Thinking-250751.88 min80.63%
DIVER (ours)Qwen3-4B-Thinking-250750.21 min81.67%
CoT Sampling + SC@3Qwen3-8B38.99 min75.83%
Soft Thinking + SC@3Qwen3-8B48.44 min73.33%
DIVER (ours)Qwen3-8B37.68 min75.21%

Under comparable computational budgets, DIVER improves accuracy on Qwen3-4B-Thinking-2507 by 1.04 and 1.88 percentage points over Soft Thinking+SC@3 and CoT Sampling+SC@3, respectively. On Qwen3-8B, DIVER achieves performance comparable to CoT Sampling+SC@3 and improves accuracy over Soft Thinking+SC@3 by 1.88 percentage points. These results demonstrate that DIVER exhibits good performance.

More importantly, DIVER can also be combined with self-consistency method. We therefore further compare DIVER+SC@8, CoT Sampling+SC@30, and Soft Thinking+SC@30 on AIME 2024 and AIME 2025, reporting the average accuracy and average runtime across the two datasets. Under this setting, DIVER+SC@8 achieves higher accuracy than both baselines on both models.

ModelMethodTimeAccuracy
Qwen3-4B-Thinking-2507CoT Sampling + SC@306.91h88.33%
Qwen3-4B-Thinking-2507Soft Thinking + SC@308.62h88.33%
Qwen3-4B-Thinking-2507DIVER + SC@86.69h90%
Qwen3-8BCoT Sampling + SC@306.38h80.00%
Qwen3-8BSoft Thinking + SC@308.03h78.33%
Qwen3-8BDIVER + SC@85.02h83.33%

These results demonstrate that DIVER not only exhibits good performance but can also be combined with self-consistency voting. We have added these supplementary experiments to the appendix of the paper and highlighted them in blue.

Q1: Are candidates whose influence is weakened by embedding mixing more likely to produce correct or efficient solutions?

Predictions after embedding mixing tend to favor the top-1 candidate, while other candidates have relatively less influence. To examine the effects of these candidates with weakened influence on subsequent reasoning, we select the 1st, 10th, and 50th high-entropy positions in the reasoning trajectories for the 30 questions in AIME 2025, replace the top-1 token at each selected position with the top-2 or top-3 token, and then continue generation. We record the final accuracy, the average number of generated tokens, and the numbers of problems that change from incorrect to correct and from correct to incorrect relative to the baseline.

Replacement PositionCandidateAccuracyAverage Number of Generated TokensBaseline Incorrect → CorrectBaseline Correct → Incorrect
No replacementtop-1 baseline66.67%18,728––
1st high-entropy positiontop-270.00%16,67143
1st high-entropy positiontop-363.33%19,02723
10th high-entropy positiontop-263.33%18,60123
10th high-entropy positiontop-370.00%17,62432
50th high-entropy positiontop-270.00%18,09932
50th high-entropy positiontop-356.67%19,43414

The results show that candidates other than top-1 can lead to correct solutions that the baseline fails to obtain, but replacing top-1 can also turn previously correct answers into incorrect ones. For example, selecting top-2 at the first high-entropy position corrects four questions answered incorrectly by the baseline while causing three previously correct answers to become incorrect. The average generation length decreases from 18,728 to 16,671. These findings indicate that alternative candidates can support successful reasoning, while the top-1 candidate also remains valuable. This motivates DIVER’s design of combining information from top-1 and alternative candidates to preserve it for subsequent reasoning.

I thank the authors for their detailed response and the substantial effort devoted to the additional experiments. These results are helpful, but several concerns remain.

  1. The accuracy–cost advantage remains unclear. (W2, Q2) At the lower budget, DIVER performs similarly to CoT+SC@3: on Qwen3-8B, accuracy is slightly lower (75.21 vs. 75.83); on Qwen3-4B, the improvement is 1.88 percentage points with approximately 22% more runtime. At the higher budget, Figure 6 appears to suggest that CoT+SC largely plateaus around SC@8, making a comparison at that operating point particularly relevant. A fuller accuracy–runtime curve would help establish the trade-off more clearly. The higher-budget accuracy margins also correspond to only one or two questions out of 60, which makes uncertainty estimates especially valuable.

  2. The statistical robustness of the improvements is difficult to assess. (W2/Q2) The reported average gains over CoT are 1.15–2.90 percentage points. Although I appreciate the eight-run evaluations added for some comparisons, means alone do not show how consistently these gains hold. For the same default Qwen3-8B configuration, the single-run AIME 2024 ablation result is 83.33, whereas the eight-run mean is 78.75. This difference does not invalidate the results, but it makes reporting variability particularly relevant when interpreting relatively small improvements.

  3. The layer-selection results and candidate analysis leave some questions unresolved. (W1, Q3; Q1) The selected window shows substantially higher patching recovery than adjacent windows (approximately 72–74% vs. 5–15%), yet the corresponding downstream accuracy differences are much smaller; a shifted window also performs better on Qwen3-8B GPQA. Thus, the connection between the recovery measure and downstream performance remains unclear to me. Similarly, the token-replacement experiment reports 15 incorrect-to-correct changes and 17 correct-to-incorrect changes. This shows that alternative candidates can affect outcomes, but provides limited evidence that preserving them yields a systematic benefit.

Beyond the issues raised in my initial review, I also have some secondary concerns about the single-run ablations, the unclear criterion for choosing K. On reconsideration, I feel my initial soundness assessment was somewhat optimistic. I will maintain my original weak-reject recommendation.

Reviewer_HFdHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies training-free use of alternative token information in LLM reasoning. Existing embedding mixtures can remain dominated by one candidate despite reweighting or regularization. DIVER independently contextualizes candidates and fuses their Values at selected layers, using entropy gating and Value-path patching. Experiments report average gains of up to 2.90 and 4.14 percentage points over CoT and Soft Thinking, respectively.

Critical strengths
  1. The motivation for delayed aggregation is reasonable. Figures 2 and 3 provide evidence of candidate dominance with aggregation depth.
  2. Value fusion is an interpretable design. Cached mixtures let subsequent tokens can access information from unselected candidates.
  3. The empirical coverage is useful. Tables 1 and 7 report the highest average accuracy among the evaluated methods on all seven backbones.
  4. The ablations help explain key design choices. Table 4 and Appendix D examine fusion depth, candidate count, mixing strength, and entropy threshold.
Critical weaknesses
  1. Sampling considers up to 30 tokens, but fusion retains only 3 or 10 candidates (Appendix A.3). Please clarify how DIVER handles a sampled token outside the fusion set.
  2. Candidate probes omit current-step fusion, but the main path can change after the first intervention (Sec. 4.1). Please explain why unmodified probe Values remain appropriate at subsequent fusion layers.

Typos:

Sec. 4.1, p. 5: “We then define the recovery ratio is” → “We then define the recovery ratio as follows.”

Questions for authors

Please see strengths and weaknesses.

1 comment
AuthorsOpenReview ↗

Response to Reviewer HFdH [1/1]

We appreciate the reviewer’s supportive comments on the motivation for delayed aggregation, the Value fusion design, and our broad evaluation. Below, we respond to the questions about candidate tokens and the use of independent candidate probes at subsequent fusion layers.

W1: Handling sampled tokens outside the fusion set

We thank the reviewer for pointing out this unclear description. In Appendix A.3, sampling top-k = 30 controls the truncation of the discrete sampling distribution, while K = 3 or 10 specifies the number of candidates used for Value fusion. These two parameters are used to select the input token for the main path and construct the candidate Value mixture, respectively.

When the sampled token is not in the fusion candidate set, DIVER still uses it as the main-path input and performs the forward computation; meanwhile, it independently computes the Values of the top-K candidates. At each fusion layer, the weighted mixture of candidate Values is injected into the main path. This requires neither resampling nor adding the sampled token to the fusion candidate set. In practice, the probability of sampling a token outside the fusion candidate set is low (<0.01%). We have added this clarification to the appendix. To further examine performance when the sampling range matches the fusion candidate set, we have added single-run experiments with sampling top-k = 10 and K = 10 on different datasets. The results show that DIVER still achieves good performance.

ModelAIME24AIME25GPQA DiamondMath 500Gsm8k
Qwen3-4B86.67%76.67%67.68%98%95.3%
Qwen3-8B80%73.33%61.11%97%95.83%
DeepSeek-14B73.33%60%59.60%95.80%94.39%

W2: Whether probe Values that have not received the current step's fusion remain applicable after the first intervention

Fusing candidate Values at the first selected layer changes the main path’s hidden states, while the subsequent probe Values come from candidate branches without being adapted to the updated main-path states. However, changes in the main-path states do not imply that candidate Values independently computed in advance become ineffective at subsequent fusion layers. For example, ITI [1] adds vectors along predetermined truthfulness directions to the outputs of selected attention heads at different layers and improves output truthfulness.

To investigate whether candidate Values in DIVER can still contribute effectively to fusion after the main path changes, we add an ablation experiment. Specifically, we fix the first fusion layer at L21 and progressively add L22, L23, and L24, while computing each candidate branch independently, to test whether injecting independently computed candidate Values at subsequent layers still provides benefits after the initial fusion.

DIVER Injection Layer SelectionAIME24AIME25GPQA
L2176.67%66.67%58.08%
L21+L2283.33%70%59.09%
L21+L22+L2380.00%70.00%61.11%
L21+L22+L23+L2483.33%73.33%60.10%

Compared with fusion only at L21, fusion across L21–24 improves accuracy on AIME24, AIME25, and GPQA from 76.67%, 66.67%, and 58.08% to 83.33%, 73.33%, and 60.10%, respectively. These results show that injecting independent candidate representations at subsequent layers can provide additional benefits even after the main path has been affected by the initial fusion. To further improve fusion, better adapting candidate information to changes in the main path is a direction well worth exploring in future work. We have added this ablation experiment and the related discussion to the paper and highlighted them in blue.

[1] Li, Kenneth, et al. "Inference-time intervention: Eliciting truthful answers from a language model."

Regarding the typo

Thank you for pointing this out. We have corrected “We then define the recovery ratio is” to “We then define the recovery ratio as follows.”

Reviewer_KUWPOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper improves Soft Thinking by independently processing candidate tokens and fusing their Attention Values at selected middle layers, only at high-entropy steps, to better use alternative candidates while keeping a discrete trajectory.

Critical strengths
  1. From the method side. The proposed method is mainly an improvement over the classic Soft Thinking method. The proposed method has simplicity and broad applicability — this is what I see as the strength of the paper; therefore, if the effectiveness of the method is further demonstrated by more follow-up work, this work will be very meaningful.

  2. The experimental coverage is relatively broad, and the amount of experiments done is relatively sufficient.

Critical weaknesses

Weaknesses and questions

  1. In Table 1, why is Soft Thinking worse than CoT in average performance in 2 out of the 3 model settings? This result shows that under this experimental setting, Soft Thinking is ineffective in most cases. I suggest that the authors use multiple different temperature settings in the main table to observe the relative strengths of CoT, Soft Thinking, the proposed method DIVER (and other core comparison methods) under different temperature settings. Currently, the authors may have used the same temperature setting for all methods in the main table; the results given can feel strange (because these results show the ineffectiveness of Soft Thinking).

  2. In Table 6, the authors avoided and did not do the experiment of letting CoT use the same computational cost as DIVER. Letting CoT sample 8 times and reporting a result that is stronger than DIVER but also requires more computation almost does not explain anything.
    In addition, strictly comparing with CoT under the same computation time and showing superiority — this is too strict a requirement. But at least the experiments currently done by the authors need improvement to provide more convincing experimental analysis.

  3. The quality of Figure 4 is relatively low and not professional enough. Moreover, the mathematical symbols in the figure are not standard and do not correspond to those in the main text. Furthermore, the formatting and figures in the appendix are also not professional enough.

Some others:
In addition, this is not a clear weakness, but a point worth noting — the authors did not include code in the supplementary material, and more importantly, they did not mention in the paper whether they will release the code after the paper is accepted.

Questions for authors

Please refer to the above comments.

2 comments
AuthorsOpenReview ↗

Response to Reviewer KUWP [1/2]

We thank the reviewer for the positive assessment of our method’s simplicity and applicability and our comprehensive experimental evaluation. In response to the comments on the fairness of comparisons, computational cost, and presentation, we have added the following experiments and clarifications.

W1: Different temperature settings

Thank you for this suggestion. We use temperature = 0.6 because this setting follows the experimental configuration in the Soft Thinking paper and is also the temperature setting recommended by Qwen.

Regarding the observation that Soft Thinking underperforms CoT on some models, its authors note in the official implementation that the method does not consistently work on small models. SwiReasoning also reports similar findings, with Soft Thinking underperforming CoT on some datasets when applied to smaller models. Therefore, our observation that Soft Thinking does not always outperform CoT on smaller models is consistent with prior studies.

We agree with the reviewer's suggestion to compare different temperatures. Since the performance gap between CoT and Soft Thinking is mainly observed on AIME 2024 and AIME 2025, we have added experiments with temperature ∈ {0.4, 0.8} on these two datasets to further examine the performance of each method under different temperature settings. The results are shown below. Each entry reports the mean accuracy over eight runs for AIME 2024 / AIME 2025. The symbols ↑ and ↓ indicate accuracy higher or lower than that of CoT Sampling at the same temperature, respectively.

ModelTemperatureCoT SamplingSoft ThinkingDIVER
Qwen3-4B0.481.67%/76.25%84.17%(↑)/72.08%(↓)85.00%(↑)/78.33%(↑)
Qwen3-4B0.681.67%/72.50%82.92%(↑)/74.58%(↑)84.58%(↑)/78.75%(↑)
Qwen3-4B0.880.83%/77.50%80.00%(↓)/69.17%(↓)82.50%(↑)/79.17%(↑)
Qwen3-8B0.477.92%/62.50%69.17%(↓)/65.83%(↑)76.67%(↓)/68.33%(↑)
Qwen3-8B0.677.08%/68.33%68.75%(↓)/70.83%(↑)78.75%(↑)/71.67%(↑)
Qwen3-8B0.877.50%/66.67%72.50%(↓)/67.50%(↑)80.42%(↑)/69.17%(↑)
DeepSeek-14B0.470.00%/49.17%68.75%(↓)/45.83%(↓)70.83%(↑)/55.83%(↑)
DeepSeek-14B0.670.42%/53.75%66.67%(↓)/44.17%(↓)72.08%(↑)/55.42%(↑)
DeepSeek-14B0.870.83%/48.33%60.00%(↓)/38.33%(↓)71.67%(↑)/56.67%(↑)

The results show that Soft Thinking underperforms CoT in some settings at each of the three temperatures, rather than only at temperature = 0.6. In comparison, across the 18 comparisons involving three models, three temperatures, and two datasets, DIVER achieves higher accuracy than CoT in 17 cases and outperforms Soft Thinking in all 18 cases. These results demonstrate that DIVER exhibits good reasoning performance. We have added this experiment to our paper and highlighted it in blue.

AuthorsOpenReview ↗

Response to Reviewer KUWP [2/2]

W2: Comparing DIVER with baselines under similar computational budgets

To more thoroughly evaluate the trade-off between accuracy and computation time, we have added experiments under comparable computational budgets, comparing DIVER, with two baselines: CoT Sampling with Self-Consistency (SC) and Soft Thinking with SC.

First, we compare DIVER with CoT Sampling+SC@3 and Soft Thinking+SC@3 on AIME 2024 and AIME 2025. We use SC@3 as a common low-budget setting for both baselines. At this setting, both baselines have runtimes close to that of DIVER. Each method is evaluated eight times, and we report the average accuracy and average runtime across the two datasets.

MethodModelAverage TimeAverage Accuracy
CoT Sampling + SC@3Qwen3-4B-Thinking-250741.26 min79.79%
Soft Thinking + SC@3Qwen3-4B-Thinking-250751.88 min80.63%
DIVER (ours)Qwen3-4B-Thinking-250750.21 min81.67%
CoT Sampling + SC@3Qwen3-8B38.99 min75.83%
Soft Thinking + SC@3Qwen3-8B48.44 min73.33%
DIVER (ours)Qwen3-8B37.68 min75.21%

On Qwen3-4B-Thinking-2507, DIVER achieves higher accuracy than both CoT Sampling+SC@3 and Soft Thinking+SC@3, while requiring less runtime than Soft Thinking+SC@3. On Qwen3-8B, DIVER achieves performance comparable to CoT Sampling+SC@3 and higher accuracy with less runtime than Soft Thinking+SC@3. These results demonstrate that DIVER offers competitive performance.

In addition, DIVER can also be combined with self-consistency. We further compare DIVER+SC@8 with CoT Sampling+SC@30 and Soft Thinking+SC@30 on AIME 2024 and AIME 2025. Under this setting, DIVER+SC@8 achieves higher accuracy in less time on both models.

ModelMethodTimeAccuracy
Qwen3-4B-Thinking-2507CoT Sampling + SC@306.91h88.33%
Qwen3-4B-Thinking-2507Soft Thinking + SC@308.62h88.33%
Qwen3-4B-Thinking-2507DIVER + SC@86.69h90%
Qwen3-8BCoT Sampling + SC@306.38h80.00%
Qwen3-8BSoft Thinking + SC@308.03h78.33%
Qwen3-8BDIVER + SC@85.02h83.33%

These results demonstrate that DIVER is competitive not only in the single-trajectory setting but can also be combined with multi-trajectory voting. We have added these supplementary experiments to the appendix of the paper and highlighted it in blue.

W3: Improving figure quality and appendix formatting

We thank the reviewer for pointing out these presentation issues. We have revised Figure 4 and aligned its mathematical notation with the main text. We have also improved the presentation of the case studies and the overall formatting in the appendix to enhance readability.

Regarding code availability, we thank the reviewer for reminding us that the original manuscript lacked a code availability statement. Upon acceptance of the paper, we will release the method implementation and evaluation code.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.