- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents DivThink, a test-time method that applies logit repulsion to common vocabulary across parallel trajectories, in order to increase diversity across the trajectories whose token distributions are primarily guided by the model.
Critical strengths
- DivThink provides a test-time parallel sampling mechanism where the samples collaborate to be dissimilar to one another in every token step via logit repulsion.
- The authors showed that controlling for eight trajectories, DivThink improves accuracy over base sampling and low-temperature sampling, and also reduces token consumption compared to MCMC Power Sampling though the benefit is small for code generation (HumanEval).
- A good number of four benchmarks and four models were used in the main experiments.
Critical weaknesses
- Comparison of DivThink with low-temperature is weak as low temperature favors greedy-like generation that therefore likely converges to similar reasoning modes. For example, low temperature T=0.25 on L796 is likely to be mode-seeking.
- L364: Whilst high temperature sampling produces less-usable results, did you tune temperature between 0 and 2? There should be a comparison between DivThink and a tuned temperature; as an example, in Tables 3 and 4 relating to GPQA Diamond, when we compare Base@8 at temperature 0.50 to DivThink@8, there is no clear winner on all four metrics, more so in the absence of standard deviation.
- Besides Base and Low-temperature sampling, are there other decoding methods targeted at improving exploration and coverage of reasoning paths?
- L395-400: Are there analysis of generated trajectories to support the claim "... with repulsion being most useful when independent trajectories remain redundant; when the base candidate set is already strong, additional diversification offers less headroom and can slightly reduce local quality" that seems to be used as an explanation for the result "... Base@8 remains stronger on HumanEval"
- Even with token budget controlled, how much more compute (or memory) overhead do the logit repulsions introduce? Are there efficiency analysis?
- Table 1 and 2: What are the standard deviation of the accuracies?
- Since DivThink diversifies parallel generations, what are the effects of DivThink on tasks such as logical reasoning and open tasks like creative writing?
- L858: Why is the lambda value of 0.4 selected when it produces the worst accuracy changes in Figure 5 (Right)?
- It is unclear how DivThink affects logits of various distribution profiles. Could you provide synthetic examples of token distribution profiles adjusted with logit repulsion?
- The references contain many errors in years and page numbers, and at least an author name.
Questions for authors
Please refer to questions, clarifications and feedback in weaknesses.
3 comments
Q6. Extension to logical reasoning and creative writing
We evaluated DIVTHINK and independent sampling on propositional reasoning and constrained creative writing. Both evaluations used and three decoding seeds, with matched generation settings within each task. On 36 propositional reasoning problems, the two methods achieved similar mean accuracy with Qwen2.5-7B. The labels were verified by exhaustive truth-table enumeration.
For creative writing, we used Qwen3-14B and measured compliance with explicit prompt requirements. The table reports aggregate compliance across prompts and decoding seeds.
| Creative-writing metric | Independent sampling | DIVTHINK |
|---|---|---|
| Required-object compliance | 31.94% | 34.03% |
| Word-budget compliance | 94.44% | 95.14% |
DIVTHINK improved adherence to both writing constraints in this exploratory evaluation. We will include the results and protocols for both tasks in the revised paper.
Q7. Choice of and interpretation of Figure 5
The right panel of Figure 5 illustrates that sensitivity to can vary across tasks even for the same backbone. This pattern is consistent with the intended trade-off: moderate repulsion discourages continuations already favored by peer trajectories while preserving the base model’s local preferences. Excessive repulsion can over-penalize otherwise plausible continuations. Because the base logits and peer preferences can differ across models and tasks, the coefficient that best balances these effects can also vary.
Q8. Peer reweighting
The update lowers a token’s odds relative to alternatives that peers favor less. When trajectories share a sharp peak, repulsion can reduce the dominant token’s probability while preserving its rank. Uniform peer probabilities shift all logits by the same amount and leave the sampling distribution unchanged. A lone active trajectory receives no peer penalty.
Consider a decoding trace for a problem that gives and and asks for . At the shared prefix If y = -12, then x =, the sole active peer strongly favors -. Using the same random draw at decoding step 384 (zero-indexed), independent sampling selects -, while DIVTHINK selects y. The resulting substitutions, x = -12 + 9 and x = y + 9 = -12 + 9, are both valid.
The continuations subsequently handle the two solution branches differently. Independent sampling combines values from different branches and reports . DIVTHINK preserves the correct pairings, and , and obtains . The shared-prefix comparison isolates the effect of peer reweighting on token selection at that step. The two continuations show how the resulting divergence played out in this example.
We will include the new experiments, probability examples, decoding trace, and corrections to the references in the revised manuscript.
I thank the authors for performing additional experiments and analyses that address some of my questions.
Could you provide the paired seed results for DIVTHINK vs temperature sampling on Qwen3-14B (Q1), and DIVTHINK vs temperature sampling for each aggregation strategy (Q3)?
While temperature-tuned DIVTHINK improved adherence to the two creative writing constraints, is there preference evaluation on paired prompts across the two methods? How many creative writing prompts were used? What are the paired seed differences between the two methods on the two creative writing constraints and the preference?
Could you quantify these? "DIVTHINK reduced exact duplicates and shortened shared prefixes"
Are you able to compare DIVTHINK on a larger model ~50-100B in parameter size of another model family?
On the balance of the new results and the above, I currently maintain my evaluation.
We thank the reviewer for the detailed feedback.
Q1. Comparison with temperature-tuned independent sampling
We selected temperatures separately for independent sampling and DIVTHINK using 50 MATH training problems and 30 GPQA-main problems outside Diamond. Both methods used the same validation grid (0.1, 0.25, 0.5, 0.75, 1.0, 1.25, 1.5), prompts, K=8, 3,072-token cap per trajectory, answer extraction, and majority voting. DIVTHINK used λ=0.4. Each row reports mean majority-vote accuracy over five decoding seeds.
DIVTHINK achieved higher mean accuracy in all four comparisons after both methods were tuned on the same validation grid. In an additional Qwen2.5-7B GPQA validation check, both methods performed poorly at T=1.75 and T=2.0. Higher temperatures flatten the next-token distribution, making low-probability continuations more likely and potentially disrupting multi-step reasoning.
Q2. Multi-seed accuracy and uncertainty
We report mean majority-vote accuracy and 95% confidence intervals for the gain over independent sampling.
DIVTHINK improved mean accuracy in all four comparisons, and the reported intervals for all four gains lie above zero.
Q3. Stronger inference-time baselines
On Qwen2.5-7B base, we applied each aggregation method to the same eight saved candidates per generator. The main comparisons average five decoding seeds, and the matched-temperature DeepConf control averages three paired seeds.
With the aggregation method held fixed, DIVTHINK achieved higher mean accuracy on both benchmarks. The highest observed MATH500 accuracy came from combining DIVTHINK with offline DeepConf. These results cover DeepConf's offline K=8 component.
For prompt-based diversification, DIVTHINK@8 achieved 66.7% and the official
VS_STANDARDprompt achieved 62.5% majority-vote accuracy on the same MATH subset, using Qwen3-14B at T=0.7 under the same generation token cap.Q4. Redundancy and headroom
DIVTHINK improved mean HumanEval pass@8 with Qwen2.5-7B under matched decoding settings for both raw completion and chat-formatted prompts. At the same candidate count, it solved more problems on average in both settings, including the chat setting where independent sampling already performed well.
In the matched Qwen3-14B evaluation, DIVTHINK reduced exact duplicates and shortened shared prefixes, alongside a modest improvement in pass@k. The trajectory metrics show less overlap among generated programs, while the pass@k gain shows that more problems had at least one test-passing candidate at the same candidate count. This connects the trajectory analysis to the outcome that matters for HumanEval: coverage of correct solutions. The gains from a strong independent baseline show that useful coverage headroom can remain even when baseline pass@k is already high.
Q5. Compute and memory overhead
On an A100 with Qwen2.5-7B, both arms decoded 256 steps from the same 64-token prefix, with identical forced tokens and forward-pass counts. Confidence and trace collection were disabled.
In this controlled decoding benchmark, DIVTHINK added less than 1% runtime at both trajectory counts, with similar peak allocated memory.