- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Recent studies have found a useful test-time effect. If we sharpen the output probability distribution of a base model, the model becomes more likely to choose answers that it already considers probable. This can achieve many effects similar to those brought by reinforcement learning.
Based on these findings, this work proposes Depth-Entropy Guided Sampling (DEGS). DEGS measures how internal entropy changes during generation. It then computes a sequence-level “collapse depth” . This depth is combined with the standard sequence probability. Together, they can be used to judge the quality of a response.
The authors also combine DEGS with an MCMC power-sampling framework. This gives DEGS-MCMC. It uses two signals to guide sampling. One signal is probability. The other signal is collapse depth. With these signals, the sampling process gradually becomes more likely to find higher-quality answers.
Critical strengths
The paper finds that the depth of internal entropy collapse in the model is correlated with response quality.
The paper proposes a training-free sampling optimization method that is effective on Qwen2.5-7B.
Critical weaknesses
Using internal entropy to analyze response quality is not a very novel idea. Moreover, although the paper finds that collapse depth needs to be combined with logits to effectively judge response quality, it does not specifically analyze how reliable this metric actually is. This further weakens the contribution of the work.
All experiments in the paper are conducted on the Qwen2.5-7B architecture, so it cannot be ensured that the findings of this work are generalizable.
Questions for authors
Are collapse depths correlated with response quality on architectures such as LLaMA3, Qwen3, and Qwen3.5, as well as at other parameter scales? Do Best-of-N-Entropy and DEGS-MCMC remain effective?
When is used as a per-candidate discriminator, what are its AUC, Spearman correlation, and pairwise-ranking accuracy?
What differences are there between responses with higher and those with lower in terms of reasoning patterns, generation length, and other behaviors? Are there any stable patterns?
Some high-entropy tokens usually appear in responses. What impact do these tokens have on collapse depth? How do different values of affect the collapse depth at these positions?
Under more hyperparameter settings involving random seeds, temperature, and max length, can Best-of-N-Entropy and DEGS-MCMC still achieve stable improvements?
Is sequence-level collapse depth related to response length? Are there differences in collapse depth among responses of different lengths?
9 comments
Official Response to Reviewer tuHY (Part 2/7)
About the Architecture and Parameter Scale
Are collapse depths correlated with response quality on architectures such as LLaMA3, Qwen3, and Qwen3.5, as well as at other parameter scales? Do Best-of-N-Entropy and DEGS-MCMC remain effective?
We add a comparison of Qwen2.5-Math-1.5B and 7B using the same first 100 MATH500 problems, generation seeds 0/1/2 and first 16 candidates per problem/seed (4,800/model). We use T=.25, top-p=1, disabled top-k, max 768 tokens, alpha=4, beta=5, tau=.25 nats, probes {4,8,12,16,20,24,28} and final RMSNorm.
| Model | Selector | Accuracy; contrast vs same-model likelihood |
|---|---|---|
| 1.5B | Likelihood | 60.00%; reference |
| 1.5B | Final entropy | 75.00%; +15.00 pp |
| 1.5B | DEGS reranking | 62.00%; +2.00 pp |
| 1.5B | Majority voting | 72.00%; +12.00 pp |
| 7B | Likelihood | 67.00%; reference |
| 7B | Final entropy | 82.00%; +15.00 pp |
| 7B | DEGS reranking | 68.00%; +1.00 pp |
| 7B | Majority voting | 79.00%; +12.00 pp |
Final entropy minimizes mean predictive entropy; DEGS reranking maximizes 4 log p+5D with post-token depth. The prompt, parser and grader are shared. Majority voting groups normalized extracted answers and breaks ties by earliest candidate; invalid groups are excluded, and all-invalid pools count incorrect. Accuracy averages all problems and generation seeds.
Best-of-N-Entropy improves on likelihood selection by 15.00 pp and majority voting by 3.00 pp at both scales. This comparison evaluates fixed-pool selection at 1.5B/7B. Part 7 evaluates both proposed decoders on Qwen3-8B-Base; the submitted Table 1 provides the DeepSeek comparison. Together, these comparisons cover both parameter scales and the additional Qwen3 and DeepSeek checkpoints.
Official Response to Reviewer tuHY (Part 3/7)
About the Discrimination and Reasoning Behavior
what are its AUC, Spearman correlation, and pairwise-ranking accuracy?
Per-candidate discrimination. Figure 2 reports all three metrics for collapse depth on Qwen2.5-Math-7B. Each dataset contains 500 problems ×16 responses at T=.8/top-p=.9:
| Submitted Fig. 2 | AUC | Spearman rho(D,correct) | Pair-ranking accuracy |
|---|---|---|---|
| MATH500 | .595 | .150 | .571 |
| DeepMind500 | .612 | .146 | .592 |
The lowest/highest-D buckets have accuracies of 17%/41% on MATH500 and 11%/29% on DeepMind500. D complements likelihood, and Tables 1–2 evaluate their combined use. The combined score has the same ranking as . The joint-matching analysis in our response to Reviewer fjAd compares this combined score directly with likelihood and likelihood plus final entropy; for EOS-ended, jointly matched pairs, its ranking accuracy is 65.92%, versus 51.93% for likelihood.
Scale comparison. For the first-100, three-seed, N=16 pools in Part 2, we compare correct and incorrect candidates within the same problem across seeds, giving ties half credit and weighting pairs equally:
| Model | D AUC / within-problem pair accuracy |
|---|---|
| Qwen2.5-Math-1.5B | 0.649/0.650 |
| Qwen2.5-Math-7B | 0.650/0.649 |
The longer-horizon random-100 configuration in Part 5 gives:
| T / cap | AUC / rho(D,correct) / pair accuracy / rho(D,length) |
|---|---|
| .25/3072 | 0.600/0.171/0.600/0.100 |
Pooled AUC and within-problem pair accuracy measure complementary aspects of discrimination. The latter compares candidates for the same question; the joint-matching analysis additionally controls likelihood, length and final-layer entropy.
Reasoning behavior. D summarizes the evolution of internal confidence across layers through the average normalized first-crossing depth. Figure 3 illustrates the entropy trajectories motivating this statistic, while Part 6 examines response length explicitly. Table 7 also links probe placement to downstream accuracy: later-layer probes yield 76.6%/42.6% on MATH500/GPQA, compared with 72.8%/38.5% for the earliest probes in the four-probe comparison.
Official Response to Reviewer tuHY (Part 1/7)
Thank you for the questions on novelty, reliability and generalization. We present the contribution and the corresponding experimental evidence below.
About the Contribution and Evaluated Models
Novelty and reliability. Best-of-N-Entropy selects complete responses by final-layer entropy; DEGS-MCMC adds normalized earliest entropy-crossing depth to a likelihood-guided sequence target, π(x)∝p(x)αexp(βD(x)). Both use the existing model without parameter updates or a trained verifier. The auxiliary DEGS reranker is a separate control.
The distinction from LED (Tan et al., ICML 2026) is the statistic and its use: LED constructs token-level exploration distributions from intermediate posteriors, whereas DEGS scores complete sequences by first-crossing depth and incorporates this score into MH acceptance. This completion-level statistic and sequence-sampling objective define the methodological contribution. Figure 2 quantifies the signal's reliability, and Table 2 tests its combined use with likelihood within each decoding mode.
Evaluated models. Table 1 includes Qwen2.5-7B, Qwen2.5-Math-7B and DeepSeek-Math-7B. Appendix E specifies 28-layer Qwen and 30-layer DeepSeek probing, with grids {4,8,12,16,20,24,28} and {4,9,14,18,22,26,30}. Depths are normalized by the number of layers. The submitted DeepSeek results are:
Both proposed decoders improve over their corresponding likelihood baselines across these four tasks. Parts 2 and 7 add the parameter-scale and Qwen3 comparisons, respectively.