acceptodds
Under review as a conference paper at ICLR 2027

Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning

Abstract

When Large Language Models produce structured outputs such as travel plans, code solutions, or multi-step proofs, individual reasoning steps may appear correct while the output as a whole is globally invalid or non-optimal. Since valid candidates often exist in the candidate pool, reliable selection is a key bottleneck. We propose a decomposed energy function that combines a learned quality scorer with deterministic constraint penalties for verifying structured LLM outputs. The quality scorer is a heterogeneous ensemble of low-rank adapters on a single frozen encoder (3% trainable parameters); the ensemble mean ranks candidates, while its standard deviation estimates epistemic uncertainty. We also experiment with a two-pass strategy that uses uncertainty and constraint violations to trigger regeneration or abstention. Across five benchmarks (GSM8K, MuSR, TravelPlanner, TACO, and Knights & Knaves), our 149M-parameter verifier orchestrating a pool of 7-26B open generators reaches 67.7% on MuSR, compared with 68.0% for Claude Sonnet 4.6, and reduces TravelPlanner constraint violations by 53% relative to Opus 4.6. The results show that structural verification and pretraining-scale priors are complementary: deterministic verification is strongest when constraints are checkable, capturing signals frontier models cannot self-detect, while frontier priors remain stronger for narrative inference and code semantics. A cross-dataset confounding analysis confirms genuine quality discrimination on four tasks and identifies a model-identity shortcut on TACO, the code-generation benchmark. Last-layer retraining reduces the dominant generator’s selection share from 94% to 34% while retaining 88.6% pass@1 on TACO.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Energy Based Models, Large Language Models, Uncertainty Quantification

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_sqm7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to select reliable structured outputs from multiple candidates generated by a large language model. The method adds a learned quality score to a deterministic constraint penalty, where the quality scorer consists of multiple heterogeneous LoRA members sharing a frozen encoder. The mean energy across members is used for ranking, while the standard deviation is used to estimate uncertainty and further determine whether to append additional generation or reject the answer. Experiments cover five task categories: math, narrative reasoning, travel planning, code generation, and logical reasoning.

Critical strengths
  1. The method is clear, and it addresses a problem of practical significance. Delegating quality judgments that are hard to verify directly to a learned model, while leaving constraints that can be checked explicitly to a program, is a sound design. A smaller encoder-based scoring model also has the potential to reduce the cost of candidate selection.
  2. The experiments cover a variety of tasks and are not confined to settings that best suit rule-based checking. On GSM8K and MuSR, the method improves the self-consistency baseline from 96.1% and 64.4% to 97.0% and 67.7%, respectively, showing that the learned scorer can indeed improve candidate selection on some tasks.
  3. The analysis of the generator-identity shortcut in TACO is quite valuable. Rather than reporting only the relatively high accuracy, the authors further examine the generator selection ratios, the influence of code style, and the changes after retraining the final layer. The paper also reports cases where uncertainty estimation and additional generation fail, and these analyses help delineate the boundaries of the method.
Critical weaknesses
  1. The proof of Proposition 3.1 does not apply to the mean-energy ranking actually used. Appendix A.1 treats "the majority of members rank correctly" as equivalent to "the mean-energy ranking is correct," but the former depends only on the sign of the energy difference, whereas the latter also depends on its magnitude. For example, if the energy differences of five members between the correct and incorrect candidates are −1, −1, −1, −1, and 10, then four members rank correctly, yet the mean energy still selects the wrong one. The current proof therefore cannot support the paper's claim about the correctness rate of mean-energy ranking.
  2. The effectiveness of the uncertainty estimation is insufficient to support the paper's central claim. In Table 15, the AUROC of the standard deviation for identifying errors is 0.54, 0.44, and 0.51 on GSM8K, MuSR, and TACO, respectively, which is close to or below chance level. The additional-generation experiments also show no consistent benefit. Moreover, the pairwise ranking loss does not constrain the additive offsets of the individual members' energies, and these offsets affect the standard deviation across members. Although the authors discuss calibration, they still need to explain more fully why this quantity can serve as a reliable uncertainty signal.
  3. The advantage of the full method over simpler alternatives is not sufficiently clear. On TravelPlanner, the constraint checker alone achieves an average violation score of 0.038, better than the full method's 0.046. In Table 11, single-member and five-member accuracy are also essentially the same on several tasks. This makes it hard for me to judge what necessary benefit energy combination and ensembling each provide. In particular, a direct comparison between homogeneous and heterogeneous LoRA ensembles under the same number of members and the same training budget is missing.
Questions for authors
  1. Does Proposition 3.1 omit an assumption about the magnitude of the members' energy differences? Please explain the equivalence between mean-energy ranking and majority voting in Appendix A.1. If this step does not hold, which conclusions in the main text would the corrected theory still support?
  2. Given the same number of members, the same training data, and the same compute budget, does a heterogeneous LoRA ensemble reliably improve error identification or selective prediction over a homogeneous LoRA ensemble? I would also like to see a comparison against a single-scorer confidence baseline, to determine whether the gains indeed come from member disagreement.
  3. On TravelPlanner, the constraint checker alone outperforms the full method. Does the learned quality term improve aspects not captured by the main metrics, such as the coherence or usefulness of the plans? Could the plans selected by the two methods be directly blind-evaluated, rather than only compared against other models or an oracle?
4 comments

Dear Reviewer,

Thank you for your detailed review, for the counterexample to the proof of Proposition 3.1, and for the concrete suggestions on the ensemble and on evaluating TravelPlanner plans. We ran the comparisons you asked for, and answer each point below.

W1, Q1: We agree that the original proof does not apply to mean-energy ranking, and thank you for pointing it out, which is what allowed us to correct it. Please find the corrected Proposition 3.1 above as a global comment, and as a correction in blue in the revised paper (§3.2 and Eq. (2), L193-207; §A.1, L763-799; Tab. 14 and its text, L1267-1289).

Your reading of Appendix A.1 is correct: L771-772 of the submitted version equate a sign count with a mean. The corrected proof starts instead from the exact identity μquality(x,y+)−μquality(x,y−)=dˉ\mu_{\text{quality}}(x, y^+) - \mu_{\text{quality}}(x, y^-) = \bar d, the average of the adapters' energy differences dkd_k, so the mean ranks a pair correctly if and only if dˉ<0\bar d < 0. Magnitude enters through each adapter's mean margin mkm_k and spread sks_k.

In your example, d=(−1,−1,−1,−1,10)d = (-1, -1, -1, -1, 10), four adapters vote correctly (V=4>K/2=2.5V = 4 > K/2 = 2.5), so the original proof counted the pair as correct; that was the error. The mean gives dˉ=+1.2>0\bar d = +1.2 > 0, so the corrected proposition counts the pair as an ensemble error, which is exactly what mean-energy ranking does.

The corrected proposition gives the correctness rate of mean-energy ranking directly: with jointly Gaussian dkd_k, Pensemble=Φ(mˉτ),τ2=Var(dˉ)=1K2∑j∑kρjk sjsk.P_{\text{ensemble}} = \Phi\left(\frac{\bar m}{\tau}\right), \qquad \tau^2 = \mathrm{Var}(\bar d) = \frac{1}{K^2}\sum_j\sum_k \rho_{jk}\, s_j s_k. Our experiments already rank by the mean energy, so the error is in the proof only and no reported number changes.

Q1, omitted assumption: The corrected Proposition 3.1 needs no assumption on the magnitudes: it models them through each adapter's mean margin mkm_k and spread sks_k, with the dkd_k jointly Gaussian and a valid correlation matrix (§A.1).

Q1, the equivalence: The corrected §A.1 does not use it. It starts from the exact identity μquality(x,y+)−μquality(x,y−)=dˉ\mu_{\text{quality}}(x, y^+) - \mu_{\text{quality}}(x, y^-) = \bar d (L769-770), so it describes mean-energy ranking directly.

Q1, what the corrected theory supports: All three conclusions in the main text hold under the corrected proposition. (C1, L204-206) τ\tau increases in every ρjk\rho_{jk}, so with each adapter's mkm_k and sks_k held fixed, lower correlation between the adapters' energy differences raises PensembleP_{\text{ensemble}}; this is the reason for the heterogeneous design. (C2, L206) For adapters of equal strength, ρ=1\rho = 1 gives the single-adapter accuracy Φ(m/s)\Phi(m/s) for every KK. (Ceiling, L199-202) For adapters of equal strength and ρ>0\rho > 0, P∞=Φ((m/s)/ρ)<1P_\infty = \Phi((m/s)/\sqrt{\rho}) < 1. The ρ\rho in these statements is now the correlation of the adapters' energy differences, which Tab. 14 reports.

The corrected proposition is also consistent with our experiments. Heterogeneous adapters are less correlated than identical ones (correlation of energy differences 0.87 against 0.93 on MuSR), and, in the direction C1 describes, the heterogeneous ensemble selects the correct answer more often (66.9% against 64.9%) (Table R2). With correlations of 0.845 to 0.941 on the four benchmarks with right-or-wrong labels (Tab. 14), the proposition's pairwise-ranking model implies diminishing returns from adding members, consistent with the limited top-1 changes in Tab. 11.

W2:

  1. The effectiveness of the uncertainty estimation is insufficient to support the paper's central claim. In Table 15, the AUROC of the standard deviation for identifying errors is 0.54, 0.44, and 0.51 on GSM8K, MuSR, and TACO, respectively, which is close to or below chance level.

We respectfully disagree: the central claim is selection (L71-72), and σquality\sigma_{\text{quality}} is claimed as a task-dependent signal (L1308-1310) that raises accuracy where the method acts on it. On TravelPlanner, σ-AUROC for the quality scorer's selection is 0.62, and abstaining on the most uncertain 40% raises zero-violation pass@1 from 28.0% to 36.7%; on TACO, abstaining on 50% raises pass@1 from 88.6% to 90.9% (Tabs. 15 and 20).

On GSM8K the selector makes only about 40 wrong selections in 1,319 problems (L1431-1432), and σquality\sigma_{\text{quality}} still finds them better than any single member's confidence in its own selection (Table R3). MuSR is the case the paper reports as a limitation (L435-436).

The additional-generation experiments also show no consistent benefit.

We respectfully disagree: regeneration helps where the feedback says what is wrong, which is the case it is designed for (L1561-1564). On TravelPlanner, where the violated constraints become the prompt, the two-pass loop raises pass@1 from 74.0% to 80.0% (Tab. 21; relaxed threshold, L368-369). On TACO, where the failing test is the feedback, it raises pass@1 from 88.6% to 93.2%, at 6 to 67 times lower cost than frontier self-verification (Tab. 22). Where the first pick is already near the oracle, the Distributional EBM keeps it (L1575-1577).

Moreover, the pairwise ranking loss does not constrain the additive offsets of the individual members' energies, and these offsets affect the standard deviation across members.

We agree, and the paper corrects for it (§F.2, L1342-1348). An offset shifts all of a member's energies equally, so it never changes the pick, only σquality\sigma_{\text{quality}}. §F.2 therefore centres and unit-scales each member's energies on the validation split before computing σquality\sigma_{\text{quality}}. This raises σ-AUROC from 0.54 to 0.58 on GSM8K and from 0.51 to 0.57 on TACO, and pass@1 also rises, by 0.1 and 1.1 pp (Tab. 16).

Although the authors discuss calibration, they still need to explain more fully why this quantity can serve as a reliable uncertainty signal.

We agree that this deserves a fuller explanation; there are three reasons.

First, σquality\sigma_{\text{quality}} is uncertainty about quality alone: the constraint term is deterministic and adds no variance (Proposition 3.2). Second, after the offset correction above, it measures on a common scale how much heterogeneous members (Tab. 6) disagree on the selected candidate. Third, it carries information a single scorer does not: on GSM8K it identifies wrong selections better than any member's own confidence (Table R3), and on TravelPlanner and TACO abstaining on the most uncertain selections raises accuracy (Tab. 20).

W3:

  1. The advantage of the full method over simpler alternatives is not sufficiently clear. On TravelPlanner, the constraint checker alone achieves an average violation score of 0.038, better than the full method's 0.046.

We agree that on TravelPlanner the constraint checker alone is the strongest selector on mean violation: TravelPlanner has an exact checker, which is sufficient for the constraints it checks. The learned scorer is for what such a checker does not measure: on GSM8K, MuSR and TACO it is the whole method and beats every pool heuristic, and on TravelPlanner it selects plans that independent judges prefer (Q3 below).

With no constraint term, the quality scorer beats self-consistency on GSM8K (97.0% against 96.1%), MuSR (67.7% against 64.4%) and TACO (88.6% against 36.4%) (Tab. 1). On TravelPlanner, mean violation counts only the five checked constraints, which the checker optimises by construction; Opus 4.6, judging 120 plans on the original TravelPlanner criteria, rates the Distributional EBM's selections above the oracle's (3.86 against 3.67, L1612-1617), although the oracle plans have the lowest mean violation (0.028). Our answer to Q3 adds the blind comparison of the two methods' plans.

In Table 11, single-member and five-member accuracy are also essentially the same on several tasks. This makes it hard for me to judge what necessary benefit energy combination and ensembling each provide.

The ensemble and the energy combination each have one job, and the paper measures both: the ensemble produces σquality\sigma_{\text{quality}}, and the combination adds exact constraints where a task has them.

Ensembling. The members are averaged for ranking, so top-1 accuracy is not what they are for (L1190). With K=1 there is no σquality\sigma_{\text{quality}}, and with K=5 σquality\sigma_{\text{quality}} supports abstention on TravelPlanner and TACO (Tab. 20). Accuracy holds as members are added (GSM8K 96.9-97.2%, K&K 99.9-100.0%), and on TravelPlanner K≥3 raises zero-violation pass@1 from 24.0% to 28.0% (Tab. 11).

Energy combination. On TravelPlanner, adding the constraint term to the quality scorer lowers mean violation from 0.058 to 0.046 (L371-372), and the combination reaches 99.9% on K&K (Tab. 1). Because the constraint term is deterministic, it does not enter σquality\sigma_{\text{quality}} (Proposition 3.2).

W3, Q2:

In particular, a direct comparison between homogeneous and heterogeneous LoRA ensembles under the same number of members and the same training budget is missing. 2. Given the same number of members, the same training data, and the same compute budget, does a heterogeneous LoRA ensemble reliably improve error identification or selective prediction over a homogeneous LoRA ensemble? I would also like to see a comparison against a single-scorer confidence baseline, to determine whether the gains indeed come from member disagreement.

We have added the comparison on MuSR, and the heterogeneous ensemble is the stronger of the two: its members' energy differences are less correlated (ρeff\rho_{\text{eff}} 0.87 against 0.93), which is the quantity the corrected Proposition 3.1 says governs the ensemble, and it selects the correct answer more often (66.9% against 64.9%).

The homogeneous ensemble is also where we started. Our first GSM8K ensembles used five identical adapters (rank 8 on WqkvW_{qkv}): every member reached its best validation loss after one epoch and then overfit (validation loss 0.056 rising to 0.211 by epoch 4; the same pattern at a five times lower learning rate), and the members' outputs stayed highly similar (mean pairwise cosine similarity 0.75). We therefore moved to heterogeneous adapters, together with stronger dropout, which is the design in the paper (Tab. 6).

For the matched comparison, the homogeneous ensemble uses five copies of adapter 0 of Tab. 6 (rank 8, alpha 16, WqkvW_{qkv}) that differ only in random seed and bagged training split. Everything else matches the heterogeneous ensemble: the same training pairs, validation split, learning rate, batch size, epoch limit and early stopping, and K=5. Both are evaluated on the same 151 MuSR test problems with 32 candidates each.

Same test pool and candidates in both rows (31,730 correct-incorrect pairs). ρeff\rho_{\text{eff}} is the sjsks_j s_k-weighted mean correlation of the members' energy differences; adapter-pair correlations range over 0.82-0.91 (heterogeneous) and 0.91-0.96 (homogeneous).

The ordering is the one Proposition 3.1 predicts: lower correlation lets the heterogeneous ensemble rank more pairs correctly than its average member, while the homogeneous ensemble does not improve on its average member. Table R2 compares the two designs on ranking and on the correlation that Proposition 3.1 depends on. For error identification, the comparison we report is our TACO ablation with identical training settings, where the heterogeneous K=5 ensemble reached σ-AUROC 0.75 against 0.36 for a uniform K=3 ensemble. For the single-scorer confidence baseline we use GSM8K, the benchmark with the most problems (1,319). Each method is scored on its own selections: σquality\sigma_{\text{quality}} for the ensemble's pick, and each single member's energy margin (best minus second-best candidate) for that member's pick (Table R3). σquality\sigma_{\text{quality}} identifies wrong selections better than any of the five single scorers' margins (AUROC 0.54 against 0.30-0.47), and abstaining by σquality\sigma_{\text{quality}} raises accuracy at every coverage we test, and at each coverage gives higher accuracy than abstaining by any single scorer's margin (97.4%, 97.3% and 97.2% at 90%, 80% and 60% coverage, against at most 97.0%, 97.0% and 96.7%). The gain in error identification therefore comes from member disagreement, not from a single scorer's confidence.

Each confidence is evaluated on its own method's selections; coverage keeps the most confident fraction of problems. The single members are the five adapters of Tab. 6, each used alone.

Q3: We ran the blind comparison you ask for. On the 40 of 50 TravelPlanner test problems where the constraint checker alone and the Distributional EBM select different plans, Opus 4.6 and Sonnet 4.6 saw both plans in both orders, without method labels, and picked the more coherent and useful plan under the original TravelPlanner criteria. A preference counts only if it holds in both orders; the rest are ties.

Table R4: Blind pairwise comparison on the 40 TravelPlanner test problems where the two methods select different plans (number of problems).

Distributional EBM's planConstraint checker's planTie or order-dependent
Preferred by Opus 4.616816
Preferred by Sonnet 4.613522
Lower violation score91318

Both judges prefer the Distributional EBM's plan more often, although the checker's plan has the lower violation score more often. The sample is small, so we report this as an indication, not a significant difference. The paper's judge validation (L1612-1617) points the same way: Opus 4.6 rates the Distributional EBM's selections above the oracle's (3.86 against 3.67).

We thank you again. We hope this response addresses your concerns, and that you will consider them in the final assessment. We would be glad to clarify anything further.

Dear Reviewer,

As the discussion period comes to a close soon, we were wondering if you were happy with our detailed responses, corrections and experiments. Kindly let us know if you require any further clarification. We would highly appreciate your feedback.

Many thanks, Authors

Reviewer_PqEcOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper discussed if there is a way to use a smaller verifier with ability to estimate uncertainty to select more reliable result from multiple candidate answers. It combines learned quality score and deterministic constraint penalties as Distributional EBM to test this assumption, with experiments on GSM8K, MuSR, TravelPlanner, TACO and Knights & Knaves to evaluate, and show improvement of this approach.

Critical strengths

This work explores a long-existing problem that is under-explored, and provides meaningful improvement in performance. The critical strengths:

  1. The candidate selection problem is practical and the work shows meaningful gains: In table 1, distributional EBM (the proposed combination) shows gains over self-consistency, and the experiment is well design to isolate the real gain of reranking, though the variance of gain is a bit high. This is the foundation of my positive overall assessment,

  2. The separation of components is clean and trustworthy: The authors clearly distinguish learned quality estimation and deterministic verification, so that the the model doesnt need to learn all computable constraints, and the constraint term can be used to exam budget or logic consistency. The authors also compare quality/constrained-only scores with combined scores and demonstrate their difference. This validate the result and justify the overall assessment.

Critical weaknesses

While there are tangible advantages of this work, some minor issues exist:

  1. The uncertainty estimation is not as stable as promoted by the authors: The authors adopted the standard deviation of 5 LoRA adapters as epistemic uncertainty, but in result it also shows the uncertainty AUROC is only 0.44 in MuSR, which is worse than random guess, so I wonder if this is really a contribution. This can weaken the overall assessment to be at most weak accept.

  2. The cross-domain generalization ability is questionable: The authors claim zero-shot transfer of quality scorer, but in fact the result still highly depends on the distribution of tasks. For example, MuSR-trained scorer achieved 93.9% and still lower than 96.1% of self-consistency, and GSM8K-trained scorer achieved 65.6%, lower than random selection (68.4%). This can hinder the claims and weaken this work's contribution, and limit this work to be at most weak accept too.

Questions for authors
  1. Can you further explain the contribution of uncertainty estimation in this paper, since it appears in the title but not yet validated by your current result?

  2. If some cross-domain results are lower than random selection or self-consistency, how will you claim the zero-shot transfer ability of quality scorer?

  3. Which contribution should be the real ones for this work, candidate reranking, analytical verification, or uncertainty estimation? Do you think all three are well supported by experiments?

3 comments

Dear Reviewer,

Thank you for your positive assessment and for the precise questions on the uncertainty estimate and on zero-shot transfer. We answer each point below.

W1, Q1:

We respectfully disagree that the uncertainty estimate is unvalidated: acting on σquality\sigma_{\text{quality}} raises accuracy on TravelPlanner and TACO, and MuSR (0.44) is the case the paper reports as a limitation (L435-436). σquality\sigma_{\text{quality}} contributes in three ways:

(a) Abstention. Abstaining on the 40% most uncertain TravelPlanner problems raises zero-violation pass@1 from 28.0% to 36.7%, and abstaining on 50% of TACO problems raises pass@1 from 88.6% to 90.9% (Tab. 20).

(b) Regeneration. In the two-pass loop, σquality\sigma_{\text{quality}} and the constraint violation together decide when to regenerate (Eq. (5)); on TravelPlanner with the Qwen-only generator pool this raises pass@1 at the paper's relaxed TravelPlanner threshold (violation score at most 0.15, L368-369) from 74.0% to 80.0% (Tab. 21).

(c) A signal about quality only. Because the constraint term is deterministic, σquality\sigma_{\text{quality}} carries no constraint-checking error (Proposition 3.2), so it can be combined with the constraint violation in one triage rule (§3.2).

On MuSR, σ-AUROC is below 0.5, and the paper reports it as anti-correlated (L436). There the members rank a (correct, incorrect) pair correctly with probability only 0.62 (Tab. 14) and the energy gap is 0.19 (Tab. 15). The triage rule never fires on MuSR (0 of 151 problems, Tab. 21; Fig. 9), so the weak signal does not lower MuSR accuracy: pass@1 stays at 67.7%.

The title says uncertainty-aware because the Distributional EBM returns σquality\sigma_{\text{quality}} with every selection and the triage rule reads it to accept, regenerate or abstain (§3.2); (a) and (b) show where that pays off.

W2, Q2:

We agree that transfer depends on the task, and the paper reports it as asymmetric and regime-dependent (§4.2). Within compatible output formats it holds on four of the five benchmarks, where a scorer trained on a different benchmark beats self-consistency with no target training (Table R1); GSM8K, which you cite, is the one where it does not, and the transferred scorer still beats random selection there.

Table R1: Zero-shot transfer to each target benchmark: the best quality scorer trained on a different benchmark, against the two training-free baselines on the same test pool.

Target benchmark (test pool)Best quality scorer trained on another benchmark (training benchmark)Self-consistency on the same test pool (no training)Random selection on the same test pool (no training)
K&K (pass@1)92.4% (GSM8K)56.4%58.7%
TACO (pass@1)56.4% (TravelPlanner)36.4%46.6%
TravelPlanner (zero-violation pass@1)43.3% (TACO)12.0%14.0%
MuSR (pass@1)64.9% (GSM8K; also K&K)64.4%60.6%
GSM8K (pass@1)94.9% (K&K)96.1%92.6%

On GSM8K, the ModernBERT scorer reaches 93.9%, above random (92.6%), without having seen a math problem (Tab. 3). GSM8K -> MATH-500 is a change of output format, which the paper reports as a limitation rather than claims as transfer (L433-435).

Q3:

Candidate reranking is the contribution, and analytical verification and uncertainty estimation are its two components; each is supported where the paper claims it.

Reranking: the contribution. The quality scorer beats the strongest pool heuristic on four of five benchmarks and matches or exceeds the strongest single-shot frontier model on three (L351-352); on MuSR it reaches 67.7% against 64.4% for self-consistency (Tab. 1).

Analytical verification: a component, where an exact checker exists. On TravelPlanner the Distributional EBM reaches 0.046 mean violation against 0.098 for Opus 4.6 (Tab. 1), and connectivity violation 0.058 against 0.375-0.418 for the frontier models (Tab. 23). On K&K it reaches 99.9% with the exact statement-consistency checker (Tab. 1).

Uncertainty estimation: a component. Acting on σquality\sigma_{\text{quality}} raises accuracy on TravelPlanner and TACO (W1 and Q1 above). It does not enter the ranking, which uses the mean energy; it decides whether to accept, regenerate or abstain (§3.2), and Proposition 3.2 makes it a signal about the learned quality term alone.

We thank you again for the positive assessment and the precise questions. Both points you named as limiting it, the uncertainty estimate and zero-shot transfer, are answered above with the corresponding revisions; we hope they strengthen your confidence in the paper, and would be glad to clarify anything further.

Thank you for the response. The clarifications are very helpful, though uncertainty reliability and cross-domain transfer remain limitations. I will maintain my positive score and raise my confidence accordingly.

We are glad we have addressed all your concerns. Thank you for the positive score and confidence raise and in engaging in this process with us.

Reviewer_nYwUOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a candidate-answer reranker. Four LLMs generate 32 candidates, which five heterogeneous ModernBERT-based LoRA members score. The system selects the answer with the lowest mean learned energy plus a deterministic constraint penalty, using the standard deviation across members to estimate uncertainty.

Critical strengths

The modular design is practical. Separating learned quality judgments from computable constraints makes task rules easier to replace and errors easier to diagnose.

Critical weaknesses

The central theoretical claim contains a clear proof error. Appendix A.1 equates correct ranking by mean energy with correct ranking by a majority of members. Averaging continuous scores is not equivalent to majority voting. For example, suppose the five correct-minus-incorrect energy differences are −1, −1, −1, 2, 2. Three members rank correctly, but the positive mean causes the ensemble to choose incorrectly. Without additional assumptions, such as equal score margins, the proof of Proposition 3.1 does not justify the mean-based scorer actually used.

The evidence for reliable uncertainty estimation is weak. MuSR’s σ-AUROC is 0.44, and abstaining on high-uncertainty cases lowers accuracy on the retained cases. On TACO, σ-AUROC is 0.51 after shortcut removal and only 0.57 after recalibration. Adding ensemble members also fails to improve top-1 accuracy consistently on most tasks. These results support a task-dependent disagreement signal, but not broadly reliable epistemic uncertainty estimates.

Combining learned scores and constraints does not consistently beat simpler alternatives. On TravelPlanner, where lower is better, the constraint checker alone scores 0.038, the full method 0.046, and the learned scorer alone 0.058. The combination improves on the learned scorer but loses to the simpler checker. As the authors acknowledge, the main value is better framed as low-cost selection than a general benefit from combining the two components.

The zero-shot transfer claims are too strong. MuSR -< GSM8K reaches 93.9%, but trails self-consistency at 96.1%. GSM8K -> MATH-500 reaches 65.6%, below random candidate selection at 68.4%. These results suggest limited transfer across compatible formats, rather than a general criterion for reasoning quality.

Questions for authors

None

4 comments

Dear Reviewer,

Thank you for your careful review and for the precise counterexample to the proof of Proposition 3.1. We have corrected the proposition, given with its proof in the global comment above, and answer each point below.

W1:

The central theoretical claim contains a clear proof error. Appendix A.1 equates correct ranking by mean energy with correct ranking by a majority of members. Averaging continuous scores is not equivalent to majority voting. For example, suppose the five correct-minus-incorrect energy differences are −1, −1, −1, 2, 2. Three members rank correctly, but the positive mean causes the ensemble to choose incorrectly. Without additional assumptions, such as equal score margins, the proof of Proposition 3.1 does not justify the mean-based scorer actually used.

We agree that the step at L771-772 of the submitted version is wrong, and thank you for pointing it out, which is what allowed us to correct it. Please find the corrected Proposition 3.1 above as a global comment, and as a correction in blue in the revised paper (§3.2 and Eq. (2), L193-207; §A.1, L763-799; Tab. 14 and its text, L1267-1289).

The corrected Proposition 3.1 needs no assumption on the magnitudes: it models them through each adapter's mean margin mkm_k and spread sks_k, with the dkd_k jointly Gaussian and a valid correlation matrix (§A.1).

The corrected proof uses no vote. It starts from the exact identity μquality(x,y+)−μquality(x,y−)=dˉ\mu_{\text{quality}}(x, y^+) - \mu_{\text{quality}}(x, y^-) = \bar d, the average of the adapters' energy differences, so the mean ranks a pair correctly if and only if dˉ<0\bar d < 0.

In your example, d=(−1,−1,−1,2,2)d = (-1, -1, -1, 2, 2), three adapters vote correctly (V=3>K/2=2.5V = 3 > K/2 = 2.5), so the original proof counted the pair as correct; that was the error. The mean gives dˉ=+0.2>0\bar d = +0.2 > 0, so the corrected proposition counts the pair as an ensemble error, which is exactly what mean-energy ranking does.

The corrected proposition is also consistent with our experiments. Heterogeneous adapters are less correlated than identical ones (correlation of energy differences 0.87 against 0.93 on MuSR), and, in the direction C1 describes, the heterogeneous ensemble selects the correct answer more often (66.9% against 64.9%) (Table R2, in our reply to sqm7). With correlations of 0.845 to 0.941 on the four benchmarks with right-or-wrong labels (Tab. 14), the proposition's pairwise-ranking model implies diminishing returns from adding members, consistent with the limited top-1 changes in Tab. 11.

W2:

The evidence for reliable uncertainty estimation is weak. MuSR’s σ-AUROC is 0.44, and abstaining on high-uncertainty cases lowers accuracy on the retained cases.

We respectfully disagree that the evidence is weak across the tasks: on TravelPlanner and TACO, abstaining on the most uncertain selections raises accuracy; MuSR is the one task where it does not, and the paper reports it as a limitation (L435-436).

On TravelPlanner, σ-AUROC for the quality scorer's selection is 0.62, and abstaining on the 40% most uncertain problems raises zero-violation pass@1 from 28.0% to 36.7% (Tabs. 15 and 20). On TACO after DFR, abstaining on 50% raises pass@1 from 88.6% to 90.9% (Tab. 20). On GSM8K the selector makes only about 40 wrong selections in 1,319 problems (L1431-1432), which leaves little for any uncertainty score to find.

Among the four benchmarks with right-or-wrong labels, MuSR is the hardest to rank: each member ranks a (correct, incorrect) pair correctly with probability 0.62, the lowest of the four (Tab. 14), and the energy gap between correct and incorrect candidates is 0.19, against 1.76-9.00 on the other four (Tab. 15). In the two-pass loop every MuSR σquality\sigma_{\text{quality}} is below the triage threshold, so the rule never fires (0 of 151 problems, Tab. 21; Fig. 9) and pass@1 stays at 67.7%.

On TACO, σ-AUROC is 0.51 after shortcut removal and only 0.57 after recalibration.

We respectfully disagree that this counts against the uncertainty estimate: shortcut removal is designed to reduce the generator-identity signal in the energies, and the σquality\sigma_{\text{quality}} that remains still adds accuracy when acted on.

Before DFR, σquality\sigma_{\text{quality}} on TACO captured which generator wrote the candidate (L1435-1436); DFR mitigates that identified generator-style shortcut. Centring and unit-scaling each member's energies then raises σ-AUROC from 0.51 to 0.57 (Tab. 16), and abstaining on the 50% most uncertain problems raises pass@1 from 88.6% to 90.9% (Tab. 20).

Adding ensemble members also fails to improve top-1 accuracy consistently on most tasks.

We agree with the numbers and disagree that this is a weakness: the members are averaged for ranking, and the paper states that the ensemble is there for uncertainty estimation rather than accuracy (L1190).

With K=1 there is no σquality\sigma_{\text{quality}} at all. Accuracy holds as members are added: across K=1, 3 and 5, pass@1 stays at 96.9-97.2% on GSM8K and 99.9-100.0% on K&K (Tab. 11). On TravelPlanner, where labels are continuous, K≥3 raises zero-violation pass@1 from 24.0% to 28.0% (Tab. 11). The extra members are low-rank adapters with their own energy heads on one shared frozen encoder.

These results support a task-dependent disagreement signal, but not broadly reliable epistemic uncertainty estimates.

We agree that σquality\sigma_{\text{quality}} is task-dependent, which is what the paper claims (L1308-1310); the paper does not claim it is reliable on every task.

Where it is acted on, it raises accuracy: abstention takes TravelPlanner from 28.0% to 36.7% (40% abstained) and TACO from 88.6% to 90.9% (50% abstained) (Tab. 20), and recalibrating the members' offsets raises σ-AUROC from 0.54 to 0.58 on GSM8K and from 0.51 to 0.57 on TACO (Tab. 16). By Proposition 3.2, σquality\sigma_{\text{quality}} carries no constraint-checking error, so it reads as uncertainty in the learned quality term alone.

W3:

Combining learned scores and constraints does not consistently beat simpler alternatives. On TravelPlanner, where lower is better, the constraint checker alone scores 0.038, the full method 0.046, and the learned scorer alone 0.058. The combination improves on the learned scorer but loses to the simpler checker.

We agree that on TravelPlanner the constraint checker alone is the strongest selector on mean violation: TravelPlanner has an exact checker, which is sufficient for the constraints it checks. The learned scorer is for what such a checker does not measure: on the three tasks with no constraint checker it is the whole method and beats every pool heuristic, and on TravelPlanner it selects plans that independent judges rate highly.

On GSM8K, MuSR and TACO there is no constraint term, so the simpler alternative is a pool heuristic. The quality scorer alone beats self-consistency on all three: 97.0% against 96.1%, 67.7% against 64.4%, and 88.6% against 36.4% (Tab. 1). These gains isolate learned ranking from hand-designed constraints (§4.2).

On TravelPlanner, mean violation counts only the five checked constraints, which the constraint checker optimises by construction. Two independent judges (Opus 4.6 and Sonnet 4.6) scoring 120 plans on the original TravelPlanner criteria agree with each other (Spearman ρ=0.67), and Opus 4.6 rates the Distributional EBM's selections above the oracle's (3.86 against 3.67, L1612-1617), although the oracle plans have the lowest mean violation of any method (0.028, Tab. 1). The violation score therefore does not capture all of plan quality, and the quality term adds to it. The blind comparison of the two methods' plans that sqm7 asks for is in our answer to sqm7 (Q3, Table R4).

As the authors acknowledge, the main value is better framed as low-cost selection than a general benefit from combining the two components.

We agree that the claim is low-cost selection (L398-399, L440-442), and the decomposition is what makes it low-cost. The constraint terms are deterministic and can be added or removed without retraining the quality scorer, so the same 149M architecture covers all five tasks. On TravelPlanner this gives 0.046 mean violation at about USD 0.003 per problem, against 0.098 for Opus 4.6 alone (Tab. 1), and against USD 0.60-0.77 per problem for Opus 4.6 and Sonnet 4.6 driven by the same constraint checker (Tab. 24).

W4:

The zero-shot transfer claims are too strong. MuSR -< GSM8K reaches 93.9%, but trails self-consistency at 96.1%. GSM8K -> MATH-500 reaches 65.6%, below random candidate selection at 68.4%. These results suggest limited transfer across compatible formats, rather than a general criterion for reasoning quality.

Table R1: Zero-shot transfer to each target benchmark: the best quality scorer trained on a different benchmark, against the two training-free baselines on the same test pool.

Target benchmark (test pool)Best quality scorer trained on another benchmark (training benchmark)Self-consistency on the same test pool (no training)Random selection on the same test pool (no training)
K&K (pass@1)92.4% (GSM8K)56.4%58.7%
TACO (pass@1)56.4% (TravelPlanner)36.4%46.6%
TravelPlanner (zero-violation pass@1)43.3% (TACO)12.0%14.0%
MuSR (pass@1)64.9% (GSM8K; also K&K)64.4%60.6%
GSM8K (pass@1)94.9% (K&K)96.1%92.6%

We agree that transfer is not universal, and the paper reports it as asymmetric and regime-dependent (§4.2). Within compatible output formats, which matches your conclusion, it is positive: on four of the five benchmarks, a scorer trained on a different benchmark beats self-consistency with no target training (Table R1); MuSR -> GSM8K, the case you cite, is the one where it does not. The ModernBERT scorer you cite reaches 93.9%, above random selection (92.6%), from a scorer that has never seen a math problem (Tab. 3). GSM8K -> MATH-500 is a change of output format, which the paper reports as a limitation rather than claims as transfer (L433-435).

We thank you again. We have corrected Proposition 3.1 and its proof, and answered each point with the corresponding revision; we hope these address your concerns, and that you will consider them in the final assessment. We would be glad to clarify anything further.

Dear Reviewer,

As the discussion period comes to a close soon, we were wondering if you were happy with our detailed responses, corrections and experiments. Kindly let us know if you require any further clarification. We would highly appreciate your feedback.

Many thanks, Authors

Comments on the paper

1 comment

We thank all three reviewers. We have uploaded a revised version of the paper that corrects Proposition 3.1 for the mean-energy ranking the method uses; the corrected proposition and its proof are given below, and the changes in the revision are marked in blue.

Corrected Proposition 3.1 (referred to in the replies to nYwU and sqm7)

Notation. For a (correct, incorrect) pair (y+,y−)(y^+, y^-) of a problem xx, adapter kk's energy difference is dk=Equalityk(x,y+)−Equalityk(x,y−)d_k = E^k_{\text{quality}}(x, y^+) - E^k_{\text{quality}}(x, y^-); dk<0d_k < 0 means adapter kk ranks the pair correctly. Write dˉ=1K∑kdk\bar d = \frac{1}{K}\sum_k d_k.

Lemma (exact). μquality(x,y+)−μquality(x,y−)=dˉ\mu_{\text{quality}}(x, y^+) - \mu_{\text{quality}}(x, y^-) = \bar d. The ensemble mean therefore ranks the pair correctly if and only if dˉ<0\bar d < 0.

Proposition 3.1 (corrected). Over random (correct, incorrect) pairs, let (d1,…,dK)(d_1, \dots, d_K) be jointly Gaussian with E[dk]=−mk\mathbb{E}[d_k] = -m_k, standard deviation sks_k and Corr(dj,dk)=ρjk\mathrm{Corr}(d_j, d_k) = \rho_{jk}, where the ρjk\rho_{jk} form a valid correlation matrix (negative values allowed), and let mˉ=1K∑kmk>0\bar m = \frac{1}{K}\sum_k m_k > 0, with τ>0\tau > 0 for τ\tau below. Then Pensemble=Φ(mˉτ),τ2=Var(dˉ)=1K2∑j∑kρjksjsk=1K2[(1−ρeff)∑ksk2+ρeff(∑ksk)2],P_{\text{ensemble}} = \Phi\left(\frac{\bar m}{\tau}\right), \qquad \tau^2 = \mathrm{Var}(\bar d) = \frac{1}{K^2}\sum_{j}\sum_{k} \rho_{jk} s_j s_k = \frac{1}{K^2}\Big[(1-\rho_{\text{eff}})\sum_k s_k^2 + \rho_{\text{eff}}\Big(\sum_k s_k\Big)^2\Big], where ρeff\rho_{\text{eff}} is the average of ρjk\rho_{jk} over adapter pairs j≠kj \neq k, weighted by sjsks_j s_k. For adapters of equal strength (mk=mm_k = m, sk=ss_k = s, ρjk=ρ\rho_{jk} = \rho), Pensemble(K)=Φ(K m/s1+(K−1)ρ),P∞=Φ(m/sρ)<1 for ρ>0.P_{\text{ensemble}}(K) = \Phi\left(\frac{\sqrt{K}\, m/s}{\sqrt{1 + (K-1)\rho}}\right), \qquad P_\infty = \Phi\left(\frac{m/s}{\sqrt{\rho}}\right) < 1 \ \text{for}\ \rho > 0. Here qq is the probability that a single adapter ranks a pair correctly. This is Eq. (2) with the single-adapter strength (q−12)/q(1−q)(q - \frac{1}{2})/\sqrt{q(1-q)} replaced by m/sm/s; under the same assumption, q=Φ(m/s)q = \Phi(m/s).

Proof. (i) By the lemma, the mean ranks the pair correctly if and only if dˉ<0\bar d < 0. (ii) dˉ\bar d is a linear combination of jointly Gaussian variables, so it is Gaussian with mean −mˉ-\bar m and, by bilinearity of covariance, variance τ2=1K2∑j∑kCov(dj,dk)=1K2∑j∑kρjk sjsk.\tau^2 = \frac{1}{K^2}\sum_j\sum_k \mathrm{Cov}(d_j, d_k) = \frac{1}{K^2}\sum_j\sum_k \rho_{jk}\, s_j s_k. Separating the KK diagonal terms (ρkk=1\rho_{kk} = 1) from the rest and using ∑j≠ksjsk=(∑ksk)2−∑ksk2\sum_{j \neq k} s_j s_k = \Big(\sum_k s_k\Big)^2 - \sum_k s_k^2 gives the ρeff\rho_{\text{eff}} form. (iii) Hence P(dˉ<0)=Φ(mˉ/τ)P(\bar d < 0) = \Phi(\bar m/\tau). (iv) For equal adapters, τ2=s2 (1+(K−1)ρ)K,\tau^2 = \frac{s^2\,\big(1 + (K-1)\rho\big)}{K}, which gives Pensemble(K)P_{\text{ensemble}}(K); as K→∞K \to \infty, τ→sρ\tau \to s\sqrt{\rho}, which gives P∞P_\infty. ■\blacksquare

What the corrected proposition says. τ\tau increases in every ρjk\rho_{jk}, since ∂τ2∂ρjk=2 sjskK2>0,\frac{\partial \tau^2}{\partial \rho_{jk}} = \frac{2\, s_j s_k}{K^2} > 0, so with each adapter's mkm_k and sks_k held fixed, lower correlation between the adapters' energy differences raises PensembleP_{\text{ensemble}} (C1). For adapters of equal strength, ρ=1\rho = 1 gives the single-adapter accuracy Φ(m/s)\Phi(m/s) for every KK (C2), and for ρ>0\rho > 0 the accuracy approaches a ceiling P∞<1P_\infty < 1 as KK grows.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.