acceptodds
Under review as a conference paper at ICLR 2027

E-CALR: Evidential Conflict-Aware Layer-wise Logit Refinement for Factuality-Oriented Decoding

Abstract

Large language models (LLMs) can generate fluent but factually incorrect outputs. Layer-wise decoding seeks to improve factuality by exploiting predictions exposed at intermediate Transformer layers. A representative method, SLED, uses directional alignment in cross-layer logit evolution to construct intermediate-layer estimates for refining the final prediction. However, directional alignment alone does not reveal whether an intermediate prediction expresses a clear preference among specific token alternatives. A diffuse prediction may therefore receive a high alignment score despite providing weak token-specific evidence. This motivates treating intermediate predictions as potential sources of interference with the final prediction, whose influence should depend on how strongly they concentrate on specific token alternatives. We refer to this degree of concentration as predictive commitment. To this end, we propose Evidential Conflict-Aware Layer-Wise Logit Refinement (E-CALR), a training-free decoding framework that identifies cross-layer conflict backed by committed evidence and uses it to selectively suppress interfering intermediate-layer information. E-CALR represents each layer prediction in terms of token-specific committed evidence and nonspecific ignorance, measures evidential conflict with the final-layer prediction, and further modulates this conflict by predictive commitment so that diffuse intermediate predictions exert less influence. The resulting weights aggregate intermediate-layer distributions into a token-level suppression signal that iteratively refines the final-layer logits while preserving the original final-layer top- candidate set. Across nine LLMs, E-CALR improves factual candidate ranking over representative layer-wise decoding baselines and also generalizes to autoregressive reasoning generation. Compared with SLED, E-CALR improves model-averaged performance across all reported factual-ranking metrics and achieves higher accuracy for all nine models on both GSM8K and StrategyQA.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Evidence-Theoretic Conflict, Layer-wise Refinement, Factuality-Oriented Decoding

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_weaZOpenReview ↗
Rating
3(+1)
Confidence
2(−2)
Informativeness
1(−1)
Presentation
2
Soundness
2
Read review
Paper summary
  • This paper proposes E-CALR, a training-free decoding method that uses predictive commitment and cross-layer evidential conflict to refine final-layer logits.
  • The method shows improvements over existing layer-wise decoding baselines across nine LLMs.
Critical strengths
  • The method requires no additional training and can be applied to existing LLMs.
  • The experiments cover multiple model families, and the ablation studies examine the contributions of commitment and conflict weighting.
Critical weaknesses

Weaknesses and Questions

  • Model scale and architecture: All evaluated models are dense models with at most 13B parameters. Experiments on MoE architectures and larger models (e.g., 20B+ parameters) would help establish whether the method generalizes to different architectures and scales.

  • Decoding efficiency: The paper provides a theoretical complexity analysis but lacks wall-clock decoding speed comparisons. Please report generation throughput (tokens/sec) and latency against Base, DoLa, and SLED, including the overhead of extracting intermediate-layer logits. If E-CALR is slower, this trade-off should be quantified and discussed as a limitation.

  • Generalization across tasks: The evaluation focuses primarily on factual QA and reasoning benchmarks. It remains unclear how the proposed logit refinement affects non-knowledge-intensive tasks. Experiments on tasks such as summarization, translation, and general instruction following would help identify potential performance degradation.

  • Intermediate-layer selection: The method selects every other intermediate layer (odd-indexed layers), but the motivation for this choice is unclear. Why this selection strategy? An ablation comparing odd/even layers, all layers, and different layer subsets would be useful, ideally under comparable computational budgets.

  • Relationship between conflict and factual errors: It is unclear when the proposed conflict score reflects genuinely harmful intermediate-layer information. Predictive commitment does not imply factual correctness, and a confident intermediate prediction may still be correct. In such cases, suppressing its contribution could hurt performance. A more systematic analysis of the relationship between conflict scores and actual prediction errors, including cases where suppression hurts, would strengthen the justification for the method.

Minor Comment

  • The description of the FACTOR evaluation in Section 4.1 could be made slightly more concise to avoid repetitive wording.
Questions for authors

I mentioned in the weakness section.

2 comments

Thank you for the additional experiments and detailed responses. The more I read the paper, the more I feel that something is still missing.

However, considering the authors' efforts, I will raise my score from 2 to 3. I will also lower my confidence, as I feel I may not have enough expertise in this area.

I appreciate the authors' hard work and wish them good luck.

AuthorsOpenReview ↗

Response to Reviewer weaZ’s Follow-up

Thank you for taking the time to reread our paper, consider the additional experiments, and update your assessment. We appreciate your thoughtful engagement and candid feedback.

Your comments helped us clarify how conflict weighting contributes to refinement, examine its effects on both correct and incorrect predictions, and extend the evaluation across model architectures, scales, and tasks. We will incorporate these analyses and experimental results into the revised manuscript, with a clearer presentation of the method’s contribution, scope, and computational cost.

We appreciate the care you put into this review and your kind wishes.

Reviewer_kbyiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes E-CALR, a training-free layer-wise decoding method for improving factuality in large language models. The method represents intermediate-layer predictions using entropy-based predictive commitment and evidential conflict with the final-layer prediction. These quantities determine layer weights for constructing a token-level suppression signal, which iteratively refines the final logits while restricting token selection to the original top-k candidates. Experiments across multiple model families evaluate factual candidate ranking and autoregressive generation, with ablations examining the weighting and refinement mechanisms.

Critical strengths
  • Consistent empirical improvements across model families. E-CALR generally improves factual candidate ranking over SLED and reports gains over this baseline across the evaluated reasoning-generation settings. This breadth provides useful evidence that the approach is not specific to a single backbone.

  • Distinguishing directional alignment from predictive concentration is reasonable. The paper translates this observation into an explicit weighting rule and includes comparisons with simpler weighting strategies and ablations of the refinement procedure.

Critical weaknesses
  • The contribution primarily consists of a new weighting rule within an established layer-wise decoding framework. Entropy-based weighting, cross-layer comparison, and logit refinement already have substantial precedent. The evidential formulation provides an interpretation of the proposed rule, but the paper does not establish a broader principle explaining when cross-layer conflict identifies harmful information that should be suppressed. The supporting propositions describe algebraic properties rather than conditions under which factuality improves.

  • The ablations show that refinement without intermediate-layer information already retains much of the improvement over the original model, while uniform and squared-commitment weighting remain competitive with the complete method. These findings leave the practical contribution of evidential conflict uncertain, particularly without broader ablations and statistical uncertainty estimates. The limited open-ended factual-generation evaluation further restricts the evidence for benefits beyond candidate ranking.

Questions for authors
  1. What justifies treating conflicting intermediate-layer predictions as harmful information to suppress, rather than potentially useful corrections to the final-layer prediction?

  2. Given that refinement without intermediate-layer signals already retains much of the improvement, what evidence establishes that the proposed conflict weighting is a meaningful source of the gains?

  3. Do the improvements in candidate ranking translate into fewer factual errors in open-ended generation across the broader set of evaluated models?

1 comment
AuthorsOpenReview ↗

Follow-up on the Generation Ablation

Dear Reviewer kbyi,

Thank you again for your thoughtful follow-up. Following your suggestion, we added a direct comparison between Full E-CALR and Commitment-Squared on open-ended TruthfulQA using the corrected evaluation pipeline, detailed in our latest response. Across nine backbones, Full E-CALR increased the mean proportion of answers judged both truthful and informative from 66.45% to 70.34%.

Preserving initially correct predictions is a central objective of layer-wise refinement. Our additional synthetic experiments identify better preservation as the clearest advantage of conflict weighting over Commitment-Squared in the evaluated settings. Across all eight configurations, this preservation benefit outweighed the reduction in corrections, yielding higher overall top-1 accuracy.

As the discussion period closes soon, we would appreciate your thoughts on whether these additional results address your remaining concern. We would be happy to clarify any outstanding points before the deadline.

Thank you for your time and consideration.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response

We sincerely thank the reviewers, ACs, SACs, and PCs for their time and effort throughout the review process.

We have submitted point-by-point responses addressing every reviewer’s comments. By the end of the rebuttal period, we will incorporate the key clarifications, additional analyses, and experimental results into the main text of the revised manuscript. We appreciate the constructive feedback and the opportunity to improve our work.

Please feel free to raise any further questions or request clarification as you review our responses. We would be happy to address them.

AuthorsOpenReview ↗

A Gentle Reminder

Dear Reviewer BZk8,

Thank you again for your time and valuable review. As the discussion period closes soon, we would appreciate your feedback on our responses and additional experiments. If any concerns remain unresolved, particularly those most relevant to your assessment, please feel free to let us know. We would be happy to provide further clarification before the deadline.

Thank you for your consideration.

we thanks the authors for your reply, i still think that this paper is not ready for accept. I will keep my score

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.