acceptodds
Withdrawn from ICLR 2027

LUCiD: Mitigating Hallucinations in Large Language Models with Learned Unilayer Contrastive Decoding

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large language models remain prone to hallucination, generating fluent but factually wrong or unsupported content. This is often traced to models falling back on pretraining priors instead of grounding each token in the available context, a failure that is costly in reasoning and knowledge-intensive tasks. Contrastive decoding methods such as DoLa improve reasoning and factuality by contrasting a model’s final-layer distribution against a premature earlier-layer distribution. They are training-free, so the premature signal is taken directly from an untrained early exit and the contrast strength is fixed across tokens, which is brittle and often lowers accuracy on tasks where earlier layers are already well calibrated. We introduce LUCiD (Learned Unilayer Contrastive Decoding), a learned contrastive decoding method. LUCiD attaches one lightweight probe to a single middle layer and reuses the model’s own frozen final RMSNorm and LM head to extract a premature distribution. This distribution is contrasted against the final layer, scaled by a per-token gate predicted from the last hidden state, under an adaptive plausibility constraint. Across five models from 0.6B to 12B and ten benchmarks, under both greedy decoding and sampling, LUCiD outperforms vanilla decoding, DoLa and SLED, in most scenarios.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
contrastive decoding, hallucination mitigation, factuality, decoding strategies

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_BCF9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Summary

  • This paper proposes LUCiD, a contrastive decoding method with a learned premature distribution and a token-wise gate.
  • The method improves SQuAD performance on some models. However, its novelty and general effectiveness remain unclear.

Overall Assessment

  • The method is computationally efficient and shows promising results on LLaMA-8B.
  • However, the methodological contribution is limited, and several design choices lack justification.
  • More competitive baselines and stronger evidence of generalization are needed.
Critical strengths
  • S1. The method adds few trainable parameters and has relatively low inference overhead.
  • S1. The paper includes experiments across five models and several ablation studies.
Critical weaknesses

W1. Limited novelty and motivation

  • The method mainly combines a learned residual probe with a gating mechanism.
  • The paper does not clearly explain why learning the premature distribution should reduce hallucinations.
  • It is unclear whether the gains come from contrastive decoding or simply from supervised logit correction. A comparison with simpler learned baselines would help.

W2. Fixed layer selection

  • The probe is always placed at the middle layer, regardless of the model or task.
  • Table 4 shows that the optimal layer varies across models. For LLaMA-3B, a later layer performs substantially better.
  • The paper should justify this fixed choice and compare it with adaptive layer selection, as in DoLa, or validation-based selection.

W3. Insufficient comparison with recent methods

  • The baselines are limited to Vanilla, DoLa, and SLED.
  • Recent methods such as ALW and ActLCD are cited but not evaluated.
  • Comparisons with more recent contrastive decoding methods are needed to establish the contribution.

W4. Unclear training and evaluation setup

  • The model is trained on MetaMathQA, ARC-Challenge, and SciQ. Yet SQuAD is used as the main evaluation benchmark.
  • Evaluating on SQuAD is reasonable for testing out-of-distribution generalization. However, the motivation for this particular setup is unclear.
  • SQuAD F1 alone is also insufficient to support broad claims about hallucination reduction.
  • More direct factuality benchmarks and a clearer comparison between in-domain and out-of-domain performance are needed.

W5. Inconsistent performance across tasks

  • The gains are largely driven by LLaMA-8B. Improvements on other models are small, and LLaMA-3B performs worse than Vanilla.
  • Several general capability benchmarks also show substantial drops. For example, Mistral-12B drops from 0.389 to 0.140 on GPQA-Diamond under greedy decoding.
  • HellaSwag performance also decreases across all five models under greedy decoding.
  • These results raise concerns about the method's general applicability. The paper provides limited analysis of these failures.
Questions for authors

See the weakness section.

Reviewer_1tHtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes LUCiD, a self-contrastive decoding method that freezes the LLM backbone, final RMSNorm, and LM head; attaches a trainable residual bottleneck probe to a fixed middle layer; reads a premature distribution through the frozen head; and subtracts it from the final-layer distribution using a per-token learned gate under an adaptive plausibility constraint. Only the probe and gate are trained by NLL on answer tokens from MetaMathQA/ARC/SciQ. On SQuAD, LUCiD improves F1 over Vanilla/DoLa/SLED for four of five 0.6B–12B models under greedy and sampling, with the largest gain on LLaMA-8B; LLaMA-3B is an exception. Additional benchmarks mostly show retention, and inference overhead is lower than DoLa/SLED.

Critical strengths

The paper's most consequential strength is the breadth and honesty of the empirical evaluation. Five model families spanning 0.6B–12B, ten benchmarks explicitly categorized as in-distribution / near-distribution / OOD relative to the training mix, both greedy and sampled decoding, and ablations (layer depth, component-wise, multi-layer pooling, inference cost) that genuinely stress-test the design rather than only showcasing wins.

Critical weaknesses
  1. The capability-retention claim is not supported. Mistral-12B GPQA-Diamond collapses from 0.342/0.389 under vanilla to 0.135/0.140 under LUCiD for sampling/greedy; LLaMA-8B HellaSwag greedy drops from 0.741 to 0.605; LLaMA-3B remains best under vanilla on SQuAD. These are material regressions, not “within about one point” noise, and they directly undermine the claim that faithfulness gains do not cost general capability.

  2. The evaluation and comparison are too narrow and statistically underspecified. The headline result is only SQuAD token-F1; there are no seeds, variance, or significance tests; baselines omit ALW/ActLCD/PruneCD despite being discussed; and the fixed middle layer plus fixed α=0.1 are not justified against the paper’s own layer ablation, which shows LLaMA-3B would prefer ℓ6. This limits support for general superiority and robustness.

Questions for authors
  1. What are the exact training-set size, answer-token count, and training compute for the probe/gate? Confirm SQuAD was never used in training.

  2. Compare against ALW/ActLCD/PruneCD and consider adaptive layer selection, especially given the LLaMA-3B exception.

  3. For other issues, please refer to the "weaknesses" section.

Reviewer_vd7uOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper targets hallucination in LLMs through contrastive decoding at inference time. Prior layer-based contrastive methods, such as DoLa, subtract a premature distribution read from an earlier layer of the frozen backbone from the final-layer distribution, but the contrast strength is the same for every token. LUCiD makes the contrast itself learnable where a lightweight probe attached to a single fixed middle layer produces the premature distribution through the model's own frozen output head, and a learned gate modulates the contrast strength per token under a plausibility constraint.

Critical strengths
  • The method is sound.

  • The paper is well-writen.

  • The empirical evaluations are generally comprehensive.

Critical weaknesses
  • The proposed method requires training, and its behavior depends heavily on the training data. The paper offers little analysis of how the choice, scale, or domain of the training corpus affects the learned contrast. It is unclear whether the learned contrast helps with hallucination in out-of-distribution domains.

  • The effectiveness of the proposed method is questionable. Evaluation results are mixed. On one of the five models the method performs worse than standard decoding, and on a knowledge-intensive benchmark the learned contrast collapses relative to vanilla. Several reported gains are small enough.

  • The probe location is hard-coded to the middle layer. The paper's own depth sweep shows this fixed choice is not optimal for at least some models, and the method offers no mechanism for selecting or learning the probe placement.

  • The paper does not analyze what the learned probe learns. The central claim is that a trained premature distribution is a better contrastive signal than the raw early exit, yet there is no evidence on how far the probe moves from its identity initialization, how the learned distribution differs from the raw one, or why the learned subtraction suppresses prior-driven tokens rather than useful context.

  • There is a train-test mismatch inside the method's own pipeline. The gate is trained only on answer positions, since the loss is computed on answer tokens, but at inference it modulates the contrast for every generated token, including positions never covered by the training signal.

  • Key training details are missing, e.g., number of training steps.

Questions for authors

See Weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.