acceptodds
Withdrawn from ICLR 2027

Hallucination Mitigation via Directional self-guided, Sample-Selective Contrastive Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Hallucination in large language models remains a major obstacle to reliable deployment. Existing fine-tuning and preference optimization methods broadly update the model, potentially perturbing otherwise reliable continuation preferences, while treating preference pairs largely uniformly. We instead formulate hallucination mitigation as self-guided selective contrastive post-training, suppressing a model-generated incorrect continuation only when it remains competitive with the correct one. The method requires no expert annotation or manual data curation and adaptively activates updates only on hallucination-active examples, with the selected fraction varying across training distributions. Despite this selective update scheme, it consistently outperforms/it is comparable to full-data SFT and standard preference-optimization baselines under cross-dataset evaluation, yielding a stronger repair–damage trade-off. These results suggest that targeting hallucination-active examples can be effective, while enabling a lightweight and scalable self-guided framework that automatically generates, filters, and corrects its own hallucination cases.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, Hallucination

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_tAe2OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a post-training objective to reduce hallucination in short-answer QA. A frozen base model greedily answers each training prompt, and answers judged different from the gold answer are kept as negatives. The loss compares answer-level cross-entropy on the gold answer and on the model's own wrong answer. It is applied only when the margin falls below a threshold, uses a bounded gate on the deficit. Evaluation covers cross-dataset transfer, with ablations over hyperparameters, layers, seeds, backbones and judges.

Critical strengths

S1. Training one layer for 100 steps adds no inference cost and leaves general ability intact. The preference baselines use the same negative pool, so the comparison isolates the objective from the data.

S2. Reporting various metrics make the trade-off visible. The DROP to HaluEval-QA gain is stable across seeds.

Critical weaknesses

W1. The premise "the model knows but is misaligned" is never operationalized. Because y−y^- is the model's own greedy output, m<0m<0 holds almost automatically, whether or not the model knows the answer. No known/unknown split is made.

W2. The metric is error rate, not hallucination. Any mismatch with the gold answer is labeled HALLUCINATED, so the reported rate is roughly 1 − accuracy.

W3. The objective can only swap one answer for another, and no calibration or abstention metric is reported, even though Section 2.3 criticizes abstention-based methods.

W4. The net improvement over the base model is small, about 1.7% (DROP) and 1.1% (HotpotQA) of the evaluation set, and the gap to the baselines is under 1 pp in the HaluEval-trained setting.

W5. The same main configuration reports inconsistent numbers: Net 158, 189, 179 and 181 across Tables 1, 3/4a, 5 and 6a.

W6. Trainable parameters, number of steps and β\beta/learning rate are not given for SFT, DPO, ORPO or KTO.

W7. Qwen judges its own outputs. Other judges shift absolute rates by 6–24 pp.

W8. ∂g/∂d=1/(1+d)2\partial g/\partial d = 1/(1+d)^2, so the most confidently wrong samples receive the weakest updates. This is not discussed.

Questions for authors

See weaknesses. I would be happy to improve my score if my questions (especially for W1, W3, W4, and W8) could be reasonably resolved.

Reviewer_DFigOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates selective post-training for mitigating hallucinations in LLMs while preserving previously correct behavior. The method uses a frozen base model to generate candidate answers and dataset-provided gold answers as positive responses. Exact matching, numerical matching, and an LLM-based semantic matcher filter out equivalent answers. A bounded contrastive correction loss is then applied only when an incorrect response remains competitive with the correct response and is not already sufficiently unlikely. Training updates selected projection matrices within a single Transformer layer.

The experiments primarily use Qwen2.5-7B-Instruct and evaluate cross-dataset transfer involving HaluEval-QA, DROP, and HotpotQA, with comparisons against SFT, DPO, ORPO, and KTO. Additional experiments examine general capability preservation, random seeds, target layers, evaluator choices, and alternative backbone models.

Critical strengths
  1. The paper addresses a practically relevant problem and explicitly considers the adverse effects of correction. Reporting Improved and Worsened examples separately helps reveal whether hallucination mitigation damages previously correct behavior.
  2. Some transfer results provide a clear positive empirical signal. In Table 1, under DROP→HaluEval-QA, the proposed method achieves a hallucination rate of 9.39%, compared with 10.89% for KTO, 12.08% for SFT, and 12.29% for DPO.
  3. The paper includes several useful robustness and capability-preservation checks. Table 2 reports largely unchanged perplexity and MMLU performance, while Tables 4–7 examine different judges, target layers, seeds, and backbones.
  4. The method introduces no additional inference-time components. Its extra processing is confined to post-training, and deployment does not require an additional retriever, verifier, or evaluator.
Critical weaknesses
  1. Limited novelty comparison. The method operates on the difference between the average token-level log-likelihoods of positive and negative responses, adding margin-dependent gating and stopping conditions. SimPO [1] already uses length-normalized log-likelihoods and a target margin, while SLiC-HF [2] uses a margin-based sequence-likelihood ranking loss. The paper neither sufficiently discusses these connections nor compares against these objectives under matched negative examples, trainable parameters, and training budgets.
  2. Unclear ablation results. The all-pair variant in Table 8 removes not only sample activation but also the bounded margin term and the bad-branch stopping term, replacing the objective with the unbounded negative margin, −m, as described in Appendix A.6. Component-wise ablations and comparisons with random or simple hard-example selection under matched effective sample budgets are needed to establish the contribution of selective updating itself.
  3. Inconsistent experimental reporting. For HaluEval-QA→DROP, Table 1 reports Net=158 and Rate=43.06%, whereas the default parameter setting in Table 3 and the Qwen-judge row in Table 4 report Net=189 and Rate=42.89%. These differences may reflect different configurations, but their correspondence is not explained clearly enough.
  4. The evaluation depends substantially on the judge, and some small improvements lack independent validation. In Table 4, the hallucination rate on HotpotQA ranges from 26.62% to 50.64% depending on the judge, and Cohen’s κ is as low as 0.291 for one judge pair. Positive Net Improved values across judges are encouraging, but they do not establish that the ranking of Ours against the training baselines is also stable. This matters particularly for HaluEval-QA→DROP, where the improvement over DPO in Table 1 is only 0.17 percentage points. Standard EM/F1, human auditing, or multi-judge comparisons covering all methods would strengthen the evidence. The primary evaluation also focuses on short-answer correctness, leaving broader hallucination-mitigation claims insufficiently supported.
  5. Limited validation on newer model generations. The main comparative experiments rely on Qwen2.5-7B-Instruct, leaving the effectiveness of the method on newer models, such as Qwen3, insufficiently established. Although Table 7 includes additional backbones, these experiments lack matched baseline comparisons.
  6. Incomplete efficiency and reproducibility evidence. The fraction of examples with nonzero loss does not directly measure total computational savings: negative generation, semantic filtering, and forward passes for activation decisions also incur costs. End-to-end training time and peak memory are not reported. Furthermore, the reproducibility statement points to Appendix A.12 for the TRL baseline settings, but that section mainly repeats the data-construction procedure rather than specifying method-specific hyperparameters and tuning ranges. The semantic matcher’s model identity, dataset splits, and hyperparameter-selection procedure are also insufficiently specified.
  7. Inconsistent citation formatting and missing references. Several author–year citations appear after sentence-ending punctuation or without appropriate parentheses and grammatical integration, particularly on page 1, lines 32–38, and in the related-work section. The original papers introducing DPO, ORPO, and KTO are also absent from the bibliography despite these methods being central baselines.

Reference:

[1] SimPO: Simple Preference Optimization with a Reference-Free Reward.

[2] SLiC-HF: Sequence Likelihood Calibration with Human Feedback.

Questions for authors
  1. What is the main advantage over SimPO or SLiC-HF? Could you test each proposed component separately?
  2. Could you explain the differences in results and sample-usage ratios across tables?
  3. Why do the main comparisons focus on Qwen2.5-7B-Instruct? Could you provide fuller comparisons on other backbones or newer models?
  4. Do the improvements over competing methods remain consistent across different judges? Could you add standard metrics or a small human evaluation?
  5. Could you standardize the citation formatting and add the original DPO, ORPO, and KTO references?
Reviewer_XZeXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces a selective contrastive learning approach to mitigate hallucination in LLMs. By creating pairs of incorrect answers generated from the base model and gold answers they can train their model on these pairs. This has been shown to improve on multiple QA tasks when compared to SFT and preference optimization methods.

Critical strengths

No sufficiently strong strengths to support acceptance.

Critical weaknesses
  • Main: The method includes mainly existing techniques, such as self generated negative pairs, semantic filtering, contrastive learning. The overall design feels engineering. The key method novelty is not clear.
  • Performance improvements over the best baselines are somewhat small.
Questions for authors
  • The main experiments only used qwen2.5-7b-instruct base model. Are there more thorough baseline comparisons on other model families or larger models?
  • Are all marginal performance gains over baseline statistically significant?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.