acceptodds
Under review as a conference paper at ICLR 2027

OMNI-TRUTH: A Unified Framework for LLM Hallucination Mitigation

Abstract

Large language models must remain faithful to supplied evidence in closed-domain generation and answer reliably from parametric knowledge in open-domain generation without external evidence. Achieving both behaviors in a single model remains challenging, as improving one can compromise the other. We introduce Omni-Truth, a unified post-training framework for closed-domain faithfulness and open-domain factuality. Omni-Truth first develops a shared reflection-and-correction policy through supervised fine-tuning, then applies closed-domain followed by open-domain reinforcement learning to acquire complementary capabilities. Task-specific multi-teacher on-policy distillation consolidates these capabilities into a single model. Across six benchmarks, Omni-Truth achieves the highest macro-average score among the evaluated Qwen3-8B-based methods (67.54), while balancing closed-domain faithfulness and open-domain factuality. On the closed-domain benchmarks, Omni-Truth achieves an average score of 83.75, only 2.69 points below DeepSeek V4 Flash under our evaluation protocol.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Hallucination, Contextual Faithfulness, Factuality, LLM Post-Training, Reinforcement Learning, On-Policy Distillation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_dtRuOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a unified framework, Omni-Truth, to mitigate hallucinations in both open-domain and closed-domain scenarios, referring to free-generation and context-based generation, respectively. The framework has three stages. First, it starts with SFT on reflection-then-correction responses. Then it applies GRPO to obtain a closed-domain specialist and a Cascade policy. Finally, it uses MOPD to complete task-specific policy consolidation. The authors empirically evaluate Omni-Truth on several benchmarks with Qwen3-8B model and compare against baselines. Experimental results along with ablation studies validate the effectiveness of Omni-Truth in improving both open-domain factuality and closed-domain faithfulness.

Critical strengths

A useful empirical improvement in contextual faithfulness and factuality. The paper addresses a practically important challenge for models that must both follow supplied evidence and answer from parametric knowledge. Relative to the Closed-to-Open Cascade, Omni-Truth improves FACTS Grounding while also improving MultiHopRAG and the open-domain average. These results provide evidence that consolidation can partially recover source grounding weakened by sequential training while preserving the benefits of that training. This demonstrated improvement in the balance between the two objectives is the main empirical strength supporting acceptance in my assessment.

Critical weaknesses
  1. Limited novelty and insufficient support for the main design choices. The Omni-Truth framework follows a pipeline similar to that of MOPD [1]. Its main changes include Cascade student initialization and task-specific use of the Closed-only and Cascade teachers πc\pi_c and πc,o\pi_{c,o}. The initialization ablation supports starting from the Cascade policy. However, the evidence for the chosen teacher pair and task assignment is less clear. The authors provide a plausible explanation based on the teachers’ benchmark scores, but the current comparisons change several parts of the configuration at once. They therefore do not clearly show whether the gains come from the proposed teacher selection and assignment. Comparisons with alternative teachers or assignments under otherwise matched conditions would provide stronger support. Moreover, the paper does not isolate whether reflection-specific supervision improves the final model over comparable SFT without reflection. These gaps leave the methodological contribution beyond adapting MOPD insufficiently established.

  2. Limited evidence that the selected configuration generalizes. All controlled training experiments use Qwen3-8B. The final method improves the Macro score by 0.74 percentage points over both Cascade and Standard MOPD, and by 0.35 points over Naive Cascade MOPD. Some individual gains are larger, but they come with lower scores on other benchmarks. The chosen teacher pair, although having strengths in this setting, may change with the backbone, which could also change the preferred initialization and teacher group. The paper does not test whether the proposed configuration remains effective under different settings. Thus, it remains unclear whether Cascade initialization and the chosen teacher pair provide a consistent advantage across models. Given the marginal performance gains, the evidence from a single backbone is insufficient to establish the broad applicability of the proposed framework.

[1] Ma, Wenhan, et al. "Mopd: Multi-teacher on-policy distillation for capability integration in llm post-training." arXiv preprint arXiv:2606.30406 (2026).

Questions for authors

Based on the two weaknesses, I have the following questions.

  1. Contribution of reflection SFT. What specific benefit does reflection-and-correction supervision provide to the final model? Beyond the behavioral analysis in Appendix E.2, is there evidence that it improves final performance compared with otherwise comparable SFT without reflection?

  2. Contribution beyond applying MOPD. Applying an existing method to a new setting can be valuable when it yields new insights. What does Omni-Truth establish about integrating closed-domain faithfulness and open-domain factuality beyond showing that MOPD is useful in this setting? In particular, what insight motivates the proposed teacher construction and task assignment, and which comparisons substantiate it? My concern is whether these adaptations provide a contribution beyond a task-specific configuration of MOPD. Clarifying this distinction would materially affect my assessment of the paper’s contribution.

  3. Evidence beyond Qwen3-8B. Do you have results on another backbone or model scale showing that the proposed configuration improves over Cascade and Standard MOPD? Such evidence would help determine whether the observed advantage extends beyond the particular teacher strengths found on Qwen3-8B.

9 comments
AuthorsOpenReview ↗

Response to Reviewer dtRu (1/3)

W1.

Limited novelty and insufficient support for the main design choices. The Omni-Truth framework follows a pipeline similar to that of MOPD [1]. Its main changes include Cascade student initialization and task-specific use of the Closed-only and Cascade teachers πc\pi_c and πc,o\pi_{c,o}. The initialization ablation supports starting from the Cascade policy. However, the evidence for the chosen teacher pair and task assignment is less clear. The authors provide a plausible explanation based on the teachers' benchmark scores, but the current comparisons change several parts of the configuration at once. They therefore do not clearly show whether the gains come from the proposed teacher selection and assignment. Comparisons with alternative teachers or assignments under otherwise matched conditions would provide stronger support. Moreover, the paper does not isolate whether reflection-specific supervision improves the final model over comparable SFT without reflection. These gaps leave the methodological contribution beyond adapting MOPD insufficiently established.

Teacher pair and task assignment under matched conditions.

Starting from the same Cascade checkpoint, with the same data and training budget, we varied only the teacher assigned to each task (Qwen3-8B):

VariantQA teacherSummary teacherFACTSMultiHopCRAGClose avgOpen avgMacro
Cascade (initialization)——70.7479.2439.2382.6451.6167.13
OMNI-TRUTHCascadeClosed-only72.6380.2440.0183.5551.5867.56
All-CascadeCascadeCascade71.3779.5639.9682.8751.3867.12
No QA distillation—Closed-only75.2176.8936.4483.3749.6366.50
SwappedClosed-onlyCascade73.2875.1638.7482.0250.4666.24
All-ClosedClosed-onlyClosed-only75.3575.6435.8782.9449.2666.10

Every alternative assignment lowers Macro, by 0.44–1.46 points. Whenever QA is not distilled from the Cascade, MultiHopRAG drops by 4.6–5.1 points and CRAG by 1.3–4.1, because the Closed-only teacher spreads over-abstention to QA (the Closed-only policy declines 26.3% of MultiHopRAG questions by citing missing input, vs. 18.5% for the Cascade). Distilling summarization from the Closed-only teacher instead of the Cascade adds 1.26 points on FACTS Grounding. QA must therefore come from the Cascade, and summarization from the Closed-only policy.

Reflection SFT vs. SFT without reflection.

OMNI-TRUTH uses reflection SFT as the shared starting point for both regimes. To test whether the reflection step matters, we sampled SFT data in the same format as the reflection SFT data but without the reflection suffix, and trained with the same SFT configuration, at both scales.

Qwen3-8B SFTFACTSHHEMMultiHopCRAGWebQAC-SQAMacro
With reflection42.7894.0675.9937.6770.0037.6859.70
Without reflection38.0692.5378.5740.5169.7536.2059.27

We repeated the comparison on Qwen3-4B with the same data pipeline, SFT recipe, and evaluation:

Qwen3-4B SFTFACTSHHEMMultiHopCRAGWebQAC-SQAMacro
With reflection40.1094.4269.6928.7562.3927.3553.78
Without reflection37.0293.5367.6331.9962.7626.0353.16

Reflection gives higher faithfulness at both scales: FACTS Grounding and HHEM are higher in every evaluation run, and Macro is higher as well. Reflection SFT teaches the model to check and revise its response against the input before answering: with reflection, most thinking traces on FACTS and HHEM contain an explicit verification step and a review step, while far fewer do without it (table below). These are the patterns that reduce closed-domain hallucination, and teaching them is the purpose of the SFT stage. The no-reflection model's advantages are limited and inconsistent. On MultiHopRAG it is ahead at 8B but behind at 4B. On CRAG it is about 3 points higher at both scales, but it shows no meaningful advantage on WebQA or Chinese SimpleQA. We therefore use reflection SFT.

Share of thinking traces (%)BaseWithout reflectionWith reflection
8B FACTS, verification step692875
8B FACTS, review step273366
8B HHEM, verification step854098
8B HHEM, review step131890
4B FACTS, verification step643075
4B FACTS, review step283366
4B HHEM, verification step875096
4B HHEM, review step172882
AuthorsOpenReview ↗

Response to Reviewer dtRu (2/3)

W2.

Limited evidence that the selected configuration generalizes. All controlled training experiments use Qwen3-8B. The final method improves the Macro score by 0.74 percentage points over both Cascade and Standard MOPD, and by 0.35 points over Naive Cascade MOPD. Some individual gains are larger, but they come with lower scores on other benchmarks. The chosen teacher pair, although having strengths in this setting, may change with the backbone, which could also change the preferred initialization and teacher group. The paper does not test whether the proposed configuration remains effective under different settings. Thus, it remains unclear whether Cascade initialization and the chosen teacher pair provide a consistent advantage across models. Given the marginal performance gains, the evidence from a single backbone is insufficient to establish the broad applicability of the proposed framework.

We replicated the full comparison on Qwen3-4B, retraining all baselines with the same data and evaluation:

Qwen3-4BFACTSHHEMMultiHopCRAGWebQAC-SQAClose avgOpen avgMacro
Closed-only RL80.6297.9574.5631.4963.3029.1484.3841.3162.84
Open-only RL45.7493.5071.4333.5067.5130.6670.2243.8957.06
Combined RL80.1997.8373.3630.7264.8429.5983.7941.7262.76
Standard MOPD80.0098.6475.3833.5166.1530.1984.6743.2963.98
Cascade (C→O)77.9198.1178.1633.2466.9630.2684.7343.4964.11
Reverse Cascade (O→C)78.6497.9971.8334.2968.2132.0482.8244.8563.83
OMNI-TRUTH78.2298.4878.7234.4066.8730.3485.1443.8764.50

The configuration transfers:

  • The trade-off reproduces. The Cascade leads standard MOPD on MultiHopRAG (78.16 vs. 75.38) but trails it on FACTS Grounding (77.91 vs. 80.00).
  • The limitation of standard MOPD lies in its teachers. It recovers its teachers almost fully (100.3% of its closed-domain teacher's average and 98.6% of its open-domain teacher's), but neither teacher has the Cascade's multi-document QA ability.
  • OMNI-TRUTH keeps its advantage. It has the highest Macro among dual-domain methods (64.50, vs. 64.11 for the Cascade and 63.98 for standard MOPD) and is the only one above 78 on both FACTS Grounding and MultiHopRAG.

At 8B, the matched ablations in W1 isolate the teacher assignment, and the MultiHopRAG gain over standard MOPD is significant (sign test over 2,548 questions, p<10−44p < 10^{-44}).

The margin over standard MOPD is concentrated by design. Standard MOPD here is the first merge of our own pipeline, built from our Closed-only and Open-only policies; since a distilled ability is largely bounded by its teacher, OMNI-TRUTH can only improve where those teachers fall short, and its gains lie exactly there:

  • MultiHopRAG: 80.16 vs. 75.34 for standard MOPD (significant, above).
  • FACTS Grounding: 73.13 vs. 70.30 for the Cascade it starts from. Its lower FACTS Grounding than standard MOPD (73.13 vs. 75.61; HHEM is within 0.16) is the cost of keeping the Cascade's multi-document QA, and OMNI-TRUTH recovers more than half of that gap (+2.83 of 5.31 points).
  • CRAG: above standard MOPD in each of the three training runs (40.05–40.93 vs. 36.65–39.93; table below). Standard MOPD's behavior depends on the run, from answering almost every question (1.3% not attempted, 59.0% wrong) to declining 19.2% of questions by citing missing input; OMNI-TRUTH cites missing input in at most 6.6%.
  • Open-domain average: 51.34, the highest among all Qwen3-8B methods in Table 1, including Open-only RL (50.21) and TruthRL (50.87).
CRAG, Qwen3-8BScoreCorrect (%)Wrong (%)Not attempted (%)Cites missing input (%)
Standard MOPD, run 139.9339.759.01.30.0
Standard MOPD, run 239.4036.850.013.24.6
Standard MOPD, run 336.6531.540.527.919.2
OMNI-TRUTH, run 140.0539.156.04.92.2
OMNI-TRUTH, run 240.1138.553.67.92.5
OMNI-TRUTH, run 340.9337.144.318.66.6

The Macro margin is small because HHEM is saturated (97.8–98.5 for all dual-domain methods) and the open-domain benchmarks are bounded by the base model's parametric knowledge, so most benchmarks barely separate the methods. Against Naive Cascade MOPD (+0.35 Macro), OMNI-TRUTH gains 4.11 points on MultiHopRAG; that comparison changes initialization and teacher routing at once, which the matched ablations in W1 separate.

AuthorsOpenReview ↗

Response to Reviewer dtRu (3/3)

Q1.

Contribution of reflection SFT. What specific benefit does reflection-and-correction supervision provide to the final model? Beyond the behavioral analysis in Appendix E.2, is there evidence that it improves final performance compared with otherwise comparable SFT without reflection?

Yes (W1). At both scales, reflection SFT gives higher FACTS Grounding and HHEM and higher Macro than SFT without reflection. It teaches the model to verify and revise its answer against the input, the behavior that reduces closed-domain hallucination.

Q2.

Contribution beyond applying MOPD. Applying an existing method to a new setting can be valuable when it yields new insights. What does Omni-Truth establish about integrating closed-domain faithfulness and open-domain factuality beyond showing that MOPD is useful in this setting? In particular, what insight motivates the proposed teacher construction and task assignment, and which comparisons substantiate it? My concern is whether these adaptations provide a contribution beyond a task-specific configuration of MOPD. Clarifying this distinction would materially affect my assessment of the paper's contribution.

The study establishes three points beyond showing that MOPD is useful:

  1. Closed- and open-domain RL conflict because each single-domain policy carries its evidence rule into the other regime (the Closed-only policy declines 14.5% of CRAG questions by citing missing input; the Open-only policy reaches only 41.59 on FACTS Grounding). Standard MOPD over the two single-domain teachers resolves this conflict.
  2. This merge is not sufficient. MultiHopRAG, QA over supplied documents, gains from sequential closed-to-open RL (79.43 for the Cascade vs. 75.34 for standard MOPD), an ability neither single-domain teacher has, while the Cascade loses grounding (FACTS Grounding 70.30 vs. 75.61).
  3. The second merge must target both data and teachers. Using the Cascade as the student carries its open-domain ability, so this stage needs only closed-domain data. Each closed-domain task is assigned the teacher that holds the required ability: QA from the Cascade, summarization from the Closed-only policy.

The matched ablations in W1 substantiate this assignment: every alternative lowers Macro (by 0.44–1.46 points); QA distilled from the Closed-only teacher loses 4.6–5.1 points on MultiHopRAG and 1.3–4.1 on CRAG, and summarization distilled from the Cascade instead of the Closed-only policy loses 1.26 points on FACTS Grounding. The resulting model improves where each merge is aimed: MultiHopRAG 80.16 vs. 75.34 for standard MOPD, FACTS Grounding 73.13 vs. 70.30 for the Cascade, and CRAG above standard MOPD in every training run (W2). The pattern reproduces at 4B (W2). We will reframe Sections 1 and 4 accordingly.

Q3.

Evidence beyond Qwen3-8B. Do you have results on another backbone or model scale showing that the proposed configuration improves over Cascade and Standard MOPD? Such evidence would help determine whether the observed advantage extends beyond the particular teacher strengths found on Qwen3-8B.

Yes, on Qwen3-4B; see W2.

References

  • [1] Ma, W., Wei, J., Zhao, L., et al. MOPD: Multi-teacher on-policy distillation for capability integration in LLM post-training. arXiv:2606.30406, 2026.
Reviewer_b7iSOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes OMNI-TRUTH, a framework for reducing hallucinations in both closed-domain and open-domain settings. The method combines SFT, RL, and distillation to improve factuality and faithfulness in a model. Experiments on 6 benchmarks show improvements over several baselines.

Critical strengths
  • The authors conduct experiments on the recent Qwen3-8B model across multiple benchmarks.
Critical weaknesses
  • The method mainly combines existing SFT, RL, and distillation techniques into a three-stage training pipeline. The key technical novelty is not clear beyond putting them together.
  • The improvements over baselines are relatively small.
Questions for authors
  • The title calls this a unified framework for LLM hallucination mitigation, but the paper only unifies closed-domain and open-domain settings. The title feels overclaimed for actual contribution.
  • The conflict between closed-domain and open-domain factuality seems partly due to how correct answers are defined for context is provided or not. Could the authors clarify why these two settings are inherently conflicting rather than just requiring different correct answers?
  • The main experiments only use Qwen3-8B. Could the authors provide results on other model families or larger models to show whether the method generalizes?
5 comments
AuthorsOpenReview ↗

Response to Reviewer b7iS (1/2)

W1.

The method mainly combines existing SFT, RL, and distillation techniques into a three-stage training pipeline. The key technical novelty is not clear beyond putting them together.

Our contribution is not a new training component but what the study establishes about integrating the two regimes:

  • Data and evaluation. Training data and a unified six-benchmark suite covering closed-domain faithfulness and open-domain factuality.
  • The conflict. Closed- and open-domain RL conflict because each single-domain policy carries its evidence rule into the other regime (Q2). MOPD over the two single-domain teachers resolves this conflict.
  • A case MOPD cannot resolve. MultiHopRAG, QA over supplied documents, gains from sequential closed-to-open RL (the Cascade: 79.43 vs. 75.34 for standard MOPD), an ability neither single-domain teacher has, while the Cascade loses FACTS Grounding (70.30 vs. 75.61).
  • OMNI-TRUTH. A second MOPD stage with the Cascade as the student assigns each closed-domain task the teacher that holds the required ability. It merges both abilities and has the highest Macro among dual-domain methods at both 8B and 4B.

Prior work either diagnoses the trade-off (Bi et al., 2025), edits shared activation subspaces at inference time (SPACE; Wang et al., 2025a), or trains one policy jointly on both settings (Wang et al., 2025b). The last corresponds to our Combined RL baseline, which is below OMNI-TRUTH at both scales. We will restate the contributions in Section 1 accordingly.

W2.

The improvements over baselines are relatively small.

The Macro margin understates the improvement. HHEM is saturated (97.8–98.5 for all dual-domain methods) and the open-domain benchmarks are bounded by the base model's parametric knowledge, so most benchmarks barely separate the methods. The gain lies on the two benchmarks that define the trade-off: MultiHopRAG improves by 4.82 points over standard MOPD (80.16 vs. 75.34; sign test over 2,548 questions, p<10−44p < 10^{-44}), and FACTS Grounding by 2.83 points over the Cascade (73.13 vs. 70.30).

Moreover, standard MOPD here is not a prior method on this problem: we built it from our own Closed-only and Open-only policies as the first merge of our pipeline, and OMNI-TRUTH is the next stage. Since a distilled ability is largely bounded by its teacher, the gain over standard MOPD can only appear where its teachers fall short.

Behavior also improves on CRAG. OMNI-TRUTH is above standard MOPD in each of the three training runs (40.05–40.93 vs. 36.65–39.93; table below), whose behavior depends on the run, from answering almost every question (1.3% not attempted, 59.0% wrong) to declining 19.2% of questions by citing missing input; OMNI-TRUTH cites missing input in at most 6.6%. Trained from the Cascade, which answers almost everything (3.2% not attempted, 58.2% wrong), OMNI-TRUTH declines more of the questions it cannot answer, cuts wrong answers to 43.5%, and raises the CRAG score (39.23 to 40.01).

CRAG, Qwen3-8BScoreCorrect (%)Wrong (%)Not attempted (%)Cites missing input (%)
Standard MOPD, run 139.9339.759.01.30.0
Standard MOPD, run 239.4036.850.013.24.6
Standard MOPD, run 336.6531.540.527.919.2
OMNI-TRUTH, run 140.0539.156.04.92.2
OMNI-TRUTH, run 240.1138.553.67.92.5
OMNI-TRUTH, run 340.9337.144.318.66.6

OMNI-TRUTH also has the highest open-domain average among all Qwen3-8B methods in Table 1, including open-domain-only RL (51.34 vs. 50.21 for Open-only RL and 50.87 for TruthRL). The same pattern holds on Qwen3-4B (Q3). We will report FACTS Grounding and MultiHopRAG alongside Macro.

Q1.

The title calls this a unified framework for LLM hallucination mitigation, but the paper only unifies closed-domain and open-domain settings. The title feels overclaimed for actual contribution.

We agree and propose to retitle the paper "OMNI-TRUTH: Unifying Closed-Domain Faithfulness and Open-Domain Factuality in LLM Post-Training", with the abstract scoped accordingly.

AuthorsOpenReview ↗

Response to Reviewer b7iS (2/2)

Q2.

The conflict between closed-domain and open-domain factuality seems partly due to how correct answers are defined for context is provided or not. Could the authors clarify why these two settings are inherently conflicting rather than just requiring different correct answers?

The two settings do require different correct answers, and one model can in principle produce both. The difficulty is that RL in one setting changes behavior in the other: each policy generalizes its evidence rule beyond the setting it was trained on.

  • Closed-domain RL carries "use only the input" into questions without input. On CRAG, the Closed-only policy declines 14.5% of questions by citing missing input, vs. 0.0% for Open-only and 3.8% for OMNI-TRUTH. These refusals are not due to missing knowledge: asked whether Cornell or Columbia Law School has the longer history, its reasoning concludes that Columbia is older, yet it answers "The input does not provide the founding years of the law schools."
  • Open-domain RL carries "answer from what you know" into grounded tasks. On about 310 of the 860 FACTS Grounding prompts, the Open-only policy is unfaithful where the Closed-only policy is faithful. Asked to list people who died in 1957 from a document that only mentions "the Master's death", it writes "Christian Dior died in 1957."
  • The same transfer occurs in distillation. With data and training budget fixed, distilling both tasks from the Closed-only teacher raises CRAG abstention to 37%, vs. about 3% when the Cascade teaches both.
  • Joint training does not avoid the transfer. Combined RL trains one policy on both settings at once. It keeps open-domain accuracy (open-domain average 50.61, vs. 50.21 for Open-only RL), but the open-domain objective still leaks into grounded tasks: FACTS Grounding falls to 69.68 (vs. 75.15 for Closed-only RL), and it gains nothing on multi-document QA (MultiHopRAG 74.54, the lowest among dual-domain methods).

The conflict therefore lies in what the policy generalizes, not in the definition of a correct answer. We will add these statistics and examples to the paper.

Q3.

The main experiments only use Qwen3-8B. Could the authors provide results on other model families or larger models to show whether the method generalizes?

We replicated the full comparison on Qwen3-4B, retraining all baselines with the same data and evaluation:

Qwen3-4BFACTSMultiHopClose avgOpen avgMacro
Closed-only RL80.6274.5684.3841.3162.84
Open-only RL45.7471.4370.2243.8957.06
Combined RL80.1973.3683.7941.7262.76
Standard MOPD80.0075.3884.6743.2963.98
Cascade (C→O)77.9178.1684.7343.4964.11
Reverse Cascade (O→C)78.6471.8382.8244.8563.83
OMNI-TRUTH78.2278.7285.1443.8764.50

The trade-off reproduces: the Cascade leads standard MOPD on MultiHopRAG (78.16 vs. 75.38) but trails it on FACTS Grounding (77.91 vs. 80.00). OMNI-TRUTH has the highest Macro among dual-domain methods and is the only one above 78 on both FACTS Grounding and MultiHopRAG.

References

  • Bi, B., Liu, S., Wang, Y., Mei, L., Fang, J., Gao, H., Ni, S., and Cheng, X. Is factuality enhancement a free lunch for LLMs? Better factuality can lead to worse context-faithfulness. ICLR, 2025.
  • Wang, P., Li, C., Wang, C., Zheng, L., Zhang, L., and Zhang, X. One SPACE to rule them all: Jointly mitigating factuality and faithfulness hallucinations in LLMs. NeurIPS, 2025a.
  • Wang, Y., Yang, Z., Ma, W., Sui, Z., and Zhao, L. Enhancing reliability across short and long-form QA via reinforcement learning. arXiv:2512.08944, 2025b.
AuthorsOpenReview ↗

Gentle Follow-up

Dear Reviewer b7iS,

Thank you again for your helpful comments. As the discussion period is coming to a close, we wanted to gently follow up on our response. In brief, we:

  • clarified our contributions and how they relate to prior work (W1);
  • showed where the gains lie: +4.82 on MultiHopRAG over standard MOPD and +2.83 on FACTS Grounding over the Cascade (W2);
  • proposed a more precise title (Q1);
  • explained why the two settings conflict beyond how correct answers are defined (Q2);
  • replicated the full comparison on Qwen3-4B (Q3).

We would greatly appreciate it if you could let us know whether these address your concerns. If they do, we would be grateful if you would consider updating your assessment.

Best regards, The Authors

Reviewer_NgXXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper aims to train a single model that does well at two things: remaining faithful to supplied evidence in closed-domain generation, and answering reliably from parametric knowledge in open-domain generation without external evidence.

Critical strengths
  1. The experiments show that the final model recovers grounding.
  2. The controlled experiments show where faithfulness is lost.
Critical weaknesses
  1. The gain over standard MOPD is small, not statistically significant, and comes from a single benchmark.
  • The gain is small and not significant: 67.54 vs. 66.80.
  • It comes entirely from MultiHopRAG. Final model minus MOPD, per benchmark: MultiHopRAG +4.82, CRAG +1.70, WebQA +0.44, Chinese SimpleQA +0.13, HHEM −0.16, FG −2.48. Without MultiHopRAG, the other five average −0.07. The QA half of the distillation adds no new teacher signal: it only anchors the student to where it started, and its measured benefit is small and confounded.
  1. The QA teacher is the starting point. The student starts as a copy of the Cascade, which is also its QA teacher. On QA prompts it therefore learns nothing at first; later, this teacher only pulls the student back toward where it started. The only new teaching comes from - Closed-only on summarization.
  • The ablation is small and confounded. Removing the QA teacher (summary-only) costs 0.69 points (p = 0.18), but that variant also doubles the summarization prompts per batch (128 vs. 64). The gain over the simpler Naive Cascade MOPD is only +0.35 (p = 0.48).
Questions for authors
  1. If the factuality–faithfulness trade-off is mainly forgetting during sequential training, what problem remains that standard MOPD does not already solve?
  2. Has the model learned when to rely on the context, or only to behave differently when a document is attached? How does it handle incomplete or conflicting context, and explicit instructions to allow or forbid world knowledge?
  3. Would standard anti-forgetting methods close the same gap?
7 comments
AuthorsOpenReview ↗

Response to Reviewer NgXX (1/3)

W1.

The gain over standard MOPD is small, not statistically significant, and comes from a single benchmark. The gain is small and not significant: 67.54 vs. 66.80. It comes entirely from MultiHopRAG. Final model minus MOPD, per benchmark: MultiHopRAG +4.82, CRAG +1.70, WebQA +0.44, Chinese SimpleQA +0.13, HHEM −0.16, FG −2.48. Without MultiHopRAG, the other five average −0.07. The QA half of the distillation adds no new teacher signal: it only anchors the student to where it started, and its measured benefit is small and confounded.

Standard MOPD is not an external baseline we aim to bypass: it is a component of our pipeline, reported as an ablation, and OMNI-TRUTH is a next-stage MOPD built on it. Our goal is to restore all abilities in one model; since a distilled ability is bounded by its teacher, the gain over standard MOPD can only appear where its teachers fall short. The gain is concentrated by design. The Cascade is strong on MultiHopRAG but loses FACTS Grounding, whereas standard MOPD preserves FACTS Grounding but misses the Cascade's MultiHopRAG gain (Section 6.3). OMNI-TRUTH combines the two abilities (Table 1):

MethodFACTS GroundingMultiHopRAG
Closed-only RL75.1575.09
Standard MOPD75.6175.34
Cascade (C→O)70.3079.43
OMNI-TRUTH73.1380.16

OMNI-TRUTH retains the Cascade's MultiHopRAG advantage (80.16, +4.82 over standard MOPD; sign test over 2,548 questions, p<10−44p < 10^{-44}) and recovers more than half of the FACTS Grounding gap between the Cascade and standard MOPD (+2.83 of 5.31 points).

The Macro margin is small because most other benchmarks barely separate the dual-domain methods: HHEM is saturated, and the open-domain benchmarks are bounded by parametric knowledge. The revision will report FACTS Grounding and MultiHopRAG alongside Macro.

Our new controlled ablations (table below) trace this gain to distilling QA from the Cascade: whenever QA is taught by the Closed-only policy (All-Closed, Swapped), MultiHopRAG drops by 4.6–5.1 points, because multi-document QA is precisely where the Closed-only policy over-abstains (it declines 26.3% of MultiHopRAG questions, vs. 18.5% for the Cascade). The comparison with Naive Cascade MOPD in Table 2 changes initialization and teacher routing at once; the new ablations fix the Cascade initialization and vary only the routing, and every alternative assignment lowers Macro, by 0.44–1.46 points.

Qwen3-8B, same Cascade initializationQA teacherSummary teacherFACTSMultiHopCRAGClose avgOpen avgMacro
Cascade (initialization)——70.7479.2439.2382.6451.6167.13
OMNI-TRUTHCascadeClosed-only72.6380.2440.0183.5551.5867.56
All-CascadeCascadeCascade71.3779.5639.9682.8751.3867.12
No QA distillation—Closed-only75.2176.8936.4483.3749.6366.50
SwappedClosed-onlyCascade73.2875.1638.7482.0250.4666.24
All-ClosedClosed-onlyClosed-only75.3575.6435.8782.9449.2666.10

The gain also extends beyond MultiHopRAG. On CRAG, OMNI-TRUTH is above standard MOPD in each of the three training runs (40.05–40.93 vs. 36.65–39.93; table below). Standard MOPD's behavior depends on the run, from answering almost every question (1.3% not attempted, 59.0% wrong) to reverting to the Closed-only failure (19.2% of questions declined by citing missing input); OMNI-TRUTH cites missing input in at most 6.6%. The Cascade answers almost every question (3.2% not attempted), so 58.2% of its answers are wrong. Trained from the same Cascade, OMNI-TRUTH declines more of the questions it cannot answer (20.6% not attempted) and cuts wrong answers to 43.5%, while correct answers change by only 2.7 points (38.6 to 35.9); its CRAG score rises from 39.23 to 40.01. Training thus moves open-domain behavior in the right direction, toward answering what the model knows and declining what it does not, while the open-domain scores themselves remain bounded by the base model's parametric knowledge.

CRAG, Qwen3-8BScoreCorrect (%)Wrong (%)Not attempted (%)Cites missing input (%)
Standard MOPD, run 139.9339.759.01.30.0
Standard MOPD, run 239.4036.850.013.24.6
Standard MOPD, run 336.6531.540.527.919.2
OMNI-TRUTH, run 140.0539.156.04.92.2
OMNI-TRUTH, run 240.1138.553.67.92.5
OMNI-TRUTH, run 340.9337.144.318.66.6
AuthorsOpenReview ↗

Response to Reviewer NgXX (2/3)

W2.

The QA teacher is the starting point. The student starts as a copy of the Cascade, which is also its QA teacher. On QA prompts it therefore learns nothing at first; later, this teacher only pulls the student back toward where it started. The only new teaching comes from Closed-only on summarization. The ablation is small and confounded. Removing the QA teacher (summary-only) costs 0.69 points (p = 0.18), but that variant also doubles the summarization prompts per batch (128 vs. 64). The gain over the simpler Naive Cascade MOPD is only +0.35 (p = 0.48).

We agree that the QA branch acts as an anchor, and we tested whether it is necessary with the confound removed (No QA distillation row in the W1 table). In the No QA distillation variant, we keep the full method's training data and summarization distillation unchanged and only remove the QA distillation signal. This variant is below OMNI-TRUTH (−1.06 Macro) and even below its own initialization, the Cascade (66.50 vs. 67.13). The loss is concentrated in open-domain and multi-document QA (Open avg −1.95, MultiHopRAG −3.35, CRAG −3.57), and the share of CRAG questions the model declines to answer rises from about 21% to about 36%.

The QA branch therefore adds no new knowledge but prevents summarization distillation from the Closed-only teacher from spreading over-abstention to QA; without it, the student gains 2.58 points on FACTS Grounding but loses more elsewhere. We will revise Section 4.4 to describe the QA branch as an anchor that preserves the Cascade's QA behavior rather than as a second source of knowledge.

Q1.

If the factuality–faithfulness trade-off is mainly forgetting during sequential training, what problem remains that standard MOPD does not already solve?

Forgetting explains only part of the trade-off. Sequential open-domain RL can erode what closed-domain RL has taught, but the two objectives also conflict in behavior, and this appears even without sequential training:

  • Abstention. The Closed-only policy declines 14.5% of CRAG questions and 26.3% of MultiHopRAG questions by citing missing input, vs. 0.0% and 12.6% for the Open-only policy.
  • Use of parametric knowledge. The Open-only policy adds outside knowledge to grounded answers: FACTS Grounding 41.59, vs. 75.15 for the Closed-only policy.
  • Reasoning and answer length. Closed-domain RL makes the model reason longer and answer more briefly, staying close to the input, which improves grounding; open-domain RL lengthens answers again to draw on parametric knowledge, which improves factuality (Appendix Table 17).

Standard MOPD merges the Closed-only and Open-only teachers and resolves most of this conflict on average. What it leaves unsolved is a case that belongs to neither side: MultiHopRAG is QA over supplied documents and needs both behaviors. Only sequential closed-to-open training (the Cascade) acquires it (79.43 vs. 75.34 for standard MOPD), and neither teacher of standard MOPD carries it. OMNI-TRUTH adds a second merge for this case. The same holds on Qwen3-4B with all baselines retrained:

Qwen3-4BFACTSMultiHopMacro
Standard MOPD80.0075.3863.98
Cascade (C→O)77.9178.1664.11
OMNI-TRUTH78.2278.7264.50
AuthorsOpenReview ↗

Response to Reviewer NgXX (3/3)

Q2.

Has the model learned when to rely on the context, or only to behave differently when a document is attached? How does it handle incomplete or conflicting context, and explicit instructions to allow or forbid world knowledge?

OMNI-TRUTH distinguishes the two settings far better than either single-domain policy. A model trained in only one setting misapplies its rule in the other: the Closed-only policy declines open-domain questions by citing missing input (14.5% of CRAG questions, vs. 0.0% for the Open-only policy), and the Open-only policy adds outside knowledge to grounded answers (FACTS Grounding 41.59 for the Open-only policy). OMNI-TRUTH reduces the first failure almost fourfold (to 3.8% of CRAG questions), so it rarely declines open-domain questions for lack of input. Examples from CRAG:

CRAG questionClosed-onlyOMNI-TRUTH
Which university did Natalie Portman attend?"The input does not specify which university Natalie Portman attended."Harvard University
What was the highest-grossing movie in the box office in 2023?"No information provided"Barbie
Which law school has a longer history, Cornell or Columbia?"The input does not provide the founding years of the law schools, so the history cannot be determined from the given information."Columbia Law School
Is ricotta drinkable?"The input does not provide information about ricotta's drinkability."No.

These refusals are not due to missing knowledge: on the Cornell/Columbia question, the Closed-only model's reasoning identifies Columbia as older, yet it declines.

Incomplete context. OMNI-TRUTH does not fill gaps in the context from parametric knowledge. On the 301 MultiHopRAG questions whose supplied articles do not contain the answer, it answers "Insufficient information" in 85.0% of cases, vs. 59.2% for the Open-only policy. Asked which North African country "expected to see a significant rise in GDP also recently unveiled an ancient artifact believed to be from the reign of a pharaoh whose name begins with T", the Open-only policy answers "Egypt" from parametric knowledge, while OMNI-TRUTH answers "Insufficient information."

Instructions that forbid world knowledge. Every FACTS Grounding prompt states "refer only to the context document. Do not employ any outside knowledge." When parametric knowledge competes with the document, the Open-only policy follows its own knowledge: asked to list people who died in 1957 from a document that only mentions "the Master's death", it writes "Christian Dior died in 1957", whereas OMNI-TRUTH answers "No names given for 1957 deaths." We will add these cases and statistics to the revision.

Q3.

Would standard anti-forgetting methods close the same gap?

Two common ways to avoid sequential forgetting, joint training on both domains (Combined RL) and standard MOPD, are already baselines in the paper, and both are below OMNI-TRUTH on Qwen3-8B. We additionally evaluated weight-space interpolation between the Closed-only policy and the Cascade trained from it, θ=αθClosed-only+(1−α)θCascade\theta = \alpha\theta_{\text{Closed-only}} + (1-\alpha)\theta_{\text{Cascade}} with α∈{0.3,0.5,0.7}\alpha \in \lbrace 0.3, 0.5, 0.7 \rbrace. Interpolation trades one domain for the other: as α increases, FACTS Grounding rises from 70.74 to 75.93, whereas MultiHopRAG falls from 79.24 to 76.21 and the open-domain average from 51.61 to 50.14. No interpolated model keeps both abilities, so the trade-off reflects a genuine conflict between the two objectives, not merely forgetting. No interpolated model exceeds the Cascade in Macro (66.65–66.99 vs. 67.13), and OMNI-TRUTH trained from the same Cascade is higher than every interpolated model on both the closed-domain average (83.55 vs. at most 83.25) and the open-domain average (51.58 vs. at most 51.18).

Qwen3-8BFACTSHHEMMultiHopCRAGWebQAC-SQAClose avgOpen avgMacro
Cascade (α = 0)70.7497.9579.2439.2374.6240.9882.6451.6167.13
α = 0.371.8697.7178.8140.0973.3240.1382.7951.1866.99
α = 0.573.4197.9177.2239.4772.9038.9782.8550.4566.65
α = 0.775.9397.5976.2139.3871.0140.0483.2550.1466.69
OMNI-TRUTH (same Cascade)72.6397.7780.2440.0173.5241.2283.5551.5867.56

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Summary of Rebuttal Response

Dear Reviewers and Area Chairs,

We sincerely thank Reviewers NgXX, b7iS, and dtRu and the Area Chair for your careful evaluation and constructive feedback.

We are encouraged that the reviewers found the problem practically important (dtRu) and the paper clear (NgXX, b7iS), and recognized the broad evaluation (b7iS), the controlled experiments that locate where faithfulness is lost (NgXX), and the improved balance between grounding and the Cascade's gains (NgXX, dtRu).

Contributions

Hallucination in LLMs spans two regimes with different evidence rules: closed-domain generation must stay faithful to supplied evidence, while open-domain generation must rely on parametric knowledge and know when to abstain. Our contributions are:

  • Data and unified evaluation. Training data and a six-benchmark evaluation suite covering both closed-domain faithfulness and open-domain factuality.
  • Conflict between the closed and open domains. Each single-domain RL policy carries its evidence rule into the other domain: the Closed-only policy declines 14.5% of CRAG questions by citing missing input (Open-only: 0.0%), and the Open-only policy adds outside knowledge to grounded answers (FACTS Grounding 41.59, vs. 75.15 for the Closed-only policy). MOPD resolves this conflict by merging the Closed-only and Open-only teachers.
  • MultiHopRAG: a case that requires a second merge. MultiHopRAG is QA over supplied documents, so it draws on both domains. It gains more from the Cascade (closed-domain RL followed by open-domain RL) than from MOPD (79.43 vs. 75.34; 75.10 on average for the single-domain policies), but the Cascade loses FACTS Grounding (70.30 vs. 75.61). The final model must keep the Cascade's MultiHopRAG ability together with the Closed-only policy's faithfulness.
  • OMNI-TRUTH: a next-stage MOPD that merges both abilities. The Cascade is the student and distills on two closed-domain subsets (QA and summarization); since it already performs well in the open domain, no open-domain data is needed, which also saves training compute. OMNI-TRUTH achieves the highest Macro among dual-domain methods, with the highest closed- and open-domain averages.

Summary of New Experiments and Manuscript Revisions

Following the reviewers' suggestions, we ran the experiments below with the paper's data and evaluation protocol; details are in the individual responses:

  • Controlled teacher-assignment ablations on Qwen3-8B (Reviewers NgXX, dtRu). Each variant changes one factor of the full method; all score below OMNI-TRUTH in Macro (by 0.44–1.46), and MultiHopRAG drops 4.6–5.1 points whenever QA is not distilled from the Cascade.
  • Full comparison on Qwen3-4B (Reviewers NgXX, b7iS, dtRu). With all baselines retrained, the trade-off reproduces; OMNI-TRUTH has the highest Macro (64.50, vs. 64.11 for the Cascade and 63.98 for MOPD).
  • Reflection SFT vs. SFT without reflection, at 8B and 4B (Reviewer dtRu). Reflection gives higher FACTS Grounding (+4.7 at 8B, +3.1 at 4B) and HHEM (+1.5, +0.9), and higher Macro.
  • Weight-space interpolation between the Closed-only policy and the Cascade (Reviewer NgXX). No interpolated model exceeds the Cascade in Macro, and OMNI-TRUTH beats all of them on both the closed- and open-domain averages.
  • Abstention and context-reliance analysis, with cases on missing, incomplete, and instruction-restricted context, and of CRAG answering behavior vs. standard MOPD and the Cascade (Reviewers NgXX, b7iS).

We will revise the manuscript to: retitle it "OMNI-TRUTH: Unifying Closed-Domain Faithfulness and Open-Domain Factuality in LLM Post-Training" (Reviewer b7iS); reframe Sections 1 and 4 around what the study establishes (Reviewers b7iS, dtRu); describe the QA branch in Section 4.4 as an anchor that preserves the Cascade's QA behavior (Reviewer NgXX); report FACTS Grounding and MultiHopRAG alongside Macro (Reviewer b7iS); and add the new results.

We hope these additions address the reviewers' concerns and welcome any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.