acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Faithfulness in LLMs: A Pairwise Context-Sensitive Perspective

Abstract

Large language models (LLMs) are expected to answer questions faithfully based on the provided context, abstaining when the context information is insufficient to answer the questions. Existing faithfulness evaluations typically assess each question-context instance in isolation; however, such instance-level evaluation fails to capture a fundamental requirement of faithful behavior: the ability to adapt model responses to changes in available contexts. In particular, a model should provide correct answers when sufficient evidence is present and abstain when it is not. In this work, we propose a Pairwise Faithfulness Benchmark (PFaithBench) that evaluates whether a model can switch between answering and abstaining for the same question under supporting versus non-supporting contexts. Our evaluations across thirty-nine models with seven model families demonstrate that faithfulness fundamentally involves a trade-off between answering and abstaining, and that most current models exhibit a strong bias toward answering, with most faithfulness errors arising from over-answering, i.e., models tend to fabricate a response even when the provided context is insufficient. We further conduct a series of studies on faithfulness training under different data constructions. Our results show that training outcomes are highly sensitive to the specific composition of answering and abstaining data. Constructing answering and abstaining data from mismatched sources can cause models to rely on dataset-specific shortcuts rather than actual context sufficiency. Moreover, increasing answer-supervised data improves answering performance but exacerbates over-answering, while increasing abstaining data reduces hallucination but leads to over-abstention.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Pairwise Faithfulness, Over-answering and Over-abstention

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_zLKQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies context-grounded question answering through a paired evaluation of answering and abstention. It introduces PFaithBench, which associates each question with sufficient and insufficient contexts, and defines joint metrics that measure whether a model answers correctly in the former condition and abstains in the latter. The benchmark contains 3,344 paired examples across multi-hop, single-hop, original factual, and counterfactual subsets, with human verification of the evaluation data. The authors evaluate 39 model configurations from seven families and report frequent over-answering under insufficient context. They further conduct GRPO-based training experiments to examine source–sufficiency correlations and different proportions of answering and abstention examples, including asymmetric and symmetric reward settings. The reported results highlight source-dependent shortcuts and the sensitivity of answering and abstention behavior to training data composition.

Critical strengths
  1. The paired evaluation design provides a useful diagnostic of context-sensitive behavior. PFaithBench holds the question fixed while varying whether the context contains sufficient evidence. Its strict pairwise accuracy requires both a correct answer under the sufficient context and abstention under the insufficient context. This captures whether both behaviors succeed for the same question, which cannot be determined from aggregate answering and abstention accuracies alone. I consider this a meaningful contribution to the evaluation of context-grounded QA.

  2. The empirical study has substantial breadth and provides useful observations about training data composition. The evaluation covers 39 model configurations across seven families and includes multi-hop, single-hop, original factual, and counterfactual contexts. The training analyses demonstrate pronounced source-dependent answering/abstention behavior under mismatched data construction and examine different proportions of answering and abstention examples. Appendix C.2 also evaluates a symmetric reward, providing additional support that the qualitative data-composition effect is not confined to the asymmetric reward used in the main experiments.

Critical weaknesses
  1. The training comparison does not isolate the contribution of preserving question-level context pairs. In Section 4.2, MHSuff-SHInsuff and SHSuff-MHInsuff make contextual sufficiency directly correlated with the QA source. PairContext simultaneously removes this source–sufficiency association and exposes each question under both sufficient and insufficient contexts. The results in Table 4 therefore support the concern that source-correlated training can induce shortcuts, but do not establish that question-level pairing itself is responsible for the improvement. A source-balanced, non-paired baseline, with answering and abstention examples drawn from different questions within each source and matched training budgets, is needed to distinguish these explanations. This matters because the paper explicitly uses the comparison to argue for the importance of preserving context pairs during training.

  2. The interpretation of the Leakage metric is insufficiently justified, particularly for counterfactual examples. Equation (3) defines Leakage as the probability of producing the reference answer under insufficient context. This is a behavioral proxy rather than direct evidence that the model relied on memorized parametric knowledge. More specifically, under the stated definition, the reference answer for PFB-ConCF is the counterfactual answer. In Appendix C.6, the model answers “Spanish” after the supporting evidence for “Catalan” is removed, and the paper interprets this as internal knowledge leakage. However, if Equation (3) is applied using the counterfactual reference, this response is classified as W rather than C and does not contribute to Leakage. The operational metric would therefore miss the very type of fallback illustrated by this case study. The reference used for this metric needs clarification, and fallback to the original factual answer should be distinguished from reproduction of the counterfactual answer. This concern affects the conclusions about internal knowledge leakage, although it does not by itself invalidate the OverAns or SFAcc metrics.

Overall, I find the benchmark perspective useful, but the evidence for the specific benefit of paired training and the interpretation of internal knowledge leakage remains insufficient. These concerns lead me to recommend weak rejection in the current form.

Questions for authors
  1. Controlled comparison for the pairing claim: Do you have results for a non-paired training condition in which each QA source contains a balanced mixture of sufficient and insufficient examples, but the two conditions use different questions? With training example counts, source proportions, and optimization budgets matched to PairContext, does question-level pairing still provide a substantial improvement? This comparison would materially affect my assessment of the training contribution.

  2. Clarification of counterfactual Leakage: For PFB-ConCF, is the C label in Equation (3) computed against the counterfactual answer or the original factual answer? How is the “Spanish” response in Appendix C.6 counted in the reported Leakage results? Please clarify the implementation and, if available, report the rates of fallback to the original factual answer separately from reproduction of the counterfactual answer.

  3. Validation of response classification: Appendix A.5 uses Qwen3-30B-A3B-Instruct-2507 to classify responses as IDK, C, or W. Was this classification validated against human judgments, particularly for counterfactual answers and responses that combine uncertainty with a concrete answer? Agreement rates and an analysis of classification errors would help establish the reliability of the benchmark results.

11 comments

Dear Reviewer zLKQ,

Thank you for your valuable comments. In response to your questions, we will address them point by point, and our replies are as follows:

W1: The training comparison does not isolate the contribution of preserving question-level context pairs. In Section 4.2, MHSuff-SHInsuff and SHSuff-MHInsuff make contextual sufficiency directly correlated with the QA source. PairContext simultaneously removes this source–sufficiency association and exposes each question under both sufficient and insufficient contexts. The results in Table 4 therefore support the concern that source-correlated training can induce shortcuts, but do not establish that question-level pairing itself is responsible for the improvement. A source-balanced, non-paired baseline, with answering and abstention examples drawn from different questions within each source and matched training budgets, is needed to distinguish these explanations. This matters because the paper explicitly uses the comparison to argue for the importance of preserving context pairs during training.

A: We thank the reviewer for pointing out that the original experiments in Section 4.2 do not fully disentangle the effects of source–sufficiency correlation and question-level context pairing. To address this concern, we conducted additional GRPO experiments with a source-balanced, non-paired baseline ∗∗NonPairContext∗∗**NonPairContext**, allowing us to separately investigate these two factors.

We constructed NonPairContext by dividing the questions within each of the SingleHop and MultiHop datasets into two disjoint halves. One half was used with sufficient contexts for answering supervision, while the other half was used with insufficient contexts for abstention supervision. Thus, both QA sources contribute equally to answering and abstention supervision, while each question appears under only one contextual condition. This eliminates the source–sufficiency association without introducing question-level pairing, enabling a more controlled comparison with PairContext.

We evaluated NonPairContext on four models and compared it with both mismatched-source settings and PairContext across all four PFaithBench subsets. The complete results are presented below.

On PFB-MultiHop

ModelSettingDFAcc ↑SFAcc ↑AnsAcc ↑AbsAcc ↑OverAns ↓OverAbs ↓
Llama-3.1-8B-InstructMH+/SH-34.2027.4081.8036.6063.402.60
SH+/MH-2.700.000.0096.703.3095.60
NonPair66.6056.5076.5076.7023.3011.00
Pair71.5061.0076.4081.9018.1010.60
Llama-3.2-3B-InstructMH+/SH-33.8026.0073.0038.2061.806.30
SH+/MH-45.0010.9018.7079.6020.4037.00
NonPair62.1047.4066.8074.6025.4013.50
Pair68.2053.4069.4080.0020.0012.50
Qwen3-4B (NoThink)MH+/SH-56.1045.5074.2064.0036.008.70
SH+/MH-58.4047.3073.3067.1032.909.60
NonPair56.5045.3073.6065.4034.609.70
Pair68.8060.5074.1084.2015.8016.00
Qwen3-1.7B (NoThink)MH+/SH-24.1014.9061.1027.2072.804.20
SH+/MH-32.2013.9026.3075.6024.4046.70
NonPair41.0025.8054.4053.3046.7014.10
Pair61.5043.7058.5077.2022.8017.80

On PFB-SingleHop

ModelSettingDFAcc ↑SFAcc ↑AnsAcc ↑AbsAcc ↑OverAns ↓OverAbs ↓
Llama-3.1-8B-InstructMH+/SH-37.1431.5032.2898.561.4461.68
SH+/MH-16.0114.7091.6016.8083.201.05
NonPair76.2572.3185.1783.9916.018.66
Pair79.1375.3386.2286.3513.657.74
Llama-3.2-3B-InstructMH+/SH-26.9023.1024.9397.512.4970.87
SH+/MH-36.3533.0782.5539.9060.104.20
NonPair64.9658.7975.7278.7421.2614.04
Pair67.4562.8679.2779.4020.6012.47
Qwen3-4B (NoThink)MH+/SH-67.7256.3061.0293.446.5626.51
SH+/MH-69.4263.7886.4873.6226.385.25
NonPair73.7566.2780.3182.0217.988.92
Pair77.9574.1581.6390.299.7112.60
Qwen3-1.7B (NoThink)MH+/SH-43.5719.1621.2693.966.0451.18
SH+/MH-38.5831.5074.5442.6557.355.12
NonPair56.0441.3459.1973.8826.1219.69
Pair58.0150.5265.8879.5320.4723.62

On PFB-ConOri

ModelSettingDFAcc ↑SFAcc ↑AnsAcc ↑AbsAcc ↑OverAns ↓OverAbs ↓
Llama-3.1-8B-InstructMH+/SH-66.7561.4466.6294.065.9427.69
SH+/MH-15.8014.2991.4016.8183.191.14
NonPair78.3872.9585.3486.7313.278.34
Pair80.4075.9885.8489.0011.009.10
Llama-3.2-3B-InstructMH+/SH-65.3652.7257.7793.306.7028.45
SH+/MH-52.8449.1888.2457.1442.864.80
NonPair76.6170.9284.2085.2114.798.72
Pair78.0073.3285.7186.2213.788.22
Qwen3-4B (NoThink)MH+/SH-76.3665.9976.3686.7313.2711.38
SH+/MH-70.8065.8786.0977.7522.256.95
NonPair75.4768.0281.6785.0814.929.61
Pair80.6675.6081.2994.185.8213.65
Qwen3-1.7B (NoThink)MH+/SH-67.8944.7549.9491.408.6023.89
SH+/MH-50.8246.9084.2057.2742.737.21
NonPair69.6660.6877.3779.7720.2311.38
Pair68.3962.8379.3981.4218.5813.40

On PFB-ConCF

ModelSettingDFAcc ↑SFAcc ↑AnsAcc ↑AbsAcc ↑OverAns ↓OverAbs ↓
Llama-3.1-8B-InstructMH+/SH-42.9824.7825.9294.945.0653.86
SH+/MH-12.397.5952.7214.7985.213.79
NonPair67.6437.6742.1085.2114.7920.99
Pair70.1642.3547.0387.7412.2619.22
Llama-3.2-3B-InstructMH+/SH-48.1719.9720.7395.074.9349.30
SH+/MH-39.1923.1438.5653.3546.6521.74
NonPair60.3036.4142.1081.9218.0825.16
Pair63.4640.8346.4084.7015.3024.40
Qwen3-4B (NoThink)MH+/SH-63.0832.4935.9087.6112.3927.18
SH+/MH-63.9734.3941.0977.3722.6315.68
NonPair65.8736.5441.4782.0517.9518.84
Pair68.6539.7040.7194.565.4428.07
Qwen3-1.7B (NoThink)MH+/SH-61.4420.8622.6391.158.8530.47
SH+/MH-50.0627.6947.1656.0143.998.98
NonPair63.5930.9738.9477.6222.3815.93
Pair61.6934.2641.7280.6619.3420.99

The additional experiments provide two important findings that directly address the reviewer's concern.

First, source balancing substantially mitigates source-conditioned behavioral biases. Compared with mismatched-source training, NonPairContext improves average DFAcc and SFAcc and substantially reduces the imbalance between over-answering and over-abstention. This suggests that eliminating source–sufficiency correlations helps prevent models from relying on source-specific cues to determine whether to answer or abstain.

Second, preserving question-level context pairs provides additional benefits beyond source balancing. Despite using sufficient and insufficient examples from both QA sources, NonPairContext consistently underperforms PairContext in strict pairwise faithfulness. Across the four models and four evaluation subsets, PairContext improves SFAcc in all 16 comparisons, with an average improvement of 6.16 percentage points. DFAcc also improves in 14 of 16 comparisons, with an average gain of 4.33 percentage points. Furthermore, the mean absolute OverAns–OverAbs difference decreases from 9.85 under NonPairContext to 6.32 under PairContext. These results indicate that question-level pairing contributes additional improvements in both joint faithfulness and behavioral balance, beyond merely eliminating source–sufficiency correlations.

Overall, the new experiments distinguish two complementary factors underlying effective faithfulness training: source balancing mitigates source-conditioned biases, while preserving question-level context pairs further improves joint answering–abstaining behavior. We will incorporate these additional experiments and revise Section 4.2 to clarify the respective contributions of both factors.

Reviewer_dEb9OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper re-evaluates the contextual faithfulness of LLMs from a pairwise, context-sensitive perspective. Existing benchmarks evaluate answerable and unanswerable instances independently across different questions, thereby confounding context sufficiency with question difficulty.

Critical strengths
  1. Shifting from isolated instance-level faithfulness to controlled pairwise context evaluation effectively disentangles context-sensitivity from question difficulty.

  2. The benchmark evaluates 39 diverse open-source LLMs across sizes

  3. Findings from Section 4.2: when positive contexts are harvested from mismatched data sources (e.g., multi-hop for answering, single-hop for abstention), the model exploits superficial dataset-level artifacts instead of learning context sufficiency

Critical weaknesses
  1. While the paper empirically identifies that reasoning models over-answer more severely, it lacks qualitative or token-level analysis of the reasoning traces. Investigating where the reasoning derailment occurs (e.g., premise fabrication vs. ungrounded deductive steps in <think>) would elevate the paper's scientific depth.

  2. The data-centric experiments in Section 4 rely exclusively on GRPO. It remains unverified whether the observed shortcut-learning vulnerability and data-ratio trade-offs generalize to other prevalent alignment paradigms, such as SFT or Preference Optimization (DPO, ContextDPO, ...).

  3. What is the empirical agreement rate (e.g., Cohen's Kappa) between the LLM judge and human experts on a randomly sampled subset of model responses, especially on border cases containing hedging or disclaimers?

Questions for authors

See Weaknesses

9 comments

Dear Reviewer dEb9,

Thank you for your valuable comments. In response to your questions, we will address them point by point, and our replies are as follows:

W1: While the paper empirically identifies that reasoning models over-answer more severely, it lacks qualitative or token-level analysis of the reasoning traces...

A: We thank the reviewer for this insightful suggestion. To better understand why LLMs tend to over-answer under insufficient contexts, we conducted an additional analysis of model reasoning traces, focusing on where and how unsupported answering emerges during the reasoning process.

Specifically, we identified three representative error patterns:

  1. Unsupported Premise Introduction: The model introduces factual premises that are not supported by the provided context and subsequently uses them to derive an answer. These premises may originate from external knowledge, fabricated or misattributed contextual information, or speculative assumptions. For example, when asked about Zsa Zsa Gabor’s native language, the context only states that she was born in Budapest. The model introduces the unsupported premise that “Budapest is the capital of Hungary” and then answers “Hungarian.” The answer is therefore derived from an externally supplied geographical premise rather than from the provided evidence.

  2. Ungrounded Deductive Step: The model draws conclusions that are not logically supported by the available premises. For example, it may infer a person's native language solely from their birthplace or nationality, even though such information is insufficient to establish the answer. As an example of an ungrounded deductive step, the context associates Herrmann with composing the score of Taxi Driver but does not identify its performer. The model explicitly notes that “it does not explicitly name the performer,” yet reasons that “usually, the composer is the primary answer” and returns “Herrmann.” Here, the derailment is an invalid relation substitution—composer is treated as performer—rather than the introduction of a new factual entity.

  3. Recognized-Gap Override: The model explicitly recognizes that the provided context lacks sufficient evidence but nevertheless produces a definitive answer. This pattern can overlap with the previous two categories and highlights a failure to maintain appropriate abstention behavior despite recognizing uncertainty. For recognized-gap override, consider a question asking who led the third division alongside Genghis Khan. The model correctly observes that “there’s no mention of anyone else leading alongside him in the third division.” Nevertheless, instead of abstaining, it replaces the missing co-leader with the supported but differently scoped answer “Genghis Khan.” The failure therefore occurs after the evidence gap has already been recognized.

We further quantified the occurrence of these error patterns across seven models, including reasoning and non-reasoning models, obtaining the following results:

ModelUnsupported Premise IntroductionUngrounded Deductive StepRecognized-Gap Override
Llama-3.1-8B-Instruct50.0%50.0%0.0%
Qwen3-4B (Think)87.0%13.0%0.0%
Qwen3.5-9B (Think)91.0%0.0%9.0%
Qwen3.6-27B (Think)69.5%30.5%0.0%
Gemma-2-9B-It42.0%58.0%0.0%
Phi-492.0%8.0%0.0%
Olmo-3.1-32B-Instruct85.0%10.0%5.0%
Average73.8%24.2%2.0%

These results provide several important insights. First, unsupported premise introduction is the most frequently observed error pattern 738˙%73\.8\%, suggesting that models tend to supplement insufficient evidence with external knowledge or speculative assumptions. Second, ungrounded deductive steps 242˙%24\.2\% reveals another common failure mechanism, revealing that over-answering does not necessarily arise from introducing entirely fabricated facts, but can also emerge when models draw unsupported conclusions from seemingly relevant contextual information. More importantly, recognized-gap override occurs in 2.0% of the analyzed cases, indicating that some models explicitly recognize missing evidence but still proceed to generate an answer rather than abstaining.

These observations offer a more fine-grained explanation of over-answering beyond the aggregate behavioral metrics reported in our original submission. In particular, they suggest that improving faithfulness requires not only detecting missing evidence, but also preventing unsupported premise introduction, invalid inference, and the override of recognized uncertainty during answer generation.

We will incorporate this additional analysis, together with representative reasoning traces and the corresponding error taxonomy, into the revised manuscript to provide a deeper understanding of the mechanisms underlying over-answering.

W2: The data-centric experiments in Section 4 rely exclusively on GRPO. It remains unverified whether the observed shortcut-learning vulnerability and data-ratio trade-offs generalize to other prevalent alignment paradigms, such as SFT or Preference Optimization DPO, ContextDPO, \.\.\..

A: To demonstrate that our findings can generalize to other paradigms, we conducted additional experiments using two widely adopted alignment paradigms, Supervised Fine-Tuning SFTSFT and Direct Preference Optimization DPODPO, under the same settings Table 4 and Figure 4.

For both SFT and DPO in the setting of Table 4, we considered MHSuff-SHInsuff, SHSuff-MHInsuff, and PairContext to examine whether mismatched answering–abstention supervision produces similar behavioral biases under different training objectives. For DPO, we constructed preference pairs by using the correct answer as the preferred response and abstention IDKIDK as the rejected response under sufficient contexts, while reversing this preference under insufficient contexts. This enables DPO to learn context-sensitive answering and abstention behavior through preference supervision. We evaluated the resulting models across all four PFaithBench subsets. The results are as follows:

PFB-MultiHop

ModelMethodTraining settingDFAcc ↑SFAcc ↑AnsAcc ↑AbsAcc ↑OverAns ↓OverAbs ↓
Llama-3.1-8B-InstructSFTPairContext61.9044.0059.5077.4022.6016.70
MHSuff-SHInsuff10.806.3065.7012.3087.704.80
SHSuff-MHInsuff45.6025.8039.2072.3027.7031.50
DPOPairContext41.5029.2059.6055.0045.0017.40
MHSuff-SHInsuff40.0027.7060.4054.1045.9017.10
SHSuff-MHInsuff39.5026.6055.9057.2042.8022.10
Llama-3.2-3B-InstructSFTPairContext45.0025.4036.5075.6024.4033.70
MHSuff-SHInsuff10.605.4050.0015.9084.1012.60
SHSuff-MHInsuff11.904.806.4084.6015.4076.80
DPOPairContext28.6013.2038.2045.6054.4027.40
MHSuff-SHInsuff25.909.9037.5038.3061.7021.70
SHSuff-MHInsuff29.2013.8033.9055.2044.8034.80
Qwen3-4B (NoThink)SFTPairContext75.3056.3071.2080.0020.004.70
MHSuff-SHInsuff14.5010.3071.8016.1083.901.70
SHSuff-MHInsuff11.509.7013.5088.7011.3077.60
DPOPairContext43.7030.2052.6065.4034.6022.40
MHSuff-SHInsuff40.1027.2052.3062.0038.0023.00
SHSuff-MHInsuff40.4027.3050.3064.9035.1025.10

PFB-SingleHop

ModelMethodTraining settingDFAcc ↑SFAcc ↑AnsAcc ↑AbsAcc ↑OverAns ↓OverAbs ↓
Llama-3.1-8B-InstructSFTPairContext47.7743.4481.5056.6943.319.45
MHSuff-SHInsuff42.5230.7148.5666.8033.2027.56
SHSuff-MHInsuff26.9024.5485.4329.2770.733.28
DPOPairContext48.0336.2261.5563.6536.3517.45
MHSuff-SHInsuff44.8833.7357.2263.6536.3521.78
SHSuff-MHInsuff47.7735.4359.5862.8637.1417.98
Llama-3.2-3B-InstructSFTPairContext42.6534.6562.6060.8939.1120.73
MHSuff-SHInsuff34.3824.8050.9256.3043.7026.12
SHSuff-MHInsuff19.9515.4972.8326.5173.498.66
DPOPairContext33.0722.7054.9947.7752.2319.95
MHSuff-SHInsuff34.7824.5454.7249.3450.6619.03
SHSuff-MHInsuff28.2217.4550.9243.1856.8222.05
Qwen3-4B (NoThink)SFTPairContext69.9566.6783.4680.1819.8210.89
MHSuff-SHInsuff51.4448.0369.2974.4125.5923.62
SHSuff-MHInsuff48.8246.3359.9783.3316.6735.17
DPOPairContext44.7542.3972.9766.5433.4622.05
MHSuff-SHInsuff44.8842.2672.0566.9333.0722.57
SHSuff-MHInsuff43.8341.2173.3664.4435.5621.26

PFB-ConOri

ModelMethodTraining settingDFAcc ↑SFAcc ↑AnsAcc ↑AbsAcc ↑OverAns ↓OverAbs ↓
Llama-3.1-8B-InstructSFTPairContext55.8852.3491.1559.6740.333.92
MHSuff-SHInsuff46.9042.8685.9752.2147.796.45
SHSuff-MHInsuff38.4335.1587.6142.3557.654.93
DPOPairContext54.8749.8184.8361.1938.817.08
MHSuff-SHInsuff56.5150.5781.9263.0836.927.96
SHSuff-MHInsuff56.1348.8080.1564.2235.788.72
Llama-3.2-3B-InstructSFTPairContext49.9446.7889.0053.7346.274.30
MHSuff-SHInsuff40.2037.4287.1043.7456.264.68
SHSuff-MHInsuff29.3326.5588.2434.0165.995.44
DPOPairContext49.9443.9979.6557.0242.989.36
MHSuff-SHInsuff49.3044.3776.9959.8040.2013.02
SHSuff-MHInsuff47.2841.3476.7456.8943.1112.64
Qwen3-4B (NoThink)SFTPairContext72.1968.6591.2875.9824.023.92
MHSuff-SHInsuff54.2451.0791.5357.1442.862.91
SHSuff-MHInsuff70.5467.6481.6783.6916.3113.27
DPOPairContext59.8057.5287.6168.0231.988.22
MHSuff-SHInsuff58.4156.3887.1066.8833.128.60
SHSuff-MHInsuff57.2755.2588.1264.9835.027.96

PFB-ConCF

ModelMethodTraining settingDFAcc ↑SFAcc ↑AnsAcc ↑AbsAcc ↑OverAns ↓OverAbs ↓
Llama-3.1-8B-InstructSFTPairContext48.0428.7048.5558.4141.5913.78
MHSuff-SHInsuff29.5817.9533.1253.1046.9033.50
SHSuff-MHInsuff32.7416.6938.6942.1057.9015.30
DPOPairContext43.9919.3429.4658.1541.8521.62
MHSuff-SHInsuff41.4719.0928.8259.4240.5826.55
SHSuff-MHInsuff40.4617.7025.5459.0440.9628.07
Llama-3.2-3B-InstructSFTPairContext40.3327.3150.7052.4747.5316.06
MHSuff-SHInsuff30.5917.7036.4142.7357.2720.48
SHSuff-MHInsuff21.6213.1536.2832.8767.1321.37
DPOPairContext40.7119.0929.8454.7445.2620.73
MHSuff-SHInsuff41.7221.7431.1059.0440.9623.39
SHSuff-MHInsuff37.0417.1928.3255.8844.1226.68
Qwen3-4B (NoThink)SFTPairContext66.1249.6864.4875.9824.029.36
MHSuff-SHInsuff47.0336.6665.9954.3645.648.34
SHSuff-MHInsuff49.0540.4649.4379.1420.8632.36
DPOPairContext41.2129.7142.6064.3535.6529.46
MHSuff-SHInsuff41.5930.7242.9864.4835.5229.58
SHSuff-MHInsuff38.9428.0741.9760.6839.3228.95

Overall, our additional experiments demonstrate that the source-conditioned behavioral biases identified in Table 4 are not unique to GRPO, and that PairContext training provides consistent benefits under SFT and DPO at most cases as well. We will incorporate these results into the revised manuscript to strengthen the generality of our findings.

Reviewer_nsyAOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors construct a benchmark called PFaithBench to evaluate whether a model correctly abstains from answering a question based on supporting vs. non-supporting contexts and measure how this impacts faithfulness errors downstream. The benchmark is created from pairs where some fact is removed leaving the original question unanswerable. Most evaluated models fail by over-answering, and training that takes supporting examples from one dataset and insufficient ones from another affects the answering-abstention balance.

Critical strengths
  • Instances in PFaithBench are human-validated.
  • Examples and illustrations in the main figures are helpful.
  • The benchmark covers a wide range of models and configurations.
Critical weaknesses
  • (W1) FaithEval (Ming et al., 2025) is cited once and then listed in Table 6 as dataset-level, but its unanswerable task takes each question from ten contextual QA datasets including HotpotQA and SQuAD, rewrites the passage so it no longer supports the gold answer while leaving the rest of the text intact, has three crowd workers check both directions (evidence present, evidence absent), and evaluates the same questions under the original and stripped context. If you put this into contrast with Section 2's statement that prior work confounds question and context variation, the novelty falls apart.
  • (W2) The downstream effects of pairwise switching remain unanalyzed. No deployed setting is examined, e.g., a retrieval pipeline where a wrong refusal drops a good answer or an eager answer ships a fabrication, no long documents, nothing outside English short-span extractive QA under one template that demands a literal "I don't know". Because of that, the scope of the conclusions about real systems is unclear.
  • (W3) The three contribution bullets point in different directions (benchmark, error taxonomy, training recipe), Section 3's experiments never state what question they answer while §4.2/§4.3 each pose one, and findings are written as possibilities ("may induce data-specific shortcuts") where the abstract states them as identifications.
    • Moreover, the paper doesn't cleanly separate the goal of dealing with unanswerability from measuring and improving faithfulness.
    • Having a lot of different metrics like answering accuracy, refusal accuracy, DFAcc and SFAcc, OverAns, OverAbs, etc. doesn't help forming a sensible picture of the aims.
    • Ergo: less is more! Consider stripping the paper a little bit and doing one or two things properly.
  • (W4) The related work suspiciously omits foundational work by the authors of SQuAD, which is their follow-up "Know What You Don’t Know: Unanswerable Questions for SQuAD" (Rajpurkar et al., ACL 2018) where they investigate exactly the problem of introducing unanswerability. Other absent references include:
    • Entity-Based Knowledge Conflicts in Question Answering (Longpre et al., EMNLP 2021): Question-context pairs where the context clashes with what the model memorized, and showed models fall back on memorized answers. Related to counterfactual setups in ConFiQA.
    • The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Models (Slobodkin et al., EMNLP 2023): Shows models answer confidently when they should not, and that hidden states often contain the right answer anyway. Prior evidence for the paper's main over-answering finding.
    • Don’t Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration (Feng et al., ACL 2024)
Questions for authors

I'm not sure if any more detailed questions help at this point, because I'd rather recommend to address the four main weaknesses first in a resubmission and only then it allows us to talk about more specific ablations and extra experiments.

  • (Q1) Could you argue for why you chose these research questions and experiments in relation to FaithEval (Ming et al.) and Slobodkin et al. (2023)?
  • (Q2) Could you outline how you would address (W2) regarding adding some kind of deployed setting?
18 comments

Dear Reviewer nsyA,

Thank you for your valuable comments. In response to your questions, we will address them point by point, and our replies are as follows:

W1: FaithEval Mingetal,˙2025Ming et al\., 2025 is cited once and then listed in Table 6 as dataset-level, but its unanswerable task takes each question from ten contextual QA datasets including HotpotQA and SQuAD, rewrites the passage so it no longer supports the gold answer while leaving the rest of the text intact, has three crowd workers check both directions (evidence present, evidence absent), and evaluates the same questions under the original and stripped context. If you put this into contrast with Section 2's statement that prior work confounds question and context variation, the novelty falls apart.

A: We thank the reviewer for highlighting the relationship between FaithEval and our work. We acknowledge that FaithEval has made an outstanding contribution to the study of LLM Faithness. Nevertheless, our work makes several distinct contributions beyond the paired context construction used in FaithEval.

  1. We introduce an explicitly pairwise evaluation framework with dedicated faithfulness metrics. Although FaithEval constructs modified contexts from original answerable contexts, its evaluation primarily reports model performance under the two contextual conditions separately. In contrast, PFaithBench explicitly treats each question with its sufficient and insufficient contexts as a unified evaluation instance, enabling us to directly assess whether a model appropriately switches between answering and abstaining for the same question. Specifically, OverAns and OverAbs characterize two opposing failure modes, while Leakage provides an additional diagnostic signal for potentially unsupported reliance on internal knowledge. More importantly, **DFAcc and SFAcc **measure the joint correctness of answering–abstaining behaviors at the question level, capturing information that cannot be determined from marginal answering and abstention accuracies alone. This distinction enables a more comprehensive characterization of context-sensitive faithfulness.

  2. The construction of PFaithBench not only considers the balance of question types (covering diverse data sources), but also the diversity of reasoning and contextual conditions. Beyond incorporating diverse QA sources, PFaithBench explicitly distinguishes different reasoning and contextual conditions through four subsets: PFB-SingleHop, PFB-MultiHop, PFB-ConOri, and PFB-ConCF. This design enables us to systematically investigate faithfulness under single-hop reasoning, multi-hop reasoning, and original versus counterfactual contexts. Although FaithEval incorporates multiple QA datasets, including HotpotQA, it does not explicitly analyze pairwise faithfulness across these reasoning categories. Our benchmark therefore enables a more fine-grained characterization of context-sensitive faithfulness across different task settings.

  3. We provide new insights into faithfulness training. Beyond evaluation, we conduct systematic experiments to investigate how existing training approaches affect faithfulness behavior. Our findings demonstrate that mismatched answering and abstention data sources can introduce dataset-specific shortcuts, while the proportion of answering and abstention supervision substantially influences the trade-off between OverAns and OverAbs. These observations highlight the importance of context coupling and balanced data composition for future faithfulness training. Furthermore, our experiments show that faithfulness-trained models maintain comparable performance on a range of general capability benchmarks, suggesting that faithfulness can be improved without substantial degradation of general capabilities.

Overall, our primary contribution lies in systematically measuring joint answering–abstaining behavior, evaluating it across diverse reasoning conditions, and identifying data-centric factors that shape faithfulness training. We will revise the related work and benchmark comparison to make these distinctions explicit.

W2: The downstream effects of pairwise switching remain unanalyzed. No deployed setting is examined, e.g., a retrieval pipeline where a wrong refusal drops a good answer or an eager answer ships a fabrication, no long documents, nothing outside English short-span extractive QA under one template that demands a literal "I don't know". Because of that, the scope of the conclusions about real systems is unclear.

A: We thank the reviewer for suggesting additional analyses of downstream effects in real-world deployment settings. However, we respectfully disagree that the practical implications of our work are unclear. We believe our framework and empirical findings have direct relevance to deployed RAG systems in the following aspects.

  1. Pairwise faithfulness captures two fundamental failure modes in deployed systems. Our evaluation explicitly considers two evidence-availability conditions: successful evidence retrieval and missing supporting evidence. Taking RAG systems as an example, our work examines a fundamental question: Given different retrieved contexts, does the generator make the appropriate answering-or-abstaining decision? Specifically, OverAbs captures cases where sufficient evidence is available but the generator unnecessarily refuses to answer, potentially causing valid answers to be lost. Conversely, OverAns captures cases where supporting evidence is missing but the generator still produces an answer, risking unsupported information being delivered to users. These failure modes directly correspond to the practical risks identified by the reviewer. Importantly, by isolating the generator's answering–abstaining decision from upstream retrieval quality, our evaluation framework is not tied to a particular retrieval architecture and can be extended to diverse RAG systems, including long-context and multilingual settings.

  2. Our findings provide actionable insights for improving deployed systems. First, our context-coupling experiments (in Table 4) show that training with mismatched answering and abstention data sources can introduce source-specific shortcuts. For example, Llama-3.1-8B-Instruct trained with SHSuff-MHInsuff exhibits 95.60% OverAbs on multi-hop questions and 83.20% OverAns on single-hop questions. This finding is particularly relevant to enterprise knowledge assistants handling heterogeneous sources, such as technical documentation, HR policies, and financial reports, where confounding document sources with answerability labels could lead to systematic answering or refusal errors. Second, our data-composition analysis (in Figure 4) reveals that increasing answering supervision generally exacerbates over-answering, whereas increasing abstention supervision can cause excessive refusals. This provides guidance for developing evidence-grounded assistants in risk-sensitive domains, such as healthcare and legal information retrieval, where unsupported answers and unnecessary refusals have different operational costs. Our results highlight the importance of jointly considering these risks rather than optimizing either answering or abstention performance alone. Finally, our pairwise metrics (DFAcc and SFAcc) provide additional criteria for evaluating and selecting context-grounded generators. Unlike separate answering and abstention accuracies, these metrics assess whether a model can handle both evidence-available and evidence-missing conditions correctly for the same question, providing a more comprehensive diagnostic of generator reliability.

Overall, we believe that our conclusions can reflect the impact on the real deployed system. However, we also understand the reviewers' need for further clarification. In the revised version, we will include specific examples to explain how our work can reflect and optimize the problems existing in the deployed system. You can find some examples in our response to Question 2 in the following.

W3: The three contribution bullets point in different directions (benchmark, error taxonomy, training recipe), Section 3's experiments never state what question they answer while §4.2/§4.3 each pose one, and findings are written as possibilities "mayinducedata−specificshortcuts""may induce data-specific shortcuts" where the abstract states them as identifications. Moreover, the paper doesn't cleanly separate the goal of dealing with unanswerability from measuring and improving faithfulness. Having a lot of different metrics like answering accuracy, refusal accuracy, DFAcc and SFAcc, OverAns, OverAbs, etc. doesn't help forming a sensible picture of the aims. Ergo: less is more! Consider stripping the paper a little bit and doing one or two things properly.

A: We thank the reviewer for the constructive suggestions on improving the focus and organization of our paper. However, we would like to clarify that our benchmark, behavioral analysis, and training experiments are not separate research directions, but rather interconnected components of a unified objective: understanding and improving context-sensitive faithfulness in LLMs.

  1. Regarding the three contribution bullets. We would like to clarify that our three contributions are not separate research directions, but rather three interconnected stages of a unified investigation into context-sensitive faithfulness: evaluation, behavioral analysis, and training analysis. Specifically, we first introduce PFaithBench to systematically evaluate context-sensitive faithfulness, then identify and analyze the key failure modes of existing LLMs, and finally investigate how training-data construction and composition influence these behaviors. Together, these three stages form a coherent research framework for evaluating, understanding, and improving faithfulness, rather than three independent contributions.

  2. Regarding the different evaluation metrics. We would like to emphasize that the proposed metrics are not redundant, but capture complementary aspects of context-sensitive faithfulness and jointly support our research objective. Specifically, AnsAcc and AbsAcc measure the model's ability to answer correctly and abstain appropriately under different contextual conditions. OverAns and OverAbs characterize the two opposing failure modes, while Leakage further identifies cases where models produce correct answers despite insufficient contextual evidence, potentially indicating reliance on internal knowledge. Building upon these individual behavioral measurements, DFAcc and SFAcc assess whether models can appropriately switch between answering and abstaining for the same question, with SFAcc additionally requiring answer correctness. Together, these metrics enable us to systematically evaluate overall faithfulness, diagnose its failure modes, and understand how training affects different aspects of faithful behavior. Each metric thus serves a distinct analytical purpose and contributes to the unified research objective outlined above.

Therefore, we believe our contributions form a coherent and complementary investigation rather than a collection of disconnected studies. Nevertheless, we appreciate the reviewer's suggestion to better highlight the central research focus. In the revised manuscript, we will further streamline the presentation, clarify the connections among the three contributions, and refine the organization and wording of our findings to make the overall research narrative more focused and accessible.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

Dear AC and All Reviewers,

We would like to express our sincere gratitude to the reviewers for their constructive suggestions and valuable comments, which have helped us strengthen our manuscript. We are pleased that the reviewers recognized several strengths of our work, including:

  • A meaningful pairwise evaluation perspective: Reviewers dEb9 and zLKQ.
  • Joint evaluation of answering and abstention beyond aggregate accuracies: Reviewer zLKQ.
  • Broad empirical coverage across diverse models and contextual settings: Reviewers nsyA, dEb9, and zLKQ.
  • Useful findings on source-dependent behavioral biases and training data composition: Reviewers dEb9 and zLKQ.
  • Human-validated benchmark instances and helpful examples and illustrations: Reviewer nsyA.

More importantly, following the reviewers’ suggestions and questions, we have incorporated additional experiments, analyses, and clarifications into the revised manuscript. We summarize the main updates below:

  • Clarification of the relationship to prior faithfulness benchmarks: revised Section 2.
  • Refinement of wording in manuscript: revised Abstract and Introduction.
  • A source-balanced, non-paired training baseline to isolate the benefits of question-level pairing: added NonPairContext in Section 4.2 and Table 4.
  • Extension of the training analyses beyond GRPO to SFT and DPO: added SFT results in Table 5 and Figure 6, and DPO results in Appendix C.3, Table 10, and Figure 8, with detailed data-composition results in Tables 14-15.
  • Leakage analysis of counterfactual setting: added Appendix C.7 and Table 17, clarifying which behaviors are captured by the Leakage metric on PFB-ConCF.
  • Human validation of the LLM response judge: added Appendix C.8 and Table 18, reporting 92.00% agreement, a Cohen’s kappa of 0.861, and a multiclass MCC of 0.864.

All revisions are highlighted in purple, including the captions of updated tables and figures. We sincerely thank all reviewers and the Area Chair for their valuable time. Their suggestions have helped us improve the manuscript and clarify both the contributions and the scope of our findings.

Our work is motivated by a fundamental requirement for context-grounded QA: a model should answer correctly when supporting evidence is available and abstain when it is absent for the same question. Throughout the rebuttal period, we have actively engaged in the discussion, made detailed clarification, and conducted substantial additional experiments to address the reviewers’ concerns as thoroughly as possible.

We would be sincerely grateful if you could consider our contribution, the revised manuscript, and the additional evidence provided during the rebuttal period when making your final assessment.

Best regards,

Authors of 40849

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.