acceptodds
Under review as a conference paper at ICLR 2027

Trust at the margin: Provenance-visibility effects and model-dependent risk localization in LLM knowledge conflicts

Abstract

LLMs increasingly rely on external information that may conflict with their parametric knowledge. Recent work demonstrates that provenance can influence conflict resolution, however, three questions remain: how provenance visibility affects model decisions, where resulting risk is concentrated, and how this information can guide mitigation. In this paper, we address these questions in three steps. First, we introduce PROVMARGIN, a benchmark that varies provenance visibility while holding others fixed and measures model epistemic margins before intervention. Second, we characterize provenance effects and their risk localization. Across eight models, removing provenance cues indicating low credibility increases false answer adoption by 9.95%, with a larger effect near decision boundary. Extensions across model families, response measures, and tasks show that overall effect generalizes more consistently than its boundary localization. Real-world evaluations further reveal heterogeneous effects for erroneous claims, while official provenance consistently supports correct updates. Finally, we translate these findings into PROVROUTE, which combines provenance, epistemic margins, and observable responses to prioritize verification. At a budget of 1/6 of requests, PROVROUTE reduces residual false-adoption risk to 27.56%, improving over the strongest static baseline by 2.07 %. Together, these results connect causal measurement, model dependent risk localization, and selective verification, while emphasizing the need to reduce errors without hindering useful knowledge updates. and task requirements further moderate provenance sensitivity. A development only study of 30 real records finds heterogeneous effects for erroneous sources, while official provenance improves adoption of correct updates. We then propose PROVROUTE, which combines model-specific margins, source risk, and observable response signals to prioritize verification. On a separate outcome-unseen panel, reviewing one sixth of requests yields 27.56% residual false-adoption risk under an ideal-verifier assumption, improving on the strongest static margin–provenance baseline by 2.07 percentage points at 83.33% automatic coverage. These findings support preserving provenance and calibrating verification to the target model while retaining the utility of reliable external evidence.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Knowledge conflicts, provenance visibility, risk localization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_SJ1DOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how provenance cues affect LLMs when external claims conflict with their parametric knowledge. It introduces PROVMARGIN, a Wikidata-based benchmark that fixes candidates and closed-book margins before comparing the same false claim with low-credibility provenance shown or removed. On four core models, removing low-credibility cues increases false adoption by 9.95pp, with a larger effect near the decision boundary, but this localization does not hold consistently across four extension models. The paper also evaluates 30 real-world records and proposes PROVROUTE, which lowers residual false-adoption risk to 27.56% at a 1/6 verification budget.

Critical strengths

The experimental protocol is careful: candidates, margins, and data splits are fixed before any outcome is observed, and confirmatory tests follow a prespecified sequence on a locked test set. The overall effect holds across all eight models and under parser-free label probabilities. The paper transparently reports mixed and negative results, such as the OLMo-2 reversal. It also evaluates both safety (rejecting erroneous claims) and utility (adopting correct updates).

Critical weaknesses
  1. The controlled study measures the removal of low-credibility cues, not provenance visibility in general. The paper describes the two controlled conditions as follows (§2.1, §3.3):
  • Visible: the cues indicate low credibility, limited authority, or out-of-scope expertise.
  • Absent: the claim appears “as an imported memory,” with origin, scope, trace, and authority unspecified.

Because the visible condition takes only the low-credibility value, Δprov estimates the effect of removing negative cues, which is how the abstract phrases it. It does not show what happens when provenance is neutral or credible. The title and RQ1, however, frame the question as provenance visibility in general. The authors should either narrow this framing or add neutral and credible-source conditions on the same claims. In addition, since the absent condition is described as an imported memory, it would help to clarify whether it serves as a no-provenance baseline.

  1. Missing discussion of and comparison with closely related prior work. PAVE [1] also stratifies conflict behavior by the model’s pre-evidence epistemic state, reports strongly model-dependent arbitration, and proposes a test-time mitigation. ClashEval [2], Adaptive Chameleon or Stubborn Sloth [3], and Context versus Prior Knowledge in Language Models [4] already relate a model’s prior confidence to its adoption of conflicting context. In addition, the task-instruction study in §5 closely resembles Task Matters [5]. The authors should position their contributions against these works, clarify what the provenance manipulation adds beyond prior-state stratification, and include the stability signal of [1] as a baseline for PROVROUTE.

  2. Model-level evidence is weak. On the locked test set, only Qwen3-4B has a near–far interval that excludes zero (Table 3). In the extension models, OLMo-2 shows the reverse pattern (Table 4), and the authors themselves conclude that localization is model-specific (§4.2). The statement in Contribution 2 that the near-boundary effect is “consistently larger” therefore holds only at the level of point estimates.

  3. The PROVROUTE evaluation does not fully support its claimed advantage. The baselines use less information. PROVROUTE uses the model’s actual response, namely whether it adopted the external claim (Eq. 7), whereas all baselines use only information available before the response (Appendix E.2). The improvement therefore shows that looking at the response helps, but not that PROVROUTE beats simpler response-based rules, such as “verify whenever the model adopts the claim.” The gain is small. PROVROUTE improves on the strongest baseline by only 2.07pp, without a reported confidence interval, and the gain is significant for only one of the four additional models (Table 9).

  4. The real-world study seems too small to support the conclusions drawn from it.

The paper contains several typos, e.g., “can ne found” (line 502) and “Figure 3 a compares” (line 314).

[1] Diagnosing LLM Arbitration Behavior over Pre-evidence Epistemic States in RAG-based Fact-Checking. arXiv:2606.01120, 2026.

[2] ClashEval: Quantifying the Tug-of-War between an LLM’s Internal Prior and External Evidence. In Advances in Neural Information Processing Systems (NeurIPS), 2024.

[3] Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts. In International Conference on Learning Representations (ICLR), 2024.

[4] Context versus Prior Knowledge in Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024.

[5] Task Matters: Knowledge Requirements Shape LLM Responses to Context–Memory Conflict. arXiv:2506.06485, 2025.

[6] Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation. In International Conference on Learning Representations (ICLR), 2023.

[7] Language Models (Mostly) Know What They Know. arXiv:2207.05221, 2022.

Questions for authors

How does PROVROUTE compare with a simple rule that verifies every response in which the model adopts the external claim?

Could Table 8 be reported separately for erroneous records and for correct updates? How much of the +24.77pp gain on correct updates comes from reduced abstention?

Reviewer_dWMKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether an LLM becomes more likely to adopt a false external claim when the provenance attached to that claim is hidden. The authors pair Wikidata facts with plausible false alternatives and record how strongly each model prefers the true answer before it sees any context. They fix the development and locked test sets before collecting any responses with provenance. Each item then presents the same false claim in two versions. In one version the claim comes from an unverified community post whose declared expertise does not cover the topic and whose topical authority is low. In the other, the same claim appears as an imported memory whose origin and authority are left unspecified.

On the locked test set, hiding the cues raises false adoption by about ten percentage points across four models, and the increase is larger for items on which the model was close to undecided. Four further models confirm the overall effect, while the pattern near the decision boundary varies between models and reverses for one of them. With real sources, provenance has no consistent effect on erroneous claims and helps the models accept correct official updates. The paper ends with ProvRoute, a rule for choosing which requests to send for verification. When one sixth of requests go to a verifier assumed to be perfect, ProvRoute lowers residual false adoption by about two percentage points compared with the strongest static baseline.

Critical strengths

The main strength is a careful design that is fixed in advance. The candidate pool, the closed book margins, the test split, and the strata are all set before any response with provenance is collected. The claim text and answer order stay the same across conditions, and abstentions remain in the denominator. I recomputed the main aggregates and p values from the printed numbers, and they agree with the tables and appendices. With this design and these checks, I find the central estimate of about ten percentage points credible, and other groups could reuse the protocol for related questions.

The paper is honest about results that cut against its own story. It reports that the boundary pattern does not hold for every model, that real provenance does not reliably protect against erroneous claims, that ProvRoute helps some models and not others, and that its second probe does not pay for itself. It further shows that provenance still moves the models when they are told to ignore the context, which matters for memory and retrieval systems that lose provenance along the way.

Critical weaknesses

I lean toward rejection because the paper claims more than its controlled results show and because the evaluation of ProvRoute does not isolate the value the method adds.

The controlled experiment shows what happens when one specific set of warning cues is removed. The visible version tells the model that the source is an unverified post outside its expertise with low authority, while the paper says that the hidden version presents the same claim as an imported memory. If that wrapper appears only in the hidden version, the warning and the framing of the record change together, and the experiment cannot say which of the two drives the effect. The discussion in the appendix concedes that the cues are tested only as a package, yet the title and the first research question still describe the result as a general effect of provenance visibility. A model that discounts a claim after being told the source is unreliable is behaving as expected, and earlier work such as Khan et al. 2026 and Schuster et al. 2026 already shows that source attributions shift adoption. When the paper turns to real metadata, the effect on erroneous records is no longer clear.

The boundary result needs the same care. Pooled across the four main models, the effect is larger near the decision boundary, yet for three of these models the individual interval includes zero. Only one of the four extension models has intervals above zero on both test sets, and another reverses the pattern. The pooled contrast holds in percentage points, but what explains it is less clear. A provenance effect that is constant on the log odds scale changes adoption most where adoption is close to one half. The same shift that raises a rate of one half by about twelve points raises a rate of one tenth by less than half as much. Because the strata are built from the absolute margin, the far stratum can mix items where the model firmly prefers the true answer with items where it firmly prefers the false one. Differences in baseline adoption could then produce the pattern on their own. The paper does not report the adoption rates in each stratum or the signed margin contrasts that the appendix says were computed, so I cannot tell whether the margin adds anything beyond the baseline rate. The abstract also states the boundary result for eight models, although the reported number comes from four of them and the pattern does not hold across all eight.

My larger concern is the evaluation of ProvRoute. In the confirmatory test every external claim is false, and both conditions count as weak or missing provenance. Under these conditions the routing score reduces to ten times the probe adoption rate plus small terms that can never outweigh it. Requests are therefore ordered first by how often the model picked the external answer on a reworded probe, and the other features only order requests that share the same probe rate. Rotating the prompt variants keeps the probe responses apart from the evaluation responses, yet the probe is still a second measurement of the same behavior, because a false adoption in this test means picking the external answer. None of the baselines uses the probe responses, and the paper does not compare against a rule that ranks by the probe alone. Without that comparison, the gain of about two percentage points cannot be credited to combining provenance, margin, and response signals. I could not find a confidence interval for this main comparison, even though the appendix says a bootstrap procedure and a threshold of two percentage points were fixed in advance.

The routing features never need to know which answer is correct. They do assume a vetted pair of candidates that always includes the correct answer, and the paper does not test how such pairs would be built in a real deployment. The part of the score meant to protect correct updates never comes into play in the confirmatory test. It is checked only on twelve development records, where the share of correct updates accepted without review stays close to that of the strongest static baseline. That is consistent with preservation, but so small a check cannot confirm the claim that the method reduces false adoption while preserving useful updates.

If the authors can report adoption rates by stratum and compare ProvRoute with a rule that uses the probe alone, with an interval for the main comparison, I would be glad to reconsider my score.

Questions for authors

Could you report, for each model and stratum, how often the model chose the false answer, the gold answer, and the abstain option with and without the cues? Please include both the strata based on the absolute margin and the signed margin tertiles mentioned in the appendix. A logistic mixed model with an interaction between condition and absolute margin, adjusted for the signed margin, would help show whether differences in baseline adoption explain the pattern.

Could you compare ProvRoute with a rule that ranks by the probe adoption rate alone and with one that adds only the margin, with the same probes and weights and the same handling of ties? Please add a bootstrap interval for the main comparison. For future work, I would be interested in how the pair of candidates could be built in a real deployment, where no vetted pair exists.

Could you share the full wording of the provenance conditions and of the three prompt variants, and say whether the imported memory wrapper appears in both conditions? Will the benchmark and the code be released?

Where do the eighteen erroneous and twelve update records come from, and how was their correctness checked? What was the competing option for the updates? The AI use statement says that synthetic data sets were generated with AI, so it would help to know which parts of the benchmark this covers. Please explain how the independent set and the new routing panel relate to the original pool of facts.

Reviewer_ZKx8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces PROVMARGIN, a benchmark for studying how the provenance of external information influences knowledge conflict resolution in LLMs. Experiments across multiple LLMs show that removing low-credibility provenance cues increases the adoption of incorrect external information, with stronger effects near decision boundaries. A selective verification strategy, PROVROUTE, is then proposed to prioritize high-risk requests for verification.

Critical strengths
  • The experiments are comprehensive, covering multiple model families, evaluation metrics, and task settings. The findings are insightful.
  • The proposed selective verification strategy, which incorporates both provenance information and model responses, is well-motivated and offers a practical approach to mitigating risks.
Critical weaknesses
  • The generalizability of the findings is somewhat limited. For example, Table 4 shows that the stronger provenance effects near decision boundaries are not consistently observed across models, with OLMo-2 exhibiting the opposite trend.
  • The benchmark construction could be better justified. PROVMARGIN primarily consists of relatively simple factual questions derived from Wikidata, and the test sets are relatively small (e.g., the locked test set contains only 60 instances). It is unclear how representative these instances are of real-world knowledge conflicts. A broader evaluation with more diverse and challenging instances would be helpful.
Questions for authors

The visible condition explicitly indicates that the external information comes from an unverified source with low authority, while the absent condition presents it as an imported memory with unspecified provenance. How to ensure that the observed effects are driven by provenance visibility rather than the explicit low-credibility cues or differences in framing?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.