acceptodds
Withdrawn from ICLR 2027

When Long Contexts Break Faithfulness: Evaluating Medical Hallucination under Adversarial Evidence

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Real-world medical dialogues require LLMs to reason over multiple sources, from retrieved documents and conversation history to family records, test reports, and personal health data. Existing medical hallucination benchmarks mainly target factual errors, but overlook faithfulness errors: misattributing external, historical, or non-user information to the current user, potentially leading to misleading user-specific medical advice. We introduce MLoHa, a long-context medical faithfulness benchmark with 2,700 items covering three objective tasks (choice, judgment, and best_answer), a fill-in task, and a five-dimensional Rubric QA task. Stratified into four length tiers (LA–LD), MLoHa evaluates 14 large language models under realistic multi-source medical contexts. Across all task formats, faithfulness degrades monotonically with context length: choice accuracy drops by 22.08 percentage points from LA to LD, and the five-task average drops by 13.22 points. An objective-only multi-factor ANOVA further shows that context length is the strongest explanatory factor (partial , ), outweighing medical scenarios and hallucination triggers. The only significant interaction is length medical scenario (), indicating that long contexts reshape how different medical risks surface, while ranking differences across tasks show that user-grounded faithfulness is not captured by a single evaluation format. We hope MLoHa can provide a reusable benchmark and analysis framework for the community to evaluate user-grounded medical faithfulness in long-context, multi-source settings.

Primary area:
datasets and benchmarks
Keywords:
medical dialogue, long-context evaluation, faithfulness hallucination, source attribution, multi-source interference, length degradation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_SGzKOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper want to evaluate long context behavior on medical text across tasks and different interventions (triggers). One part of the paper is the construction of a dataset from a private medical app including the task definition and evaluation schema with an LLM-as-a-judge. Based on teh dataset several LLMs are evaluated and a statistical analysis is performed to isolate relevant components of the performance.

Critical strengths
  1. Long context evaluation in medical / clinical settings is very important to assess the usefulness of LLMs for many relevant tasks. Characterizing the performance of different LLMs and common failure cases is very important.
  2. Broad evaluation of LLMs, tasks, triggers and context incuding statistical analysis.
Critical weaknesses
  1. Unclear construction of the dataset. This paper has high relevance, but its main contribution should be the clear and transparent construction of the dataset and tasks. All result depend on this. However, I have severe issues to understand each step that was taken for dataset construction. Also, to assess the relevance of the downstream results it is very important to know what data serves as input. I have a clinical background and struggle to understand what data was used. It seems the data is constructed accross several (bad) cases of a medical app, which was then normalized, rewritten and enriched with other sources and triggers.
  2. Limited generlizability without any human review of the source data. As far as I understand it there was only a human review of the LLM-as-a-judge component. However, when the central piece of the paper is a contribution in the space of ML for healthcare and there is no human validity of the source data or tasks the generalizability to real-world and relevant settings is questionable.
Questions for authors

Clarifications:

  1. Can you give us a clean flow chart of the data? What was taken from the app, what were all steps and numbers until you arrived at you final benchmark.
  2. Please carefully describe the context and setting of the medical data from the app? This is very relevant to assess the usefulness of this benchmark.
  3. What are online bad cases from the medical app?
  4. Are all ground truth labels LLM generated?
  5. What exactly did human reviewers see and label during the annotation process?
  6. Was there any human review of the source data, task definition, and labels?
  7. What is the population this data was drawn from, what is the scraped data about?

Suggestions (future work):

  1. Add targeted human review at critical places to ensure data quality and generalizability.
  2. Common reporting standard like flow charts and data descriptions should be used for the dataset reporting.
  3. You should widen your discussion of related work regarding medical text tasks and datasets.
Reviewer_j56COpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces MLoHa, a benchmark for evaluating user-grounded faithfulness in long-context medical dialogue. Unlike conventional hallucination benchmarks that primarily focus on factual correctness, MLoHa targets errors in which models misattribute information from retrieved documents, historical conversations, family members, or fabricated records to the current user.

The benchmark contains 2,700 items across five evaluation formats and four context-length tiers, and evaluates 14 language models. The authors report monotonic performance degradation as context length increases, with a 22.08 percentage-point drop in multiple-choice QA accuracy and a 13.22-point decrease in the average score across five tasks. They also analyze medical scenarios, hallucination triggers, model-ranking differences across evaluation formats, and the reliability of their evaluation procedures.

The paper addresses an important problem and provides a substantial evaluation effort. However, I have concerns about the extent to which the experimental design isolates the effect of context length, the ecological validity of the synthetic benchmark, and the reliability and interpretation of some evaluation results. These issues limit the strength of the paper's broader conclusions.

Critical strengths

Important and clinically relevant problem.

Distinguishing information about the current user from information about family members, retrieved documents, and historical conversations is essential for safe medical dialogue. The paper appropriately emphasizes that a medically correct fact can still lead to unsafe advice when attributed to the wrong person or time.

A useful evaluation perspective.

The focus on source attribution and user-grounded faithfulness complements conventional factuality-oriented medical hallucination benchmarks. The distinction between factual correctness and whether evidence actually belongs to the current user is conceptually valuable.

Broad evaluation coverage.

The benchmark includes 2,700 items, five task formats, four context-length tiers, and 14 models. The authors also report task-specific performance, robustness checks, qualitative examples, and analyses of differences across medical scenarios and hallucination triggers.

Recognition of important limitations.

The paper acknowledges that LLM-generated interference may differ from real clinical confusion patterns, that using Claude as a generator, judge, and evaluated model may introduce bias, and that context length is confounded with retrieval volume. These are important limitations to discuss when interpreting the findings.

Critical weaknesses
  1. The effect of context length is not sufficiently isolated from increasing context complexity.

This is my primary concern. Across the four length tiers, the benchmark progressively activates additional interference sources and increases the RAG recall cap, from 0–20 in LA to approximately 150 in LD. Consequently, context length, the number of competing information sources, and the volume of retrieved material change together.

The observed degradation therefore cannot be attributed specifically to context length. It may instead reflect some combination of longer inputs, greater evidence density, additional opportunities for source conflict, and exposure to more fabricated or misleading information. Although the paper explicitly frames its design as measuring compound complexity rather than raw token count alone, several central claims emphasize length as the dominant explanatory factor.

The paper would be substantially stronger with controlled comparisons that vary token length while holding the underlying evidence and source configuration fixed, as well as comparisons that vary source count or retrieval volume at approximately fixed token length. Matched or minimally modified versions of the same underlying cases would also help control for differences in item difficulty. Without such experiments, the findings support degradation across increasingly complex context tiers, but a stronger causal interpretation of the independent effect of context length is not fully established.

  1. The ecological validity of the synthetic adversarial contexts remains uncertain.

The benchmark is motivated by real failure cases from third-party medical applications, which is a useful starting point. However, the interference materials, including fabricated profiles, reports, historical dialogue, memory snippets, and retrieved content, are largely generated synthetically by language models.

This construction enables controlled evaluation, but it also raises questions about whether the benchmark reflects the distribution and difficulty of source-attribution failures in actual medical interactions. The paper does not sufficiently establish how frequently these specific failure modes occur in real-world deployments or whether clinicians consider the constructed scenarios representative of realistic workflows.

The authors acknowledge the need for blinded clinical validation. I consider this more than a minor limitation because the paper makes clinically motivated claims about the significance of the observed failures. Independent clinician review of scenario realism, source ambiguity, and the plausibility of the injected errors would substantially strengthen the benchmark. Where feasible, validation on naturally occurring, appropriately de-identified cases would provide additional evidence of external validity.

  1. The evaluation protocol introduces potential scoring and model-family biases.

The use of five task formats is a strength, but the scoring procedures have different limitations. Exact-match evaluation may penalize semantically equivalent responses, while LLM-as-judge evaluation introduces dependence on the judge's interpretation of the rubric.

In particular, Claude is used for content generation and as the judge for the generative tasks, and Claude Opus 5 is also among the evaluated models. The authors acknowledge this potential homology bias, but the reported human–machine agreement analysis does not fully establish that the resulting scores are unbiased across model families.

The reported agreement is approximately 78–81% for the three-level faithfulness dimensions, while the much higher agreement on the binary advice and safety dimensions is partly attributable to their simpler scoring criteria. These results are useful, but stronger validation would include independent judges from different model families, blinded human assessment on a larger and more representative sample, and sensitivity analyses showing whether model rankings and the principal conclusions remain stable across judges and scoring schemes.

The paper should also clarify how exact-match scoring handles semantically correct alternatives and how disagreements between automated and human judgments affect aggregate results.

  1. The statistical analysis requires clearer treatment of item dependence and uncertainty.

The paper provides extensive statistical analyses, including bootstrap confidence intervals, ANOVA, and logistic regression. The repeated observation of a degradation trend across tasks and models is encouraging. Nevertheless, the statistical interpretation could be more rigorous.

The reported analyses involve multiple models evaluated on a shared set of items, and the benchmark's construction may introduce dependencies among cases. The paper should clearly explain how item-level dependence and repeated model observations are handled when estimating uncertainty. In particular, resampling model–item observations independently may not adequately reflect uncertainty associated with the sampled items or the evaluated model set. A cluster-aware analysis that preserves the relevant dependencies would provide a stronger basis for the reported confidence intervals.

There is also an apparent reporting inconsistency in Appendix A.9: the FCC degradation estimate is reported as −10.40 percentage points, whereas its listed confidence interval is [−15.7, −11.2], which does not contain the point estimate. This should be corrected or clarified.

Finally, the paper should distinguish more carefully between statistical significance and practical explanatory importance. A small p-value establishes evidence against a specified null hypothesis; it does not, by itself, establish that context length is the principal causal mechanism behind the observed failures.

  1. The interpretation of the proposed failure mechanisms is stronger than the direct evidence.

The discussion attributes the observed failures to attention dilution, instruction-following degradation, and competition between sources. The qualitative examples are useful illustrations of these possibilities, but they do not directly establish the underlying mechanisms.

For example, failures in very long contexts are consistent with attention dilution, but they are also consistent with the increased number of misleading sources and the greater complexity of the decision. Similarly, a model accepting a fabricated premise does not necessarily demonstrate general instruction-following degradation without a controlled comparison that isolates the relevant instructions and context components.

The authors appropriately describe these explanations as hypotheses in parts of the discussion. I recommend maintaining this distinction throughout the paper and avoiding causal language unless supported by targeted experiments. Controlled source-removal, source-tagging, and matched-length interventions could help distinguish these mechanisms.

Questions for authors

I wrote all the questions in the weakness part.

Reviewer_ZqTkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces MLoHa, a 2,700-item benchmark for testing whether medical LLMs remain faithful to the correct user/source in long, multi-source contexts. Across 14 models and five task formats, performance consistently worsens as contexts become longer; the average drop from the shortest to longest tier is 13.22 points, with MCQA dropping 22.08 points

Critical strengths
  1. The paper distinguishes factual hallucination from source/provenance hallucination, which is especially consequential in personalized medical settings.
  2. It covers five task formats, four context-length tiers up to 340K tokens, and 14 models rather than relying on one evaluation setting.
  3. The authors include bootstrap tests, ANOVA/logistic regression, repeated runs, human–machine agreement, and qualitative failure cases rather than reporting only aggregate accuracy.
Critical weaknesses
  1. Longer tiers contain more RAG recalls and more activated interference sources, so the experiments do not cleanly establish that token length itself causes the degradation. The authors acknowledge this limitation.
  2. Claude is involved in benchmark generation and automatic judging, while a Claude model is also the strongest evaluated system. Human agreement helps, but R1–R3 agreement is only about 78–81%, leaving room for systematic judge effects.
  3. Responses from multiple models on the same benchmark items are not fully independent, while the primary ANOVA does not explicitly model item/model random effects. The appendix contains a smaller mixed-effects analysis, but a full hierarchical analysis would make the central statistical claim more convincing.
Questions for authors

q1. Can you disentangle context length from interference complexity? q2. Can you reproduce the generative-task results using an independent judge family and/or substantially more human evaluation? q3. Can you rerun the main significance analysis using a hierarchical model or cluster-robust inference over items/seeds and models?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.