acceptodds
Under review as a conference paper at ICLR 2027

Where Medical Responses First Go Wrong: Logic-Threaded Semantic Tree for Hallucination Detection and Correction

Abstract

Hallucinations in clinical large vision-language model responses arise when claims are unsupported by medical images, patient context, or medical knowledge, and may further cascade toxically when subsequent claims depend on earlier erroneous statements. However, existing methods largely verify clinical claims independently, overlooking both inter-claim dependencies and the multimodal contextual evidence needed to assess each claim. To address these limitations, we propose MedFH, a unified framework for First-Hallucination-aware detection and correction. Specifically, MedFH explicitly models claim dependencies through logic threads over a unified semantic tree, which serves as a structural backbone for query and response sentences and claims. Building on this representation, MedFH subsequently verifies each claim against two complementary sources of evidence: grounding evidence obtained from external tools and predecessor evidence selected through logic threads. Based on the verification results, MedFH locates the first hallucinated claim and uses its direct logical successors as cascade proposals to guide revision from its source sentence onward. We also introduce MedLHallu-Bench, a multimodal benchmark for medical hallucination detection, first hallucination localization, and correction in radiology report generation and diagnostic reasoning. Experiments on MedLHallu-Bench show that MedFH consistently outperforms state-of-the-art baselines, and external evaluation further supports its effectiveness in correcting hallucinations in naturally generated responses. Code and datasets will be released.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Clinical Large Vision-language Models, Hallucination Detection, Hallucination Correction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_NH5zOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MedFH for detecting and correcting hallucinations in medical responses. The authors decompose responses into claims and link each claim to relevant earlier claims. They verify claims using medical images, retrieved knowledge, and those links. The first detected error determines where correction begins. Evaluation uses 1,209 constructed cases across radiology reporting and diagnostic reasoning. The paper also tests correction on naturally generated responses from MedHEval.

Critical strengths
  • It records sentence errors, the first error, and affected later claims. This permits a more precise evaluation of correction than a single response score.

  • In diagnostic reasoning, removing logic threads lowers first error localization accuracy from 64.35 to 57.22. On MedHEval, correction lowers the reported visual hallucination measure from 15.44 to 9.84 while increasing key finding recall from 2.50 to 37.08. The joint change makes the result more persuasive than a reduction in hallucinations alone.

Critical weaknesses
  • Every constructed response receives two injected errors. Later dependent statements are edited as part of the same construction process. This design closely matches the proposed localization and correction strategy. It contains no clean responses for measuring unnecessary corrections. The 200 case benchmark audit uses language models, without clinician adjudication. The human assessment covers 100 corrected MedFH outputs, but does not validate the benchmark error positions or compare baseline corrections. MedHEval contains natural responses, yet provides no reference first error positions or dependency links. Thus, the distinctive localization and cascade claims remain untested on natural errors.

  • In radiology reporting, removing logic threads changes localization accuracy from 70.17 to 70.48, a slight increase. The benefit is clearer in diagnostic reasoning. MedFH also uses specialized medical grounding and verification tools and averages 69.31 model calls per response. Comparable resource use is not reported for the correction baselines. The large correction gains therefore cannot be assigned specifically to the claim links. The related work omits TIGER [1], which uses claim graphs and evidence guided local repair, and Dialectic-Med [2], which addresses diagnostic hallucinations through visual evidence and graph based reasoning. Their settings differ, but both require a clearer account of the proposed novelty.

[1] Zhao, K., Yu, T., Huang, S., Jenkins, P., Dong, Y., and Hughes, A. “Self-Correction Can Amplify Hallucinations: Fact-Level Repair with Graph-Based Evidence Routing in Multimodal Generation.” arXiv:2606.00232, 2026.

[2] Lu, Z., and Su, J. “Dialectic-Med: Mitigating Diagnostic Hallucinations via Counterfactual Adversarial Multi-Agent Debate.” Findings of the Association for Computational Linguistics: ACL 2026, pp. 36877–36894, 2026.

Questions for authors
  • On clinician annotated, naturally generated responses that include correct answers, what are the first error localization accuracy and the rate of unnecessary correction?
  • How much of the correction gain remains when a comparator receives the same medical evidence tools, correction model, and model call budget? Can the authors report this separately for radiology reporting and diagnostic reasoning?
4 comments
AuthorsOpenReview ↗

Response 1

We thank the reviewer for the careful reading and valuable suggestions on natural-error evaluation, controlled comparisons, and related work.

W1. Physician assessment of benchmark annotations and correction quality. We randomly sampled 100 benchmark cases and asked a physician to review their sentence-level hallucination labels, cascade annotations, and corresponding corrections from MedFH and Woodpecker. The physician agreed with all audited labels and cascade annotations in this sample. The table below reports physician-rated correction quality on the same cases, together with agreement between physician and automated assessments.

Table 1. Physician-rated correction quality and agreement with automated evaluation.

MethodPhysician-rated FCA ↑\boldsymbol{\uparrow}Physician-rated Sc\boldsymbol{S_c} ↓\boldsymbol{\downarrow}FCA agreementSc\boldsymbol{S_c} linear-weighted κ\boldsymbol{\kappa}Sc\boldsymbol{S_c} MAE
MedFH39%3.3376%0.7640.415
Woodpecker22%4.3598%0.7910.369
Pooled agreement——87%0.7860.392

W1 / Q1. Evaluation on naturally generated responses. We evaluated 100 responses generated by GPT-4.1-mini, comprising 50 radiology reports (RG) and 50 diagnostic reasoning responses (Dx). GPT-6-Astra produced the annotations, which a physician checked and revised. Hallucinations were identified in 29/50 RG and 27/50 Dx responses, leaving 44 clean responses. The table below reports MedFH’s results using the Intern-S1 backbone on these 100 responses.

Table 2. Evaluation on naturally generated responses with Intern-S1.

Results
FLA on hallucinated responses60.71%
FCA on hallucinated responses46.43%
Unnecessary correction on clean responses9.09%
Sentence-level hallucination F1 / macro-F171.23 / 83.14
ScS_c, before →\rightarrow after correction2.68 →\rightarrow 1.75
AggNLG, before →\rightarrow after correction84.85 →\rightarrow 84.86

The annotations enable direct evaluation of first-error localization and correction on natural errors. The decrease in ScS_c indicates improved response-level correction quality, while reference agreement remains essentially stable. Logic-thread accuracy is 86.89%. The localization and unnecessary-correction results jointly assess whether MedFH intervenes at the correct position in erroneous responses and preserves naturally correct answers.

AuthorsOpenReview ↗

Response 2

W2 / Q2. Detection and correction with shared tools and models. To test whether MedFH retains its advantages with shared medical tools and models, we conducted additional detection and correction experiments. We provided UNIHD, RAC, and Woodpecker with MedFH’s medical evidence tools and used Intern-S1 as the detection or correction backbone for all methods. The two tables below report the detection and correction results, respectively. Compared with UNIHD, MedFH improves FLA by 4.78 percentage points on RG and 3.20 on Dx, and hallucination F1 by 10.24 and 1.70 points, respectively. Compared with the strongest correction baseline on each task, MedFH improves FCA by 25.61 percentage points on RG and 2.29 on Dx, while reducing ScS_c by 1.19 and 0.15, respectively. These results show that MedFH retains advantages in both detection and correction when the medical tools and corresponding models are shared.

Table 3. Hallucination detection with shared medical tools and Intern-S1.

TaskMethodFLA ↑\boldsymbol{\uparrow}Hall. F1 ↑\boldsymbol{\uparrow}Macro-F1 ↑\boldsymbol{\uparrow}
RGUNIHD60.9671.3978.15
RGMedFH65.7481.6385.69
DxUNIHD64.7176.2883.40
DxMedFH67.9177.9884.36

Table 4. Hallucination correction with shared medical tools and Intern-S1.

TaskMethodFCA ↑\boldsymbol{\uparrow}Sc\boldsymbol{S_c} ↓\boldsymbol{\downarrow}AggNLG ↑\boldsymbol{\uparrow}
RGRAC6.024.7948.15
RGWoodpecker12.354.6521.24
RGMedFH37.963.4680.29
DxRAC49.942.5675.61
DxWoodpecker35.833.5753.39
DxMedFH52.232.4186.07
AuthorsOpenReview ↗

Response 3

W2. Innovation relative to TIGER and Dialectic-Med.

Relative to TIGER [1], our innovation lies in premise-dependent verification and first-error-guided correction. TIGER propagates support through shared-entity links and selects high-risk facts for local repair. MedFH explicitly models which patient facts and earlier claims each claim depends on, combining medical tools with the selected response predecessors’ verification verdicts and rationales. These dependencies also control correction. The sentence containing the first detected hallucination defines the repair boundary. MedFH preserves the preceding text, uses direct logical successors to guide revision of the continuation, and rebuilds the affected structure for re-verification. This connects the reliability of a claim’s premises to both its verification and the repair of downstream content.

Relative to Dialectic-Med [2], our innovation lies in explicit hallucination detection and localization within an existing response. Both methods use visual evidence and graph-based reasoning. Dialectic-Med connects diagnostic hypotheses and counter-evidence through falsification and rectification edges, then aggregates path credibility to select a diagnosis and produce its explanation. MedFH models premise dependencies among patient facts and ordered response claims, produces claim-level hallucination judgments and sentence labels, and locates the first detected hallucination. These outputs directly determine the correction boundary and subsequent reassessment, supporting detection, localization, and correction of an existing response alongside hallucination mitigation.

MedFH also uses a unified semantic tree to handle different medical tasks. MedFH organizes query and response content into a shared source–sentence–claim hierarchy that preserves textual order. This accommodates radiology reporting with no query claims and diagnostic reasoning with patient-specific query facts. The tree maps claim judgments to sentences and correction boundaries, while added logic threads encode premise dependencies. Together, they support the same verification, localization, and iterative correction procedure across RG and Dx.

We will cite both papers and explicitly compare their methods with MedFH in the related-work paragraph of the main text.

[1] Zhao, K., Yu, T., Huang, S., Jenkins, P., Dong, Y., and Hughes, A. “Self-Correction Can Amplify Hallucinations: Fact-Level Repair with Graph-Based Evidence Routing in Multimodal Generation.” arXiv:2606.00232, 2026.

[2] Lu, Z., and Su, J. “Dialectic-Med: Mitigating Diagnostic Hallucinations via Counterfactual Adversarial Multi-Agent Debate.” Findings of the Association for Computational Linguistics: ACL 2026, pp. 36877–36894, 2026.

Dear Reviewer,

Thank you for your thoughtful review. We would like to follow up on the points you raised and highlight where we have addressed them in the revised manuscript:

  • Natural errors and unnecessary correction (Appendix A.8, Tables 11–12). On 100 naturally generated responses with physician-reviewed annotations, MedFH localized the first hallucination in 60.71% of the 56 responses containing errors. The unnecessary correction rate was 9.09% on the 44 clean responses. Table 12 also compares correction methods on 200 additional clean responses per task.

  • Physician assessment (Appendix B.5, Table 16). A physician reviewed the sentence-level labels and cascade annotations for 100 benchmark cases and rated corrections from MedFH and Woodpecker on those same cases. Table 16 reports both physician-rated correction quality and agreement with automated evaluation.

  • Contribution of claim dependencies and related work. Section 3.4 and Appendices F.2–F.3 clarify how predecessor selection and cascade guidance contribute to detection and correction. Our earlier reply reports the shared-tool comparisons separately for radiology reporting and diagnostic reasoning. We also added TIGER and Dialectic-Med to the related-work discussion in Section 1.

We would appreciate your feedback on whether the completed revisions resolve your main concerns. If they do, we would be grateful if you could consider updating your assessment. We welcome further discussion.

Best regards,
The Authors

Reviewer_XorrOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MedFH, a framework for detecting and correcting hallucinations in medical vision-language model responses. It organizes medical claims into a semantic tree with logic threads to capture dependencies between claims. Each claim is verified using external grounding evidence and relevant preceding claims. Once the first hallucination is detected, the method revises the response from that point onward, using logical successors to guide correction. The authors also introduce MedLHallu-Bench, covering radiology report generation and diagnostic reasoning. Experiments show improvements in hallucination detection, localization, and correction, with additional evaluation on MedHEval.

Critical strengths
  1. A coherent and meaningful methodological contribution. The paper addresses a relevant problem that is not fully captured by independent claim verification: how an early unsupported claim can affect subsequent medical reasoning. Using explicit dependencies both to select verification evidence and to guide downstream correction is a coherent idea, and the first-hallucination formulation gives the framework a clear focus.
  2. Evidence extending beyond overall benchmark improvements. The evaluation includes controlled component ablations and an external benchmark with naturally generated medical responses. In particular, the improvements from first-hallucination-guided correction and the diagnostic-reasoning logic-thread ablations support the value of the proposed design, rather than relying solely on comparisons against existing systems.
Critical weaknesses
  1. The contribution of logic threading is not fully clear. MedFH relies on several specialized medical grounding and retrieval tools, which may account for much of the improvement. In Table 4, removing grounding evidence causes a much larger performance drop than removing logic threading. On MedLHallu-RG, removing logic threading even slightly improves FLA. It remains unclear how much the proposed dependency modeling contributes beyond these additional tools.

  2. The benchmark may favor the proposed method. MedLHallu-Bench is constructed by injecting hallucinations and modifying dependent downstream claims, which closely matches the assumptions of MedFH. Although the external MedHEval results are encouraging, they do not directly validate first-hallucination localization or dependency tracking on naturally generated errors.

Questions for authors
  1. How much of MedFH's improvement comes from logic threading rather than the additional medical grounding tools? The current ablations do not fully separate these effects.

  2. In Table 4, why does removing logic threading slightly improve FLA on MedLHallu-RG but reduce it on MedLHallu-Dx?

  3. Since MedLHallu-Bench explicitly constructs downstream hallucinations, how well do these dependency patterns reflect naturally generated errors?

6 comments
AuthorsOpenReview ↗

Response 1

We thank the reviewer for the careful reading and valuable suggestions. We address your three questions below.

W1 / Q1. How much does logic threading contribute beyond medical grounding? We conducted a staged ablation that introduces grounding evidence, selected query predecessors, and verified response predecessors in sequence. Grounding increases FLA by 31.01 points on RG and 5.52 on Dx. With grounding retained, adding both predecessor sources yields a further 1.35 FLA points on RG and 7.49 on Dx, with corresponding hallucination-F1 gains of 1.34 and 7.08. These conditional gains show that grounding contributes most of the RG improvement, while predecessor evidence provides a larger additional benefit on Dx.

Table 1. Incremental contributions of grounding and predecessor evidence.

Verification inputsRG FLA ↑\boldsymbol{\uparrow}RG Hall. F1 ↑\boldsymbol{\uparrow}Dx FLA ↑\boldsymbol{\uparrow}Dx Hall. F1 ↑\boldsymbol{\uparrow}
Direct Judge37.8151.6551.3460.54
+ Grounding evidence68.8278.3656.8661.36
+ Selected query predecessors68.8278.3661.8565.49
+ Verified response predecessors (MedFH)70.1779.7064.3568.44
AuthorsOpenReview ↗

Response 2

W1 / Q2. Why does removing threading slightly improve RG FLA but reduce Dx FLA? Many RG statements describe anatomical structures or imaging findings that can be assessed directly from the image, so their verification has relatively limited dependence on preceding statements. Dx, however, requires combining patient-specific facts and earlier conclusions to form a diagnosis, making relevant premises more important.

In the original Table 4, removing logic threading replaces selected predecessors with the full predecessor context. Its small effect on RG localization and larger effect on Dx are therefore consistent with the role of logic threading in selecting the premises needed for context-dependent verification.

W2 / Q3. How well do the constructed dependency patterns reflect natural errors? We generated 50 responses for each of RG and Dx using GPT-4.1-mini. GPT-6-Astra produced the annotations, which a physician checked and revised. Among responses containing hallucinations, 71.43% contain annotated cascade errors, showing that the dependency patterns targeted by the benchmark also occur in natural outputs. The results below additionally evaluate MedFH’s localization, correction, and dependency prediction on these responses using Intern-S1.

Table 2. Evaluation on naturally generated responses with Intern-S1.

Results
Logic-thread accuracy86.89%
FLA on hallucinated responses60.71%
FCA on hallucinated responses46.43%
Unnecessary correction on clean responses9.09%
Sentence-level hallucination F1 / macro-F171.23 / 83.14
ScS_c, before →\rightarrow after correction2.68 →\rightarrow 1.75
AggNLG, before →\rightarrow after correction84.85 →\rightarrow 84.86

Thanks for the additional experiments. The new results help address my concerns, particularly regarding performance on naturally generated errors. Although the independent contribution of logic threading remains somewhat unclear, the ablations and additional evaluations provide reasonable support for the proposed approach. I will maintain my rating of 3.

Reviewer_hKrbOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3(+1)
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MedFH for detecting and correcting hallucinations in medical multimodal responses. It organizes sentences and atomic claims into a semantic tree, adds dependency links between claims, and verifies each claim using external grounding tools and selected predecessor evidence. Correction begins at the first detected hallucinated sentence, with logical successors guiding revision and iterative reverification. The authors also introduce MedLHallu-Bench, covering radiology reports and diagnostic reasoning. Experiments report improvements in detection, localization, and correction, with additional correction results on natural MedHEval responses.

Critical strengths
  1. The paper addresses a meaningful limitation of independent claim verification: a clinically plausible statement can become unsupported when its patient-specific premises are incorrect. Combining direct grounding with selected predecessor evidence provides a coherent approach to detecting such errors and revising their downstream consequences.

  2. The evaluation covers detection, first-error localization, correction, and intermediate components across two backbones. The diagnostic-reasoning ablation supports the value of predecessor evidence, while external MedHEval results provide evidence beyond the controlled benchmark. The reported human–judge agreement is also useful, although its scope is limited.

Critical weaknesses
  1. The benchmark’s clinical validity and realism remain insufficiently established. Errors and downstream propagation are synthetically constructed, and benchmark validation relies on model judges rather than clinician adjudication. The human study uses one medically trained annotator and only 100 MedFH-corrected outputs, excluding baseline outputs and benchmark-label validation. Consequently, it does not establish the clinical reliability of the annotations or the comparative correction rankings. The controlled benchmark also contains injected errors in every case, limiting assessment of unnecessary intervention on clean responses.

  2. The contribution of dependency modeling is difficult to separate from stronger evidence tools and additional computation. Baselines do not receive equivalent medical grounding modules, and external MedHEval comparisons give MedFH an additional GPT-5.4-nano detector. Removing grounding causes the largest detection loss, whereas removing logic threading leaves RG localization essentially unchanged. Given the reported average of 69.31 model calls per corrected response, tool- and budget-matched comparisons are important for establishing the specific benefit of the proposed structure.

Questions for authors
  1. Does MedFH retain its advantage when the strongest claim-verification and correction baselines receive the same medical grounding tools, detector backbone, and comparable inference budgets? Please isolate the gains from predecessor selection and cascade guidance.

  2. Can the authors provide blinded clinician assessment of benchmark labels, dependency relations, and corrected outputs from both MedFH and the strongest baselines? Evaluation on naturally occurring errors and clean responses would materially strengthen the evidence.

  3. Appendix B.3 excludes invalid cases separately for each method. Please report evaluation coverage and failure rates, together with comparisons on a common case set and an analysis counting failures as unsuccessful outcomes. Do the main conclusions remain unchanged?

5 comments
AuthorsOpenReview ↗

Response 1

We thank the reviewer for the careful review and valuable suggestions on controlled comparisons, clinical validation, and evaluation coverage.

W2 / Q1. Shared-tool comparisons and component contributions.

(a) Matched baselines. We equipped UNIHD, RAC, and Woodpecker with the same medical evidence tools as MedFH and used Intern-S1 as the detection and correction backbone. The new comparisons are below:

Table 1. Hallucination detection with shared medical tools and Intern-S1.

TaskMethodFLA ↑\boldsymbol{\uparrow}Hall. F1 ↑\boldsymbol{\uparrow}Macro-F1 ↑\boldsymbol{\uparrow}
RGUNIHD60.9671.3978.15
RGMedFH65.7481.6385.69
DxUNIHD64.7176.2883.40
DxMedFH67.9177.9884.36

Table 2. Hallucination correction with shared medical tools and Intern-S1.

TaskMethodFCA ↑\boldsymbol{\uparrow}Sc\boldsymbol{S_c} ↓\boldsymbol{\downarrow}AggNLG ↑\boldsymbol{\uparrow}
RGRAC6.024.7948.15
RGWoodpecker12.354.6521.24
RGMedFH37.963.4680.29
DxRAC49.942.5675.61
DxWoodpecker35.833.5753.39
DxMedFH52.232.4186.07

MedFH retains FLA gains of 4.78 on RG and 3.20 on Dx, and FCA gains over the strongest comparator of 25.61 and 2.29, respectively. Its detection and correction advantages therefore persist with shared tools and backbones.

(b) Predecessor evidence and cascade guidance. The original Table 4 compares selected predecessors with the full predecessor context. We conducted a staged detection ablation to further distinguish the contributions of query predecessors and verified response predecessors while retaining grounding evidence:

Table 3. Incremental contributions of grounding and predecessor evidence.

MethodRG FLA ↑\boldsymbol{\uparrow}RG Hall. F1 ↑\boldsymbol{\uparrow}Dx FLA ↑\boldsymbol{\uparrow}Dx Hall. F1 ↑\boldsymbol{\uparrow}
Direct Judge37.8151.6551.3460.54
+ Grounding evidence68.8278.3656.8661.36
+ Selected query predecessors68.8278.3661.8565.49
+ Verified response predecessors70.1779.7064.3568.44

On Dx, selected query predecessors add 4.99 FLA points, and verified response predecessors add a further 2.50 points. These increments support using both relevant query context and the verification records of earlier claims. RG has no query context, so the query-predecessor step leaves its results unchanged, while verified response predecessors add 1.35 FLA points. The additional detection benefit is larger on Dx, where verification depends more strongly on patient-specific premises.

To isolate the contribution of cascade guidance during correction, we compared correction with and without successor proposals while holding the detected first error location and correction boundary fixed. Adding successor proposals improves FCA by 4.37 on RG and 2.42 on Dx, while reducing ScS_c by 0.22 and 0.17, respectively. This measures the additional correction benefit of downstream guidance.

AuthorsOpenReview ↗

Response 2

W1 / Q2. Physician assessment, natural errors, and clean responses. We randomly sampled 100 benchmark cases and asked one physician to assess their sentence-level hallucination labels, cascade annotations, and corresponding MedFH and Woodpecker corrections. The physician agreed with all audited sentence-level hallucination labels and cascade annotations. The table below reports two aspects of correction assessment. The first two metric columns show physician-rated correction quality, while the remaining columns measure agreement with automated evaluation. Under physician assessment, MedFH achieves higher FCA and a lower mean ScS_c than Woodpecker. The pooled FCA agreement is 87%.

Table 4. Physician-rated correction quality and agreement with automated evaluation.

MethodPhysician-rated FCA ↑\boldsymbol{\uparrow}Physician-rated Sc\boldsymbol{S_c} ↓\boldsymbol{\downarrow}FCA agreementSc\boldsymbol{S_c} linear-weighted κ\boldsymbol{\kappa}Sc\boldsymbol{S_c} MAE
MedFH39%3.3376%0.7640.415
Woodpecker22%4.3598%0.7910.369
Pooled agreement——87%0.7860.392

We also evaluated hallucination detection and correction on 100 naturally generated responses. GPT-4.1-mini generated the responses from the images and available query context. GPT-6-Astra produced the annotations, which a physician checked and revised. With Intern-S1, MedFH achieves 60.71% FLA and 46.43% FCA on hallucinated responses.

To examine unnecessary correction of clean answers, we additionally constructed 200 clean responses for each of RG and Dx and evaluated MedFH and the baselines. MedFH obtains the lowest ScS_c, at 1.11 and 0.62, compared with 2.39 and 1.25 for the strongest respective baselines. These results support MedFH’s ability to detect and correct natural errors and show better preservation of clean responses than the evaluated baselines.

AuthorsOpenReview ↗

Response 3

Q3. Coverage and sensitivity to failures. The following table lists all detection settings with coverage below 100%. All other settings have complete coverage. For correction, coverage is 100% except for GPT-5.4-nano MedFH on RG, with 647/648 valid outputs.

Table 5. Detection coverage and failure rates for configurations with incomplete coverage.

BackboneTaskMethodValid / totalCoverageFailure
GPT-5.4-nanoRGSelfCheckGPT639/64898.61%1.39%
GPT-5.4-nanoRGUNIHD643/64899.23%0.77%
GPT-5.4-nanoRGFactSelfCheck646/64899.69%0.31%
GPT-5.4-nanoRGMedFH647/64899.85%0.15%
GPT-5.4-nanoDxVLM-as-Judge557/56199.29%0.71%
GPT-5.4-nanoDxUNIHD505/56190.02%9.98%
Intern-S1DxVLM-as-Judge557/56199.29%0.71%
Intern-S1DxUNIHD522/56193.05%6.95%

We further evaluated the original baseline configurations on common case sets and with failures counted as unsuccessful outcomes. MedFH retains the highest FLA and FCA across all four task–backbone settings under both evaluations, supporting the robustness of the localization and correction conclusions. Each cell below reports best baseline →\rightarrow MedFH, with the best baseline selected separately for each metric and evaluation setting.

Table 6. Localization and correction on common cases and with failures counted as unsuccessful.

Backbone / TaskCommon cases: FLA ↑\boldsymbol{\uparrow}Common cases: FCA ↑\boldsymbol{\uparrow}Failures counted as unsuccessful: FLA ↑\boldsymbol{\uparrow}Failures counted as unsuccessful: FCA ↑\boldsymbol{\uparrow}
GPT-5.4-nano / RG49.79 →\rightarrow 70.2729.25 →\rightarrow 34.6949.69 →\rightarrow 70.0627.10 →\rightarrow 34.57
GPT-5.4-nano / Dx54.75 →\rightarrow 64.4544.29 →\rightarrow 57.9652.22 →\rightarrow 64.3543.52 →\rightarrow 57.76
Intern-S1 / RG49.54 →\rightarrow 65.7412.63 →\rightarrow 38.0649.54 →\rightarrow 65.7412.53 →\rightarrow 37.90
Intern-S1 / Dx64.59 →\rightarrow 68.0137.87 →\rightarrow 52.3363.45 →\rightarrow 67.9137.97 →\rightarrow 52.23

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the discussion.

Dear Area Chair and Reviewers,

Thank you for the careful reviews and constructive discussion. The main questions concerned the clinical validity of our benchmark, the contribution of logic threading beyond medical grounding, and MedFH’s performance on naturally generated responses. We have revised the manuscript to address the reviewers’ questions and highlighted all changes in blue. We summarize the discussion with each reviewer below.

Reviewer hKrb — benchmark validation and evaluation reliability. A physician reviewed the sentence-level hallucination labels and cascade annotations for 100 randomly sampled benchmark cases and agreed with all audited annotations. On the same cases, physician-rated first-hallucination correction accuracy was 39% for MedFH and 22% for Woodpecker (Appendix B.5, Table 16). We also reported coverage and failure rates. MedFH retained the highest localization and correction accuracy across both tasks and backbones when evaluated on common cases and when failures were counted as unsuccessful (Appendix B.3, Tables 13–14). Reviewer hKrb stated that our response addressed most of his concerns and updated the assessment.

Reviewer Xorr — the contribution of logic threading and the realism of error propagation. With medical grounding held fixed, selecting relevant predecessors increased first-hallucination localization accuracy from 57.22% to 64.35% in diagnostic reasoning. The change was small in radiology report generation, from 70.48% to 70.17%, consistent with many imaging findings being directly verifiable from the image. With the first error and correction boundary fixed, guidance from the error’s direct logical successors improved correction accuracy by 4.37 percentage points on radiology reports and 2.42 points on diagnostic reasoning (Section 3.4, Appendices F.2–F.3). Cascade errors also occurred in 40 of the 56 naturally generated responses containing hallucinations (Appendix A.8). Reviewer Xorr confirmed that our clarification addressed his main concern about logic threading, particularly in diagnostic reasoning.

Reviewer NH5z — natural-error localization, unnecessary correction, shared-tool comparisons, and novelty. We evaluated 100 naturally generated responses with physician-reviewed annotations. MedFH correctly localized the first hallucination in 60.71% of the 56 responses containing hallucinations, and the unnecessary correction rate was 9.09% on the 44 clean responses (Appendix A.8, Table 11). To assess how much of the correction gain remained when baselines used the same medical evidence tools and correction backbone (Intern-S1), we reported results separately for each task. MedFH exceeded the strongest baseline in first-hallucination correction accuracy by 25.61 percentage points on radiology report generation and 2.29 points on diagnostic reasoning. We added TIGER and Dialectic-Med to Section 1 and clarified MedFH’s contributions relative to each in our reply. We have not yet heard from this reviewer whether our response addresses his concerns.

We appreciate the time the Area Chair and reviewers have devoted to our submission.

Sincerely,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.