- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes MedFH for detecting and correcting hallucinations in medical responses. The authors decompose responses into claims and link each claim to relevant earlier claims. They verify claims using medical images, retrieved knowledge, and those links. The first detected error determines where correction begins. Evaluation uses 1,209 constructed cases across radiology reporting and diagnostic reasoning. The paper also tests correction on naturally generated responses from MedHEval.
Critical strengths
-
It records sentence errors, the first error, and affected later claims. This permits a more precise evaluation of correction than a single response score.
-
In diagnostic reasoning, removing logic threads lowers first error localization accuracy from 64.35 to 57.22. On MedHEval, correction lowers the reported visual hallucination measure from 15.44 to 9.84 while increasing key finding recall from 2.50 to 37.08. The joint change makes the result more persuasive than a reduction in hallucinations alone.
Critical weaknesses
-
Every constructed response receives two injected errors. Later dependent statements are edited as part of the same construction process. This design closely matches the proposed localization and correction strategy. It contains no clean responses for measuring unnecessary corrections. The 200 case benchmark audit uses language models, without clinician adjudication. The human assessment covers 100 corrected MedFH outputs, but does not validate the benchmark error positions or compare baseline corrections. MedHEval contains natural responses, yet provides no reference first error positions or dependency links. Thus, the distinctive localization and cascade claims remain untested on natural errors.
-
In radiology reporting, removing logic threads changes localization accuracy from 70.17 to 70.48, a slight increase. The benefit is clearer in diagnostic reasoning. MedFH also uses specialized medical grounding and verification tools and averages 69.31 model calls per response. Comparable resource use is not reported for the correction baselines. The large correction gains therefore cannot be assigned specifically to the claim links. The related work omits TIGER [1], which uses claim graphs and evidence guided local repair, and Dialectic-Med [2], which addresses diagnostic hallucinations through visual evidence and graph based reasoning. Their settings differ, but both require a clearer account of the proposed novelty.
[1] Zhao, K., Yu, T., Huang, S., Jenkins, P., Dong, Y., and Hughes, A. “Self-Correction Can Amplify Hallucinations: Fact-Level Repair with Graph-Based Evidence Routing in Multimodal Generation.” arXiv:2606.00232, 2026.
[2] Lu, Z., and Su, J. “Dialectic-Med: Mitigating Diagnostic Hallucinations via Counterfactual Adversarial Multi-Agent Debate.” Findings of the Association for Computational Linguistics: ACL 2026, pp. 36877–36894, 2026.
Questions for authors
- On clinician annotated, naturally generated responses that include correct answers, what are the first error localization accuracy and the rate of unnecessary correction?
- How much of the correction gain remains when a comparator receives the same medical evidence tools, correction model, and model call budget? Can the authors report this separately for radiology reporting and diagnostic reasoning?
4 comments
Response 2
W2 / Q2. Detection and correction with shared tools and models. To test whether MedFH retains its advantages with shared medical tools and models, we conducted additional detection and correction experiments. We provided UNIHD, RAC, and Woodpecker with MedFH’s medical evidence tools and used Intern-S1 as the detection or correction backbone for all methods. The two tables below report the detection and correction results, respectively. Compared with UNIHD, MedFH improves FLA by 4.78 percentage points on RG and 3.20 on Dx, and hallucination F1 by 10.24 and 1.70 points, respectively. Compared with the strongest correction baseline on each task, MedFH improves FCA by 25.61 percentage points on RG and 2.29 on Dx, while reducing by 1.19 and 0.15, respectively. These results show that MedFH retains advantages in both detection and correction when the medical tools and corresponding models are shared.
Table 3. Hallucination detection with shared medical tools and Intern-S1.
| Task | Method | FLA | Hall. F1 | Macro-F1 |
|---|---|---|---|---|
| RG | UNIHD | 60.96 | 71.39 | 78.15 |
| RG | MedFH | 65.74 | 81.63 | 85.69 |
| Dx | UNIHD | 64.71 | 76.28 | 83.40 |
| Dx | MedFH | 67.91 | 77.98 | 84.36 |
Table 4. Hallucination correction with shared medical tools and Intern-S1.
| Task | Method | FCA | AggNLG | |
|---|---|---|---|---|
| RG | RAC | 6.02 | 4.79 | 48.15 |
| RG | Woodpecker | 12.35 | 4.65 | 21.24 |
| RG | MedFH | 37.96 | 3.46 | 80.29 |
| Dx | RAC | 49.94 | 2.56 | 75.61 |
| Dx | Woodpecker | 35.83 | 3.57 | 53.39 |
| Dx | MedFH | 52.23 | 2.41 | 86.07 |
Response 3
W2. Innovation relative to TIGER and Dialectic-Med.
Relative to TIGER [1], our innovation lies in premise-dependent verification and first-error-guided correction. TIGER propagates support through shared-entity links and selects high-risk facts for local repair. MedFH explicitly models which patient facts and earlier claims each claim depends on, combining medical tools with the selected response predecessors’ verification verdicts and rationales. These dependencies also control correction. The sentence containing the first detected hallucination defines the repair boundary. MedFH preserves the preceding text, uses direct logical successors to guide revision of the continuation, and rebuilds the affected structure for re-verification. This connects the reliability of a claim’s premises to both its verification and the repair of downstream content.
Relative to Dialectic-Med [2], our innovation lies in explicit hallucination detection and localization within an existing response. Both methods use visual evidence and graph-based reasoning. Dialectic-Med connects diagnostic hypotheses and counter-evidence through falsification and rectification edges, then aggregates path credibility to select a diagnosis and produce its explanation. MedFH models premise dependencies among patient facts and ordered response claims, produces claim-level hallucination judgments and sentence labels, and locates the first detected hallucination. These outputs directly determine the correction boundary and subsequent reassessment, supporting detection, localization, and correction of an existing response alongside hallucination mitigation.
MedFH also uses a unified semantic tree to handle different medical tasks. MedFH organizes query and response content into a shared source–sentence–claim hierarchy that preserves textual order. This accommodates radiology reporting with no query claims and diagnostic reasoning with patient-specific query facts. The tree maps claim judgments to sentences and correction boundaries, while added logic threads encode premise dependencies. Together, they support the same verification, localization, and iterative correction procedure across RG and Dx.
We will cite both papers and explicitly compare their methods with MedFH in the related-work paragraph of the main text.
[1] Zhao, K., Yu, T., Huang, S., Jenkins, P., Dong, Y., and Hughes, A. “Self-Correction Can Amplify Hallucinations: Fact-Level Repair with Graph-Based Evidence Routing in Multimodal Generation.” arXiv:2606.00232, 2026.
[2] Lu, Z., and Su, J. “Dialectic-Med: Mitigating Diagnostic Hallucinations via Counterfactual Adversarial Multi-Agent Debate.” Findings of the Association for Computational Linguistics: ACL 2026, pp. 36877–36894, 2026.
Dear Reviewer,
Thank you for your thoughtful review. We would like to follow up on the points you raised and highlight where we have addressed them in the revised manuscript:
-
Natural errors and unnecessary correction (Appendix A.8, Tables 11–12). On 100 naturally generated responses with physician-reviewed annotations, MedFH localized the first hallucination in 60.71% of the 56 responses containing errors. The unnecessary correction rate was 9.09% on the 44 clean responses. Table 12 also compares correction methods on 200 additional clean responses per task.
-
Physician assessment (Appendix B.5, Table 16). A physician reviewed the sentence-level labels and cascade annotations for 100 benchmark cases and rated corrections from MedFH and Woodpecker on those same cases. Table 16 reports both physician-rated correction quality and agreement with automated evaluation.
-
Contribution of claim dependencies and related work. Section 3.4 and Appendices F.2–F.3 clarify how predecessor selection and cascade guidance contribute to detection and correction. Our earlier reply reports the shared-tool comparisons separately for radiology reporting and diagnostic reasoning. We also added TIGER and Dialectic-Med to the related-work discussion in Section 1.
We would appreciate your feedback on whether the completed revisions resolve your main concerns. If they do, we would be grateful if you could consider updating your assessment. We welcome further discussion.
Best regards,
The Authors
Response 1
We thank the reviewer for the careful reading and valuable suggestions on natural-error evaluation, controlled comparisons, and related work.
W1. Physician assessment of benchmark annotations and correction quality. We randomly sampled 100 benchmark cases and asked a physician to review their sentence-level hallucination labels, cascade annotations, and corresponding corrections from MedFH and Woodpecker. The physician agreed with all audited labels and cascade annotations in this sample. The table below reports physician-rated correction quality on the same cases, together with agreement between physician and automated assessments.
Table 1. Physician-rated correction quality and agreement with automated evaluation.
W1 / Q1. Evaluation on naturally generated responses. We evaluated 100 responses generated by GPT-4.1-mini, comprising 50 radiology reports (RG) and 50 diagnostic reasoning responses (Dx). GPT-6-Astra produced the annotations, which a physician checked and revised. Hallucinations were identified in 29/50 RG and 27/50 Dx responses, leaving 44 clean responses. The table below reports MedFH’s results using the Intern-S1 backbone on these 100 responses.
Table 2. Evaluation on naturally generated responses with Intern-S1.
The annotations enable direct evaluation of first-error localization and correction on natural errors. The decrease in Sc indicates improved response-level correction quality, while reference agreement remains essentially stable. Logic-thread accuracy is 86.89%. The localization and unnecessary-correction results jointly assess whether MedFH intervenes at the correct position in erroneous responses and preserves naturally correct answers.