- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies hallucination detection and proposes UAHV. It combines global LLM screening with local lexical and semantic verification, as well as additional LLM verification for borderline cases. It aims to improve detection performance while reducing LLM calls.
Critical strengths
- UAHV achieves the highest reported macro-F1 among the evaluated methods in 11 of the 12 dataset–model settings.
- The ablation results provide support for the method’s main design choices.
Critical weaknesses
- The novelty is modest, and the verification rules need further justification. For example, HFV cannot correct false positives after the global screening. Errors missed globally may also be accepted when too few keywords remain unsupported to trigger arbitration, even if those keywords represent a critical fact.
- Key implementation details are unclear. The paper does not clearly explain keyword extraction or common-noun identification. Although search ranges and sensitivity results are provided, it remains unclear how the default fuzzy matching threshold of 0.75 and missing-count threshold of 4 were selected.
Questions for authors
- Please explain how keywords are extracted and common nouns are identified, and how the default fuzzy matching threshold of 0.75 and missing-count threshold of 4 were selected.
- In Table 2, adding rescue increases SUM FPR from 16.60% to 28.00%. As defined in the paper, rescue can only reduce or maintain the missing count, so FPR should not increase in this setting. Please explain this result.
- Please report how often the global stage incorrectly rejects supported responses and how much these errors contribute to the final false-positive rate.
- How well does the fixed missing-count threshold work across samples with different numbers of keywords? How does UAHV handle a response with one incorrect critical fact when the remaining facts are supported?
2 comments
A Brief Follow-up Before the Discussion Closes
Dear Reviewer ibGn,
We apologize for following up once more, as the discussion period will close in a few hours. We have carefully addressed all of your comments and would greatly appreciate any brief feedback on whether your main concerns have been resolved. If so, we would also be grateful if you could reconsider your evaluation accordingly.
Thank you again for your time and thoughtful review.
Request for Further Feedback on Our Rebuttal
Dear Reviewer ibGn,
Thank you once again for your valuable comments on our submission. As the discussion phase is approaching its end, we would like to kindly confirm whether we have sufficiently addressed all of your concerns (or at least part of them). Should there be any remaining questions or areas requiring further clarification, please do not hesitate to let us know. If you are satisfied with our responses, we would greatly appreciate your consideration in adjusting the evaluation scores accordingly.
We sincerely look forward to your feedback.