acceptodds
Under review as a conference paper at ICLR 2027

When Rubrics Fail: Hallucinations Reveal Blind Spots in Medical AI Evaluation

Abstract

Hallucinations can undermine clinician trust in LLMs, making it important that evaluation methods capture clinically relevant errors. Rubric-based evaluation has become the leading approach for assessing LLMs in medicine, but it is unclear whether rubric scores reflect such errors. We first study this in a controlled setting using MedHallu, finding that more specific rubrics better distinguish correct from hallucinated responses. To test this systematically, we develop a taxonomy of medical hallucination types and a clinician-validated error-injection pipeline that creates matched original and error-injected responses. Across HealthBench, HealthBench Professional, and LiveMedBench, our clinically relevant hallucinations are missed by rubrics, often leaving scores unchanged. We find that rubrics are most effective when explicitly checking facts, and are less effective for additional or unexpected errors they do not anticipate. Targeted rubric criteria and retrieval-based factuality checks recover some rubric-blind errors highlighting complementary directions for improving error detection. These findings reveal systematic blind spots in current medical evaluation of LLMs and suggest that rubric scores alone are insufficient to establish clinical reliability, potentially undermining clinician trust and confidence in clinical deployment.

Primary area:
datasets and benchmarks
Keywords:
clinical AI evaluation, rubric-based evaluation, hallucinations, LLMs in medicine, LLM-as-a-judge, medical benchmarks

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eCibOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the limitations of rubric-based evaluation in detecting clinically meaningful hallucinations in medical LLM responses. Overall, this paper is clearly motivated and addressed an important problem. The authors develop a clinician-validated hallucination taxonomy and a controlled error-injection pipeline to examine rubric sensitivity across several medical benchmarks. The results reveal substantial blind spots in existing rubrics and explore targeted criteria and retrieval-based factuality checking as potential improvements.

Critical strengths

The paper addresses an important and timely problem in medical AI evaluation. The controlled error-injection framework and clinician validation provide meaningful evidence of the limitations of existing rubric-based benchmarks. The experiments cover multiple datasets and hallucination types, and the findings offer useful insights into the relationship between rubric specificity and error detection. Overall, the paper is clearly written and the experimental analysis is relatively comprehensive.

Critical weaknesses

My main concern is the novelty relative to closely related work, particularly RubricRobustness (ICML 2026), which has already studied rubric sensitivity to controlled response perturbations on HealthBench. Although this work extends the analysis to clinically meaningful errors, a more thorough comparison would help establish its additional contributions. Authors mentioned this work in the paper briefly, but I think more detailed comparisons are needed.

Another concern is the scalability and reliability of the proposed evaluation approach. Clinical expert availability, annotation costs, and inter-reviewer disagreement remain important challenges. Furthermore, the proposed hallucination taxonomy relies on predefined error categories, which may introduce coverage limitations similar to those of existing rubrics, particularly when considering previously unanticipated errors. The experiment results also show that experts cannot agree with each other on some instances.

Questions for authors

Could the authors provide a more detailed comparison with RubricRobustness and clarify what new insights the proposed framework reveals beyond existing perturbation-based evaluation methods?

How could the proposed approach scale given the cost, limited availability, and potential disagreement of clinical experts?

Since the hallucination taxonomy also relies on predefined error categories, how would the framework identify clinically important errors that fall outside the existing taxonomy?

Reviewer_dcFAOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Medical AI benchmarks like HealthBench grade a model’s answer against a checklist (a “rubric”) written by physicians. The paper asks whether a rubric score drops when an answer contains a clinically dangerous error. Mostly it doesn’t. The authors take a good answer, quietly insert one realistic error (e.g. a 100× insulin dose, a wrong triage level, a fabricated citation), and re-score it. Disturbingly often the two answers get identical scores. Rubrics are best at catching errors they explicitly anticipate, but unexpected errors can fall outside their coverage. It is hard to anticipate every possible type of error.

As a preliminary remedy they tested a rubric with one criterion per error type, and a simplified search-based fact-checker that breaks the entire answer into claims and checks each against Google results.

In testing this remedy, they show that "incorporating known error types directly into rubric criteria improves their detection, but requires specifying the errors in advance. We further find that retrieval-grounded factuality checks can detect hallucinations missed by rubric evaluation (Section 5.3)."

I'd characterize the paper as mostly shining a light on the problem, with some early preliminary steps towards proposing a solution.

Critical strengths

A clear question about a widely used methodology, important because rubrics are now training signals.

A well-designed controlled probe that holds everything fixed except one injected error, giving matched pairs and clean attribution.

Honest reporting of context leakage, false positives, and clinician disagreement.

I think the impact of publishing this paper would be good. Medical LLM evaluation is important. This paper, along with Dixit, pushes the rubric-based evaluation community in a good direction.

Critical weaknesses

The injected errors are synthetic: controlled LLM edits, not naturally occurring hallucinations. It seems that naturally occurring errors could be larger, more entangled, or more detectable. To their credit the authors acknowledge this, but it would nice to see more spot-checks or anecdotal examples of naturally occurring model hallucinations, even with a small number of clinician-reviewed examples, to see how representative the injected errors are.

Comparison to alternatives could be better. For example, there is no direct comparison against a plain pairwise LLM-judge or “find errors in this answer” prompt to a strong frontier-model reviewer, or even a few examples where a physician is expcliity asked to give likely safety-critical errors. Without these, it is unclear whether rubrics are worse than the alternatives.

As mentioned above, the proposed remedies are preliminary.

No downstream test shows whether these errors actually distort RL training with rubric rewards, which is the most consequential claim implied.

The paper may be less influential than hoped if follow-ups show that stronger judges or natural errors behave differently. To me the paper's biggest unresolved question is how this blind spot affects models trained on rubric rewards. But this could be a natural next paper.

Questions for authors

Among the false negatives, how often is it a weakness in Google retrieval?

How would a physician-written error checklist do? For example, if clinicians were asked to write a list of “things that would make this answer dangerous” for each question, would coverage improve, and at what cost?

Reviewer_6CdROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether rubric-based LLM evaluation correctly identify clinically relevant hallucinations in medical answers. The authors build a clinician validated taxonomy of 13 medical hallucination error types. They generate adversarial responses by injecting one error at a time into otherwise unchanged responses, using a pipeline with programmatic and LLM quality filters followed by clinician review. The original and error-injected answers are scored against the benchmark’s rubric on 3 datasets (HealthBench, LiveMedBench, and HealthBench Professional). Tie rates are high and paired AUROC is low, suggesting that these rubrics miss hallucinations. The empirical story is interesting. However, the evaluation design leaves open how much of the result comes from the rubrics and how much from the injection method and grader. Some parts of the writing are also hard to follow, and a few design choices need more justification.

Critical strengths
  1. This is a timely paper. Rubric meta-evaluation is under-studied, and this paper tests rubrics directly instead of asking whether they agree with humans.
  2. The error type taxonomy is a useful contribution in its own right.
  3. The error injection method is simple and modular, with intuitively motivated components.
  4. The per-error-type breakdown is very informative. Under-triage and treatment errors are caught (AUROC 0.85 and 0.77), while dosage, threshold and evidence fabrication sit around 0.52 to 0.54. The cross-benchmark consistency with Kendall tau of 0.687 supports the generalization across benchmarks.
  5. Using several injectors and graders on HealthBench adds additional robustness check.
  6. Its great that the authors report their own negative and ambiguous findings!
Critical weaknesses
  1. Ties don’t show whether the rubric missed the error or never asked about it. They are an expected outcome of a mostly irrelevant edit and a short edit to a long response can often points to no error criterion. As indicated in the paper, HealthBench Professional has about 2.16 criteria per question and an 89% tie rate, so that number probably says more about sparse rubrics than about rubric failure. I’d like to see detection rates split into pairs where a criterion covers the edited fact and pairs where none does. Right now,’blind spot conflates “the rubric does not cover this” with “the rubric fails to penalise this”, which are different failures with different fixes.
  2. There is no way to tell how much of the score movement is noise. There exist cases where the rubric prefers the injected answer, and the paper doesn’t explain why. It could be grader noise, or the injection may have changed length, hedging or detail along the way. A paraphrase control that leaves the clinical meaning alone would give a noise floor. Without one, it is difficult to read the wins as detection or the losses as anything.
  3. The original responses are unverified. o3 scores 0.61 on HealthBench, so many of its answers likely already contain errors. Injecting a new error on top of an existing one muddies the comparison.
  4. Injected errors are not like real hallucinations. Real hallucinations are rarely isolated single span substitutions in an otherwise correct text. They tend to be correlated, consistent, and confidently stated. A small experiment using natural errors that clinicians found in o3 outputs would make the paper far more convincing.
  5. The paper is very dense with appendix pointers (E.4, E.7, E.9, E.10, Table 36). Key results, such as the no-effect of irrelevant criteria and the false-positive rates, only appear there.
Questions for authors

N/A

1 comment
AuthorsOpenReview ↗

Paraphrase Control Experiment

Paraphrase Control Experiment

We again thank the reviewer for suggesting a paraphrase control, which measures how much a meaning-preserving rewording changes the score. Injected errors lower the score significantly more often than a meaning-preserving rewording of the same answer (30.7% vs 20.0% of pairs), and change the score by about twice as much on average. The rubric is therefore responding to the error itself, not just to the change in wording, though the win rates in the paper are likely an upper bound on detection. Injected answers sometimes score higher than the original, but no more often than a paraphrase does (8.6% vs 10.7%), and most of these cases disappear when the error is reworded. They therefore mostly reflect the rubric’s sensitivity to wording rather than a preference for incorrect content. Rewording the injected errors themselves gives results consistent with the main experiment (57.9% of reworded injected answers tie with the original, against 59.9% in the main o3 results), so the paper’s central finding, that most injected errors leave the score unchanged, does not depend on how each error was worded.

Experimental Detail

For each of 140 o3 pairs from our main experiment, we took the passage of the original answer that the injection had edited and reworded it without changing its meaning. This gives a version of the original answer that is changed in the same place as the injected answer but contains no error. Rewordings stayed within 15% of the original length, and were rejected by a separate LLM judge if they changed clinical content, certainty, urgency or detail. All the other text was left unchanged. We also took each injected answer and reworded the passage containing the error, keeping the same error with the same certainty and detail. This tests whether detection depends on how the error is worded. All answers were graded with the paper's original pipeline (GPT-4.1, temperature 0).

The table compares each answer's score with the original answer's, on the same 140 pairs (counts in brackets):

Mean absolute changeScored lowerSameScored higher
Paraphrase, no error0.04320.0% (28)69.3% (97)10.7% (15)
Injected error (main result)0.09130.7% (43)60.7% (85)8.6% (12)
Injected error, reworded0.09634.3% (48)57.9% (81)7.9% (11)

An injected error lowers the score significantly more often than a meaning-preserving paraphrase. Rewording without changing the meaning (first row) changes the score in 30.7% of pairs (20.0% lower, 10.7% higher), by 0.043 on average. This is more often than re-grading the same text (10%, Appendix E.6), but less often than injecting an error (39.3%). Injected answers (second row) score lower than the original in 30.7% of pairs, against 20.0% for paraphrases of the same answers (paired difference 10.7 percentage points, 95% CI [2.9, 18.6]), and change the score about twice as much on average (paired difference in mean absolute change 0.048, 95% CI [0.017, 0.083]). The injections themselves changed answer length by a median of 1 word (IQR 0 to 12), and length change was not significantly associated with score change (Spearman's ρ\rho = 0.11, 95% CI [-0.06, 0.27]).

"Losses", where the error-injected answer scores higher than the original, are no more frequent than paraphrases scoring higher (8.6% vs 10.7%). When the error was reworded, most wins survived (34 of 43), but only 5 of 12 losses did, and most of the rest became ties (6 of the remaining 7). Losses are therefore stable for a given text but sensitive to its wording, which is why most replicated on re-grading. Because the rewording kept the error's certainty and detail, those side effects of the injection do not explain the losses. We read the losses as mostly reflecting sensitivity to wording, not a preference for incorrect content.

Rewording the error (third row) produced win/tie/loss rates of 34.3/57.9/7.9%, consistent with the main o3 results (30.8/59.9/9.3%), and no detectable change in aggregate rates or AUROC against the original injections. The reported aggregate rates in the paper therefore do not depend on how each error was worded. Pooling o3 and Kimi-K3 answers gives the same pattern.

We will add this control to the paper, using the additional page in the camera-ready version. The main win and loss rates will be reported alongside the paraphrase rates, so readers can compare them directly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.