acceptodds
Under review as a conference paper at ICLR 2027

Risk-Controlled Detection of Hallucinated References with Aggregated Detectors

Abstract

Large language models frequently fabricate academic references that appear authentic but correspond to no real publication. Existing detectors typically rely on a single aspect of verification and offer no formal guarantees on their error rates. We design a risk-controlled framework that assigns each reference a continuous suspicion score by aggregating heterogeneous verification signals, and then labels it as valid, hallucinated, or deferred to a human, controlling the false positive and false negative rates with as few deferrals as possible. Our guiding principle is that validity comes from calibration and efficiency from aggregation. We calibrate two thresholds with per-class conformal quantiles of the suspicion score, which delivers finite-sample control at user-specified levels. We then show that the deferral rate depends on the score only through its ROC curve, and that the posterior over detector outputs is the optimal aggregator. Experiments on the HallMark benchmark and on StatRef, a new dataset of statistics references, confirm that calibration secures validity and aggregation secures efficiency, with recalibration restoring control under domain shift.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
hallucinated reference detection, conformal prediction, risk control, aggregation, selective classification

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_895oOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper integrates rule-based checks, bibliographic verification tools, and multiple LLM-based detectors. A learned aggregation model produces a suspicion score for each reference, while class-conditional conformal calibration provides statistical guarantees on false positive and false negative rates. The paper further theoretically analyzes the relationship between detector aggregation and human review rates, and validates the approach on the HallMark and StatRef datasets.

Critical strengths
  1. The paper addresses a practically important problem, balancing detection reliability with human review efficiency.

  2. The theoretical analysis is relatively comprehensive, providing theoretical support for dual-risk control and the effectiveness of detector aggregation.

  3. The experiments are extensive, covering multiple detectors, aggregation strategies, and cross-domain evaluations, with substantial performance improvements on both datasets.

Critical weaknesses
  1. The methodological novelty is somewhat limited, as the approach mainly relies on existing detector aggregation and conformal prediction techniques.

  2. Practical deployment costs are insufficiently considered. The method relies on multiple LLMs and external verification tools, but lacks a systematic evaluation of inference latency, API costs, and overall efficiency.

  3. The deferral rate does not fully reflect the actual human review burden. If references flagged as hallucinated also require manual verification, the reduction in practical review workload may be considerably smaller than the reported results suggest.

Questions for authors
  1. Compared with using only a small number of high-performing bibliographic verification tools, do the benefits of integrating multiple LLMs justify the additional computational and API costs?

  2. If references flagged as hallucinated are also included in the human review workload, how much efficiency improvement does the proposed method provide in realistic deployment scenarios?

3 comments

Thank you for the positive assessment. Changes are in blue in the revised PDF.

W1. Novelty

We build on PAC-Wrap's calibration and Sadinle et al.'s oracle. For PAC-Wrap, it calibrates an anomaly score to provide PAC guarantees on FPR and FNR, and reduces ambiguity by loosening error tolerances. We keep the tolerances fixed and contribute a finite-sample analysis of deferral under aggregation.

For Sadinle et al., they provide the oracle and split-conformal class-specific coverage. Our theoretical contribution is the finite-sample deferral bound in Theorem 3, which separates the learned, PAC-calibrated rule's deferral into an information term, aggregation regret, and the unused budget. Corollary 4 gives the exact distribution of that budget.

 

W2. Runtime, API costs, and overall efficiency

We recorded API costs and running times on StatRef-LLM, a new dataset of 300 LLM-generated references added to evaluate actual LLM citation errors (new Sec. 5.5 and Table 24).

The table below compares the percentage of references flagged+deferred for human review.

ConfigurationHallMarkStatRefStatRef-LLMEstimated median parallel latencyAPI fees
Two-tool stacker59.7%60.2%46.3%51 s/reference$0/1,000 references
Full pool (11 detectors)60.0%46.8%46.7%51 s/reference$8.94/1,000 references

Verification tools dominate latency, while LLM judges account for API fees. The tools incur no API fees, but the slower tool takes a median of 51 seconds per reference. The seven judges together cost USD 8.94 per 1,000 references, with median response times ranging from 1 to 20 seconds.

 

Q1. Do multiple LLMs justify their cost?

Their benefit depends on the dataset. Compared with the two-tool stacker, the full pool reduces 134 human reviews per 1,000 references on StatRef, but provides little additional benefit on HallMark or StatRef-LLM. All seven judges are unnecessary: on StatRef, two tools plus Claude already perform similarly to the full pool. Claude's measured API cost is USD 3.78 per 1,000 references (new Table 24).

 

W3 / Q2. Human review including flagged+deferred references

We also include the best learned single detector in the comparison of total review burden below.

DatasetBest learned singleTwo-tool stackerFull poolSource
HallMark61.4%59.7%60.0%Table 6
StatRef61.5%60.2%46.8%Table 6
StatRef-LLM91.0%46.3%46.7%New Table 23

The benefit of additional LLM judges is data-dependent. On HallMark, all three configurations have similar total review burden. On StatRef, the full pool improves on both alternatives. On StatRef-LLM, aggregation helps, but the two tools already perform similarly to the full pool. These differences likely reflect error types and detector complementarity. HallMark already has a strong single detector. StatRef contains varied metadata errors for which LLM judges may provide complementary evidence. In StatRef-LLM, 85% of hallucinations involve DOI errors, which bibliographic tools can directly check.

Thank you to the authors for the thorough rebuttal and additional experiments. I appreciate the new evaluations on real LLM-generated references, the analysis of actual human review burden, and the measurements of API costs and inference latency. These additions have substantially addressed my practical concerns and strengthened the empirical evaluation.

My main remaining concern is the limited methodological novelty, as the framework largely builds upon existing calibration and aggregation techniques. Moreover, the new results suggest that the additional benefits of integrating multiple LLM judges over a small number of strong verification tools are dataset-dependent and sometimes marginal.

Nevertheless, I believe the paper addresses a practically important problem, and the rebuttal has demonstrated the value and feasibility of the proposed framework. I also believe that research emphasizing practical utility and rigorous empirical validation deserves encouragement, even when the methodological novelty is relatively modest.

Therefore, I will maintain my score of 3. I would also respectfully encourage the other reviewers and the AC to consider the paper's practical significance, and to give it favorable consideration for acceptance.

Thank you for your careful review and for recognizing the practical value of our work.

 

We agree that our method builds on existing calibration and aggregation work. Our theoretical contribution is the finite-sample deferral analysis (Theorem 3 and Corollary 4), which shows when aggregation improves on a learned single detector.

 

We agree that the benefit of adding LLM judges depends on the dataset. Since Theorem 1 holds for any detector pool, one can start from the two free verification tools and add judges only when they reduce deferral, as on StatRef (Table 24).

Reviewer_2Ep9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to audit references generated by LLMs, where each reference is accepted, flagged as hallucinated, or sent to a human for checking. The authors combine the outputs of eleven detectors, including DOI rules, two verification tools, and seven zero-shot LLM judges, into one suspicion score, and set two thresholds on it so that the false positive and false negative rates are controlled with high probability, following PAC-Wrap. They also show that the smallest possible deferral rate is decided by the ROC curve of the score, and break the actual deferral down into an information term, a regret term, and an unused-budget term. Experiments on HallMark and a new StatRef benchmark show that the error guarantees hold, and that boosted stacking defers 2.3 to 3.9 times fewer references than the stronger of the raw and re-encoded single detectors, though the gains do not transfer across benchmarks.

Critical strengths
  1. The error guarantees are checked very carefully. Besides 1,000 random splits, the authors compare the observed failure rates with the exact ones on known populations, and the two match closely. This is useful because it leaves little doubt that the guarantees hold when the calibration and deployment data are similar.
  2. The paper is honest about its limitations. It runs a wide range of robustness tests, e.g., removing single detectors and transferring across benchmarks, and reports the unfavorable results as well. This is useful because readers can easily see where the method works and where it does not.
Critical weaknesses
  1. The major concern is that the technical novelty over prior work seems limited. The paper itself states that its calibration step follows PAC-Wrap, and that the method of Sadinle et al. (2019) is the same algorithm with a different ranking. The aggregator is also a standard gradient-boosted stacker. The paper lists a few differences from PAC-Wrap, e.g., a product bound instead of a union bound and tie-breaking, but they barely change the results in the experiments. In fact, the idea that guarantees come from calibration and efficiency comes from combining detectors is already common in conformal prediction. So the new part mainly comes down to the analysis of the deferral rate, and the method looks more like a combination of existing pieces.

  2. Another concern is that the practical gains are small and mostly tied to one benchmark. If flagged references still need human confirmation, the share of references reviewed on HallMark only drops from 61.4% to 60.0% compared with the best learned single detector, and a stacker using only the two verification tools, without any LLM queries, does about as well as the full pool. Besides, the gain depends heavily on one tool, and it does not transfer: when calibrated on HallMark and used on StatRef, the method violates the false positive tolerance in 98% of the repeated runs. In addition, most hallucinated references in both benchmarks are created by rule-based perturbations rather than real LLM errors, and the cost of the seven LLM judges is not reported. So it is hard to tell whether the method is worth its cost in practice. The authors are encouraged to address these issues.

Questions for authors
  1. Can the authors explain the technical contribution of the proposed method compared with PAC-Wrap and Sadinle et al. (2019)?
  2. Since the improvement is not very significant, what is the computational cost, e.g., API cost and running time, of the proposed method?
3 comments

Thank you for the careful reading. Changes are in blue in the revised PDF.

W1 / Q1. Contribution relative to PAC-Wrap and Sadinle et al.

PAC-Wrap. It calibrates an anomaly score to provide PAC guarantees on FPR and FNR. It reduces ambiguity by loosening error tolerances. We keep the tolerances fixed and contribute a finite-sample analysis of deferral under aggregation.

Sadinle et al. They provide the oracle and split-conformal class-specific coverage. Our theoretical contribution is the finite-sample deferral bound in Theorem 3, which separates the learned, PAC-calibrated rule's deferral into an information term, aggregation regret, and the unused budget. Corollary 4 gives the exact distribution of that budget.

Tie-breaking. Its effect is substantial for discrete scores. On HallMark, it reduces deferral from 50.5% to 39.9% for majority vote and from 13.9% to 8.8% for the two-tool stacker (Table 5, newly stated in Sec. 5.1).

 

W2. Practical gains depend on the dataset

We add a new StatRef-LLM (new Sec. 5.5) to show real LLM errors: 300 references that GPT-5.1, Claude Sonnet 4.6, and Gemini 2.5 Flash returned for 20 statistics topics.

The table below compares the percentage of references flagged+deferred for human review.

DatasetBest learned singleTwo-tool stackerFull poolSource
HallMark61.4%59.7%60.0%Table 6
StatRef61.5%60.2%46.8%Table 6
StatRef-LLM91.0%46.3%46.7%New Table 23

The benefit of additional LLM judges is data-dependent. On HallMark, all three configurations have similar total review burden. On StatRef, the full pool improves on both alternatives. On StatRef-LLM, aggregation helps, but the two tools already perform similarly to the full pool. These differences likely reflect error types and detector complementarity. HallMark already has a strong single detector. StatRef contains varied metadata errors for which LLM judges may provide complementary evidence. In StatRef-LLM, 85% of hallucinations involve DOI errors, which bibliographic tools can directly check.

Tool dependence. We acknowledge that the gains depend strongly on bibtex-updater, as shown by the detector-removal ablation (Fig. 3b). It detects most hallucinated references, while other detectors may help prevent valid references from being wrongly flagged (Table 3).

Transfer. Source thresholds can fail under distribution shift (Prop. 6), recalibrating on 200 target labels restores control of both error rates, with violation frequencies ≤ 0.096 at δ = 0.1 (Table 19).

 

Q2. API cost and running time

Verification tools dominate latency, while LLM judges account for API fees. We recorded API costs and running times on the 300 StatRef-LLM references (new Sec. 5.5 and new Table 24). The tools incur no API fees, but the slower tool takes a median of 51 seconds per reference. The seven judges together cost USD 8.94 per 1,000 references, with median response times ranging from 1 to 20 seconds.

On StatRef-LLM, the two-tool stacker performs similarly to the full pool, so tool runtime suffices without LLM fees (new Table 23). On StatRef, LLM judges help, but all seven are unnecessary: two tools plus Claude already perform similarly to the full pool. Claude's measured API cost is USD 3.78 per 1,000 references (new Table 24).

Dear Reviewer 2Ep9,

Thank you again for your thoughtful review.

  • We clarified our contribution relative to PAC-Wrap and Sadinle et al.
  • We added StatRef-LLM, a dataset of 300 LLM-generated references, to test performance on actual citation errors.
  • We compared human review workload (flagged+deferred references) across HallMark, StatRef, and StatRef-LLM: aggregation reduces it on StatRef and StatRef-LLM, while the benefit of additional LLM judges is data-dependent.
  • We measured API costs and running times on StatRef-LLM, and found that verification tools dominate latency, while LLM judges account for API fees.

We would welcome your feedback on whether these revisions address your concerns.

Best regards,
The Authors

Thank you for the detailed response and the additional experiments. The API cost and latency measurements answer my second question, and the total review-burden comparison and StatRef-LLM are useful additions.

My main concern remains the limited technical novelty. The response restates the finite-sample deferral analysis (Theorem 3 and Corollary 4), which I had already credited as the new part. The calibration and aggregation steps still follow PAC-Wrap, Sadinle et al. (2019), and standard gradient-boosted stacking. Moreover, the new results show that the two-tool stacker matches or outperforms the full pool on HallMark and StatRef-LLM, so the benefit of aggregating the seven LLM judges is limited to StatRef.

I therefore keep my score. That said, given the practical relevance of the problem and the careful empirical validation, I would not object if the AC decides to accept the paper.

Reviewer_WDPnOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper considers detecting hallucinated references by combining 11 tools from different perspectives. The method then trains a model to combine their outputs into a suspicion score. It then uses labeled calibration data to set two thresholds: references are accepted, flagged as hallucinated, or passed to a human for checking. Experiments on HallMark and StatRef show lower deferral rates than individual detectors.

Critical strengths

S1. The problem is insteresting for sure. Reference checkers can make mistakes in both directions, so controlling false accusations and missed hallucinations is useful. Allowing the system to flag uncertain cases also makes sense.

S2. The experiments compare against both raw detectors and models trained on a single detector’s outputs. I appreciate this comparison, since it helps separate the benefit of combining detectors from simply learning a better way to use one detector.

Critical weaknesses

W1. I find the main contribution limited. The calibration step is basically PAC-Wrap, and the result about the best possible scoring rule comes from earlier work, as the authors acknowledge. The new analysis explains how score quality affects deferral, but it does not show when the trained boosting model should improve on a learned single detector. Combining the detectors is generally fine, but I am not convinced that the paper adds enough beyond existing calibration and widely-used ensemble methods.

W2. My main concern with StatRef is how its labels are determined. Appendix B states that the labels follow the generation procedure rather than independent expert checking. Some changes, such as shortening an author list, can resemble ordinary bibliographic variation and need closer examination. The authors also did not retain the perturbation type for each reference, so we cannot check which types account for the improvement. The error guarantee only holds relative to the supplied labels. If those labels are questionable, calibration does not fix that. I would like to see an independent label audit and stronger evaluation on citations actually generated by LLMs.

Questions for authors

See W1-W2.

2 comments

Thank you for the review. Both concerns led to new material (blue in the revised PDF).

W1. Contribution, and when boosted stacking beats a learned single detector

PAC-Wrap. It calibrates an anomaly score to provide PAC guarantees on FPR and FNR. It reduces ambiguity by loosening error tolerances. We keep the tolerances fixed and contribute a finite-sample analysis of how aggregation affects deferral.

Sadinle et al. They provide the oracle and split-conformal class-specific coverage. Our theoretical contribution is a finite-sample deferral bound for the learned, PAC-calibrated rule. Theorem 3 bounds the deferral by an information term, aggregation regret, and the unused budget. Corollary 4 gives the exact distribution of that budget.

The added paragraph after Corollary 4 clarifies when boosted stacking helps: its information gain must exceed its additional aggregation regret, both averaged over the common distribution of calibrated error levels. This is a score-level criterion rather than a boosting-specific learning guarantee. Empirically, with the same total labeling budget and a 50/50 split between training and calibration, boosted stacking outperforms the best learned single detector from 200 total labels on HallMark and about 300 on StatRef (Fig. 3c). Majority vote and mean evidence do not reliably beat the best learned single detector (Table 5).

 

W2. Label reliability and LLM-generated references

  • Types. We recovered the per-record perturbation types from the generation log of StatRef (new Table 21). Stacking reduces deferrals on valid references (7% vs 29% for the best single detector) and on subtle types (near-miss titles 14% vs 36%, partial author lists 19% vs 39%).

  • Audit. The first author manually checked a stratified sample of 200 StatRef references, including all 42 partial author lists (new Table 21, new Sec. 5.5). The manual labels matched the original labels in 194 cases. Of the partial lists, 37 omit authors without an abbreviation marker, while five were unchanged by the generator but incorrectly labeled as hallucinated. One reference originally labeled valid contained an incorrect author name.

  • Sensitivity. After correcting the six labels, boosted stacking defers 4.4% vs 19.9% for the best single detector. Excluding all 42 partial author lists gives 3.5% vs 15.6%, and extrapolating the audited disagreement rates gives 14.7% vs 27.3% (new Table 22).

  • Citations generated by LLMs. HallMark already contains 155 LLM-generated and 80 real-world hallucinations (Table 16). We add a new StatRef-LLM (new Sec. 5.5): 300 references that GPT-5.1, Claude Sonnet 4.6, and Gemini 2.5 Flash returned for 20 statistics topics. Labels were assigned by an LLM agent that checked every field against Crossref, OpenAlex, and DOI records and stored the evidence, and the first author audited 100 labels and agreed with all. As Sec. 5.4 warns, thresholds from StatRef do not transfer, and recalibration on 200 of these labels restores control for every aggregator (violation frequency at most 0.074 at δ = 0.1). Boosted stacking then defers 14.3%, vs 62.7% for the best single detector (new Table 23).

Section 7 explicitly states the dependence on label quality and recommends manually verifying calibration labels.

Dear Reviewer WDPn,

Thank you again for your thoughtful review.

  • We clarified our contribution and when stacking reduces deferral relative to a learned single detector.
  • We manually audited 200 StatRef references to assess label reliability.
  • We added StatRef-LLM, a dataset of 300 LLM-generated references, to test performance on actual citation errors.
  • We added information on perturbation types in StatRef and sensitivity analyses of label changes.

We would welcome your feedback on whether these revisions address your concerns.

Best regards,
The Authors

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

Thank you for your careful reviews. As the discussion draws to a close, we would like to highlight three points from our responses. Revisions are marked in blue in the manuscript.

 

Contribution, and when boosted stacking beats a learned single detector. We build on PAC-Wrap's calibration and Sadinle et al.'s oracle. For PAC-Wrap, it calibrates an anomaly score to provide PAC guarantees on FPR and FNR, and reduces ambiguity by loosening error tolerances. We keep the tolerances fixed and contribute a finite-sample analysis of deferral under aggregation. For Sadinle et al., they provide the oracle and split-conformal class-specific coverage. Our theoretical contribution is the finite-sample deferral bound in Theorem 3, which separates the learned, PAC-calibrated rule's deferral into an information term, aggregation regret, and the unused budget. Corollary 4 gives the exact distribution of that budget. The added paragraph after Corollary 4 clarifies when boosted stacking helps: its information gain must exceed its additional aggregation regret.

 

Label quality and LLM-generated references. The first author manually audited 200 StatRef references. After correcting the six label errors found, boosted stacking defers 4.4% vs 19.9% for the best single detector (new Table 22). We also added StatRef-LLM, comprising 300 references generated by three LLMs. After recalibration, boosted stacking defers 14.3%, vs 62.7% for the best single detector (new Table 23).

 

Human workload, API cost, and running time. The table below compares the percentage of references flagged+deferred for human review.

DatasetBest learned singleTwo-tool stackerFull poolSource
HallMark61.4%59.7%60.0%Table 6
StatRef61.5%60.2%46.8%Table 6
StatRef-LLM91.0%46.3%46.7%New Table 23

The benefit of additional LLM judges is data-dependent. On HallMark, all three configurations have similar total review burden. On StatRef, the full pool improves on both alternatives. On StatRef-LLM, aggregation helps, but the two tools already perform similarly to the full pool. At the usage measured on StatRef-LLM, the seven LLM judges together cost $8.94 per 1,000 references, while the slower verification tool accounted for most of the latency (new Table 24).

 

We thank Reviewer 895o for the thoughtful follow-up and for highlighting the practical value of our work. We would be grateful if Reviewers WDPn and 2Ep9 could let us know whether these additions address their concerns, and we welcome any remaining questions from the Area Chair and reviewers.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.