acceptodds
Under review as a conference paper at ICLR 2027

HalluciBench: Benchmarking Citation Faithfulness in Large Language Models

Abstract

Large language models are increasingly used to draft scholarly text and assemble bibliographies, yet citation hallucinations have become a visible failure mode in the ML publication pipeline, with recent venue guidance and community scrutiny around NeurIPS, ICLR, and AAAI highlighting how incorrect or unverifiable references can propagate into submissions. We introduce HalluciBench, a benchmark for citation faithfulness where models must output a single BibTeX entry or abstain with REFUSE. HalluciBench contains a dataset of 6,000 records and five prompt variants ranging from exact-title requests to progressively noisier partial-metadata cues, enabling controlled analysis of when models retrieve faithfully versus guess. We evaluate outputs using a deterministic, field-level protocol covering parseability, author accuracy, title fidelity, venue, year, and DOI. Across evaluated models, citation formatting and factual correctness remain unreliable: BibTeX parseability ranges from 0.8 % to 99.8 %, year exact-match reaches only 0.6 % to 79.7 %, and author-set F1 varies substantially. Models also differ sharply in calibration, with correct refusal rates on no-answer prompts ranging from 1.0 % to 100 %. HalluciBench provides a transparent, reproducible platform for tracking progress toward verifiable citation generation and appropriately calibrated abstention.

Primary area:
datasets and benchmarks
Keywords:
Citation hallucination, large language models, bibliographic faithfulness, BibTeX generation, parametric recall, refusal calibration, benchmarking, scientific integrity

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_UFFfOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

HalluciBench gives models a paper title and asks them to produce one BibTeX entry or refuse. The dataset contains 1,000 real ML papers and 200 synthetic papers, with five prompt variants for each. Some variants also provide an author hint, the venue and year, or the first author and year. The paper tests 33 locally deployable models without search, RAG, or access to the papers themselves. It reports BibTeX parsing, metadata accuracy, and refusal scores.

Critical strengths
  1. The paper uses five prompts for the same paper, so readers can compare what happens when some author, venue, or year information is added.

  2. It reports format and individual metadata scores separately, which gives more information than a single overall accuracy score.

Critical weaknesses
  1. I find the practical relevance and advance over prior work insufficient.

    In actual citation work, a user who knows the paper title can consult the paper or look up its bibliographic record. This study removes access to those sources and asks local models to reconstruct the reference from memory. The authors explicitly justify this restriction as a way to isolate parametric recall and describe the benchmark as complementary to tool-assisted citation systems. I remain unconvinced that the insights from this restricted task establish sufficient practical value to warrant a new benchmark.

    The code link has expired, and I find the dataset contribution limited. The code link in the abstract currently returns HTTP 410 with repository_expired, so I cannot obtain the code or dataset through it. Existing resources address related but different needs: S2ORC is a scholarly corpus, ArXivBench evaluates research-paper and arXiv-link generation, and ArxEval includes jumbled- and mixed-title tasks. HalluciBench's exact-title-to-BibTeX task and matched no-answer controls are distinct. However, I find the evidence that this combination yields sufficiently new insights beyond existing citation-fabrication and bibliographic-recall studies unconvincing.

    The evaluation also leaves out current frontier proprietary models. It does not show how those commercial systems perform, so its value as a reference for current citation tools is limited.

    Several central questions have already been studied. Agrawal et al. examined fabricated references and author consistency. Niimi examined the relationship between citation count and recall, and differences in how bibliographic fields are remembered. HalluciBench cites both and adds a fixed title-anchored pool, matched refusal controls, prompt variants, and broader model comparisons. These additions provide a distinct evaluation setup, but I find the resulting analysis of failure mechanisms too limited to constitute a substantial advance over the earlier evidence on citation fabrication and uneven bibliographic recall.

  2. The analysis does not sufficiently disentangle failure causes and lacks answer-level joint error statistics.

    Much of this task depends on memory. A model may never have seen the paper. Even if it has seen the main text, it may not have seen the complete authors, final venue, or DOI. Even if it has seen those details, it may not remember them all or be able to put them into BibTeX. The two-year rule does not solve this: publication at least two years before a model's public release does not establish training-corpus inclusion. The authors acknowledge this, but different training coverage can still affect the model comparison. This is a problem with several possible causes, but the authors do not use enough experiments to separate them. The reported results still mix these causes and the resulting factual errors.

    The analysis of memory differences stops too early. Well-known papers may be remembered better, and different kinds of bibliographic information may be remembered differently. The paper groups results by citation count and year, but does not further examine when models are more likely to make up the remaining details, depending on which details they can already get right.

    The existing metrics need answer-level error breakdowns. The paper separately reports refusal, parseability, and field-level precision/recall/F1. Its worked example gives full author precision and low recall to “Vaswani et al.”, acknowledging correct but incomplete authorship. However, these summaries do not show how omissions and false additions coexist within individual answers. Non-parseable, non-refusal attempts also receive zero field credit; this measures output usability and does not establish that their factual content is false. Complete correct answers, correct but incomplete answers, mixtures of correct and false details, and answers with only false new details should be counted separately. This breakdown should identify answers that only copy the prompt and retain the paper's separate refusal and format-failure strata.

    Additional comparisons would clarify practical relevance and failure causes. The closed-book restriction is deliberate. To assess how far the findings extend to practical citation systems, I would like the same papers tested in four settings: title only; the full paper in context; internet search enabled; and RAG enabled. Under the same information, the model should also be asked to list the metadata directly and to generate BibTeX. This would help tell whether the problem is obtaining the information or writing the format. In the paper-context condition, only details actually present in the paper should count as supplied information. The existing partial-metadata prompts do not replace these comparisons.

    The synthetic papers raise a separate issue about the refusal labels. The authors did manual title checks and Semantic Scholar searches. However, more evidence is needed that the controls do not match real papers and resemble realistic citation mistakes. If a model returns a real paper with a similar title, that is a wrong match and should be distinguished from inventing a nonexistent paper. The paper also does not show whether the synthetic titles are easy to identify through style. These questions affect what the refusal scores mean.

Questions for authors
  1. Beyond the existing field-level metrics, among parseable citations that add metadata beyond the prompt, what proportions are completely correct, correct but incomplete, mixed correct/incorrect, or contain only incorrect new details? Please distinguish answers adding no new information and retain the existing separate refusal and format-failure strata. Does the proportion of false remaining details change depending on which details were correctly recovered?

  2. Can you provide the full-paper, internet-search, and RAG comparisons described above, at least on a representative subset, and compare direct metadata output with BibTeX?

  3. Were model outputs pointing to real papers with similar titles checked, and how were possible collisions and stylistic cues in the synthetic controls assessed?

Reviewer_wjLxOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper has developed a benchmark called HalluciBench for evaluating the generation of bibliographies. This benchmark provides the titles of papers to the models, sometimes along with the authors' names, years, etc., allowing the models to generate BibTeX references without the ability to search the internet. It checks whether the fields such as authors, publication years, conferences, and DOI are correct. At the same time, it observes whether the models will reject fabricated papers. The authors collected 1,000 real papers and 200 synthetic paper records, and evaluated 33 models. The results showed that even if the models could generate correctly formatted references, the authors and identifiers in them might still be incorrect; some models reduced fabrication by frequently rejecting responses, but they also rejected a large number of real paper requests.

Critical strengths

Evaluating the accuracy of fields such as BibTeX format, author and year, as well as the behavior of refusing to answer, can help distinguish between format errors, factual errors, and unwillingness to answer.

The real papers and synthetic negative examples use matching five types of prompts and cover 33 models. This allows us to observe how the quality of the model's responses and the behavior of refusing to answer change when different metadata is added.

Critical weaknesses
  • After time filtering, the sets of papers used by different models are different, and the field scoring has excluded some rejected response samples. Therefore, high scores may only represent a small portion of the questions that the models are willing to answer; currently, there is a lack of overall results on a common sample set.

  • Key experimental configuration and original records are missing. The papers admit not having saved complete model weight versions, quantization methods, actual reasoning parameters, and individual response records. This affects the result verification and makes it difficult to distinguish whether the low scores are due to model capabilities or running configuration.

  • The prompt allows returning DOI or stable URL, but the main identifier indicators only look for exact DOI matches. Correct URLs, missing DOIs, and incorrect DOIs may all be recorded as zero. The current results are insufficient to distinguish between legitimate alternative answers and true identifier hallucinations.

  • Real entries are mainly automatically converted from Semantic Scholar metadata, and the results of cross-verifying with the conference website or publisher records are not reported. There is also a lack of systematic processing for the differences between preprints and formal published versions, which may confuse model errors with source data or version differences.

Questions for authors

Improve his ability in drawing.

Reviewer_yUEvOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes HalluciBench, a benchmark for citation faithfulness where models must output a single BibTeX entry or abstain with REFUSE. HalluciBench includes 6,000 records with five prompt variants ranging from exact-title requests to progressively noisier partial-metadata cues. Its fixed pool contains 1,000 real machine-learning papers and 200 synthetically constructed no-answer controls. The papers are collected from Semantic Scholar. During evaluation, models had no internet access, bibliographic search, retrieval system, external tools, function calls, RAG, agentic loop, or communication with other models. It tests 24 checkpoints in the LLM suite and 9 in the SLM suite.

Critical strengths
  1. The paper includes both real-paper and no-answer records. The paper fixed the title to improve ablation with five conditions. Each setting also distinguishes between providing additional information and requiring additional output. The research topic is interesting. The paper also provides implementation details.
  2. The experiment provides useful analysis. It tests on very broad current language models. The results separate syntactic success from factual correctness. For example, Gemma 3 27B achieves 99.8% parseability but only 15.6 author F1. Qwen 2.5 7B refuses every synthetic request and 81.7% of real requests. This analysis is very useful for the real world. DOI scoring excludes the 444 papers without valid gold metadata for better accuracy.
Critical weaknesses
  1. The dataset construction has some problems. The main scores are not directly comparable across models. The paper said that temporal eligibility is applied per model at evaluation time. Models are evaluated on different temporally eligible paper subsets as shown in Table 3. Explicit refusals are excluded from field-level scoring. Based on this, it seems that different models are not comparable.
  2. The paper said "the resolved values, runtime version, and immutable weight identifiers were not archived". It also said immutable weight digests, instruction/base variants, or quantization were not preserved in the archived execution manifest". This significantly reduces reproducibility. The whole paper also reads like AI written.
Questions for authors
  1. What is the subset size or the per-model denominator for Table 1?
  2. What is the quality of generated No-Answer records?Is there any human quality check>

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.