acceptodds
Under review as a conference paper at ICLR 2027

HALT: Held-Out-Source Evaluation for Hallucination Detection in Forensic Video Reporting

Abstract

Hallucination detectors for long-form multimodal generation are almost always evaluated on text from the same source LLMs they were trained on. In forensic video reporting we show this conflates detection with identifying which LLM wrote the text. A detector that silently stops working when the underlying model is updated offers false assurance, in a setting where inventing a detail and omitting a crime are two different evidentiary failures. We build HALT (Hallucination Across LLMs and Techniques) on a previously released corpus of 19,361 forensic video reports (21.8M words) in which three frontier multimodal LLMs each describe all 807 surveillance videos, spanning eleven crime categories, under all eight prompting strategies, labelled for six hallucination types by a cross-judging panel in which no model evaluates its own output. Our central result is that much of what a detector appears to know is the identity of the model that wrote the report: a predictor reading no report text at all, returning only the writing model’s base rate, captures 70 to 73% of a detector’s above-chance signal on the fabrication labels against 13% on omission. Withholding a source LLM accordingly costs the fabrication labels 0.104 to 0.191 AUC against 0.035 to 0.105 for omission and distortion, consistently across four detector families from bag-of-words to a fine tuned long-context encoder, and again when a source LLM is updated rather than replaced, though the tendency being detected is unchanged across that update. Two further findings concern the panel’s H1–H6 labels: (i) report length predicts the panel’s scene-fabrication label at AUC 0.727 but the consensus of two human annotators who each independently labelled the same 130 reports at only 0.566, and (ii) models are not allowed to judge their own output, so each source LLM is scored by a different pair of judges; those pairs differ in how often they agree, so comparing hallucination rates across models partly compares their judges. We release the reliability tiers, the four splits, and reference implementations of every detector.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
hallucination detection, held-out-source evaluation, shortcut learning, multimodal large language models, LLM-as-judge, benchmark, distribution shift, forensic video analysis

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_AiSXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

HALT introduces held-out-source evaluation to test whether hallucination detectors for forensic video reports transfer to unseen generating models, and examines reliance on source identity, report length, and the labeling mechanism. It builds on an existing corpus of 807 UCF-Crime videos and 19,361 reports from three generators and eight prompting strategies, adding splits, detector comparisons, reliability tiers, and diagnostic analyses. TF-IDF ANY AUC falls from 0.820 in standard evaluation to 0.566 under source holdout, with larger degradation for fabrication labels than for omission and distortion. Strict video-disjoint evaluation, human agreement, length associations, and judge-pair confounding provide further context. The main contribution is an evaluation and diagnostic framework, rather than a wholly new corpus or generation model.

Critical strengths
  1. The evaluation question matters to the community. When the same generators appear on both sides of a split, high scores do not establish transfer to an unseen generator. HALT combines source holdout, a source-base-rate control, and type-specific transfer analysis into a reusable diagnostic protocol. The fall in TF-IDF ANY AUC from 0.820 to 0.566 makes the concern concrete and testable. Its value does not depend on proposing a complex new model.
  2. The experiments and self-audit provide useful evidence. The crossed corpus enables comparisons across videos, generators, and prompting strategies; linear, reference-conditioned, and fine-tuned encoder detectors show a consistent pattern of cross-source degradation. Fabrication AUC drops by 0.104–0.191, versus 0.035–0.105 for omission/distortion, showing what aggregate scores can hide. The authors disclose human agreement, aggregation sensitivity, and judge-pair confounding, and release splits and detector implementations; the existing per-judge verdicts also enable reanalysis. These candid limitations and reusable resources strengthen the contribution.
Critical weaknesses
  1. Label validity and source–judge confounding limit the core interpretation. In Table 1, H1 panel–human κ is 0.150, while H5/H6 human–human κ is only 0.082/0.192. Higher panel–consensus agreement is calculated on a subset where the humans already agree and therefore does not establish reliability across all reports. MACRO-HQ excludes only H1 and retains H5/H6 despite weak human agreement. More fundamentally, preventing self-evaluation assigns a different judge pair to each generator; H1 pairwise κ values are 0.850, 0.415, and 0.148. Source holdout thus changes both report distributions and the label mechanism, preventing separation of factual errors, writing style, and judge preferences. The authors acknowledge this and appropriately qualify the result as degradation of “fabrication labels,” but the identifiability limitation remains. In particular, Section 4.1’s assertion that a shared bias leaves relative source comparisons unaffected needs evidence that the bias is source-invariant. This does not invalidate the observed degradation in label transfer, but it limits interpretation as a benchmark for factual video hallucination detection. Narrower claims, per-judge and aggregation analyses, and results excluding H1/H5/H6 are feasible revisions. Stronger factual claims require fixed external judges or a stratified sample independently annotated from the videos. This is a structural limitation of the labeling design; adding detectors alone cannot resolve it.
  2. Evidence for strict generalization and uncertainty remains incompletely reported. The main source-holdout experiment deliberately shares videos to hold the input fixed, but it does not directly represent deployment on both a new generator and new videos. Appendix D already provides strict video-disjoint corroboration: TF-IDF fabrication/omission/ANY AUC is 0.668/0.780/0.559. The concern is not absence of this check, but absence of complete results across detectors, types, folds, and training settings. Each video yields up to 24 correlated reports, yet the bootstrap resampling unit is unspecified. Independent report-level resampling could produce overly narrow intervals; the manuscript alone does not establish that this error actually occurred. Complete strict-split results, paired bootstrap intervals clustered by video, and variation across training seeds would establish the stability of the type differences and detector ranking. The cross-generation pilot has only 15 videos and 360 reports, with generators and judges changing together, so it offers exploratory corroboration rather than a standalone basis for broad cross-generation claims. Most issues here are addressable through reporting and reanalysis; the pilot’s representativeness requires a larger sample with controlled judges.
Questions for authors
  1. Labels and confounding: Can you report per-judge metrics and cross-source differences under AND/OR aggregation within matched video–prompt conditions, with sensitivity results excluding H1 alone versus H1/H5/H6? Do fixed external judges or independent video annotations support the ranking? If unavailable, will you explicitly limit the claims to transfer under this labeling pipeline and revise the assertion that relative comparisons are unaffected?
  2. Strict splits and statistics: Please report fold-level and type-level AUC, sample sizes, and class prevalence for all detectors with both source and video disjointness, and explain video grouping in training/validation/test. Is bootstrap resampling performed over reports or videos? Please provide paired difference intervals clustered by video, and clarify seeds, hyperparameter selection, and reference-text isolation across folds.
  3. The “70–73%” interpretation: (AUC_base − 0.5) / (AUC_tfidf − 0.5) is a descriptive ratio of above-chance AUC, not a causal contribution or explained-variance decomposition. Can you provide paired confidence intervals for the ratio, test sensitivity to fold/type aggregation, and adjust the “signal share” wording accordingly?
  4. Length association: On H1, length predicts panel and human labels at AUC 0.727 and 0.566. Does the difference persist after controlling for source, video, prompting strategy, and truncation? Can you report paired uncertainty on the same human-annotated sample? This determines whether the evidence supports a length association or a stronger attribution to judge length bias.
  5. Reproducibility and comparisons: Please complete model versions, epochs, seeds, validation selection, truncation rules, and tuning procedures for the strict splits. The existing external zero-shot Qwen detector is a useful control. Any added baseline should match report/reference inputs and available generation information; semantic entropy requiring multiple sampled generations is not directly comparable to a single-report detector. PR-AUC, calibration, and false-positive rates at fixed recall are useful future additions, not independent grounds for rejection. The cross-generation pilot should retain its exploratory framing; a larger sample with fixed judges is future work.
3 comments

Part 1 of 2

We thank the reviewer for an exceptionally careful review. We agree with the central point. Because no model judges itself, holding out a source also changes who labels it, so the result must be tested with the labelling held fixed. We ran those tests, and where the effect shrank, we narrowed our claims. (The two terms we use below, the fabrication asymmetry ratio and the base-rate control, are defined in our general comment.)

W1 / Q1: Labels and source-judge confounding

"Source holdout thus changes both report distributions and the label mechanism... Section 4.1's assertion that a shared bias leaves relative source comparisons unaffected needs evidence that the bias is source-invariant." "Do fixed external judges or independent video annotations support the ranking?"

  • A correction first: Table 1's panel column used a different combination of judge votes from the released labels used in every experiment. On the released labels, panel-human agreement (Cohen's κ) is 0.173 (H1), 0.631 (H2), 0.495 (H3), 0.622 (H4), 0.564 (H5) and 0.588 (H6). As the reviewer anticipated, H5 and H6 move from the high to the medium tier.
  • Agreement on all reports (Appendix I): The reviewer is right that agreement measured only where the two raters agree is an upper bound. Against each rater separately, on all 130 reports, the panel's κ is 0.19 for H1, 0.28 for H5 and 0.37 for H6, compared with 0.44 to 0.47 for the other types. On H5 and H6 the panel agrees with each rater at least as well as the two raters agree with each other (0.08 and 0.19). Still, on this measure all three fabrication types are weakly labelled, so every headline result is also reported without H1, H5 and H6 (below).
  • Test 1, the same judge for training and testing (Appendix F): Each judge labels exactly two source LLMs. We train a detector on one judge's labels for one source and test it on the same judge's labels for the other source. Averaged over the six such pairs, AUC drops 0.128 on fabrication, 0.090 on omission and 0.102 on distortion. The ratio is 1.25 [1.08, 1.47], compared with about 3 when the judges change. So part of the asymmetry comes from the change of judges, and the revision says so.
  • Test 2, one outside judge for everything (Appendix F): Qwen3.8-27B, from a model family not in the study, relabelled 2,419 reports using the panel's instructions: one prompting strategy for each of the 807 videos, with the reports of all three source LLMs. With these labels, holding out a source costs fabrication 0.171 [0.153, 0.189] AUC, against 0.105 for omission and 0.102 for distortion. The ratio is 1.63 [1.38, 1.82], compared with 2.47 [2.13, 2.87] using the panel's labels on the same reports. The base-rate control still reaches 51 to 65% of TF-IDF's above-chance AUC on the fabrication types, against 21 to 25% on the others.
  • Does the ranking of source LLMs hold? Partly. Under the outside judge, GPT still fabricates much less than Claude and Gemini (by 0.358 [0.33, 0.38] and 0.357 [0.33, 0.39]) and omits more, as under the panel. But Claude and Gemini become indistinguishable on fabrication (difference 0.001 [−0.03, 0.03]), whereas the panel separates them by 0.105. So "GPT differs from the other two" holds under both labellings, while the Claude-versus-Gemini order comes from the judges. Qwen is consistent across sources, not more accurate: it agrees with the human raters less than the panel does on five of six types. We use it only to test whether the result depends on the judges.
  • Per judge, and AND versus OR (Appendix F): Across all 6,449 video-and-strategy combinations where all three sources wrote a report, GPT is the least fabricating and most omitting source under the AND rule, the OR rule and every individual judge. The held-out-source ratio is the same under AND (3.24) and OR (3.19).
  • Without H1, and without all three fabrication types (Appendix I): Holding out a source lowers the average AUC by 0.100 [0.086, 0.114] without H1, and by 0.056 [0.035, 0.077] without H1, H5 and H6.
  • Test 3, human labels only (Appendix J): In RAGTruth, six LLMs answer the same prompts and people label every response, so no LLM judge is involved. The base-rate control reaches 69% [62, 78] of TF-IDF's above-chance AUC on baseless information (the analogue of fabrication) and 57% [50, 65] on conflicts. Holding a generator out costs baseless detection 0.061 AUC and conflict detection 0.029. The direction matches our corpus, but the difference is smaller and its interval just reaches zero (−0.032 [−0.067, 0.003]).

(continued in Part 2)

Part 2 of 2

W1 / Q1 (continued)

  • Where the asymmetry does not appear: The judges read at most 3,000 characters of each report. On the 6,712 reports short enough to be read in full, the asymmetry disappears (ratio 1.10 [0.65, 1.54]). This subset is 68% GPT, so it does not prove that truncation causes the asymmetry, but it is consistent with the reviewer's concern, and we report it.
  • What we changed in the claims: We removed the Section 4.1 sentence saying a shared bias leaves comparisons between sources unaffected. The main result is now stated for this labelling pipeline, with the same-judge ratios reported beside the original.

W2 / Q2: Strict splits and statistics

"Is bootstrap resampling performed over reports or videos? Please provide paired difference intervals clustered by video, and clarify seeds, hyperparameter selection, and reference-text isolation."

  • Bootstrap: The submission resampled reports, so the reviewer's concern was correct. All intervals now resample whole videos, with the same resamples for every comparison. The main gap becomes −0.134 [−0.159, −0.110] (submission: [−0.148, −0.121]).
  • Strict splits, all detectors (Appendix D): Here training and test share neither a source LLM nor a video. We train on the other two sources' training videos (about 9,000 reports) and test on the held-out source's test videos (about 1,015 reports from 127 videos). The ratio is 2.33 [1.97, 2.79] for TF-IDF, 1.75 [1.45, 2.14] for DeBERTa (1.70 across three seeds, SD 0.05) and 1.69 [1.45, 2.00] for ModernBERT. Appendix D tabulates AUC and prevalence for every held-out source and type.
  • Seeds and settings (Appendices D and K): Across three seeds, DeBERTa's fabrication AUC on the standard split is 0.879 (SD 0.003). The encoders use one fixed training schedule, with no model selection on validation data and no tuning on any test fold. A second ModernBERT seed showed that one failed fold in the first run was a one-off.
  • Video grouping: The standard split puts 563 videos in training, 117 in validation and 127 in test, with all 24 reports of a video in the same part. Five other random video splits change the results by an SD of at most 0.017.
  • Reference-text isolation (Appendix K): Each expert reference belongs to one video. Under the standard and strict splits, no test reference is seen in training. Under held-out source, the video and its reference are intentionally the same and only the source LLM changes; the strict split removes even that overlap, and the asymmetry remains.
  • TF-IDF vocabulary (Appendix I): The vocabulary is built from all report text, without labels. Building it from training text only leaves every result pointing the same way (ratio 3.24).

Q3: The "70 to 73%" interpretation

"A descriptive ratio of above-chance AUC, not a causal contribution or explained-variance decomposition."

Agreed. It is a descriptive ratio, not a causal share. We renamed it the base-rate fraction, removed "signal share", and report all six types with intervals: H1 73% [68, 77], H5 70% [65, 76], H6 45% [40, 51], H2 35% [26, 43], H4 31% [24, 37] and H3 13% [7, 19].

It is also stable (Appendix I). Across six different video splits it changes by an SD of at most 0.043. Computed per error group instead of per type, it is 63% [59, 67] for fabrication, 33% [27, 38] for distortion and 13% [7, 18] for omission, the same order.

Q4: Length association

"Does the difference persist after controlling for source, video, prompting strategy, and truncation?"

Partly, and not once the video is controlled. On the released labels, report length predicts the panel's scene-fabrication label with AUC 0.651 and the human label with 0.566 (130 reports from 62 videos). The difference is 0.085 [−0.13, 0.30] as tabulated, and 0.007 within each source LLM. Controlling for source, prompting strategy and truncation, it is 0.264 [0.03, 0.48]. Severity minimization shows a difference both before (0.178 [0.03, 0.33]) and after (0.156 [0.04, 0.31]) these controls.

When we also control for video, comparing only reports of the same video, the difference disappears: 0.105 [−0.11, 0.31] for scene fabrication and 0.066 [−0.09, 0.24] for severity minimization. So the link with length runs between videos, not within one. Section 4.6 now reports a length association, not a judge bias.

Q5: Reproducibility and comparisons

Appendix K lists the model versions, epochs, seeds, model selection, truncation and tuning for every detector. We keep the zero-shot Qwen detector as the outside control and add no baseline that needs inputs this task lacks. PR-AUC, Brier score, calibration error (ECE) and the false-positive rate at 80% recall are in Appendix I. The cross-generation pilot is still labelled exploratory and is no longer in the abstract.

Thank you again. In a final update to the revised PDF:

  • Strict splits for all detectors (Appendix D, Table 11): we added the remaining linear detectors. The asymmetry holds for each, with every ratio interval above one: surface 1.40 [1.16, 1.66], surface-noid 1.34 [1.04, 1.67], grounded 1.77 [1.29, 2.47]. The appendix also explains the three configurations not in the strict tables (the prompted detector is never trained, so a strict split changes only its test set; the embedding features are not part of the released corpus; ModernBERT at 2,048 tokens was not rerun with saved predictions).
  • A correction: running these showed that the "surface" row of the held-out-source feature table reported one fold rather than the three-fold mean. It is corrected (0.616 / 0.507 / 0.546), with no effect on any conclusion.
  • Independent video annotations: the limitations now state explicitly that no new labels were collected from the videos. We answer this question with the fixed external judge, the existing 130 human-labelled reports and RAGTruth's human labels.
Reviewer_4ntNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that hallucination detectors for long-form multimodal generation are predominantly evaluated on outputs from the same source LLMs used during training, potentially conflating hallucination detection with recognition of the generating model. Using a released corpus of 19,361 forensic video reports covering 807 videos, generated by three frontier MLLMs under eight prompting strategies and annotated for six hallucination types through cross-judging without self-evaluation, it introduces HALT: a protocol combining held-out-source evaluation with a text-blind baseline that predicts solely from the source model’s label prevalence. For fabrication labels, this baseline recovers 70–73% of the full detector’s above-chance AUC, compared with just 13% for omission. Holding out a source LLM reduces fabrication-detection AUC by 0.104–0.191, versus 0.035–0.105 for omission and distortion, with a consistent pattern across four detector families. Further analysis shows that report length alone predicts the panel’s scene-fabrication labels with an AUC of 0.727, but achieves only 0.566 against independent human annotations. Finally, excluding self-evaluation assigns each source LLM a different pair of judges, whose inter-judge agreement varies by more than fivefold.

Critical strengths
  1. The problem is real, important, and genuinely underexamined. SelfCheckGPT, semantic entropy, and FActScore are evaluated within a single source LLM or across several pooled without a held-out condition. The argument that model families differ sharply in both surface style and hallucination rate, so a detector can score well by recognizing the family and applying its base rate, is clear and correct — and invisible under a random split.
  2. The base-rate control is an exceptionally clean diagnostic. A predictor reading no report text and returning only the source LLM's training-set prevalence must derive all signal from source identity, converting "how large is the shortcut" from an inference into a measurement. Combined with a TF-IDF source classifier reaching 1.000 accuracy against a chance rate of 0.333, the argument is complete. 3.The delineation from the prior study is explicit, with a dedicated paragraph listing what is inherited and what is new, and noting that the prior work itself names the absence of an out-of-distribution test among its limitations.
Critical weaknesses

1.The three-source limitation runs deeper than acknowledged, and the bootstrap does not cover source-level uncertainty.The bootstrap resamples reports, not source LLMs, so [−0.148, −0.121] characterizes sampling uncertainty conditional on these three models and does not propagate variation from the choice of model family. Relatedly, the ratio in Table 2 varies from 1.4 to 3.1 across detectors (3.13, 2.42, 1.68, 1.38), and across-fold variation cannot be separated from across-detector variation without a variance decomposition. The paper's primary phrasing concerns detector families, which is appropriate; any inference toward model families in general should state the conditional nature of the interval. 2. The "70 to 73%" figure omits H6 (45%), which belongs to the same axis, and this conflicts with Section 4.3. Table 3 reports fabrication-axis shares of 73% (H1), 70% (H5), and 45% (H6) — a range of 45–73%. The abstract states "70 to 73% ... on the fabrication labels," and Section 4.4 discusses only H1 and H5. This conflicts with Section 4.3, which — after dropping H1 — states that "Entity fabrication and phantom actors alone carry the effect," i.e. the conclusion rests on H5 and H6. H6 is thus excluded from the share discussion in one place and treated as a principal carrier in another. 3.Appendix H is numbered twice while the body cites it for both purposes.

Questions for authors

1.Could the held-out-source experiment be rerun on the untruncated subset? What is the truncation rate over the full corpus? 2.Could you report within-source AUC?

1 comment

We thank the reviewer for a precise review. All three weaknesses are correct. Both questions led to new results, and one of them narrows our claim. (The fabrication asymmetry ratio used below is defined in our general comment.)

W1: Source-level uncertainty and the ratio's variation

"The bootstrap resamples reports, not source LLMs, so [-0.148, -0.121] characterizes sampling uncertainty conditional on these three models... across-fold variation cannot be separated from across-detector variation without a variance decomposition."

  • Uncertainty: We now resample whole videos instead of reports, with the same resamples for every comparison. The main gap becomes −0.134 [−0.159, −0.110], about 1.8 times wider, and still excludes zero. With only three source LLMs, resampling the LLMs themselves is not meaningful, so we report each held-out source separately and state that the intervals apply to these three LLMs.
  • Each held-out source separately: For TF-IDF, the ratio is 2.90 [2.00, 5.13] with Claude held out, 2.70 [1.78, 5.12] with GPT and 2.84 [2.14, 4.02] with Gemini. Every interval excludes 1.
  • The decomposition you asked for: We split the variation in the 12 ratios (four detectors × three held-out sources). Which source is held out explains 45%, which detector 20%, and their interaction 35%. This exposed an exception that the averages hid. For the fine-tuned encoders, the asymmetry appears when GPT or Gemini is held out, but not when Claude is. With Claude held out, fabrication AUC barely changes (+0.011 [−0.003, +0.023] for DeBERTa, −0.026 for ModernBERT), while omission and distortion drop by 0.04 to 0.13. This holds in every seed and in the strict split. With GPT or Gemini held out, every encoder interval is above 1 (DeBERTa 3.31 and 4.46).
  • What we changed: "Consistently across four detector families" now reads "in every fold for the lexical detector and in two of three folds for the encoders." We also found that the size of each drop tracks how far the held-out source's hallucination rate is from the training sources' rate (Spearman 0.51, p = 0.031, across 18 source-by-type cells). That is what a base-rate shortcut predicts.

W2: The 70 to 73% figure omits H6

"Table 3 reports fabrication-axis shares of 73% (H1), 70% (H5), and 45% (H6), a range of 45-73%... H6 is thus excluded from the share discussion in one place and treated as a principal carrier in another."

Correct. All three fabrication types are now reported everywhere, with intervals: H1 73% [68, 77], H5 70% [65, 76] and H6 45% [40, 51], against 13% [7, 19] for omission (H3). The abstract now says 45 to 73%.

H6 fits the mechanism rather than contradicting it. Across the seven targets in Table 3, the types where the base rate explains more are also the types that degrade more when a source is held out (Spearman 0.82, p = 0.024). Among the fabrication types, H6 is lowest on both. Section 4.3 now says the asymmetry persists without H1 but is weaker, and is carried more by H5 than by H6.

W3: Appendix H numbered twice

Fixed. Held-out generation is now its own appendix (Appendix M), and the judge-pair material moved to Appendix E.

Q1: Untruncated subset and truncation rate

"Could the held-out-source experiment be rerun on the untruncated subset? What is the truncation rate over the full corpus?"

65.3% of the 19,361 reports are longer than the judges' 3,000-character limit: 29.4% of GPT's, 76.3% of Gemini's and 90.4% of Claude's. This also corrects a sentence in Section 4.1: Gemini writes the longest reports, but Claude's are truncated most often.

On the 6,712 reports that were not truncated, the asymmetry does not appear. Fabrication, omission and distortion drop by similar amounts (0.110, 0.098 and 0.100), a ratio of 1.10 [0.65, 1.54]. We read this with caution in both directions. The subset is 68% GPT, with only 622 Claude reports, so it cannot prove that truncation causes the asymmetry. But it is consistent with the fabrication labels depending on how much of each report the judge saw. We added it to Section 4.6 and the limitations.

Q2: Within-source AUC

Added (Appendix I). Training and testing within a single source LLM, with no shared videos, gives AUC 0.793 for fabrication, 0.849 for omission and 0.786 for distortion. Compared with pooling all three sources, fabrication loses 0.083 [0.071, 0.097] and omission only 0.005 [−0.001, 0.012]. So when the writer cannot vary, fabrication detection loses signal and omission does not. Each source always has the same pair of judges, so this also holds with the judges fixed.

Reviewer_zzjsOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper is motivated a problem in hallucination detection, which is: a detector stops working when it does not know the underlying model. The real significance of this is that in detector applications, if a new model is released which writes the texts, the detector would not be able to detect them. The paper's central result is that knowing the model recovers most of the hallucination detection. Extensive experiments are done across different detectors, including hand-built features with logistic regressions.

Critical strengths
  1. the paper considers a wide range of methods, making me feel that authors have put in much work and thought
  2. the analysis section of this paper is very comprehensive and trying to address the research questions well
Critical weaknesses
  1. the critical weakness for this paper is presentation. I felt the abstract is written in a very confusing way with tons of words not defined. Moreover Figure 1 just feels AI-generated without much human feedback. There's no point in displaying the contrast between your setup and the pipeline you inherited, because that should not be the main focus of this figure. Also i don't feel that I have seen any single qualitative example that helps me understand the problem.
  2. how did you come up with the hallucination taxonomy? was there any prior literature support? Also all the experiments were done on one dataset, which weakens the result validity.
Questions for authors

I don't really understand how the logistic regressors connect with the paper's story - because the intro seems to be motivated by LLM detectors. Can you explain how these fit coherently with the main motivation.

2 comments

We thank the reviewer for the time spent on the paper, and we agree that the presentation was the main obstacle. We have revised it substantially.

What the paper shows, in brief: Hallucination detectors are usually trained and tested on reports from the same LLMs. We find that a large part of what they detect on fabrication is simply which LLM wrote the report. A predictor that never reads the report, and only returns the writer's usual hallucination rate, reaches 45 to 73% of a trained detector's performance above chance on the three fabrication types, against 13% on omission. When the writer is left out of training, this shortcut stops working, and fabrication detection drops the most.

W1: Presentation

"The abstract is written in a very confusing way with tons of words not defined. Moreover Figure 1 just feels AI-generated... I don't feel that I have seen any single qualitative example."

  • Abstract: Rewritten so that every term (source LLM, held-out source, base-rate control, the three error groups) is explained in plain words where it first appears.
  • Figure 1: We agree. The new Figure 1 shows the problem directly: one video, reports from three LLMs, a detector trained on two and tested on the third, and how much its accuracy changes for each error group. The comparison with the earlier pipeline is removed.
  • A worked example (Appendix L): We walk through one video (Robbery027, chain-of-thought prompting): the expert's description, the three LLM reports and their labels. Claude's report contains a fabrication. When Claude's reports are in training, the detector scores this report 0.93 (likely hallucinated). When Claude is held out, it scores 0.24, below the average for that fold's reports with no fabrication. Across that fold, reports with and without fabrication average 0.55 and 0.52, almost indistinguishable, compared with 0.93 and 0.23 when Claude is in training.

W2: The taxonomy and the single dataset

"How did you come up with the hallucination taxonomy? Was there any prior literature support? Also all the experiments were done on one dataset."

  • Taxonomy (Section 3.1): The six types come from the study that released the corpus, and each matches an established category in the literature. Entity fabrication (H5) is object hallucination, as measured by CHAIR (Rohrbach et al., 2018) and POPE (Li et al., 2023). Scene fabrication (H1) and phantom actors (H6) are extrinsic hallucinations, meaning content the video does not support, as in VideoHallucer (Wang et al., 2024). Crime misclassification (H2) and severity minimization (H4) are intrinsic hallucinations, meaning content that misstates what the video shows. Crime missed (H3) is an omission, which ARGUS evaluates separately (Rawal et al., 2025). The forensic setting adds one thing: the types are grouped by consequence, because inventing a detail, misstating an act and missing a crime are different failures for evidence.
  • A second dataset (Section 4.4, Appendix J): Our question needs a dataset in which several LLMs describe exactly the same inputs, so that leaving one out changes nothing else. No video hallucination benchmark has this. We therefore added RAGTruth (Niu et al., 2024), where six LLMs answer the same 2,965 prompts and people label every response. The shortcut appears there too: knowing only which LLM wrote a response reaches 69% [62, 78] of TF-IDF's above-chance AUC on baseless information and 57% [50, 65] on conflicts. Leaving one LLM out of training costs baseless detection 0.061 AUC, against 0.029 for conflicts. That is the same direction as in our corpus, though smaller.

Q: How do the logistic regressions fit the story?

"The intro seems to be motivated by LLM detectors. Can you explain how these fit coherently with the main motivation."

In the paper, a "hallucination detector" is any model that reads a report (and optionally the expert reference) and outputs a hallucination score. It does not mean a detector built from an LLM. Our question is whether such detectors still work on reports from an LLM they were not trained on, and that applies to every kind of detector. We include logistic regressions on purpose, for three reasons:

  • They are what the prior study used, so we can show that its conclusion does not hold for an unseen LLM.
  • Their features are interpretable, so we can see what carries the signal. The base-rate control is the simplest case: a single feature.
  • To show that the result is not caused by weak models, we repeat it with fine-tuned encoders (DeBERTa-v3, ModernBERT) and a prompted 27B LLM.

Thank you again. Following your questions, the revised PDF now states both points in the paper itself, not only in our reply:

  • Taxonomy (Section 3.1): we now explain first why the six types form three groups (fabrication adds content absent from the footage, omission fails to report the criminal act, distortion misstates an act that is reported), and then how each type maps to the established categories.
  • Logistic regressions (Section 3.3): the section now opens by stating that we do not propose a new detector; the detectors are reference instruments for testing whether apparent detection depends on source identity, running from fully interpretable (logistic regressions, where the base-rate control is the one-feature case) to high-capacity (fine-tuned encoders).

Comments on the paper

1 comment

We thank the reviewers for their careful reviews. We have uploaded a revised PDF. The reviews raised four shared concerns, and we address them below.

Two terms are used throughout our replies:

  • Fabrication asymmetry ratio: When a detector is tested on reports from a source LLM it was not trained on, it loses AUC on every hallucination type. The ratio divides the fabrication loss by the larger of the omission and distortion losses. A ratio above 1 means fabrication detection suffers most.
  • Base-rate control: A predictor that never reads the report. It only knows which LLM wrote the report and returns that LLM's hallucination rate in the training data. We report its score as a share of a trained detector's score above chance: (control AUC − 0.5) / (detector AUC − 0.5). A high share means a detector can do well simply by recognising the writer.
  1. Uncertainty (AiSX, 4ntN): The submission resampled individual reports, but reports of the same video are not independent. We now resample whole videos and use the same resamples for every comparison. The main gap (fabrication loss minus the other losses) is −0.134 [−0.159, −0.110]. The interval is wider than before but still excludes zero. The result holds in every held-out fold for TF-IDF, and for every detector when training and test also share no videos. These intervals describe the three source LLMs we studied, and the paper now says so.

  2. The judge confound (AiSX): No LLM judges its own reports, so each source LLM is labelled by a different pair of judges. Holding out a source therefore also changes its judges. We tested this three ways:

    • Same judge for training and testing: the ratio falls from about 3 to 1.25 [1.08, 1.47].
    • One outside judge (Qwen3.8-27B) relabelling 2,419 reports from all three sources: 1.63 [1.38, 1.82], against 2.47 with the panel's labels on the same reports.
    • Training and testing within a single source LLM, so the judges never change: fabrication loses 0.083 AUC compared with pooling sources, and omission only 0.005.

    Part of the effect therefore comes from the judges, and the revision says so. The base-rate shortcut survives all three tests, and it also appears on RAGTruth, a dataset labelled entirely by people. We also report one case where the asymmetry disappears: reports short enough for the judges to read in full.

  3. A correction (AiSX): While recomputing Tables 1 and 4, we found that their panel column used a different combination of the same judge votes from the released labels that every detector is trained on. Both tables now use the released labels. Two types drop in reliability, from the high to the medium tier: entity fabrication (0.811 to 0.564) and phantom actors (0.658 to 0.588). Only H1 is excluded from the headline average, as before.

  4. Presentation (zzjs, AiSX, 4ntN): The abstract is rewritten with every term defined. Figure 1 is redrawn to show the problem itself. We added a worked example (Appendix L) and a reproducibility table (Appendix K), and fixed the "70 to 73%" inconsistency and the duplicated appendix.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.