acceptodds
Under review as a conference paper at ICLR 2027

CAN FACTUAL PROBES DETECT CORPUS EVIDENCE? AN AUDITABLE BENCHMARK

Abstract

Can factual probes distinguish reference-corpus evidence from answer plausibility? We introduce the Factual Exposure Benchmark (FEB), which records subject–object co-occurrences in an exhaustive scan of 6,407,814 English Wikipedia documents. Its labels describe this reference snapshot, not membership in the evaluated models’ training data. The benchmark pairs 878 evidence-absent facts with evidence-present facts matched on entity frequency and length, and includes subject-preserving object corruptions. Across five Pythia checkpoints and GPT-Neo-2.7B, pooled probe features reach mean AUC 0.5583 over five overlapping entity-grouped splits on the matched slice, compared with 0.9230 against corrupted objects. Single-model matched AUCs range from 0.5236 to 0.5676. GPT-Neo continuation-likelihood statistics also yield modest discrimination, with AUCs of 0.5488–0.5572. Pythia deduplication twins have nearly identical true-versus-corrupted answer gaps. When labels are instead defined by exact token-window presence in the released Pythia shards, the five checkpoints’ scalar answer probabilities yield AUCs of 0.501–0.518. A fixed 60-feature classifier yields 0.5545/0.5235 on all facts but 0.5026/0.4807 after exact-relation covariate matching. These contrasts show how the negative pool changes the apparent strength of factual auditing signals. They support matched evaluation of reference-corpus evidence, while leaving detection of actual training exposure unresolved.

Primary area:
datasets and benchmarks
Keywords:
corpus auditing, pretraining data detection, factual recall, membership inference, data contamination, language model memorization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_QovpOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper examines whether a model’s ability to provide plausible-sounding factual answers is the same as its ability to determine whether evidence for those facts exists within a specific corpus. The paper constructed the "Factual Exposure Benchmark" (FEB) by exhaustively searching approximately 6.4 million Wikipedia articles to record the co-occurrence of subjects and objects. The authors then compared "positive" samples, where the subject and object appear together against two types of control samples: those with matched occurrence frequencies and those with altered objects. A classifier integrating probe features from six different models yielded an AUC of 0.5583 for the first condition and 0.9230 for the second. Paper also conducted further analysis based on whether specific tokens appeared in the training data for the Pythia model. The primary conclusion is that the evaluation results for these probes vary significantly depending on how the samples and labels are constructed.

Critical strengths
  1. The design allows for the verification of labels at the unit level. It specifies the corpus used for exhaustive searching, the conditions for occurrence, the supporting context, and frequency information. This can enable other researchers to check the results using the same labels and control groups, which is beneficial for the research community rather than just reporting the detection performance alone.

  2. Instead of interpreting an initially high AUC as proof of successful exposure detection, this work runs various controlled experiments for verification. This includes not only frequency matching but also the application of a classifier, based on the same 60 features to the labels present in the Pythia training data. The study also include data partitioning through entity and uncertainty estimation; specifically, it doesn't overclaim that low AUC equates to inherent undetectability.

Critical weaknesses
  1. The experimental conditions were not sufficiently controlled to isolate the observed difference in AUC as being solely due to the effect of negative samples. Specifically, there is no indication that the set of positive facts remained consistent across the two conditions, which utilized 3,039 pairs for the "object-modified" condition and 878 pairs for the "frequency matched" condition. Consequently, the difference in AUC (0.9230 vs. 0.5583) could be influenced not only by the negative samples but also by the inherent difficulty of the positive facts and the number of training samples used. While the sensitivity to the choice of evaluation set is evident, the explanation provided in Section 4.4 for these results requires a more direct, controlled comparison.

  2. Furthermore, the study fails to quantitatively demonstrate how the specific characteristics of the appearing labels influence the results. By definition, the approach does not examine the semantic nature of the relationship but instead relies solely on the co-occurrence of two strings. In some instances, such as the example "Pennsylvania Route 696 / Pennsylvania", the object is contained within the subject. If such matches are permitted, a simple mention of the subject could suffice to classify a pair as a positive instance. While the authors appropriately acknowledge the distinction between semantic grounding and mere string co-occurrence, a more precise assessment of the benchmark’s informational value would require reporting the prevalence of such cases and providing results excluding them.

Questions for authors
  1. If we keep the same number of training samples (878 positive instances), features, and data-splitting procedures, but change only the type of negative samples, do the observed differences in AUC persist? Please also provide instance-level confidence intervals for the differences found in this comparison.

  2. Are cases where the object is contained within the subject, or where the spans of the two entities overlap also classified as positive? I would like to know the proportion of such cases and see the results when they are excluded.

  3. In Table 5, to what extent can the decline in performance after matching be distinguished from the effect of a reduction in sample size? Comparing the results against a randomly selected subset of the same size for each relationship type would help clarify the specific impact of the matching process itself.

Reviewer_tNnUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work presents Factual Exposure Benchmark (FEB), from subject-relation-object facts from LAMA-TREx, 878 are labeled as evidence-absent facts (true negatives; subject and object do not co-occur within a 512 char window in 6.4M wiki docs) and 3825 are labeled as evidence-present facts, from which 3175 object-corrupted negatives (same subject with object replaced to a non-co-occurring one) are constructed. The paper compare probe-based features with continuation-likelihood baselines. Probe-based methods achieve a near-chance discrimination performance of AUC=.56 on frequency matched fact pairs, while reaching AUC=.92 when the negatives are object-corrupted negatives.

Critical strengths
  • The paper tackles an important question of distinguishing LLMs factual knowledge from exposure during training vs from other means like generalization.
  • The paper introduces a new benchmark that is largely reproducible given the details.
  • The experiments involve multiple models (albeit mostly from a single family) and multiple methodologies for probing.
Critical weaknesses
  • Because there are no ground-truth labels for fact-level exposure during training, the paper does not establish whether fact-level exposure can or cannot be detected.
  • The experimental results are largely negative, e.g. AUC=.56 on distinguishing true negatives from paired positives, which is near chance. Drawing generalizable conclusions from these results is especially concerning given the lack of ground-truth exposure labels.
  • The facts are limited to subj-rel-obj style, with Wiki as the single evidence source.
Questions for authors
  • Are there any evidence that the presented results based on reference-corpus correlate with exposure at training time?
  • Are there particular relations (or perhaps subjects or objects) for which the distinction between the positives and negatives is easy?
Reviewer_e4QPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces the Factual Exposure Benchmark (FEB) to investigate whether factual probes can distinguish corpus evidence from factual plausibility. The authors construct the benchmark by scanning Wikipedia for subject–object co-occurrences, and introduce both frequency-matched true negatives and object-corrupted negatives. Experiments on Pythia and GPT-Neo show that probing features achieve high AUC against corrupted negatives, but perform only slightly above chance on frequency-matched facts. The paper suggests that negative-sample construction can substantially affect the apparent effectiveness of factual exposure detection.

Critical strengths
  • A useful and auditable benchmark. The full-corpus scan, evidence records, and matched negatives provide a reasonably transparent evaluation setup for studying factual exposure.

  • A meaningful empirical observation. The large performance gap between corrupted and frequency-matched negatives highlights how easily factual knowledge can be mistaken for exposure signals.

Critical weaknesses
  • Limited validity of the evidence labels. The benchmark defines evidence through subject–object co-occurrence within a fixed window, which does not necessarily mean the text supports the intended relation. More importantly, the Wikipedia reference corpus is not the models' actual training corpus, limiting the interpretation of the results.

  • Insufficient control of factual knowledge confounding. Matching entity frequency does not necessarily control how familiar a model is with a specific fact. Even among true facts, differences in factual familiarity could explain some of the remaining discrimination. Also, the corrupted-negative and matched-negative evaluations use different positive sets, making the main AUC comparison less controlled than it could be.

Questions for authors
  • Have the authors checked how often subject–object co-occurrence actually provides evidence for the intended relation, rather than an incidental mention or entity-name overlap?

  • Could the authors compare corrupted and frequency-matched negatives using the same positive facts and evaluation splits, to better isolate the effect of negative construction?

  • Beyond entity-frequency matching, is there any way to assess whether differences in fact-level familiarity explain the remaining above-chance performance on the matched subset?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.