acceptodds
Under review as a conference paper at ICLR 2027

Judging by the Cover: Cleaning LLM Truthfulness Benchmarks to Avoid Surface-Level Feature Leakage

Abstract

Binary-choice truth benchmarks ask models to choose between a correct and an incorrect answer, but if the two answers differ systematically in surface-level features, models can exceed chance without performing the intended reasoning. We show that this failure mode is detectable and can be exploited by downstream classifiers. In TruthfulQA, a simple six-feature logistic classifier achieves substantial accuracy in separating correct from incorrect answers. We further show that similar surface-level artifacts are present in additional benchmarks. To counteract this, we developed a general mechanism to clean them by removing the most leakage-reinforcing pairs. We release a version of TruthfulQA with surface-feature leakage reduced close to chance and provide a mechanism, Audit-Prune, so that datasets can be cleaned before release.

Primary area:
datasets and benchmarks
Keywords:
evaluation, benchmarks, dataset artifacts, shortcut learning, TruthfulQA, data leakage, benchmark validity

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_GpHaOpenReview ↗
Rating
2
Confidence
1
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether simple features of answer wording predict correctness in truthfulness benchmarks. It proposes six surface features and an iterative filtering procedure that removes question pairs reinforcing these associations. Applied to binary-choice TruthfulQA, the procedure produces a smaller subset on which the surface classifier performs close to random. Additional experiments examine other benchmarks and robustness to reversed surface cues.

Critical strengths
  • The paper addresses a relevant problem using a simple, interpretable diagnostic.
  • The release of a filtered benchmark provides practical value.
Critical weaknesses
  • The conceptual and methodological contribution seems modest to me. The approach combines a small set of hand-designed features, logistic regression, and greedy filtering.
  • The justification for treating the detected associations as undesirable bias is not fully convincing. TruthfulQA targets misconceptions, so correct answers may naturally contain more negation. Demonstrating that negation predicts correctness does not by itself establish that removing this association improves the intended evaluation.
  • Near-chance performance on a filtered dataset provides little evidence that the resulting subset is a better truthfulness benchmark overall.
Questions for authors
  1. What criterion distinguishes an unwanted surface artifact from a legitimate property of a misconception-focused task?

  2. Can the authors provide independent evidence, beyond the targeted six-feature audit, that filtering improves evaluation validity without removing important aspects of the task?

4 comments
AuthorsOpenReview ↗

Response to Reviewer GpHa

Q1 — Criterion: artifact vs legitimate property

"What criterion distinguishes an unwanted surface artifact from a legitimate property of a misconception-focused task?"

The goal of many well-known dataset tasks is to evaluate proper reasoning, such as evaluating the truth of a statement. Starting an answer with "No ..." does not imply it should be more truthful. In the case of TruthfulQA its statistical correlation is a construct of the phrasing of responses about misconceptions.
We demonstrate specifically how this can be exploited in the Surface-Inversion Generalization Test (Sec 3.3), and the generative extension in Appendix L.

The Surface6 features all have correlation with the TruthfulQA responses, and specifically do not address the core task (Truthfulness).

So you are right, this is an important question. And extending these Surface features to other datasets and reasoning tasks will require care and (for now) human judgement to determine if there is a definitional or only surface-statistical relationship to the task intended to be measured. A practical test is the one we use: move the cue to the other answer, keep the question and the labels fixed, and see whether model choices change.

Q2 — Independent evidence that filtering improves validity

"Can the authors provide independent evidence, beyond the targeted six-feature audit, that filtering improves evaluation validity without removing important aspects of the task?" / "Near-chance performance on a filtered dataset provides little evidence that the resulting subset is a better truthfulness benchmark overall."

  • (a) The Surface-Inversion test (Section 3.3, Table 3): classifiers trained on TruthfulQA-476 transfer better to the inverted cohort (significant for four of nine encoders).
    On the natural cohort the reported tests detect no significant difference, though the cohort is small and seven of nine point estimates are lower, so we claim no more than "no detected change".
    A generative extension shows similar results in Appendix L.

  • (b) On 16 embedding classifiers never used in pruning (see reply to j8yP Q1) and 300 surface features proposed by four LLMs (see reply to D77L W3), TruthfulQA-476 is less predictable than a size-matched random subset.

  • (c ) Question difficulty and ranking are retained, as well as coverage. Specifically we keep:

    • all 37 categories,
    • per-item difficulty 0.909 vs 0.910 under five instruments including four frontier models (Appendix C), and
    • ranking agreement ρ = 0.915 with the full set (Appendix D).

Moreover, our methodology (surface features and AuditPrune) can be used as part of the creation process for new datasets. There the goal is to prevent the sort of explicitly identified leakage before these datasets become part of the substrate.

W1 — Contribution seems modest

"The approach combines a small set of hand-designed features, logistic regression, and greedy filtering."

We intentionally built our method on standard classic tools because they work. We did not add complexity for complexity's sake.

The contribution is in the problem identification, the careful combination of such tools, and the experimental design and demonstrations.

Thanks for the constructive comments. I do appreciate it.

Thank you for your kind reply. If our answers have resolved your concerns, we would appreciate it if you would consider revisiting your score.

I do appreciate the clarifications you made although my view on the work has not changed much. My main concerns, which were (1) the removal of negation is debatable while (2) the overall contribution significance seems limited, somewhat remain.

However, I have to admit that I am no expert in this particular area and I am open to increasing my rating of your submission, if the other PC members and the AC (who hopefully are experts) believe it makes sense to do.

Reviewer_D77LOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work looks into the potential confounding factors within truthfulness related benchmarks, then designs and applies 6 surface features to detect answer-label correlations in these benchmarks. It uses the audit AUC, calculated by predicting answers from these 6 features, to measure how clean the benchmark is. Then it proposes an algorithm named Audit-Prune which removes question pairs until the audit AUC meets some threshold. This work also releases cleaned benchmarks with lower surface feature predictability.

Critical strengths

S1. The audit and fix are simple and cheap.

S2. Deconfounding for benchmarks is an important problem that helps ensure models don't rely on non-causal clues for making judgements.

Critical weaknesses

W1. The metric seems to be not standard. This paper measures whether the features distinguish true from false answers across the dataset, while the benchmark asks models to choose between two answers to the same question. Why choose the "global" audit metric instead of within-question? Could the authors also report how often their audit model chooses the true answer within each question, before and after cleaning?

W2. The necessity for this degree of cleaning seems unclear to me. An AUC of 0.7 shows some degree of confounding, but does not directly demonstrate how much it inflates scores and conceals actual model behavior. Negation can also be appropriate for misconception questions. But I am not fully convinced that reducing AUC to 0.53 justifies removing 40% of the questions, especially given the uneven category retention (it might further worsen the finite sample variance).

W3. The feature set is fixed and small. So the seemingly "safe" benchmark could also contain some confounding factor not included in this set. Would it be more reasonable and effective to let an LLM auto-propose and test possible confounding factors instead of relying only on these features?

Questions for authors

See weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer D77L

W1 — Within-question metric

"Why choose the 'global' audit metric instead of within-question? Could the authors also report how often their audit model chooses the true answer within each question, before and after cleaning?"

The global metric is used since the goal is to check if there is statistical leakage that can be inferred from the dataset. The method does not care which way (Yes or No) a feature (e.g., negation) is associated with; we believe it is an issue if it can be used to predict the answer without addressing the core task (Truthfulness).
Moreover, this is a more general method since not all datasets have a paired set of responses.

For each answer pair, how often does a classifier from Surface6 predict the correct one? It picks the truthful answer in

  • 75.1% of all TruthfulQA questions (before cleaning)
  • 61.6% on the TruthfulQA-476 questions (after cleaning).
  • 75.2% on average over random subsets of size 476 from TruthfulQA (random cleaning).

So the AuditPrune-cleaned dataset leaks much less information through these Surface6 features. What remains is almost entirely answer length: without the length feature, the classifier picks the true answer in 48.8% of the TruthfulQA-476 questions.

W2 — Is this degree of cleaning necessary?

"The necessity for this degree of cleaning seems unclear to me. An AUC of 0.7 shows some degree of confounding, but does not directly demonstrate how much it inflates scores and conceals actual model behavior. Negation can also be appropriate for misconception questions. But I am not fully convinced that reducing AUC to 0.53 justifies removing 40% of the questions, especially given the uneven category retention (it might further worsen the finite sample variance)."

Our method allows the user to choose their own threshold θ\theta or size (for example, θ=0.60\theta = 0.60 keeps 598 of the 790 questions).

We highlight and use the θ=0.53\theta = 0.53, size 476 option in the paper since it is about the point it reaches statistical indistinguishability. At θ = 0.53 all 37 categories are kept, the retained questions are no easier (0.909 vs 0.910; Appendix C), and the model ranking agrees with the full set (ρ = 0.915; Appendix D).

We agree negation may be appropriate for refuting misconceptions, but it is not directly indicative of truth, and we show it can be confounding.
Please revisit the Surface-Inversion Generalization Test (Sec 3.3); we generate a new dataset with this relation statistically inverted. Then classifiers trained on embedded full TruthfulQA perform worse than ones trained on the pruned TruthfulQA-476.
This demonstrates that training on the non-cleaned TruthfulQA (the most widely used truthfulness dataset) can lead to worse performance than using the cleaned one. Generative models show the same sensitivity: in Appendix L, 6 of 8 instruction-tuned models lose 12–21 points when the cues are moved to the false answer.
The benefit does not depend on our threshold: for every θ from 0.50 to 0.65 we rebuilt the cleaned set and retrained the classifiers, and they beat the full-trained ones on the inverted cohort at every θ.

W3 — Fixed, small feature set

"The feature set is fixed and small. So the seemingly 'safe' benchmark could also contain some confounding factor not included in this set. Would it be more reasonable and effective to let an LLM auto-propose and test possible confounding factors instead of relying only on these features?"

The reason we did this is two-fold:

  • The size 6 is intentionally small, and so its effect is interpretable.
  • We need the features to not imply the task at hand (Truthfulness)

But Yes, LLM-proposed features could certainly be used, and this would be another way to automate the extension of our method designed for TruthfulQA to broader types of datasets.

As an experiment, we ran your suggestion with four models from two families (Claude Opus 5.5, Claude Fable 5.1, GPT-5.4 and GPT-5.6) and three prompts: they were asked to propose answer-only surface features when shown (1) no data, (2) when told the benchmark is TruthfulQA, and (3) after reading 40 labelled pairs. Each model answered each prompt once: 4 × 3 = 12, each generating 25 features, so 300 in total. This included checking for quantifiers, certainty and attribution words, corrective framing, numbers and dates, voice, modals, punctuation, sentence structure, TruthfulQA idioms such as "nothing happens" and "no comment". No single feature reaches AUC 0.56 on TruthfulQA-476, and the few above 0.53 are length measures; and in 11 of the 12 runs the 25 features together predict the label less well on TruthfulQA-476 than on random 476-pair subsets (below all 20 random draws in 10 runs); the twelfth set shows no difference (AUC 0.57 vs 0.57).

So while we think this is worth exploring more, it would probably need careful prompting and its own pruning to be part of an effective automated process.

Thanks again for your review. If our response has addressed your concerns, we would be grateful if you would consider updating your score.

Reviewer_j8yPOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

In binary QA benchmarks, the options are written and presented in a way that may reveal the answer just based on how the answer was written. The model may answer the question correctly based on this heuristic without doing the required reasoning to actually answer the question. The authors show this with a simple logistic classifier, which uses just six simple features. Omitting the question and just showing the answer options can distinguish correct from incorrect answers effectively.

Audit Prone removes examples which contribute to this leakage, which reduces the classifier’s AUC. The authors do a similar investigation across about a dozen other benchmarks.

Critical strengths

Many existing academic benchmarks are fiercely flawed, and this paper correctly identifies one of the many problems that exist in 1. “uncleaned” academic benchmarks. These are benchmarks that people climb, and the finding that simple classifiers can effectively answer these questions while omitting the question is an impressive find. 2. The proposed method is simple and understandable. It's good that this study is done across many commonly used benchmarks.

Critical weaknesses

The authors mention rewriting answer options as an alternative to pruning, but argue that it requires additional factual verification. Given that Audit-Prune removes approximately 40% of the benchmark and substantially changes category representation, could the authors elaborate on the tradeoffs between pruning and rewriting? In particular, would rewriting and re-verifying a subset of answers be preferable to removing entire categories' worth of examples?

Questions for authors

The authors show that they can reduce leakage using SURFACE6, but they evaluate the cleaned benchmark using the same six features they optimize for. I would be interested in seeing whether the improvement holds for other features that were not considered during pruning. For example, could the authors test TruthfulQA-476 using additional stylistic features or a stronger answer-only classifier? This would help show whether the benchmark is genuinely less susceptible to shortcuts beyond those captured by SURFACE6.

2 comments
AuthorsOpenReview ↗

Response to Reviewer j8yP

W1 — Pruning vs rewriting

"Given that Audit-Prune removes approximately 40% of the benchmark and substantially changes category representation, could the authors elaborate on the tradeoffs between pruning and rewriting? In particular, would rewriting and re-verifying a subset of answers be preferable to removing entire categories' worth of examples?"

Yes — rewriting fits the framework: Audit-Prune scores pairs, so a rewritten pair can be re-scored and kept if it passes. It could be used to rank which pairs should be first re-written.

The tradeoff as we see it:

  • Pruning keeps only original, already-verified text and is reproducible and automatable from one script; its costs are in size reduction (476/790), the wider confidence intervals that come with it, and perhaps uneven coverage.
  • Rewriting keeps size and coverage, but every rewritten answer must be fact-checked again, and re-verified as a possible confound.

We saw this re-writing cost ourselves: when a frontier model rewrote the answers for our surface-flipped cohort, a "drift check" flagged 40 of 111 rewrites as changing the claim, and our own human-fact-check still corrected 16 of the 131 released pairs.

While the paper provides an automated pruning approach, the take-away a reader may get from the paper is a hybrid approach: prune, then rewrite and re-verify only in the depleted categories.

Q1 — Features not used in pruning

"The authors show that they can reduce leakage using SURFACE6, but they evaluate the cleaned benchmark using the same six features they optimize for. I would be interested in seeing whether the improvement holds for other features that were not considered during pruning. For example, could the authors test TruthfulQA-476 using additional stylistic features or a stronger answer-only classifier? This would help show whether the benchmark is genuinely less susceptible to shortcuts beyond those captured by SURFACE6."

We have explored two variants of this.

  1. We started with a larger set of potential features that we deemed independent of the truth of the statements. We reduced to the 6 in the paper, since they were the ones with meaningful association with the TruthfulQA dataset. The others (such as authority phrases, sentence count and punctuation rate) had almost no predictive power on this dataset, before or after pruning (AUC 0.50–0.51 on the full set, 0.49–0.50 on TruthfulQA-476). We also tested 300 further stylistic features proposed by four LLMs (details in our reply to Reviewer D77L, W3): none reaches AUC 0.56 on TruthfulQA-476, and in 11 of the 12 feature sets they predict the label less well on TruthfulQA-476 than on random subsets of the same size.

  2. We considered 16 embedding classifiers (BGE, E5, DeBERTa-NLI, Phi-3.5, ...). For these the AUC on our pruned TruthfulQA-476 is still detectable (on average 0.73). However we note two important points:

    • this is less than on a random subset of size 476, where it averages 0.77; all 16 are lower on TruthfulQA-476, so it shows some reduction.
    • as we point out for AFLite family of approaches, these embedded features encode world knowledge and were often built after TruthfulQA was released, and so it is not clear how to disentangle this effect. So we urge caution in reading too much into this.

These are the reasons we stuck with Surface6 evaluation.

Thanks for the detailed response and for running the additional experiments, particularly around generalization beyond SURFACE6. I'll maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.