- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors show that the concept "plausibility" might be spuriously correlated and entangled with the concept "truth" in the latent space of LLMs. They present a procedure for constructing plausible but false statements. They construct a direction as the mass mean between the plausible false and false statements and show that this direction can separate true and false statements reasonably well. Moreover, they show that standard truth probes, trained on true and false statements, achieve a lower AUC on the plausible false/false statements compared to the true/false statements. They propose the method "local contrastive readout" which achieves a higher AUC between plausible false and false statements.
Critical strengths
- The authors demonstrate the existence of a direction constructed only from false statements, that can nevertheless separate true and false statements with high AUC. That is a significant contribution.
- Local contrastive readout seems to be a promising technique for more accurate classification of true and plausible false statements. The method has a number of limitations but these are communicated very transparently by the authors.
Critical weaknesses
Novelty and Significance
- While the finding of the authors is interesting, it is quite narrow and rests heavily on prior work by Marks and Tegmark.
- The authors narrowly investigate one potential confound to truth but there may be many others.
- True/P in Table 2 is still quite high. I do not find it that surprising that more plausible falsehoods are harder to distinguish from true statements than blatant falsehoods. For sp_en_trans and cities the AUC of True/P is even 1, showing that can separate true from plausible false.
Conceptual and Methodological concerns
-
Marks and Tegmark specifically study truth representations for statements that are "simple, uncontroversial, and unambiguous", i.e. for statements that the model "knows" to be true or false. The claim that truth directions separate true and false statements can only possibly be made for statements for which the model "knows" to some extent whether the given statement is true or false (possibly with some uncertainty). For the "plausible but false" statements you construct, it remains unclear whether the model "knows" that these are false or whether it takes them to be true. Could you please evaluate this, e.g. by asking the model to classify a given statement as true or false? Is the model able to accurately classify the plausible false statements as false? This would significantly clarify the results in my opinion. If the model can accurately predict that the plausible false statements are false but the truth probe classifies them as true, this points to some surprising confounding factors in truth probe training. If the model is uncertain about these statements (e.g. make a few prompt variations + sample), this is evidence that truth directions capture some notion of model uncertainty, which would be a cool result! However, if the model consistently classifies the plausible false statements as true, then the model is simply mistaken about these statements and we probably also should not expect a truth probe to classify them accurately. For this check, please focus specifically on the false positives of the standard truth probe.
-
Line 348: I do not understand why you use the plausible false statements generated with Llama for all other models as well. Wouldn't it be cleaner to generate the plausible false statements for each model individually?
Presentation
The presentation of the paper could be improved. For instance, a nice Figure 1 could help with the understanding, the notation is sometimes a bit confusing (e.g. "render" in Line 408 is never explained), and some explanations are not very clear (e.g. "We also remove multi-valued relations" in Line 194 is not clear).
Questions for authors
- In equation (2) you describe the construction of the plausible but false statements. Did you run a check on each of these statements that it is actually false (e.g. using a strong LLM judge)? I saw that you did an alias screening but maybe there could be other ways in which the LLM generates a statement that is not false, i.e. either true or without any truth-value? If you did not check that each plausible statement is clearly false, please include such a check in the rebuttal.
- What is the saliency of the plausibility direction compared to the truth direction? I.e. what is compared to ?
Update: CounterFact plausible falsehoods verified by two judges
Following Reviewer fcMy's suggestion, we re-audited the mined plausible falsehoods of CounterFact with two strong judges, Claude Opus 5.5 and GPT-6.1, and kept only the statements that both label FALSE (see our response to Reviewer fcMy for the audit). We re-ran the comparison of decision rules, which is the most important set of results in our responses, on this verified subset. The same filter is applied to the training negatives of the plausible- and mixed-negative probes, while the original true and random false statements are unchanged. The other three datasets are not changed.
As shown in the updated Tables 4a/4b below, the conclusions do not change. On CounterFact, the local contrastive readout remains comparable to the likelihood-only rule on T/P (F1 0.811 vs. 0.813) and better than all three global-threshold baselines (0.750-0.768). Averaged over the four datasets, the readout still attains the highest F1 on T/P among all the rules compared (0.910 vs. 0.862-0.883). Due to time constraints, the other results that involve the CounterFact plausible-false subset will be updated in the revised paper, but we believe the updates above are sufficient to support the core claims of the paper.
Table 4a (updated). Average performance of the decision rules over the four datasets (CounterFact, Cities, sp_en_trans, and Companies). Each entry is shown as excluding → including ranking-ambiguous items. All rules except likelihood-only are evaluated at the same TPR, matched to the local contrastive readout. CounterFact plausible statements are kept only if both judges label them FALSE.
Table 4b (partially updated). Only CounterFact rows. The rows for Cities, sp_en_trans and Companies are unchanged.