acceptodds
Under review as a conference paper at ICLR 2027

The Truthfulness Spectrum: How General Are Truth Directions?

Abstract

Large language models (LLMs) have been reported to linearly encode truthfulness, but recent work questions whether there exists a domain-general truth encoding. We reconcile these findings through the truthfulness spectrum: truth-related linear directions lie on a spectrum of generality, from broadly domain-general to narrowly domain-specific. To characterize this spectrum, we evaluate probe generalization across five truth categories (definitional, empirical, logical, fictional, and ethical), sycophantic and expectation-inverted lying, and existing honesty benchmarks. We find that linear probes generalize across most domains, while sycophancy transfer is near-chance and expectation-inverted transfer is often reversed. Yet joint training on all domains yields a single linear direction with strong held-out performance; poor transfer between domains thus does not preclude a domain-general detector. Prior evidence against a shared truth direction also includes comparing probe weights, but this too can obscure shared signals: nearly orthogonal probes can behave similarly on the same data, and cosine similarity of probe weights explains only about half the variance in cross-domain transfer (R^2=0.56). To further characterize the spectrum constructively, using novel concept-erasure methods, we isolate directions that are highly domain-general, domain-specific, or shared across certain domain subsets. These directions serve different causal roles. In steering experiments, several domain-specific directions increase the correct answer margin, while the domain-general direction is null or harmful despite strong detection performance. Finally, we show the spectrum is reshaped by post-training, which reduces alignment between sycophancy and other truth types. Together, our results reconcile prior seemingly conflicting findings and further characterize truth directions on a spectrum, with directions of varying generality coexisting.

Primary area:
interpretability and explainable AI
Keywords:
interpretability, truthfulness, monitoring, linear probing, representation geometry, sycophancy, post-training

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_HesrOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes the truthfulness spectrum for investigating how truth directions generalize across domains. The authors evaluate probe generalization across multiple truth categories. Extensive analyses show that domain-general and domain-specific detection directions can coexist, therefore reconciling prior seemingly conflicting findings and further characterizing truth directions on a spectrum, with directions of varying generality coexisting.

Critical strengths
  • The analysis in this paper helps researchers understand the generalization of using probes to detect the truthfulness of content generated by LLMs. It reveals the coexistence and mechanisms of both domain-general and domain-specific truth directions, offering valuable insights for future research.
  • The datasets employed cover a diverse range of "truth" types, including basic epistemological truth types, sycophantic lying, expectation-inverted scenarios, and existing honesty benchmarks.
Critical weaknesses
  • Although the analysis covers a variety of truth types, the type selection process appears somewhat arbitrary—combining epistemological truth types and existing honesty benchmarks with a few additional types conceived by the authors. The paper would be more robust if the authors had first established a formal taxonomy of "truth" based on epistemological or other theoretical frameworks, and subsequently constructed datasets for analysis derived from that taxonomy.
  • As the authors acknowledge in the limitations, this study focuses exclusively on linear structures, testing three types of probe architectures: Difference of Means, Logistic Regression, and Linear Discriminant Analysis. The final experiments were conducted using a specific configuration—layer 33, logistic regression, and training on averaged tokens—which to some extent limits the generalizability of the findings.
Questions for authors

None

2 comments

We thank the reviewer for the positive review! We are glad that you find that "the coexistence and mechanisms of both domain-general and domain-specific truth directions" offers "valuable insights for future research", and that our datasets "cover a diverse range of truth types".

"the type selection process appears somewhat arbitrary ... The paper would be more robust if the authors had first established a formal taxonomy of 'truth' based on epistemological or other theoretical frameworks"

We aimed to stress-test generality across deliberately diverse notions of truthfulness (L129–130). The truth types in one paper are unlikely to be exhaustive, as we note in the Limitations. Our FLEED follows well-established philosophical distinctions in the domain of truth (Kant; Hume; Quine 1951; Lewis 1978; Wright 1992). The remaining datasets vary along other axes: whether the statement is the model's own generation (on-policy), whether truth is objective or relative to the user's expectation, and where the pressure to lie comes from.

We will add the epistemological paper citations and add a table organizing the ten datasets along these axes, so the selection is easier to follow.

"The final experiments were conducted using a specific configuration ... which to some extent limits the generalizability of the findings."

Our main probe configuration was selected by cross-domain performance (App. B), but we test the main findings well beyond the main configuration: using LDA and difference-of-means (Figs. 19, 20), across layers and token aggregation (Figs. 12, 13, 27), model families (Figs. 15, 22), and various datasets (Figs. 16, 17, 18).

Reference

  • Kant, Critique of Pure Reason, 1781 (trans. Guyer & Wood, Cambridge University Press 1998)
  • Hume, An Enquiry Concerning Human Understanding, 1748
  • Hume, A Treatise of Human Nature, 1739
  • Quine, Two Dogmas of Empiricism, Philosophical Review 1951
  • Lewis, Truth in Fiction, American Philosophical Quarterly 1978
  • Wright, Truth and Objectivity, Harvard University Press 1992

We hope these responses address your concerns. Thank you again for your positive assessment!

Thanks for your response. I have no further questions.

Reviewer_MaFyOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper argues that linear truth directions in LLMs span a spectrum from domain-general to domain-specific, reconciling conflicting reports on their universality. It introduces FLEED (five truth types) plus on-policy sycophancy and expectation-inverted datasets. On Llama-3.3-70B, FLEED probes transfer to each other but not to sycophancy, while a jointly trained probe scores well everywhere. Mahalanobis cosine (R²=0.98), unlike raw cosine (R²=0.56), predicts transfer, and Stratified INLP/LEACE extract directions of differing generality. Steering favors domain-specific over general directions, and post-training (localized to SFT/DPO on OLMo-3) weakens sycophancy–truth alignment.

Critical strengths

S1. On-policy, paired sycophancy dataset. Pairs differ only in whether the user's answer matches the model's own paraphrase-stable baseline, isolating lying against belief; the most distinctive findings rest on it.

S2. Useful diagnostic for weight-cosine arguments.

S3. Replication breadth. Results hold on a Claude-regenerated FLEED, against a TF-IDF baseline and across six model families, and the OLMo-3 checkpoints give a controlled within-lineage post-training design.

Critical weaknesses

W1. The paper is very poorly written. A lot of sentences are vague and unclear. Just a few examples:

  • when models represent their generations to be false
  • what probes can recover from model activations,
  • which features the model uses for truthfulness-related behavior

W2. The evidence does not yet establish a domain-general detector beyond prior work. The combined probe is evaluated in-distribution, and the held-out test (Fig. 21) omits sycophancy and expectation-inverted lying, which extra training domains do not help (Fig. 14). Without a null control (e.g., label-shuffled tasks), one shared direction cannot be distinguished from a superposition.

W3. It is unclear whether the sycophancy and post-training findings hold across probe types. With difference-of-means (DoM) probes, sycophancy→FLEED transfer is 0.97–0.99 (Fig. 20), versus about 0.55 with logistic regression (LR). Replicating the base-vs-chat gap with DoM probes on matched question sets would strengthen it.

Questions for authors

Address the weaknesses

2 comments

We thank the reviewer for their time! We are glad you found that the on-policy, paired sycophancy dataset supports "the most distinctive findings", that our analysis is a "useful diagnostic for weight-cosine arguments", and that the results show "replication breadth" across generators, baselines, and six model families.

"The evidence does not yet establish a domain-general detector beyond prior work. The combined probe is evaluated in-distribution, and the held-out test (Fig. 21) omits sycophancy and expectation-inverted lying, which extra training domains do not help (Fig. 14). Without a null control (e.g., label-shuffled tasks), one shared direction cannot be distinguished from a superposition."

This comment raises three related concerns: the joint probe is evaluated in-distribution, the held-out test omits the two hardest domains, and there is no null control to separate a shared direction from a superposition. We first restate what we claim, then answer each with existing and new evidence.

  • We distinguish detection-level and causal-level generality. We do not claim a single causal feature shared by all truth types. By "domain-general" we mean "a single linear direction that performs well across the evaluated domains" (L80–83), and Footnote 1 explicitly discusses that such a detection direction may combine domain-specific causal features.
  • We characterize and quantify the graded shared truth structure. The effective dimensionality of the ten single-domain probes is far below a label-permutation control (Tab. 4), so shared structure exists. The low-dimensional structure is interpretable: a shared truth direction for FLEED, a shared on-policy deception direction, and separate sycophancy and expectation-inverted directions (Tab. 5). A new label-flip experiment shows the same structure inside the joint probe: flipping one FLEED domain's labels and retraining the joint probe costs 0.11–0.24 AUROC on that domain and also degrades the other FLEED domains, while flipping sycophancy costs essentially nothing.
  • Held-out generalization follows the spectrum. As shown in Fig. 14, training on more domains helps FLEED and the on-policy datasets, but does nothing for sycophancy and even hurts exp. inverted lying. This generalization pattern is consistent with all other analyses.
  • However, the joint probe is not a simple superposition of single-domain probes (new). We show that the 10 single-domain probes' scores do not predict the joint probe's scores well. The best combination of single-domain probes reaches only AUROC 0.74 on sycophancy and 0.77 on exp. inverted lying, vs. 0.97 and 0.90 for the joint probe. The joint probe thus finds signal that no single-domain probe contains and is not a simple superposition of the single-domain probes we find.
  • The joint probe still matters in practice. For building white-box monitors, our results suggest one does not need to establish whether the model represents truthfulness as one shared feature. A probe trained on pooled data can detect well across the evaluated domains regardless, provided domains are covered in training.

In short, beyond prior work, we carefully quantify the rich graded structure (Tab. 4, 5, App. E.1, new analyses). Despite the rich truth structure, we can find multiple directions that classify all domains well (Sec. 5,6), which are not a simple superposition of single-domain directions we find (new). This is important in practice, as a successful white-box monitor does not depend on the model using one shared causal feature. We will add a new section analyzing superposition by moving App. E.1 and Fig. 14 to the main text together with the new analyses.

"It is unclear whether the sycophancy and post-training findings hold across probe types. "

The DoM probes perform worse than LR during our probe design tuning stage, and are thus excluded from main experiments. Notably, the DoM sycophancy probe fails in-distribution (AUROC 0.63, Fig. 20). Its high transfer performance to FLEED thus cannot reflect a shared sycophancy–truth signal. For this reason, we do not think a base-vs-chat comparison with DoM sycophancy probes would be informative.

Our post-training result already replicates across 14 models from 3 families (Fig. 7, 28), different truth datasets (Fig. 29), and layers (Fig. 27). Since the DoM sycophancy probe does not even perform well in-distribution, we do not think analyzing its transfer behavior is informative.

Writing clarity

Thank you for pointing to specific sentences for clarity. We will rewrite them. We are happy to revise any other passages you found unclear.

We hope these responses address your concerns about the superposition of domain-specific features and the null control, robustness to DoM probes, and writing clarity. We sincerely hope you may consider a more positive score for our work!

Dear Reviewer,

As the discussion period closes soon, we wanted to follow up briefly.

We have uploaded a revised PDF (changes in purple). Here is a recap of how we addressed each point in the revision:

  • W2 (domain-general detector, null control, superposition). New Sec. 4.1 and Fig. 3 analyze the joint probe superposition directly. A label-flip test shows the FLEED domains are strongly coupled in the joint probe, while sycophancy is not. A regression analysis shows that the joint probe is largely a superposition of single-domain probes on FLEED and the on-policy datasets (R2=0.72–0.93), but not on sycophancy or expectation-inverted lying.

    Beyond prior work on domain-general detectors, we characterize the rich truth structure with novel datasets and multiple complementary analyses, reconcile seemingly contradictory prior findings, and show that a single joint probe can detect orthogonal or even anti-correlated domains, without being merely a superposition of single-domain probes.

  • W3 (post-training result robustness). Our post-training result replicates across 14 models from 3 families, truth datasets, and layers (Fig. 8, 28–30). The DoM sycophancy probe reaches only 0.63 in-distribution (Fig. 21), so its transfer cannot reflect a shared sycophancy-truth signal.

  • W1 (writing). We rewrote the sentences you flagged (L41–42, L80–81) and revised Sec. 1, 4, and 7 for clarity.

We would be really grateful if you could take a look at the revised paper. We hope it addresses your concerns and that you may consider re-evaluating our work!

Reviewer_zSSaOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether linear truth representations in LLMs are domain-general or task-specific. The authors propose that truth directions lie on a generality spectrum and distinguish between detection generality (probing accuracy across domains) and causal generality (steering efficacy). Using a new benchmark of five truth categories (FLEED), an on-policy sycophantic lying dataset, an expectation-inverted dataset, and existing honesty benchmarks across Llama, Qwen, and OLMo-3 models, the paper shows:

  1. Probes trained on single factual categories transfer well to each other, fail on sycophancy, and invert on expectation-inverted prompts, whereas a probe trained jointly on all 10 domains achieves high held-out AUROC across all domains (Fig. 2).
  2. Standard cosine similarity poorly predicts cross-domain transfer (R2=0.56R^2 = 0.56) due to low-variance noise dimensions, while Mahalanobis cosine similarity (CosΣ(wA,wB)\text{Cos}_{\Sigma}(w_A, w_B)) aligns closely with transfer AUROC (R2=0.98R^2 = 0.98, Sec. 5).
  3. Concept erasure via Stratified INLP and LEACE isolates directions at different levels of generality (Sec. 6).
  4. Steering SimpleQA and PopQA with the jointly trained general direction degrades performance (Δdiff=−0.07\Delta\text{diff} = -0.07), while directions extracted from individual domains improve the correct-answer margin (Sec. 7).
  5. Post-training (specifically SFT and DPO in OLMo-3-7B) reduces alignment between sycophancy and factual truth probes compared to base models (Sec. 8).
Critical strengths
  • Geometric reconciliation of prior probing results (Sec. 5 & App. E.1): Showing why Euclidean cosine similarity fails in d=8192d=8192 (R2=0.56R^2=0.56) and analyzing the eigenspectrum of the 10-domain probe Gram matrix (App. E.1, where regularized probes have participation ratio PR≈4.0\text{PR} \approx 4.0 and whitened Fisher directions have PR≈8.4\text{PR} \approx 8.4) provides a rigorous explanation for why prior studies reached opposite conclusions. The controls against synthetic artifacts (Claude-3.5-Sonnet regeneration in Fig. 17; TF-IDF baseline in Fig. 18) are also solid.
  • Disconnect between detection generality and steering, plus post-training trajectory analysis (Sec. 7 & Sec. 8): Showing that a high-AUROC joint detector fails as a steering vector on SimpleQA/PopQA because it mixes factual truth and sycophancy variance is an important practical takeaway. Tracking the drop in sycophancy--truth alignment across OLMo-3-7B checkpoints (Base -> SFT -> DPO -> RLVR, Fig. 7c) cleanly localizes this shift to SFT/DPO.
Critical weaknesses
  • Superposition vs. true domain-general directions (Sec. 4, Sec. 6.1, & App. E.1): In R8192\mathbb{R}^{8192}, a linear classifier trained on K=10K=10 domains with orthogonal discriminative directions u1,…,u10u_1, \dots, u_{10} can simply learn wjoint=∑kαkukw_{\text{joint}} = \sum_k \alpha_k u_k and achieve high AUROC on in-distribution test folds of those same 10 domains (acknowledged in Footnote 1). Two results in the appendix show that wjointw_{\text{joint}} is acting as a multi-task superposition rather than a shared feature for the outlier domains:

    1. In the domain-scaling experiment (Fig. 14), holding out sycophancy or expectation-inverted lying yields near-chance (0.56–0.600.56\text{--}0.60) or anti-correlated (0.260.26) transfer even at K=9K=9.
    2. In App. E.1 (Table 4 & Table 5), the 10 probes decompose into ∼3–4\sim 3\text{--}4 nearly orthogonal subspaces (v1v_1 for FLEED claims, v2v_2 for roleplaying/insider trading, and v3,v4v_3, v_4 for sycophancy and expectation-inverted lying, with sycophancy loading only 0.080.08 on v1v_1). App. E.1 should be brought into the main text, and claims that joint training recovers a "domain-general direction" should be qualified to distinguish shared subspaces from linear superposition.
  • Confound in Stratified INLP + Causal Steering (Sec. 6.1 & Sec. 7): Projecting out a joint probe direction v∝u1+u2v \propto u_1 + u_2 via I−vv⊤I - vv^\top only removes a 1D subspace, leaving u1−u2u_1 - u_2 intact; moreover, iterative LR nullspace projection (INLP) does not fully erase linear concept subspaces (Belrose et al., 2023). In Sec. 7 (Fig. 6), Stratified INLP is run over 14 datasets, ∼10\sim 10 of which are factual knowledge datasets (A&M-*, Marks&Tegmark, empirical). The residual directions after 16 INLP steps are called "domain-specific," yet they steer general factual QA on held-out SimpleQA and PopQA. This indicates that shared factual truth signal survived the 16 INLP steps (while sycophancy was separated out), rather than the steering effect coming from truly domain-specific features.

Questions for authors
  1. Inconsistency in Sycophancy dataset description (Sec. 2.2 vs. App. A.2): Sec. 2.2 (lines 160–162) and Fig. 9 state that the model agrees with the user in both honest and deceptive samples (where the honest prompt states the model's baseline answer (A)). However, App. A.2 (line 801) says sycophantic samples are "paired with control cases where the model remains steadfast." Which pairing was actually used? Furthermore, in Sec. 8 (Fig. 7), since sycophancy data are filtered on-policy per model, does the Base →\to SFT →\to DPO drop on OLMo-3-7B still hold if evaluated on a fixed, shared set of MMLU question/completion pairs across checkpoints?

  2. Sign flip in Expectation-Inverted MCS and transfer (Fig. 2 & Fig. 24): When exp inverted is the training set (row), transfer to FLEED is strongly inverted (AUROC 0.09--0.39, MCS −0.82-0.82 to −0.19-0.19 under ΣFLEED\Sigma_{\mathrm{FLEED}}). When FLEED probes are tested on exp inverted (column), AUROCs are mostly >0.5\gt 0.5 (0.58--0.79) and MCS under Σexp-inv\Sigma_{\mathrm{exp\text{-}inv}} is positive (+0.19+0.19 to +0.40+0.40). Why does changing the covariance matrix from ΣFLEED\Sigma_{\mathrm{FLEED}} to Σexp-inv\Sigma_{\mathrm{exp\text{-}inv}} flip the sign of wA⊤ΣwBw_{A}^{\top} \Sigma w_{B} for the same pair of probes?

  3. Pre-INLP factual steering baseline (Sec. 7): How does steering SimpleQA/PopQA with a probe trained jointly only on factual datasets (excluding sycophancy, or using eigenvector v1v_{1} from Table 5) compare to the post-INLP directions in Fig. 6?

  4. Minor: Line 208 typo ("leveld" →\to "level"). Please also consider moving Table 4 and Table 5 from App. E.1 into the main text.

1 comment

Thank you for the positive and thorough review. We are glad you found that our analysis "provides a rigorous explanation for why prior studies reached opposite conclusions", that "the controls against synthetic artifacts...are also solid", and that the detection–steering gap is "an important practical takeaway".

"Two results in the appendix show that w_joint is acting as a multi-task superposition rather than a shared feature for the outlier domains ... App. E.1 should be brought into the main text, and claims that joint training recovers a 'domain-general direction' should be qualified to distinguish shared subspaces from linear superposition."

Thank you for this careful reading of App. E.1 and Fig. 14. Please also see the General Response for a full response.

  • We agree the rich truth structure shows that some domains are not shared with the rest. As shown in Tab. 5, the FLEED domains rely on v1v_1, roleplaying and insider trading share v2v_2, and sycophancy and expectation-inverted lying rely on their own signals (v3v_3, v4v_4), consistent with the held-out transfer results in Fig. 14. A new label-flip experiment shows the same truth structure inside the joint probes: flipping sycophancy labels and retraining the joint probe costs no performance, while flipping a FLEED domain degrades the other FLEED domains. Flipping expectation-inverted lying even improves its AUROC, consistent with it being anti-correlated with other domains.
  • However, the joint probe is not a simple superposition of single-domain probes (new). We show that using the 10 single-domain probes' scores does not predict the joint probe's scores well. The best combination of single-domain probes reaches only AUROC 0.74 on sycophancy and 0.77 on exp. inverted lying, vs. 0.97 and 0.90 for the joint probe. The joint probe thus finds signal that no single-domain probe we trained contains.
  • The joint probe still matters in practice. For building white-box monitors, our results suggest one does not need to establish whether the model represents truthfulness as one shared feature. A probe trained on pooled data can detect well across the evaluated domains regardless, provided domains are covered in training.

We will follow your suggestions to add a new section on superposition and bring Fig. 14 and Tab. 4,5 to the main text together with the new analyses.

"iterative LR nullspace projection (INLP) does not fully erase linear concept subspaces (Belrose et al., 2023)."

As we discuss in footnote 4, "INLP is not an ideal method for achieving linear concept erasure. We use it here because it provides a practical procedure for identifying multiple, mutually orthogonal truth directions with high performance. Since our work focuses directly on the existence and generalization of multiple directions, this capability is the key consideration." Where guaranteed erasure matters, we use LEACE (Sec. 6.2) to identify intersections across domains.

"The residual directions after 16 INLP steps are called 'domain-specific,' yet they steer general factual QA on held-out SimpleQA and PopQA. This indicates that shared factual truth signal survived the 16 INLP steps"

We address this concern in the three points below:

  • "Domain-specific" is not limited to a dataset, but refers to a truth type. These directions transfer to independently built datasets of the same type (Fig. 32). SimpleQA is factual QA, so it is in-domain for the factual directions, and good detection and steering there is expected. In addition, we mean "domain-specific" in the detection sense (L80–83).
  • The detection vs. steering divergence is itself evidence supporting our main claim in Sec.7. The domain-specific directions do not detect truthfulness in other training domains, yet they steer held-out factual QA. We cannot rule out that some factual signal shared across the factual datasets survives erasure (because of INLP). This is itself a case of detection and causality diverging, further supporting our main claim and contribution in the causal section.
  • New on Llama-3.3-70B. We switch the main 70B plus 10-domain configuration for steering, as we use in other sections. The same pattern holds at the main model scale: the domain-general direction hurts steering (-0.03 vs. random mean −0.04), whereas domain-specific directions show positive steering (6 of 10).

Minor clarifications

  • Sycophancy design: Sec. 2.2 and Fig. 9 are correct. The model agrees with the user in both positive and negative cases, which is what we meant by "remains steadfast" (L801). We will reword it to avoid confusion.
  • Sign flip for MCS: MCS is asymmetric for a pair of probes, since they use different test-data covariance for reweighting (e.g. cov_FLEED vs. cov_expectation-inverted), so they are different quantities and need not share a sign, just like the transfer performance also being asymmetric.
  • Typo at L208 will be fixed
Reviewer_6umSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether truth representations in large language models are domain-general or domain-specific by evaluating linear probes across five synthetic or adapted truth categories (FLEED), on-policy sycophantic and expectation-inverted lying datasets, and three honesty benchmarks. Using Mahalanobis cosine similarity, Stratified INLP, LEACE, causal steering, and Base/SFT/DPO comparisons, the authors argue that truth representations form a hierarchical spectrum combining shared and domain-specific directions.

Critical strengths
  • Studying truth probing across ten diverse datasets—spanning factual claims, on-policy sycophancy, expectation-inverted lying, and strategic deception—provides a comprehensive empirical view of where linear probe transfer succeeds and where it breaks down across layers and model scales.
  • Comparing standard Euclidean cosine similarity against covariance-aware alignment and iterative concept erasure offers thoughtful geometric tools for analyzing linear separability in high-dimensional activation spaces.
Critical weaknesses
  • While joint probe training across all ten domains achieves high in-distribution AUROC (Section 4, Figure 2, p. 5), an 8192-dimensional linear probe can fit a union of nearly orthogonal domain subspaces without relying on a single shared direction; on held-out domains in Figure 14 (p. 21), expanding training domain breadth leaves sycophancy near chance (0.56–0.60 AUROC) and reduces expectation-inverted lying to 0.26, suggesting limited out-of-domain generalization for deceptive behaviors.
  • In Section 5 (p. 6), weighting probe inner products by full test-domain covariance makes Mahalanobis cosine similarity largely reflect prediction correlation with the in-domain probe, whereas Appendix E.1 (Table 4 and Figure 26, pp. 28–29) shows that whitening class-mean differences by within-class covariance raises the participation ratio to 8.4–8.9 out of 10 with near-zero cross-domain similarity (∣CAB∣≤0.07|C_{AB}| \le 0.07) outside FLEED, which seems closer to orthogonal task subspaces than to a low-dimensional shared spectrum (PR ≈4\approx 4).
  • Causal steering experiments in Section 7 (pp. 7–8, Figure 6) switch from the 10-domain Llama-3.3-70B setup to a 14-dataset Llama-8B setup that omits strategic deception and expectation-inverted lying, and evaluate only small teacher-forced log-probability shifts (+0.02 to +0.05) on factual QA rather than open-ended generation across the deceptive domains.
Questions for authors
  • How do the authors reconcile the main-text conclusion of low effective dimensionality (PR ≈4\approx 4) with the within-class covariance whitening results in Appendix E.1 (Table 4, p. 28), where PR reaches 8.4–8.9 out of 10 and cross-domain alignment outside FLEED is near zero?
  • Why did the causal steering experiments in Section 7 switch to Llama-8B and omit the strategic deception and expectation-inverted lying datasets, and do the extracted directions causally steer open-ended responses on those deceptive tasks?
  • Could the authors clarify the slight differences between Figure 2 (0.97) and Figure 22 (0.86) for Llama-3.3-70B layer-33 combined-to-sycophancy AUROC, and between Figure 4a (5 global iterations) and Section 6.1 (4 global iterations) for Stratified INLP?
3 comments

Thank you for the thoughtful review! We appreciate you finding our ten datasets give "a comprehensive empirical view of where linear probe transfer succeeds and where it breaks down", and that MCS and concept erasure are "thoughtful geometric tools".

"an 8192-dimensional linear probe can fit a union of nearly orthogonal domain subspaces without relying on a single shared direction"

Please also see the General Response for more. We agree that a high-dimensional probe is partly a superposition of orthogonal subspaces. We regard it as a contribution instead of a weakness, where all old and new analyses converge. Specifically, we:

  • Characterize and quantify the graded shared truth structure across domains. The four FLEED domains rely on one direction (v1v_1), roleplaying and insider trading share a second (v2v_2), while sycophancy and expectation-inverted lying load on their own (v3v_3, v4v_4) (Tab. 4, 5). A new label-flip experiment shows the same structure for the joint probes: flipping a FLEED domain and retraining joint probes degrades other FLEED domains, while flipping sycophancy costs nothing. This is consistent with sycophancy and expectation-inverted lying not benefiting from more training domains (Fig. 14).
  • However, the joint probe is not a simple superposition of single-domain probes (new). We show that the 10 single-domain probes' scores do not predict the joint probe's scores well. The best combination of single-domain probes reaches only AUROC 0.74 on sycophancy and 0.77 on exp. inverted lying, vs. 0.97 and 0.90 for the joint probe. The joint probe thus finds signal that no single-domain probe contains and is not simply a superposition of domain-specific probe directions we identify.
  • The joint probe still matters in practice. For building white-box monitors, our results suggest one does not need to establish whether the model represents truthfulness as one shared feature. A probe trained on pooled data can detect well across the evaluated domains regardless, provided domains are covered in training.

We regard these results as an important contribution of our paper rather than a weakness: all four analyses (held-out domain transfer, effective dimensionality, label flipping, and regression) quantify the same rich truth structures. We will add a new section on superposition with these analyses.

"reconcile the main-text conclusion of low effective dimensionality (PR=4) with the within-class covariance whitening results in Appendix E.1 (Table 4, p. 28), where PR reaches 8.4–8.9 out of 10"

The two participation ratios measure different things. Both apply the whitened Gram, but to different sets of vectors:

  • To trained LR probes (PR=4). The LR probes we actually use share substantial structure, which reflects cross-domain transfer.
  • To Fisher linear discriminants (PR=8.5). The whitened diff-mean directions are Fisher discriminants, which often rely on low-variance domain-specific features, so these directions appear nearly orthogonal. They are not what our LR probes pick up. Domains whose Fisher directions are nearly orthogonal still have strong transfer performance with LR (e.g., Roleplaying probes detect Insider Trading at AUROC 1.00 despite C_AB=0.07).

The two analyses are thus complementary. It suggests domains share high-variance truth directions, and each also retains low-variance domain-specific structure, consistent with our spectrum view and matches the Stratified INLP and LEACE results (Sec. 6).

"Why did the causal steering experiments in Section 7 switch to Llama-8B and omit the strategic deception and expectation-inverted lying datasets, and do the extracted directions causally steer open-ended responses on those deceptive tasks?"

  • Model and datasets (new). We rerun the steering experiments on the 70B model with the same 10 domains as the rest of the paper. We see consistent results with the 8B model: steering domain-general direction hurts (−0.03) despite its strong detection performance, while many domain-specific directions show positive effects (6 of 10, up to +0.06).
  • Effect sizes against a random baseline (new). The absolute effects are small, but they should be compared to a random-direction baseline. Steering along random directions lowers the LPD (mean over 100 random directions −0.04). 6 of the 10 domain-specific directions exceed the 95th percentile of the random baseline.
  • Evaluation. Log-probability differences on QA are a standard protocol in recent work with truth steering (Adarsh et al., ACL 2026; Schouten et al., COLM 2025; Park et al., ICML 2025). We will discuss open-ended evaluation on deceptive tasks as limitation and future work.

We hope these responses address your concerns about the joint probe superposition, the two participation ratios, and the scope and effect sizes of the steering experiments. We sincerely hope you may consider a more positive score of our work!

Minor clarifications

"weighting probe inner products by full test-domain covariance makes Mahalanobis cosine similarity largely reflect prediction correlation with the in-domain probe"

Yes, MCS is equivalent to the correlation of test-data projections onto the two probe directions, as we explicitly discussed (L97–99). That is intended: it gives a geometric interpretation of the probe's behavioral alignment. We use MCS to diagnose why standard cosine can be misleading, since standard cosine ignores the test data distribution.

We will correct Fig. 22, which uses an earlier probe configuration with regularization of 1 instead of 1e-4.

We will correct Fig. 4a, which should use 4 global iterations instead of 5.

Reference

  • Adarsh et al., How Context Shapes Truth, ACL 2026
  • Schouten et al., Truth-value judgment in LMs: ‘truth directions’ are context sensitive, COLM 2025
  • Park et al., Steer LLM Latents for Hallucination Detection, ICML 2025

Dear Reviewer,

As the discussion period closes soon, we want to follow up briefly.

We have uploaded a revised PDF (changes in purple). Here is a recap of how we addressed each point in the revision:

  • W1 (joint probe as a union of orthogonal subspaces). New Sec. 4.1 and Fig. 3 test superposition directly, showing that the joint probe is not merely a superposition of single-domain probes. A label-flip test shows that the FLEED domains are strongly coupled in the joint probe, while sycophancy is not. A regression analysis shows that the joint probe is partly a superposition of single-domain probes on FLEED and the on-policy datasets (R2=0.72–0.93), but not on sycophancy or expectation-inverted lying.

    Thanks to the reviewer's comment, we now present this as one of our main contributions. Using novel datasets and several complementary analyses, we 1) characterize a rich graded truth structure across domains, and 2) show that a single joint probe can detect orthogonal and even anti-correlated domains, without being a simple superposition of single-domain probes.

  • W2 (two participation ratios). New Sec. 4.1 reports both participation ratios and explains the difference: trained LR probes share high-variance directions (PR=4.0), while Fisher directions capture low-variance domain-specific structure (PR=8.5) (L255–260; App. E.1). Sec. 5 also explicitly states that MCS equals output correlation and serves as a diagnostic of why standard cosine fails (L352–358).

  • W3 (steering). Sec. 7 now uses Llama-3.3-70B with all ten domains (Fig. 7). Against a random-direction baseline, the domain-general direction is no better than random, while several domain-specific directions steer positively. The Limitations discuss open-ended evaluation on deceptive tasks (L532–535).

  • Figures. Fig. 23 (formerly 22) and Fig. 5a (formerly 4a) are corrected.

We would be really grateful if you could take a look at the revised paper. We hope it addresses your concerns and that you may consider re-evaluating our work!

Comments on the paper

1 comment

We thank all four reviewers for their careful, thoughtful feedback and the AC for overseeing our submission.

We appreciate reviewers noting that our work "provides a comprehensive empirical view of where linear probe transfer succeeds and where it breaks down" (6umS), with datasets that "cover a diverse range of 'truth' types" (Hesr), and that it "reveals the coexistence and mechanisms of both domain-general and domain-specific truth directions, offering valuable insights for future research" (Hesr). They find our geometric analysis a "useful diagnostic for weight-cosine arguments" (MaFy) that "provides a rigorous explanation for why prior studies reached opposite conclusions" (zSSa), and that showing "a high-AUROC joint detector fails as a steering vector" is "an important practical takeaway" (zSSa).

We want to address a common question raised by several reviewers in this comment.

Domain-general detection and superposition

Several reviewers ask whether the jointly trained probe reflects one shared causal feature or a superposition of domain-specific directions. We clarify our claim and bring together four analyses.

  • Conceptually, we distinguish detection-level from causal-level generality (L75–78). By "domain-general" we mean generality at the detection level: "a single linear direction that performs well across the evaluated domains" (L80–83). Footnote 1 (L106–107) explicitly discusses how a superposition of domain-specific causal directions can constitute a domain-general detection direction. We separately test the causal role of these detection directions in Sec. 7. A concurrent survey (Lee et al., BlackboxNLP 2026) draws the same distinction between representation formation and representation use, noting that the literature "often conflates" the two. We will make the wording more careful throughout and bring Footnote 1 into the main text.

  • We characterize and quantify the graded truth structure (App. D, App. E.1). The effective dimensionality of the ten single-domain probes is far below a null control (Tab. 4), so shared structure exists. The low-dimensional structure is interpretable: a shared truth direction for FLEED, a shared on-policy deception direction, and separate sycophancy and expectation-inverted directions (Tab. 5). Held-out transfer shows the same pattern: adding training domains improves transfer for seven domains, but not for sycophancy and lowers transfer to exp. inverted (Fig. 14).

  • The joint probe shows the same truth structure through a label-flipping test (NEW). We retrain the joint probe after flipping the labels of one dataset at a time. Flipping a FLEED domain costs 0.11–0.24 AUROC on all FLEED domains, suggesting the FLEED domains are strongly coupled. Roleplaying is coupled but weaker (drop of 0.09), while flipping sycophancy costs nothing, so their signals can lie in orthogonal subspaces to other domains. Flipping exp inverted improves its AUROC (0.90 to 0.98), consistent with it being anti-correlated with other domains.

  • However, regression analysis shows that the joint probe is not a superposition of the single-domain probes (NEW). If the joint probe were simply a superposition of single-domain probe directions, combining the single-domain probes should reproduce it. However, regressing the joint probe's scores on the ten single-domain probes' scores gives high R^2 on FLEED and on-policy data, but R^2 of only −0.78 on sycophancy and −0.23 on exp. inverted. Using the ten single-domain scores to predict the labels across ten domains directly reaches only AUROC 0.74 on sycophancy and 0.77 on exp. inverted, compared with 0.97 and 0.90 for the joint probe, even though each of these domains has its own near-perfect in-distribution probe. The joint probe therefore finds signal that does not appear in single-domain probes and is not a simple superposition of single-domain probes.

  • The joint probe still matters in practice. For building white-box monitors, our results suggest one does not need to establish whether the model represents truthfulness as one shared feature. A probe trained on pooled data can detect well across the evaluated domains regardless, provided domains are covered in training.

In short, these four analyses (held-out domain transfer, effective dimensionality, label flipping, and regression) tell a consistent story: truth signals show graded structure, which we carefully quantify (Tab. 4, 5, App. E.1, new analyses). Despite the rich truth structure, we can find multiple directions that classify all domains well (Sec. 5,6), which are not simple superpositions of single-domain directions (new). This is important in practice, as a successful white-box monitor does not depend on the model using one shared causal feature. We will add a new section analyzing superposition by moving App. E.1 and Fig. 14 to the main text together with the new analyses.

We respond to each reviewer individually below.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.