acceptodds
Under review as a conference paper at ICLR 2027

Knowing Is Not Saying: How Language Models Decouple Truth from Deceptive Reporting

Abstract

Large Language Models can produce incorrect reports under deceptive instructions even when the truthful answer remains internally available, raising a fundamental question: how can knowledge and report diverge within the same model? We study instructed lying as a controlled setting for this problem and refer to the resulting separation between available knowledge and reported behavior as knowledge–report decoupling. Layerwise decoding shows that deceptive instructions reshape the answer space through exclusion rather than substitution: they broaden incorrect competition while the truthful target retains predictive support and remains linearly decodable deep into answer formation. Per-layer transcoder (PLT)-based circuit tracing further shows that truthful and reported incorrect targets share highly similar input-side attribution but diverge substantially in downstream target-directed influence, revealing a distributed target-dependent pattern rather than a dedicated internal state that partially transfers to broader deception-related settings. Feature interventions further distinguish answer-specific from shared report control: full downstream effects are most effective for individual questions, whereas mediated influence is strongest across questions, suggesting that shared report control lies in the transformation from knowledge to report, rather than at the output endpoint.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Models, Deception, Mechanistic Interpretability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_faWQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies what happens when you instruct Qwen3-8B to lie by using factual fill-in-the-blank questions crossed with different lie instructions and sorts inputs into three sets: always truthful, always incorrect, and mixed. The first component of the paper looks at the answer space and finds that removing the line instruction shrinks the number of incorrect first-token candidates by roughly 70% to 90%. It shows that when the model consistently lies, the truthful token still sits in the top 20 at the same median layer as the reported lie, which the authors refer to as "exclusion rather than substitution." The second component of the paper utilizes per-layer transcoder circuit tracing, where two attribution graphs are built from the same forward pass: one is traced to the truthful token, and the other to the incorrect token. Across a large number of recurrent features, they find that input-side attribution is basically identical between these two graphs, but direct influence on the target diverges. A linear probe can distinguish the paired graphs, and a frozen version of the classifier transfers to six other deception settings at reasonably high accuracy. The third component of the paper documents various interventions. For example, they find that amplifying truth-oriented features and suppressing incorrect-oriented features works best with total influence when features are selected per question. The authors conclude in this section that shared control over the report lives in intermediate computation rather than at the output.

Critical strengths

The paired graph design is the strongest part of the paper, and the methodological thought to trace two targets from the same forward pass successfully cancels out potential differences in activation state as a confound. The finding that direct influence fails to transfer across questions while mediated influence does is particularly interesting. The paper also runs strong controls, including a count-only control and a row shuffle control on the presence probe, as well as a hidden state probe with question group splits to prevent leakage.

Critical weaknesses

The core separation in Section 4 seems almost tautological, and the control that would be required to demonstrate otherwise is nowhere to be found in the paper. Because the two graphs share the same input and every activation, and differ only in which output logic they are traced to, the input-side attribution must be nearly identical and influence, defined relative to the target logit, therefore logically has to differ. Relatedly, the truthful and incorrect targets are different tokens. For example, "Paris" versus "London". Any statistic conditioned on the target will separate graphs traced to different tokens, whether or not deception has anything to do with this difference. The core control that would have been helpful here would have been to compare graphs traced to two different incorrect candidates, or truthful versus incorrect targets, with no lie instruction. Without a control like this, a "distributed target-dependent" pattern, rather than a dedicated state, seems like the logically expected result for any pair of competing answer tokens. For the intervention, the paper also has no random feature control at match dose. It does not seem to give any rationale for the dose parameters and seems to not report whether the generated answers actually flip to the truth.

Questions for authors

Does the probe separation hold when both graphs are traced to two different incorrect candidates and/or to truthful versus incorrect targets with no lie instruction, as highlighted in the weaknesses?

What does a random feature baseline at matched dose look like on the interventions? What fraction of interventions flip the greedy output to the truthful answer? What are the top shared mediated features here?

For clarification, which position counts as the "first answer token" in the always incorrect set, and how did the 100-example consistent subset in Figure 3 get selected?

5 comments
AuthorsOpenReview ↗

Official Comment for Reviewer faWQ (PART I)

We sincerely thank the reviewer faWQ for the careful evaluation and constructive feedback. We particularly appreciate the recognition of our paired-graph design, which controls for differences in activation states, the interesting contrast between direct and mediated influence in cross-question transfer, and the rigorous experimental controls. These observations highlight the central motivation of our work: understanding how available factual knowledge is transformed into reported answers through downstream computation and intervention.

We have organized our responses into four focused points to address the reviewer faWQ's concerns more clearly and systematically:

  1. Beyond target-conditioned attribution differences: Whether the findings in Sec. 4 go beyond expected differences between distinct output targets, and how non-deceptive controls inform their mechanistic interpretation. (W1, Q1)
  2. Feature-specific intervention and behavioral effects: Whether influence-selected features outperform layer- and dose-matched random controls, and whether interventions change actual predictions and complete answers. (W1, Q2)
  3. Shared mediated features and dose sensitivity: Which features contribute to shared mediated intervention and how the results vary across intervention strengths. (W1, Q2)
  4. Answer-token position and sample selection: How the first answer-token position is defined in the always-incorrect set and how the 100 examples in Fig. 3 were selected. (Q3)

We respond to each point below.


1. Beyond Target-Conditioned Attribution Differences (W1, Q1)

We appreciate the reviewer faWQ's important concern regarding the interpretation of target-conditioned attribution graphs. While different output targets naturally produce different attribution patterns, their quantitative organization and causal consequences are not predetermined by graph construction. Our central finding goes beyond target separability to characterize how available knowledge is transformed into the reported answer under deceptive instructions.

Specifically, Fig. 4c reveals high input-side correlations (0.941 and 0.933), but substantially different downstream correlations for direct (0.263) and mediated (0.798) influence. More importantly, these differences have distinct intervention consequences. At K=80K=80, shared interventions selected by mediated influence achieve 55.01% joint success, compared with only 28.40% for direct influence (Appendix E.3). Such differences in computational organization and intervention effectiveness do not follow automatically from choosing different attribution targets.

Non-deceptive control. To further examine whether target-conditioned differences also arise during ordinary answer competition, we sampled ten responses per question from the original 1,012-question pool without deceptive instructions. We obtained 47 truthful/incorrect pairs satisfying the same shared-prefix criterion, yielding 94 attribution graphs with balanced labels. The frozen direct ⊕\oplus mediated influence classifier from Sec. 4.2 produced the following results.

Evaluation conditionPairsAUCACCPaired ACC
Deceptive held-out1680.900083.93%86.90%
Non-deceptive control470.869680.85%91.49%

Remarkably, the classifier trained under deceptive instructions retains strong discriminative ability even when the deceptive instruction is entirely absent. This suggests that the observed target-dependent structure reflects more general computations supporting competing answers, rather than features created specifically by deceptive instructions. The classifier identifies differences between answer targets, and the two evaluation sets are not strictly matched across instructions, so their AUC difference cannot isolate an instruction-specific effect.

More importantly, this observation provides further insight into our original findings. If truthful and incorrect answers already have distinguishable computational pathways without deception, deceptive instructions need not construct new incorrect representations to alter the report. Instead, they can reshape competition among existing alternatives, consistent with Sec. 3, where removing deceptive instructions reduces viable incorrect candidates from 7.02 to 1.48 (−78.9%). Combined with the distinct causal effects of direct and mediated influence in Sec. 5, these results reinforce our central account of knowledge–report decoupling, in which available knowledge and the final report diverge through downstream selection and transformation. We will summarize this implication in Sec. 4 and report the full control in a new Appendix D.4, “Non-Deceptive Control: Competing Answer Targets.”

AuthorsOpenReview ↗

Official Comment for Reviewer faWQ (PART II)

2. Feature-Specific Intervention and Behavioral Effects (W1, Q2)

We appreciate the reviewer faWQ's valuable suggestions regarding matched random-feature controls and actual output changes, which help strengthen the rigor of our intervention analysis. We conducted additional experiments to examine whether influence-guided feature selection produces meaningful effects beyond generic activation perturbations and whether these effects translate into actual reporting behavior.

Matched random-feature controls. We introduced bidirectional random-feature interventions, matching the number of selected features in each layer and intervention direction, as well as total absolute activation displacement. Dose was matched at the dataset-average level for shared interventions and per input for sample-specific interventions. All experiments used the original 838 inputs.

InterventionFeature selectionRaw joint successSampling joint success
Shared (I=B−AI=B-A, K=80K=80)Influence-guided55.01%54.30%
SharedMatched random38.54%37.71%
Sample-specific (BB, K=40K=40)Influence-guided63.48%61.81%
Sample-specificMatched random35.56%35.56%

Influence-guided features consistently outperform matched random features in both intervention regimes, indicating that the observed effects depend on the selected computational features rather than generic activation perturbations alone. We will summarize this control in Secs. 5.2–5.3 and report the full analysis in a new Appendix E.4, “Layer- and Dose-Matched Random-Feature Controls.”

First-token greedy prediction transitions (Experiment 4). We first examined whether the probability changes translate into changes in greedy token selection. Using the stored logits before and after intervention, we evaluated prediction transitions across all 838 inputs.

Greedy prediction transitionShared cross-sampleSample-specific
Non-truthful → Truthful129/376 (34.31%)78/376 (20.74%)
Specified incorrect → Truthful107/313 (34.19%)64/313 (20.45%)
Truthful → Non-truthful59/462 (12.77%)30/462 (6.49%)
Baseline truthful-token rate55.13%55.13%
Post-intervention truthful-token rate63.48%60.86%
Net improvement+8.35 pp+5.73 pp

Both interventions increase the proportion of truthful greedy predictions, with more non-truthful-to-truthful transitions than reverse transitions. These results establish behavioral changes at the first answer-token prediction position, beyond favorable probability shifts alone. We will summarize these results in Secs. 5.2–5.3 and provide the full analysis in a new Appendix E.6, “First-Token Greedy Prediction Transitions.”

Complete-answer evaluation with native-model intervention (Experiment 8). We separately examined whether feature interventions improve complete responses when applied to Qwen3-8B's native MLP computation [1]. The selected feature activation changes were projected through the PLT decoder and added to the original MLP outputs. The native MLP and downstream normalization computations remained active, allowing the injected perturbations to propagate through subsequent model computation. Unlike the attention-frozen setting used in our earlier first-token probability and greedy-transition analyses, Experiment 8 recomputes attention queries, keys, values, attention probability matrices, and attention outputs from the intervened hidden states, together with RMSNorm statistics, downstream MLP outputs, and final logits (freeze_attention=False, constrained_layers=None). This allows attention patterns to respond to the feature perturbation at the first answer-token prediction step. Feature injection is applied only at this step; subsequent tokens are generated through ordinary autoregressive computation without further feature intervention.

On 126 stratified inputs, we generated up to ten responses per input using the original decoding settings (temperature 0.7, top-p 0.8, top-k 20). GPT-5 evaluated complete-answer correctness using the reference-answer criterion from our main experiments.

InterventionEvaluated inputsBaseline ACCIntervention ACCImprovement
Shared (I=B−AI=B-A, K=80K=80)12549.68%70.85%+21.17 pp
Sample-specific (BB, K=40K=40)12449.27%55.62%+6.34 pp

ACC is calculated as the mean per-input response accuracy. Inputs with no completed intervention runs were excluded from both conditions.

AuthorsOpenReview ↗

Official Comment for Reviewer faWQ (PART III)

These additional results are consistent with our original findings in Sec. 5. The matched controls and greedy transitions demonstrate the functional relevance of influence-guided feature selection, while the separate native-model evaluation shows that interventions at the answer-commitment position can improve complete reporting behavior. Together, they provide further support for our mechanistic interpretation of knowledge–report decoupling. We will report the complete-answer evaluation in a new Appendix E.7, “Native-Model Intervention and Complete-Answer Evaluation.”

[1] Michael Hanna and Emmanuel Ameisen, "Latent Planning Emerges with Scale," International Conference on Learning Representations (ICLR), 2026. Paper.


3. Shared Mediated Features and Dose Sensitivity (W1, Q2)

We thank the reviewer faWQ for the valuable questions regarding the identities of shared mediated features and the rationale for our intervention parameters. We conducted additional analyses to provide a more concrete characterization of these features and examine the sensitivity of our intervention results.

Top shared mediated features. We examined the dataset-level Top-80 and Bottom-80 features selected by mediated influence (I=B−AI=B-A), corresponding to the shared intervention configuration in Sec. 5. For semantic characterization, we used GPT-5-generated feature annotations following the established LLM-based automated neuron interpretation approach of Bills et al. [1], as also described in Appendix D.7. These automated explanations are not independently validated functional labels, so we treat them as qualitative evidence for potential feature roles rather than definitive interpretations.

OrientationRankFeature (layer, ID)Automated interpretation
Truthful1(33, 35155)Grammatical relations and comparisons
Truthful3(34, 76578)Structured-information delimiters
Truthful5(34, 145627)Date/time markers and separators
Incorrect1(0, 31392)Virus/disease-name suffixes
Incorrect2(27, 91570)Function words and list punctuation
Incorrect3(31, 49265)Newlines, parentheses, and separators

The automated descriptions suggest heterogeneous and partly overlapping semantic roles across both orientations, including grammatical relations, structural delimiters, and entity-related information. This tentatively suggests that shared mediated control involves general intermediate computations rather than a small set of explicit truth- or deception-related features. Importantly, the truthful/incorrect orientation is determined by measured signed mediated influence, not by these semantic annotations. We therefore regard this interpretation as suggestive rather than a validated functional characterization.

Dose sensitivity. We additionally examined the intervention parameters DD, denoting total absolute activation displacement, and qq, specifying the fraction allocated to truthful-oriented amplification, with the remainder allocated to incorrect-oriented suppression. On 126 stratified inputs, we varied DD across seven values while fixing q=0.075q=0.075, and varied qq across eight values while fixing D=675D=675.

Parameter sweepShared raw joint successSample-specific raw joint success
DD: 300–105044.44%–57.14%54.76%–69.84%
qq: 0.025–0.946.03%–56.35%61.11%–79.37%

The ranges report the minimum and maximum observed joint-success rates across each sweep. Across all tested configurations, both intervention schemes consistently produced positive mean changes in truthful-target raw probability and negative mean changes in incorrect-target raw probability, although the magnitude and joint-success rate varied. Thus, the original setting (q=0.075q=0.075, D=675D=675) serves as a common reference configuration rather than a claimed global optimum.

Together, these observations are consistent with our findings in Sec. 5. The selected shared features exhibit diverse, potentially reusable computational roles, while their intervention effects retain the expected directional behavior across different dose settings. This provides additional support for our interpretation that shared report control is associated with mediated computation, while recognizing the inferential nature of automated feature explanations.

Reference

[1] Steven Bills et al. Language Models Can Explain Neurons in Language Models. 2023.

Reviewer_6fMqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how factual knowledge and reported answers can diverge under deceptive instructions in Qwen3-8B. Combining token-level measurements, feature attribution, and interventions, the authors find that support for truthful answer tokens can remain detectable during incorrect reporting and report that differences in downstream feature contributions help determine which answer is reported.

Critical strengths
  • Strength 1: Clear presentation and detailed documentation. The paper is well organized, with experimental procedures documented in detail and findings clearly communicated through figures and explanations.

  • Strength 2: Comprehensive experimental analysis. The paper combines output-probability measurements, internal feature analysis, and intervention experiments, providing complementary perspectives on truthful and incorrect reporting.

  • Strength 3: An important research question. The paper investigates the distinction between a model’s factual knowledge and its reported answers, addressing a meaningful question for understanding truthfulness and deception in language models.

Critical weaknesses
  • Weakness 1: Limited evidence of generalization across models. The experiments use only one model, leaving it unclear whether the findings extend to other model families or scales. Replication on additional models, ideally two or three more, would strengthen the broader conclusions.

  • Weakness 2: Insufficient justification for the token-position choice. The analysis focuses on the first token. Although the authors describe it as a “precise decision point,” they provide insufficient evidence explaining why this position is appropriate or comparing token positions to justify selecting it. Such analysis would help establish whether the findings depend on this particular choice.

  • Weakness 3: Missing baseline to distinguish deception-related effects from answer-related differences. The observed feature differences could reflect the concepts or entities associated with the correct and incorrect answers, rather than the deceptive instruction itself. This alternative explanation affects two interpretations:

    • Feature distribution: Ordinary answer-related differences should be accounted for before concluding that deception-related differences involve a broad feature population. The subset specifically associated with deception could be substantially smaller.
    • Transferability: Answer-related features could remain predictive across different prompts. The observed transfer could therefore reflect persistent answer-related signals rather than a pattern specific to deceptive reporting.

    A useful control would keep the questions and answer identities fixed, replace deceptive instructions with neutral or unrelated instructions, and repeat the attribution analysis and frozen-classifier evaluation. Comparing these conditions, while checking label balance and candidate competitiveness, would help distinguish ordinary answer-related patterns from changes associated with deceptive instructions.

Questions for authors

Please see the Weaknesses section.

5 comments
AuthorsOpenReview ↗

Official Comment for Reviewer 6fMq (PART I)

We sincerely thank the reviewer 6fMq for the careful evaluation and constructive feedback. We particularly appreciate the recognition of our clear presentation and detailed documentation, comprehensive experimental analysis combining output-level measurements, feature attribution, and interventions, and the importance of investigating the distinction between factual knowledge and reported answers.

We have organized our responses into three focused points to address the reviewer 6fMq's concerns more clearly and systematically:

  1. Generalization across models: Whether the observed knowledge–report decoupling and underlying mechanisms extend to other model families and scales. (W1)
  2. Justification for first-token analysis: Why the first answer-token position provides an appropriate decision point, and whether the findings depend on this choice. (W2)
  3. Distinguishing deception-related and answer-related effects: Whether the observed feature distribution and transferability reflect deceptive reporting or ordinary differences between competing answer targets, and how non-deceptive controls clarify this distinction. (W3)

We respond to each point below.


1. Generalization Across Models (W1)

We appreciate the reviewer 6fMq's valuable suggestion regarding cross-model generalization. Our current study focuses on an in-depth mechanistic analysis of Qwen3-8B, combining output-level measurements, PLT-based attribution graphs, and feature-level interventions. Extending this framework to other models requires model-specific transcoders and extensive circuit construction, making systematic replication considerably more involved. Nevertheless, prior research [1] has shown that high-level computational mechanisms can remain consistent across Transformer model scales, even when their underlying components and circuit structures differ. Our study similarly focuses on higher-level mechanisms rather than model-specific feature realizations.

While model-specific differences may exist in the realization of these mechanisms, we acknowledge that broader generalizability remains an important direction for further investigation. We thank the reviewer 6fMq for highlighting this valuable perspective and plan to extend our analysis to different model families and scales, particularly to examine whether knowledge–report decoupling exhibits shared mechanistic properties across models.

Reference

[1] Curt Tigges, Michael Hanna, Qinan Yu, and Stella Biderman. LLM Circuit Analyses Are Consistent Across Training and Scale. NeurIPS 2024.


2. First Answer-Token Position (W2)

We recognize the importance of clarifying our choice of token position. By first answer-token position, we mean the prediction of the first token of the answer entity, after any shared generated response prefix; it need not be the first token of the entire response. For paired analyses on DmixD _ {\mathrm{mix}}, truthful and incorrect continuations are retained only when their generated-token prefixes before the distinct targets are identical (Appendix B.1). Thus, yTy ^ {T} and yIy ^ {I} are evaluated under the same preceding context and forward-pass activations, before either answer enters the context. This enables controlled comparisons of their probabilities, ranks, and attribution graphs without confounding effects from divergent generated histories.

Prior research provides complementary evidence for the importance of the answer-commitment position. Sok et al. [1] show that uncertainty peaks at the first answer token and that first-token entropy predicts answer correctness substantially better than second- or third-token entropy (Fig. 2). Across 16 models, the first correct token also accounts for the largest share of recoverable errors in 13 models (Sec. 5.4; Appendix B.2). Yeom et al. [2] find that answer-concept probability and correctness-related signals peak at the answer-commitment step, including in long-form responses where the answer follows an introductory prefix (Fig. 3). Complementarily, Gabriel [3] shows that confidence at the first content-bearing answer token predicts complete-answer correctness across three model families (mean AUROC 0.820). Zhang et al. [4] further demonstrate that early answer commitments can shape subsequent generation, leading to hallucination snowballing. Together, these findings support focusing on the answer boundary as a meaningful point of prediction and intervention.

AuthorsOpenReview ↗

Official Comment for Reviewer 6fMq (PART II)

We further validated the behavioral significance of the first answer-token position through complete-answer evaluation. On a stratified subset of 126 inputs from DmixD _ {\mathrm{mix}}, we applied interventions only at the first answer-token prediction step, followed by ordinary autoregressive generation. We generated up to ten responses per input using the original decoding settings (temperature 0.7, top-p 0.8, top-k 20). GPT-5 evaluated complete-answer correctness using the reference-answer criterion from our main experiments.

InterventionInputsBaseline ACCIntervention ACCImprovement
Shared cross-sample (I=B−AI=B-A, K=80K=80)12549.68%70.85%+21.17 pp
Sample-specific (BB, K=40K=40)12449.27%55.62%+6.34 pp

ACC is calculated as the mean per-input response accuracy. Inputs with no completed intervention runs were excluded from both conditions. These results demonstrate that intervening solely at the first answer-token position can substantially improve complete-answer correctness, further supporting the behavioral significance of this decision point. We will report the complete-answer evaluation in a new Appendix E.7, “Native-Model Intervention and Complete-Answer Evaluation,” and clarify the rationale and shared-prefix criterion in the revised manuscript.

References

[1] J. Sok et al., "On the Tip of the Tongue: First-Token Cueing as a Behavioral Probe of Recall Failures in LLMs",

[2] J. Yeom et al., "Hallucination as Commitment Failure: Larger LLMs Misfire Despite Knowing the Answer",

[3] M. Gabriel, "The First Token Knows: Single-Decode Confidence for Hallucination Detection",

[4] M. Zhang et al., "How Language Model Hallucinations Can Snowball" ICML, 2024.


3. Answer-Related and Instruction-Related Feature Differences (W3)

We appreciate the reviewer 6fMq's important distinction between answer-related and instruction-related feature differences. This distinction is closely aligned with our central finding of knowledge–report decoupling. Our paper shows that deceptive instructions reshape competition among existing answer candidates (Sec. 3), while truthful and incorrect targets exhibit distinct downstream computations within the same deceptive forward pass (Sec. 4). Crucially, targeted interventions on these computations can change the reported answer (Sec. 5). The instruction-removal experiment further supports this account, showing that removing deceptive instructions reduces viable incorrect alternatives from 7.02 to 1.48 (−78.9%).

Non-deceptive control. To examine whether target-dependent computations also exist without deception, we constructed 47 truthful/incorrect pairs (94 attribution graphs) from the same pool of 1,012 factual questions, using the original shared-prefix criterion and balanced labels (47/47). Both targets remain competitive, with median raw ranks of 1 and 2 and probabilities of 38.94% and 23.85%, respectively. We evaluated these graphs using the frozen direct ⊕\oplus mediated influence classifier trained under deceptive instructions.

Evaluation conditionPairsAUCACCPaired ACC
Deceptive held-out1680.900083.93%86.90%
Non-deceptive control470.869680.85%91.49%

The strong non-deceptive separability supports our original interpretation that truthful and incorrect answers already have distinguishable computational pathways, which deceptive instructions can exploit to alter reporting. Importantly, our classifier distinguishes answer targets, not the presence of deceptive instructions. Since the two evaluation sets are not strictly matched across instructions, their AUC difference is not interpreted as an instruction-specific effect.

Feature distribution and transferability. We further tested the distributed pattern reported in Sec. 4.2 through feature deletion using the separate binary graph-presence classifier.

Feature basisRemaining featuresTest AUC
Full basis1,7420.7636
Top-693 removed1,0490.6797
Random-693 removed1,0490.7571

The remaining predictive signal after removing features accounting for 80% of classifier coefficient contribution further supports our original finding that report-target discrimination is distributed across a broad feature population. Likewise, the 74.49% accuracy across six deception-related settings (Sec. 4.4) supports the partial transfer of target-dependent computational patterns beyond instructed lying. Neither conclusion requires these features to be exclusive to deception.

AuthorsOpenReview ↗

Official Comment for Reviewer 6fMq (PART III)

Together, these results reinforce our paper's central account: deceptive reporting arises from changes in how available knowledge is expressed and selected, rather than requiring a separate set of deception-specific representations. In the revised manuscript, we will summarize the non-deceptive-control implication in Sec. 4 and report the full control in a new Appendix D.4, “Non-Deceptive Control: Competing Answer Targets.” We will also briefly summarize the feature-deletion result in Sec. 4.2 and report its full analysis in a new Appendix D.6, “Feature-Deletion Analysis of the Distributed Predictive Signal.” These additions will clarify the distinction while preserving the original mechanistic interpretation.


At its core, our study investigates how factual knowledge and reported answers can diverge under deceptive instructions. As discussed in our rebuttal, most of our experimental configurations are grounded in well-established practices in prior work, and we fully share the reviewer's emphasis on methodological rigor. We sincerely appreciate the thoughtful questions, which have encouraged us to examine alternative explanations more comprehensively and refine the scope of our conclusions. We sincerely hope that this work can, in some small way, contribute to the community's ongoing efforts to understand knowledge–report decoupling and truthful and deceptive behaviors in LLMs.

Reviewer_DMRdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies instructed lying in Qwen3-8B to examine how truthful answers remain represented while incorrect answers are reported. Candidate-space and logit-lens analyses suggest that deceptive instructions expand incorrect alternatives while truthful answer tokens often remain highly ranked before decoding truncation. Per-layer-transcoder attribution graphs traced toward truthful and incorrect targets share input-side structure but differ downstream. Linear classifiers distinguish these target-conditioned graphs and partially transfer to additional deception-related prompt settings. Feature interventions compare direct, mediated, and total influence: total influence usually gives the strongest sample-specific joint probability shifts, while mediated influence is more effective when feature identities are shared across examples.

Critical strengths
  1. The separation of predictive support from sampling availability is informative. Sections 3.2-3.4 combine instruction-removal controls with raw probability, rank, decoding support, and layerwise readouts. For example, the truthful first token has median raw rank four in the incorrect-report set, yet only 5.1% remain sampleable under the chosen decoder. This usefully illustrates why an incorrect report need not imply that the correct candidate has vanished from the model's representations.

  2. The direct-versus-mediated comparison provides a concrete intervention hypothesis. Sections 4-5 compare targets within the same forward pass and distinguish single-question feature selection from shared feature identities. Appendices E.1-E.3 specify dose allocation and report both directional success rates and mean probability changes. The reversal between total influence in the sample-specific setting and mediated influence in the shared setting is interesting and merits investigation beyond purely correlational decoding of truthfulness.

Critical weaknesses
  1. The graph analyses do not yet isolate a mechanism specific to deceptive reporting. In Section 4, the input and activations are fixed while the attribution target changes. Similar upstream attribution and different downstream influence can also arise when tracing any two distinct output tokens, particularly because graph construction prioritizes nodes by influence on the selected target (Appendix D.2). The graph-size and shuffled-presence controls in Appendix D.4 do not distinguish this explanation from a deception-specific one. Furthermore, the claim that the mechanism is broadly distributed is inferred within a basis already restricted to features present in at least half the graphs, using regularized classifier coefficients rather than a causal test of minimal feature sets. Matched non-deceptive competing-target controls are needed to support the stronger mechanistic interpretation.

  2. The shared-intervention evidence is narrower than the claimed reusable report control. Appendix E.1 computes feature rankings and dose calibration over all of Dmix, and Appendix E.3 evaluates on those same 838 inputs. Occurrence selection also uses each example's truthful- and incorrect-target graphs (Equation 20), so sharing feature identities does not remove all target-specific information. Finally, joint success requires only favorable first-token probability changes above a small threshold, rather than an actual change to a correct completed answer; the reported mean probability shifts are useful but do not resolve this distinction. These results establish a useful within-cohort intervention pattern, but do not yet demonstrate that a fixed controller transfers to unseen questions or corrects deceptive reports.

Questions for authors
  1. Does the upstream/downstream contrast and mediated-influence advantage remain distinctive when compared with non-deceptive prompts and pairs of competing answer tokens matched for probability, rank, and answer type? This would help separate a deception mechanism from general target-conditioned attribution structure.
  2. What happens when recurrent-feature selection, rankings, intervention scales, and occurrence-selection rules are fixed using only training questions, then evaluated on question-disjoint examples? Can the shared intervention be applied without consulting the test example's truthful-target graph?
  3. Do these interventions increase complete-answer correctness under the original model and decoder, and are the effects larger than those of layer- and dose-matched random-feature controls? Please clarify which nonlinear components are recomputed during intervention versus held fixed in the local replacement model. These results would determine how strongly the probability shifts support causal report correction.
6 comments
AuthorsOpenReview ↗

Official Comment for Reviewer DMRd(PART I)

We sincerely thank the reviewer DMRd for the careful evaluation and constructive feedback. We particularly appreciate the recognition that separating predictive support from sampling availability provides an informative perspective, and that the direct-versus-mediated intervention comparison offers an interesting hypothesis for further investigation. Since several concerns raised in the review involve multiple related but distinct aspects, we have organized our responses into six focused points to address them more clearly and systematically:

  1. Specificity of the graph results: Whether the upstream/downstream contrast and mediated-influence advantage distinguish deceptive reporting from ordinary competition between answer targets. (W1, Q1)
  2. Evidence for a distributed pattern: Whether the recurrent-feature selection and classifier coefficients support the conclusion that report separation is distributed across a broad feature population. (W1)
  3. Transfer of shared cross-sample intervention: Whether feature selection and intervention scales generalize to unseen questions, and what information is required at test time. (W2, Q2)
  4. Feature-specific intervention effects: Whether influence-selected features produce stronger intervention effects than layer- and dose-matched random features. (W2, Q3)
  5. Behavioral effects on complete answers: Whether the interventions improve complete-answer correctness under the original model and decoding settings. (W2, Q3)
  6. Intervention implementation and dose robustness: Which computational components are recomputed or held fixed during intervention, and how robust the effects are to different intervention scales. (W2, Q3)

We respond to each point below.


1. Specificity of the Graph Results (W1, Q1)

We thank the reviewer DMRd for highlighting the distinction between general target-conditioned attribution structures and those associated with deceptive reporting. To further examine this distinction, we added a non-deceptive control to investigate whether similar graph patterns also arise when the model considers competing answers without deceptive instructions.

Specifically, we sampled ten responses to each of the same 1,012 questions after removing the deceptive instruction. Following Appendix B.1, we retained 47 questions with truthful and incorrect continuations sharing the same pre-target token prefix. Each pair therefore shares the same question, answer type, and pre-target context. The truthful and incorrect targets have median raw ranks of 1 and 2, and median raw probabilities of 38.94% and 23.85%, respectively. We then evaluated the frozen direct ⊕ mediated influence classifier from Sec. 4.2 on these attribution graphs.

Evaluation setPairsAUCACCPaired ACC
Deceptive instruction, original held-out set1680.900083.93%86.90%
No deceptive instruction, new paired set470.869680.85%91.49%

The classifier retains strong discrimination without deceptive instructions (AUC 0.8696 vs. 0.9000). Importantly, this finding is consistent with our original account in Secs. 3.2–3.4. Our analysis already suggests that deceptive instructions do not necessarily create entirely new incorrect-answer representations; instead, they strengthen the competition from incorrect alternatives while truthful answers retain predictive support. The strong discrimination in the non-deceptive setting further suggests that the target-conditioned computational patterns identified by our probe are already present in ordinary answer competition. Thus, the probe is not intended to detect whether a prompt contains a deceptive instruction, but to distinguish the computational patterns associated with truthful and incorrect target continuations. This distinction is important: the contribution of our work lies in revealing how deceptive instructions exploit existing target-conditioned computational structures to alter reporting behavior, rather than in identifying a computational structure that exists exclusively under deception.

We thank the reviewer DMRd again for suggesting this additional comparison. It has helped us articulate more precisely the relationship between general target-conditioned computation and deceptive reporting, and further strengthens our central conclusion that deceptive instructions can alter the model's reporting behavior while leaving truthful knowledge available. In the revised manuscript, we will summarize this implication in Sec. 4 and report the full non-deceptive control in a new Appendix D.4, “Non-Deceptive Control: Competing Answer Targets.”

AuthorsOpenReview ↗

Official Comment for Reviewer DMRd (PART II)

2. Evidence for a Distributed Pattern (W1)

We recognize the importance of the reviewer DMRd's distinction between the distribution of a classifier's predictive signal and the size of a causally sufficient feature set. To further examine whether the observed coefficient distribution reflects contributions beyond a small set of dominant features, we conducted a feature-deletion analysis.

Using the binary feature-presence classifier from Table 1, we ranked features by their training-set coefficient contributions, removed the highest-ranked features, and retrained the classifier on the remaining feature basis. The held-out examples were not used for feature ranking. As controls, we randomly removed an equal number of features at each cutoff. We evaluated four cumulative coefficient contribution thresholds (50%, 60%, 70%, and 80%).

Feature basisRemaining featuresTest AUCPaired ACCThreshold ACC
Full recurrent basis1,7420.763675.60%69.35%
Remove Top-309 (50%)1,4330.706674.40%67.56%
Randomly remove 3091,4330.743873.81%68.15%
Remove Top-409 (60%)1,3330.697474.40%66.07%
Randomly remove 4091,3330.755969.64%67.26%
Remove Top-534 (70%)1,2080.690474.40%63.10%
Randomly remove 5341,2080.745575.60%68.45%
Remove Top-693 (80%)1,0490.679772.02%64.29%
Randomly remove 6931,0490.757176.19%68.75%

Removing the leading features consistently causes a larger AUC reduction than randomly removing the same number of features across all four cutoffs, confirming their disproportionate contributions. Nevertheless, even after removing the 693 features accounting for 80% of the cumulative coefficient contribution, a classifier retrained on the remaining 1,049 features still achieves 0.6797 AUC and 72.02% paired accuracy. These findings further support our original conclusion in Sec. 4.2 that "report separation is distributed across a broad feature population rather than localized to a small dedicated feature set." Although individual features contribute unequally, the discriminative signal is not concentrated exclusively in a small subset of dominant features. This analysis further substantiates the distributed nature of the predictive signal within the recurrent-feature basis, without inferring a minimal causal feature set from classifier coefficients. In the revised manuscript, we will briefly summarize this result in Sec. 4.2 and report the full analysis in a new Appendix D.6, “Feature-Deletion Analysis of the Distributed Predictive Signal.”


3. Transfer of Shared Cross-Sample Intervention (W2, Q2)

We agree that whether shared feature identities and intervention scales generalize to unseen questions is an important question. To examine this, we conducted a question-disjoint evaluation, separating feature selection and calibration from intervention testing.

We split the 838 inputs in DmixD_{\mathrm{mix}}, representing 515 distinct questions, into 671 training and 167 test inputs, ensuring that all deceptive-instruction variants of each question remained in the same partition.

For this question-disjoint evaluation, we independently constructed the recurrent-feature basis using only the 671 training inputs. Each training input contributes one truthful-target graph and one incorrect-target graph. We retained a feature if it appeared in at least 50% of the truthful-target graphs or in at least 50% of the incorrect-target graphs, yielding 1,756 features. This training-only basis is distinct from the 1,742-feature basis selected using the full 838-input cohort in the main analysis.

Using this training-derived basis, we ranked features by the dataset-level signed mediated-influence contrast (I=B−AI=B-A), treating missing graph contributions as zero, selected the Top-80 and Bottom-80 features with their respective orientations, and calibrated a single pair of shared intervention scales using the training-set mean activation masses. All feature identities, rankings, and intervention scales were then fixed and applied to the 167 test inputs.

MetricOriginal 838-input cohortQuestion-disjoint test (167)
Mean raw-probability change, truthful target+0.068234+0.067107
Mean raw-probability change, reported incorrect target-0.028598-0.033028
Raw joint success55.01%53.29%
Mean sampling-probability change, truthful target+0.081530+0.073671
Mean sampling-probability change, reported incorrect target-0.071301-0.071965
Sampling joint success54.30%54.49%
AuthorsOpenReview ↗

Official Comment for Reviewer DMRd(PART III)

On the question-disjoint test set, the intervention increases the truthful-target probability while decreasing the reported incorrect-target probability, achieving 53.29% raw joint success (89/167) and 54.49% sampling joint success (91/167). Both results remain close to those obtained on the original cohort (55.01% and 54.30%, respectively). These results demonstrate that the shared feature identities and intervention scales generalize across questions, further supporting our original finding that mediated influence provides a transferable basis for shared cross-sample intervention.

One aspect remains target-conditioned: Eq. 20 uses each test input's truthful- and incorrect-target graphs to locate feature occurrences. Thus, the demonstrated generalization concerns feature selection, ranking, and intervention scales under the same occurrence-selection rule, rather than a fully target-independent controller. We will make this scope condition explicit in the revised manuscript.


4. Feature-specific Intervention Effects (W2, Q3)

We consider this comparison valuable for strengthening the rigor of our intervention analysis, as it helps distinguish the effects of influence-guided feature selection from those of generic activation perturbations. We therefore introduced bidirectional random-feature controls for both shared cross-sample and sample-specific interventions.

Specifically, we randomly sampled feature identities without replacement from the same layer-specific candidate pools, without using influence scores. At each layer, the random control matched the numbers of features assigned to truthful-oriented amplification and incorrect-oriented suppression, respectively. We further matched the total absolute activation displacement relative to the clean baseline: the dataset-average dose for shared cross-sample intervention and the per-input dose for sample-specific intervention. The control does not separately match feature-occurrence counts or the realized dose at every layer.

Setting / SelectionRaw JointRaw UpRaw DownSampling JointSampling UpSampling Down
Shared / Selected55.01%73.03%62.65%54.30%64.20%62.29%
Shared / Random38.54%55.73%49.16%37.71%47.02%46.18%
Sample-specific / Selected63.48%71.00%80.19%61.81%67.30%70.64%
Sample-specific / Random35.56%46.42%61.81%35.56%42.84%48.09%

Raw/Sampling Joint: joint success rate; Up: truthful-target probability increases; Down: reported incorrect-target probability decreases.

The corresponding mean probability changes are:

Setting / SelectionRaw TruthfulRaw IncorrectSampling TruthfulSampling Incorrect
Shared / Selected+0.068234−0.028598+0.081530−0.071301
Shared / Random+0.015117+0.005615+0.005180−0.002633
Sample-specific / Selected+0.032381−0.038568+0.057631−0.068315
Sample-specific / Random−0.002374−0.017519+0.006942−0.024359

The results consistently show that influence-selected features outperform random features in both intervention settings. In particular, random features produce substantially weaker bidirectional probability shifts, despite matching the layer-wise feature counts and total intervention dose. These findings indicate that the observed intervention effects depend on the influence-guided selection of feature identities, rather than generic activation perturbations alone. We will summarize this control alongside the intervention results in Secs. 5.2–5.3 and report the full analysis in a new Appendix E.4, “Layer- and Dose-Matched Random-Feature Controls.”


5. Greedy Prediction Transitions (W2, Q3)

We further examined whether first-token probability shifts translate into changes in the model's greedy predictions. We compared the predictions before and after intervention across all 838 inputs, with results summarized below.

Greedy prediction transitionShared cross-sampleSample-specific
Non-truthful → Truthful129/376 (34.31%)78/376 (20.74%)
Specified incorrect → Truthful (subset)107/313 (34.19%)64/313 (20.45%)
Truthful → Non-truthful59/462 (12.77%)30/462 (6.49%)
Baseline truthful-token rate55.13%55.13%
Post-intervention truthful-token rate63.48%60.86%
Net improvement+8.35 pp+5.73 pp

Both interventions produce substantially more non-truthful-to-truthful transitions than reverse transitions. Notably, among the 313 inputs initially selecting the specified incorrect target, 107 (34.19%) and 64 (20.45%) switch to the truthful target under shared and sample-specific interventions, respectively. Consequently, the overall truthful greedy prediction rate increases by 8.35 and 5.73 percentage points.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.