- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies what happens when you instruct Qwen3-8B to lie by using factual fill-in-the-blank questions crossed with different lie instructions and sorts inputs into three sets: always truthful, always incorrect, and mixed. The first component of the paper looks at the answer space and finds that removing the line instruction shrinks the number of incorrect first-token candidates by roughly 70% to 90%. It shows that when the model consistently lies, the truthful token still sits in the top 20 at the same median layer as the reported lie, which the authors refer to as "exclusion rather than substitution." The second component of the paper utilizes per-layer transcoder circuit tracing, where two attribution graphs are built from the same forward pass: one is traced to the truthful token, and the other to the incorrect token. Across a large number of recurrent features, they find that input-side attribution is basically identical between these two graphs, but direct influence on the target diverges. A linear probe can distinguish the paired graphs, and a frozen version of the classifier transfers to six other deception settings at reasonably high accuracy. The third component of the paper documents various interventions. For example, they find that amplifying truth-oriented features and suppressing incorrect-oriented features works best with total influence when features are selected per question. The authors conclude in this section that shared control over the report lives in intermediate computation rather than at the output.
Critical strengths
The paired graph design is the strongest part of the paper, and the methodological thought to trace two targets from the same forward pass successfully cancels out potential differences in activation state as a confound. The finding that direct influence fails to transfer across questions while mediated influence does is particularly interesting. The paper also runs strong controls, including a count-only control and a row shuffle control on the presence probe, as well as a hidden state probe with question group splits to prevent leakage.
Critical weaknesses
The core separation in Section 4 seems almost tautological, and the control that would be required to demonstrate otherwise is nowhere to be found in the paper. Because the two graphs share the same input and every activation, and differ only in which output logic they are traced to, the input-side attribution must be nearly identical and influence, defined relative to the target logit, therefore logically has to differ. Relatedly, the truthful and incorrect targets are different tokens. For example, "Paris" versus "London". Any statistic conditioned on the target will separate graphs traced to different tokens, whether or not deception has anything to do with this difference. The core control that would have been helpful here would have been to compare graphs traced to two different incorrect candidates, or truthful versus incorrect targets, with no lie instruction. Without a control like this, a "distributed target-dependent" pattern, rather than a dedicated state, seems like the logically expected result for any pair of competing answer tokens. For the intervention, the paper also has no random feature control at match dose. It does not seem to give any rationale for the dose parameters and seems to not report whether the generated answers actually flip to the truth.
Questions for authors
Does the probe separation hold when both graphs are traced to two different incorrect candidates and/or to truthful versus incorrect targets with no lie instruction, as highlighted in the weaknesses?
What does a random feature baseline at matched dose look like on the interventions? What fraction of interventions flip the greedy output to the truthful answer? What are the top shared mediated features here?
For clarification, which position counts as the "first answer token" in the always incorrect set, and how did the 100-example consistent subset in Figure 3 get selected?
5 comments
Official Comment for Reviewer faWQ (PART II)
2. Feature-Specific Intervention and Behavioral Effects (W1, Q2)
We appreciate the reviewer faWQ's valuable suggestions regarding matched random-feature controls and actual output changes, which help strengthen the rigor of our intervention analysis. We conducted additional experiments to examine whether influence-guided feature selection produces meaningful effects beyond generic activation perturbations and whether these effects translate into actual reporting behavior.
Matched random-feature controls. We introduced bidirectional random-feature interventions, matching the number of selected features in each layer and intervention direction, as well as total absolute activation displacement. Dose was matched at the dataset-average level for shared interventions and per input for sample-specific interventions. All experiments used the original 838 inputs.
| Intervention | Feature selection | Raw joint success | Sampling joint success |
|---|---|---|---|
| Shared (, ) | Influence-guided | 55.01% | 54.30% |
| Shared | Matched random | 38.54% | 37.71% |
| Sample-specific (, ) | Influence-guided | 63.48% | 61.81% |
| Sample-specific | Matched random | 35.56% | 35.56% |
Influence-guided features consistently outperform matched random features in both intervention regimes, indicating that the observed effects depend on the selected computational features rather than generic activation perturbations alone. We will summarize this control in Secs. 5.2–5.3 and report the full analysis in a new Appendix E.4, “Layer- and Dose-Matched Random-Feature Controls.”
First-token greedy prediction transitions (Experiment 4). We first examined whether the probability changes translate into changes in greedy token selection. Using the stored logits before and after intervention, we evaluated prediction transitions across all 838 inputs.
| Greedy prediction transition | Shared cross-sample | Sample-specific |
|---|---|---|
| Non-truthful → Truthful | 129/376 (34.31%) | 78/376 (20.74%) |
| Specified incorrect → Truthful | 107/313 (34.19%) | 64/313 (20.45%) |
| Truthful → Non-truthful | 59/462 (12.77%) | 30/462 (6.49%) |
| Baseline truthful-token rate | 55.13% | 55.13% |
| Post-intervention truthful-token rate | 63.48% | 60.86% |
| Net improvement | +8.35 pp | +5.73 pp |
Both interventions increase the proportion of truthful greedy predictions, with more non-truthful-to-truthful transitions than reverse transitions. These results establish behavioral changes at the first answer-token prediction position, beyond favorable probability shifts alone. We will summarize these results in Secs. 5.2–5.3 and provide the full analysis in a new Appendix E.6, “First-Token Greedy Prediction Transitions.”
Complete-answer evaluation with native-model intervention (Experiment 8). We separately examined whether feature interventions improve complete responses when applied to Qwen3-8B's native MLP computation [1]. The selected feature activation changes were projected through the PLT decoder and added to the original MLP outputs. The native MLP and downstream normalization computations remained active, allowing the injected perturbations to propagate through subsequent model computation. Unlike the attention-frozen setting used in our earlier first-token probability and greedy-transition analyses, Experiment 8 recomputes attention queries, keys, values, attention probability matrices, and attention outputs from the intervened hidden states, together with RMSNorm statistics, downstream MLP outputs, and final logits (freeze_attention=False, constrained_layers=None). This allows attention patterns to respond to the feature perturbation at the first answer-token prediction step. Feature injection is applied only at this step; subsequent tokens are generated through ordinary autoregressive computation without further feature intervention.
On 126 stratified inputs, we generated up to ten responses per input using the original decoding settings (temperature 0.7, top-p 0.8, top-k 20). GPT-5 evaluated complete-answer correctness using the reference-answer criterion from our main experiments.
| Intervention | Evaluated inputs | Baseline ACC | Intervention ACC | Improvement |
|---|---|---|---|---|
| Shared (, ) | 125 | 49.68% | 70.85% | +21.17 pp |
| Sample-specific (, ) | 124 | 49.27% | 55.62% | +6.34 pp |
ACC is calculated as the mean per-input response accuracy. Inputs with no completed intervention runs were excluded from both conditions.
Official Comment for Reviewer faWQ (PART III)
These additional results are consistent with our original findings in Sec. 5. The matched controls and greedy transitions demonstrate the functional relevance of influence-guided feature selection, while the separate native-model evaluation shows that interventions at the answer-commitment position can improve complete reporting behavior. Together, they provide further support for our mechanistic interpretation of knowledge–report decoupling. We will report the complete-answer evaluation in a new Appendix E.7, “Native-Model Intervention and Complete-Answer Evaluation.”
[1] Michael Hanna and Emmanuel Ameisen, "Latent Planning Emerges with Scale," International Conference on Learning Representations (ICLR), 2026. Paper.
3. Shared Mediated Features and Dose Sensitivity (W1, Q2)
We thank the reviewer faWQ for the valuable questions regarding the identities of shared mediated features and the rationale for our intervention parameters. We conducted additional analyses to provide a more concrete characterization of these features and examine the sensitivity of our intervention results.
Top shared mediated features. We examined the dataset-level Top-80 and Bottom-80 features selected by mediated influence (), corresponding to the shared intervention configuration in Sec. 5. For semantic characterization, we used GPT-5-generated feature annotations following the established LLM-based automated neuron interpretation approach of Bills et al. [1], as also described in Appendix D.7. These automated explanations are not independently validated functional labels, so we treat them as qualitative evidence for potential feature roles rather than definitive interpretations.
| Orientation | Rank | Feature (layer, ID) | Automated interpretation |
|---|---|---|---|
| Truthful | 1 | (33, 35155) | Grammatical relations and comparisons |
| Truthful | 3 | (34, 76578) | Structured-information delimiters |
| Truthful | 5 | (34, 145627) | Date/time markers and separators |
| Incorrect | 1 | (0, 31392) | Virus/disease-name suffixes |
| Incorrect | 2 | (27, 91570) | Function words and list punctuation |
| Incorrect | 3 | (31, 49265) | Newlines, parentheses, and separators |
The automated descriptions suggest heterogeneous and partly overlapping semantic roles across both orientations, including grammatical relations, structural delimiters, and entity-related information. This tentatively suggests that shared mediated control involves general intermediate computations rather than a small set of explicit truth- or deception-related features. Importantly, the truthful/incorrect orientation is determined by measured signed mediated influence, not by these semantic annotations. We therefore regard this interpretation as suggestive rather than a validated functional characterization.
Dose sensitivity. We additionally examined the intervention parameters , denoting total absolute activation displacement, and , specifying the fraction allocated to truthful-oriented amplification, with the remainder allocated to incorrect-oriented suppression. On 126 stratified inputs, we varied across seven values while fixing , and varied across eight values while fixing .
| Parameter sweep | Shared raw joint success | Sample-specific raw joint success |
|---|---|---|
| : 300–1050 | 44.44%–57.14% | 54.76%–69.84% |
| : 0.025–0.9 | 46.03%–56.35% | 61.11%–79.37% |
The ranges report the minimum and maximum observed joint-success rates across each sweep. Across all tested configurations, both intervention schemes consistently produced positive mean changes in truthful-target raw probability and negative mean changes in incorrect-target raw probability, although the magnitude and joint-success rate varied. Thus, the original setting (, ) serves as a common reference configuration rather than a claimed global optimum.
Together, these observations are consistent with our findings in Sec. 5. The selected shared features exhibit diverse, potentially reusable computational roles, while their intervention effects retain the expected directional behavior across different dose settings. This provides additional support for our interpretation that shared report control is associated with mediated computation, while recognizing the inferential nature of automated feature explanations.
Reference
[1] Steven Bills et al. Language Models Can Explain Neurons in Language Models. 2023.
Official Comment for Reviewer faWQ (PART I)
We sincerely thank the reviewer faWQ for the careful evaluation and constructive feedback. We particularly appreciate the recognition of our paired-graph design, which controls for differences in activation states, the interesting contrast between direct and mediated influence in cross-question transfer, and the rigorous experimental controls. These observations highlight the central motivation of our work: understanding how available factual knowledge is transformed into reported answers through downstream computation and intervention.
We have organized our responses into four focused points to address the reviewer faWQ's concerns more clearly and systematically:
We respond to each point below.
1. Beyond Target-Conditioned Attribution Differences (W1, Q1)
We appreciate the reviewer faWQ's important concern regarding the interpretation of target-conditioned attribution graphs. While different output targets naturally produce different attribution patterns, their quantitative organization and causal consequences are not predetermined by graph construction. Our central finding goes beyond target separability to characterize how available knowledge is transformed into the reported answer under deceptive instructions.
Specifically, Fig. 4c reveals high input-side correlations (0.941 and 0.933), but substantially different downstream correlations for direct (0.263) and mediated (0.798) influence. More importantly, these differences have distinct intervention consequences. At K=80, shared interventions selected by mediated influence achieve 55.01% joint success, compared with only 28.40% for direct influence (Appendix E.3). Such differences in computational organization and intervention effectiveness do not follow automatically from choosing different attribution targets.
Non-deceptive control. To further examine whether target-conditioned differences also arise during ordinary answer competition, we sampled ten responses per question from the original 1,012-question pool without deceptive instructions. We obtained 47 truthful/incorrect pairs satisfying the same shared-prefix criterion, yielding 94 attribution graphs with balanced labels. The frozen direct ⊕ mediated influence classifier from Sec. 4.2 produced the following results.
Remarkably, the classifier trained under deceptive instructions retains strong discriminative ability even when the deceptive instruction is entirely absent. This suggests that the observed target-dependent structure reflects more general computations supporting competing answers, rather than features created specifically by deceptive instructions. The classifier identifies differences between answer targets, and the two evaluation sets are not strictly matched across instructions, so their AUC difference cannot isolate an instruction-specific effect.
More importantly, this observation provides further insight into our original findings. If truthful and incorrect answers already have distinguishable computational pathways without deception, deceptive instructions need not construct new incorrect representations to alter the report. Instead, they can reshape competition among existing alternatives, consistent with Sec. 3, where removing deceptive instructions reduces viable incorrect candidates from 7.02 to 1.48 (−78.9%). Combined with the distinct causal effects of direct and mediated influence in Sec. 5, these results reinforce our central account of knowledge–report decoupling, in which available knowledge and the final report diverge through downstream selection and transformation. We will summarize this implication in Sec. 4 and report the full control in a new Appendix D.4, “Non-Deceptive Control: Competing Answer Targets.”