acceptodds
Under review as a conference paper at ICLR 2027

Cross-Model In-Context Neurofeedback Reveals (Limited) Activation Control in LLMs

Abstract

Probes on activation vectors are widely used to detect misbehavior in language models, but their success relies on the assumption that models cannot control what their activations encode. However, humans can learn, through neurofeedback, to control the firing of even single neurons; therefore, recent studies investigated whether this ability extends to language models, reaching conflicting conclusions. Notably, no experimental design so far disentangles competence at the task from genuine access to one's own activations. We fill this gap with in-context neurofeedback sessions, in which a "subject" model is asked to maximize (or minimize) a score derived from its activations, which we show back to it at every turn. In our first experiment, the subject generates the sentences to be scored. However, to disentangle competence at the task from actual control, we also let a second "writer" model generate them, while still deriving the score from the activations of the subject. We test ten open-weight models from five families, with sizes from 9B to 32B. In the first experiment, some models move their own scores considerably (even modulo controls), but no more than a writer of comparable competence moves them. Is this because succeeding at the task requires no privileged access, or because the subject controls its activations while processing the sentences, whoever wrote them? To answer this, we fix the sentence, prefilled as the subject's reply at every turn, and ask the subject to increase or decrease the score. Strikingly, three out of ten models show small but consistent control effects. We further ask whether models can control their activations to the point of fooling an activation monitor. To this aim, we make models repeat sentences a monitor would flag, such as a deceptive answer for a deception monitor, and ask them to raise or lower the monitor's score. No model learns to lower it, but, notably, one model can raise it. In-context control of a model's own activations might not yet be a threat to activation monitors, but our results show it exists in a few models, and it needs to be tracked as models evolve. Code and data will be released upon acceptance.

Primary area:
interpretability and explainable AI
Keywords:
interpretability, activation monitoring, linear probes, introspection, neurofeedback, AI safety, in-context learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_gj4vOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether LLMs can control their own activations in context via a neurofeedback-style protocol. It separates three hypotheses — (i) in-context pattern matching over sentences, (ii) sentence selection based on privileged access to activations, and (iii) changing how a fixed sentence is processed — using a writer/subject decoupling (Exp. 1), a fixed-sentence setting (Exp. 2), and fixed-sentence sessions against six safety monitors with and without CoT (Exp. 3–4). Across 10 open-weight models and 42 readouts, free-generation effects are explained by writer competence rather than self-access; with the sentence fixed, three models (Qwen3.5-27B, Qwen3.6-27B, Gemma-4-31B) show small but significant effects (≤2 points). Against monitors, only Qwen3.5-27B shows an effect, and only in the direction of raising the score.

Critical strengths
  • The writer/subject decoupling and the additive fit (competence + susceptibility) cleanly separate (i) from (ii), closing the gap left by Aoki et al., who refuted the earlier positive finding but did not directly establish its cause.
  • Thorough evaluation: 42 readouts × 2 depths × disclosure conditions × 10 models, 168,000 sessions.
  • Effect sizes are reported without exaggeration, and the safety implications are stated conservatively.
Critical weaknesses
  • Effect size and control validity (Exp. 2). The 1.8-point effect is 2% of the scale. That activations on a fixed sentence differ when the preceding numbers differ is trivial; the question is only whether the shift depends on the instruction. A significant objective difference could still arise from an interaction between the “max”/”min” wording and the different statistics of real vs. uniform-random scores. The Limitations section acknowledges this, but it is a fundamental weakness of the control design.
  • Additivity assumption. The diagonal excess depends on s_o τ = m + a_W + b_S. Any W × S interaction (a writer that is specifically effective on a particular subject) would invalidate it. Fig. 5 is consistent with additivity, but no test is reported.
  • No account of model dependence. Why only Qwen3.5/3.6 and Gemma-4-31B? Training data, post-training, and scale are all plausible, but the paper offers no mechanistic analysis or discussion.
Questions for authors
  • How would the results generalize to other model families and larger sizes?
  • Do training phases (base / SFT / RL) differ in their ability to control activations?
  • What are the practical or safety implications that developers should monitor today, and what would constitute a real threat?
3 comments
AuthorsOpenReview ↗

[1/2] Effect size and control validity

We thank the reviewers for their careful reviews. We are glad that they found that our cross-model writer comparison resolves an ambiguity left by prior work (Am4h, gj4v), that the fixed-sentence experiments provide new insights (Am4h, XxMZ), and that our evaluation is thorough (gj4v).

Effect size and control validity (Exp. 2). The 1.8-point effect is 2% of the scale. A significant objective difference could still arise from an interaction between the “max”/”min” wording and the different statistics of real vs. uniform-random scores.

We thank the reviewer for raising this point, which led us to run a new control. We agree that the effect is small on average, and we claim that it is statistically significant, not that it is large.

To test the interaction you describe, we reran part of our experiments with a shuffled control, in which each session is shown the scores of a real session in a random order, so that they have the same statistics as real feedback (new Appendix F.1 and Figure 12). We find that, with this control in place, the effect of Qwen3.5-27B and Qwen3.6-27B is weaker, about half of what we reported, but remains significant, while the effect of Gemma-4-31B disappears. The half that remains requires the feedback to come from the model's own activations, turn by turn. The half that disappears does not depend on the feedback, but we cannot tell whether it comes from the interaction you describe or from the model trying to follow the instruction without using the scores, which would mean zero-shot control. Since our question is whether models use the feedback, our metric should not count it.

The average over our 42 readouts, however, understates this control, since some of the readouts we designed are especially hard to move. As our paper already shows (Figure 8, and now also in the main text, Figure 3, left), the effect is concentrated on the readouts read at 66% of the depth, in agreement with previous work [1, 2]. The shuffled control confirms this: the readouts defined as a direction in the residual stream space move by two to three times the average at 66% of the depth, and by less than 1 point at 33% in every model (the metric is the objective difference at turn 30, measured against the shuffled control, as in the new Figure 12):

ModelAll 42 readoutsReadouts at 33% of the depthReadouts at 66% of the depth
Qwen3.5-27B0.9 [0.7, 1.2]−0.2 [−0.34, −0.03]2.8 [2.2, 3.5]
Qwen3.6-27B0.5 [0.3, 0.8]0.2 [0.01, 0.37]1.3 [0.8, 1.8]
Gemma-4-31B0.1 [−0.2, 0.3]0.0 [−0.1, 0.0]0.3 [−0.2, 0.8]
Mistral-Small-3.2-24B0.1 [−0.5, 0.7]−0.8 [−2.1, 0.5]0.4 [−0.5, 1.3]

As is well known, no control condition is perfect [3], and we keep the random scores as the main control of the paper, because they allow a comparison at every turn and, since they do not depend on the real sessions, can be shared by all the readouts and run for every experiment.

Our main finding still holds: with the sentence fixed, a few models show a small but significant control effect.

References

[1] Ji-An Li, Hua-Dong Xiong, Robert C. Wilson, Marcelo G. Mattar, and Marcus K. Benna. Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations. NeurIPS 2025.

[2] Marek Mateusz Kowalski, Joshua Fonseca Rivera, Uzay Macar, and David Demitri Africa. Measuring Activation Control in Large Language Models. arXiv:2608.21664, 2026.

[3] Bettina Sorger, Frank Scharnowski, David E. J. Linden, Michelle Hampson, and Kymberly D. Young. Control freaks: Towards optimal selection of control conditions for fMRI neurofeedback studies. NeuroImage 2019.

AuthorsOpenReview ↗

[2/2] Additivity, model dependence, and safety implications

Additivity assumption. ... Fig. 5 is consistent with additivity, but no test is reported.

We thank the reviewer for pointing this out. We agree that this needed a test, and we added one (new paragraph "Additivity" and Table 6 in Appendix E.1). We predicted every pair with W ≠ S by fitting the additive model on the other 19 pairs with W ≠ S, in all four combinations of objective and disclosure (we exclude the pairs with W = S, since they are the ones whose excess we measure). The prediction errors are within sampling noise, so we find no evidence of a W × S interaction, that is, of a writer that is specifically effective on a particular subject. We built the additive model to separate two reasons why a model can move its own readouts without privileged access. It can be a good writer, which moves the readouts of any subject (a_W), or a susceptible subject, whose readouts move whoever writes the sentences (b_S). However, the fit itself shows a posteriori that subjects differ little in susceptibility, so our conclusion does not actually need the model, and when we compare each model's effect on itself with its average effect on the other four subjects, no model shows a significant advantage.

No account of model dependence. Why only Qwen3.5/3.6 and Gemma-4-31B?

How would the results generalize to other model families and larger sizes?

Do training phases (base / SFT / RL) differ in their ability to control activations?

We thank the reviewer for these questions, which also led us to test a larger model. We honestly cannot say yet what explains the differences across models. Training data and post-training are plausible causes, but the training data of these models is not public, and no base checkpoint is released for Qwen3.5-27B or Qwen3.6-27B, so we cannot test them. OLMo-3.1-32B releases a checkpoint after each training phase (SFT, DPO, and reinforcement learning), but its final model doesn't show control, so comparing them would not tell us when control emerges. We also find that scale does not explain it, because within the Qwen3.5 family the effect appears in the 27B model but neither in the smaller Qwen3.5-9B nor in the larger Qwen3.5-122B-A10B, which we added for this revision (−0.1 points, Section 3.2, Figure 3). Kowalski et al. [2] also find that the ability of models to control their activations does not track their number of parameters. Across the five families we tested (Qwen, Gemma, GPT-OSS, Mistral, and OLMo), robust control appears only in the Qwen family, so we cannot claim that it generalizes to other families.

However, when we observe a control effect, we characterize it. Our paper already shows that it appears within the first few turns and does not require telling the model what is measured (Section 3.4, Figures 3 and 7), and the analyses we added for the revision show that it is targeted (new Appendix F.2) and that it only appears in the layers between 55 and 85% of the depth (new Appendix F.3).

What are the practical or safety implications that developers should monitor today, and what would constitute a real threat?

We appreciate this question, since it is the motivation of our work. The threat we study is a model that evades an activation monitor by controlling what its activations encode. McGuinness et al. [4] show that finetuning can teach this, and that the evasion transfers to unseen monitors. We ask whether it can also arise in context, where neurofeedback gives an upper bound, since the model sees the monitor's score at every turn. A real threat would be a model that lowers the score of a monitor enough to change its decisions, and no model does so in our experiments (Section 4). Developers should therefore track this capability as new models are released: we find control only in recent models, and our fixed-sentence protocol is cheap to rerun with the released code.

References

[2] Marek Mateusz Kowalski, Joshua Fonseca Rivera, Uzay Macar, and David Demitri Africa. Measuring Activation Control in Large Language Models. arXiv:2608.21664, 2026.

[4] Max McGuinness, Alex Serrano, Luke Bailey, and Scott Emmons. Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitors. arXiv:2512.11949, 2025.

Thank you for the detailed response and the additional experiments. My main concerns have been addressed.

  • Control validity (Exp. 2): The shuffled control directly tests the confound I raised. I appreciate that you report the outcome transparently: the effect for Qwen3.5/3.6 is roughly halved but remains significant, and the Gemma-4-31B effect disappears. The breakdown by depth also makes clearer where the effect is concentrated.
  • Additivity: The leave-one-pair-out test resolves my concern, and the comparison of each model's effect on itself against its average effect on other subjects shows that the conclusion does not depend on the additive model.
  • Model dependence: The cause remains open, but the addition of Qwen3.5-122B-A10B rules out scale as a simple explanation, and the limits of what can be tested are stated clearly.

One suggestion for the final version: since the shuffled control changes the picture for Gemma-4-31B, please state this in the main text (not only in Appendix F.1) and adjust the claim that three models show the effect accordingly.

I am satisfied with the rebuttal and maintain my positive score.

Reviewer_Am4hOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether language models can use in-context feedback to control their internal activations. The authors introduce a cross-model writer–subject comparison to distinguish sentence-selection strategies from privileged self-access, and conduct fixed-sentence experiments using general activation readouts and safety-relevant monitors, with and without generated reasoning.

Critical strengths
  1. Improved experimental controls: The cross-model writer comparison addresses an important ambiguity in prior work that improving an activation-derived score through text generation does not necessarily require privileged access to one’s own activations. Separately testing fixed sentences provides a useful methodological clarification.

  2. Clear presentation: The paper clearly states its hypotheses, connects them to the experimental conditions, and distinguishes activation control from stronger claims about introspection. This makes its experimental reasoning relatively easy to assess.

Critical weaknesses
  1. Limited incremental insight: Beyond the improved experimental design, the empirical contribution over prior work remains insufficiently established. Aoki et al. already reported small, inconsistent effects under fixed-output neurofeedback. The current findings remain largely model-specific, with limited broader insight into when activation control should occur or how these observations should guide future research.

  2. Limited mechanistic analysis: The analysis primarily measures changes in readout scores, without identifying the internal processes responsible for them. It remains unclear which activation components drive the observed effects and whether they reflect targeted modulation or broader context-induced changes. This limits how informative the results are about activation control itself.

Questions for authors
  1. Comparison with prior work: Could the authors provide a detailed comparison with prior works, separating inherited experimental elements from new contributions?

  2. Mechanistic explanation: Which layers or components drive the observed effects, and what explains the differences across models?

  3. Feedback controls: Since real and random feedback differ in their distributions and temporal structure, would the positive effects survive shuffled or replayed feedback that better matches these properties? This would strengthen the interpretation that the effects reflect feedback-driven control.

3 comments
AuthorsOpenReview ↗

[1/2] Prior works and model differences

We thank the reviewers for their careful reviews. We are glad that they found that our cross-model writer comparison resolves an ambiguity left by prior work (Am4h, gj4v), that the fixed-sentence experiments provide new insights (Am4h, XxMZ), and that our evaluation is thorough (gj4v).

Could the authors provide a detailed comparison with prior works [...]?

We agree that Aoki et al. [1] found small effects in some settings, and we made it clearer in the Discussion (Section 5). Our work adds three things to theirs.

  1. Why models succeed when they write the sentences. Aoki et al. suspected that the control reported by Li et al. [2] comes from in-context pattern matching, that is, from learning in the conversation which sentences raise the score, but did not test it. We developed an experimental setup to separate this from access to the model's own activations, and found that being the model whose activations are scored gives the writer no advantage in choosing the sentences (Section 3.4, Figure 2).
  2. A more extensive test with the sentence fixed. We use 10 models (11 in the revision) instead of 4 and 42 readouts instead of 3 probes, and we run every session with and without telling the model what the readout measures. Aoki et al. conclude that models "do not demonstrate reliable control". We find two models, Qwen3.5-27B and Qwen3.6-27B, whose control survives statistical tests, including a new shuffled-feedback control (Appendix F.1).
  3. Activation monitors. We directly test the threat model, by evaluating whether and how much activation control lets models evade six trained monitoring probes, with and without chain of thought. Neither earlier work does this (Section 4), and we find that no model lowers a monitor's score enough to change its decisions, although Qwen3.5-27B lowers the scores of the deception and truthfulness monitors when asked to (by up to 14.5 points against random feedback).

For future research, our results suggest that success when models write freely should be compared with other models writing the same sentences before it is read as access to activations, and that activation monitors should be re-tested on new models, which our fixed-sentence protocol makes cheap.

[...] limited broader insight into when activation control should occur [...]

[...] what explains the differences across models?

We agree that the findings are model-specific, and we honestly cannot say yet what explains the differences across models. Training data and post-training are plausible causes, but the training data of these models is not public, and no base checkpoint is released for Qwen3.5-27B or Qwen3.6-27B, so we cannot test them. We also find that scale does not explain it, because within the Qwen3.5 family the effect appears in the 27B model but neither in the smaller Qwen3.5-9B nor in the larger Qwen3.5-122B-A10B, which we added for this rebuttal (−0.1 points, Section 3.2, Figure 3). Kowalski et al. [3] also find that the ability of models to control their activations does not track their number of parameters. Across the five families we tested (Qwen, Gemma, GPT-OSS, Mistral, and OLMo), robust control appears only in the Qwen family, so we cannot claim that it generalizes to other families.

However, when we observe a control effect, we characterize it. Our paper already shows that it appears within the first few turns and does not require telling the model what is measured (Section 3.4, Figures 3 and 7), and the analyses we added for the revision show that it is targeted (new Appendix F.2) and that it only appears in the layers between 55 and 85% of the depth (new Appendix F.3).

References

[1] Koshiro Aoki, Ryota Takatsuki, Gouki Minegishi, Yusuke Haruki, and Daisuke Kawahara. In-Context Neurofeedback: Can LLMs Control Their Internal Representations through Privileged Access? Findings of EMNLP 2026.

[2] Ji-An Li, Hua-Dong Xiong, Robert C. Wilson, Marcelo G. Mattar, and Marcus K. Benna. Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations. NeurIPS 2025.

[3] Marek Mateusz Kowalski, Joshua Fonseca Rivera, Uzay Macar, and David Demitri Africa. Measuring Activation Control in Large Language Models. arXiv:2608.21664, 2026.

AuthorsOpenReview ↗

[2/2] Mechanistic analysis and feedback controls

Limited mechanistic analysis: ... whether they reflect targeted modulation or broader context-induced changes.

We are grateful for this suggestion, which led us to a new analysis that makes our result more precise. For this rebuttal, we recorded all 42 readouts in every session of Experiment 2, not only the one used as the score, since a general change in the state of the model would move a readout whether or not it is the score. In Qwen3.5-27B, by turn 30, a readout moves on average by 1.7 points in the instructed direction when it is the score, and by −0.1 points when another readout is the score (p = 0.008), which suggests that the modulation is targeted. Qwen3.6-27B shows the same pattern, but weaker (1.0 and 0.3 points) and not significant (new Appendix F.2).

Mechanistic explanation: Which layers or components drive the observed effects [...]?

We thank the reviewer for this question, which led us to run a new experiment that strengthens our results. In the paper, every readout is read at 33% or 66% of the depth of the model. To see which layers matter, we reran Experiment 2 with the readouts defined as a direction in the residual stream space read at five more depths (45, 55, 75, 85, and 95%), and measured the effect against the shuffled-feedback control (see below). In Qwen3.5-27B, the model with the largest effect, control appears only when the score is read between 55 and 85% of the depth: there its readouts move by 1.5 to 3 points, against less than 1 point at 33, 45, and 95%. This agrees with similar findings in activation control [3]. The results are in the new Appendix F.3 (Figure 14).

Feedback controls: ... would the positive effects survive shuffled or replayed feedback that better matches these properties?

We appreciate the reviewer's suggestion, and reran part of our experiments with a shuffled control, which let us clarify some subtleties. In short, the effect under such control is weaker but still significant (new Appendix F.1 and Figure 12).

To see what this control measures, it helps to list why the readouts of the maximize and the minimize sessions can end up apart. (i) The words of the two instructions differ, which changes the activations by itself, which means no control. (ii) The model may follow the instruction without using the scores, which means zero-shot control. (iii) The model may use the scores to adjust its activations, which is the control through neurofeedback that we want to measure. In a control session, the scores carry no information about the activations, so only (i) and (ii) can move the readouts. Our metric subtracts the control sessions from the real ones, which removes (i) and (ii) and keeps (iii).

In Qwen3.5-27B, (i) and (ii) alone set the maximize and the minimize control sessions 0.89 points apart with random scores, but 1.79 points apart with shuffled real scores, so subtracting the random controls left about 0.9 points of (i) and (ii) in our effect.

Against the shuffled control, the effect of Qwen3.5-27B and Qwen3.6-27B is about half of what we reported and remains significant, whereas the effect of Gemma-4-31B disappears. The half that remains requires the feedback to come from the model's own activations, turn by turn, so it is control through neurofeedback, which is what the paper claims. The half that disappears does not depend on the feedback, and we do not know whether it comes from (i) or (ii). Since our question is whether models use the feedback, our metric should not count it.

We want to clarify that, as is well known, no control condition is perfect [4], and we keep the random scores as the main control of the paper, because they allow a comparison at every turn and, since they do not depend on the real sessions, can be shared by all the readouts and run for every experiment.

Finally, replaying a session's own scores in their original order cannot serve as a control in our setting: since the sentence is fixed and the model generates nothing, a replayed session receives exactly the same input as the real one and yields the same readouts.

References

[3] Marek Mateusz Kowalski, Joshua Fonseca Rivera, Uzay Macar, and David Demitri Africa. Measuring Activation Control in Large Language Models. arXiv:2608.21664, 2026.

[4] Bettina Sorger, Frank Scharnowski, David E. J. Linden, Michelle Hampson, and Kymberly D. Young. Control freaks: Towards optimal selection of control conditions for fMRI neurofeedback studies. NeuroImage 2019.

Thank you for the prompt response. I appreciate the discussion of the model’s scope and the additional mechanistic insights. I think adding the discussed experiments to the final revision would strengthen the paper and I have adjusted my score accordingly.

Reviewer_XxMZOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether transformer-based language models can control their internal activations through in-context neurofeedback. The study evaluates models ranging from 9B to 32B parameters across various model families and finds that some models exhibit limited but consistent control over their activations under specific experimental conditions.

Critical strengths
  • This is an interesting research direction for exploring the potential of controlling model activations through in-context signals.
  • The fixed-sentence experiments reveal intriguing model behaviors, providing new insights into this research direction.
  • Although the findings do not indicate an immediate safety threat, this topic deserves further investigation as new models continue to emerge and their internal behaviors remain not fully understood.
Critical weaknesses
  • From an AI safety perspective, this research direction warrants further attention. However, the observed effects are too limited to support strong conclusions.
  • The current experimental evidence is insufficient to demonstrate that models can reliably and meaningfully control their internal activations.
Questions for authors

.

3 comments
AuthorsOpenReview ↗

Effect size and strength of the evidence

We thank the reviewers for their careful reviews. We are glad that they found that our cross-model writer comparison resolves an ambiguity left by prior work (Am4h, gj4v), that the fixed-sentence experiments provide new insights (Am4h, XxMZ), and that our evaluation is thorough (gj4v).

However, the observed effects are too limited to support strong conclusions.

The current experimental evidence is insufficient to demonstrate that models can reliably and meaningfully control their internal activations.

We agree that the effects are small, and indeed we claim that there is some control, not that it is large. Such effects are too small to affect activation monitors today, but the fact that they appear even when the model cannot choose its words indicates that some current models already have a weak ability to move their own activations, which is worth tracking as models evolve. In particular, our experimental results allow us to conclude two things.

First, in every setting we tested, we can bound how much models control their activations. When models write the sentences freely, being the model whose activations are scored gives the writer no advantage, so access to its own activations does not help the model in this setting, even though the effects are large (Section 3.4 and Figure 2). With the sentence fixed, the data rule out an average effect larger than 0.2 to 0.4 points for five of the seven models without an effect, and a larger model that we added for this revision, Qwen3.5-122B-A10B, shows no control either (we now report these bounds in the Discussion and in Appendix E.2).

Second, the control we observe with the sentence fixed, in Qwen3.5-27B and Qwen3.6-27B, is reliable, although small on average. In both models, the effect is significant (p = 0.002 and p = 0.02, also accounting for our choice of readouts). Furthermore, the average over our 42 readouts understates this control, since some of the readouts we designed are especially hard to move, and the readouts read at 33% of the depth do not move in any model, as Figure 8 in the appendix shows, in agreement with previous work [1, 2]. Focusing on the readouts that are defined as a direction in the residual stream space, the two Qwen models move them by several points at 66% of the depth, and no model moves them at 33% (objective difference at turn 30, against random feedback, as in the paper):

ModelRead at 33% of the depthRead at 66% of the depth
Qwen3.5-27B−0.3 [−0.7, 0.1]5.0 [2.1, 8.2]
Qwen3.6-27B0.1 [−0.4, 0.5]2.8 [0.3, 5.2]
Gemma-4-31B0.0 [−0.1, 0.1]0.3 [−1.2, 2.1]
Mistral-Small-3.2-24B−1.4 [−5.1, 2.1]1.0 [−1.7, 4.0]

Also, against the monitors (Section 4), Qwen3.5-27B, the model with the strongest control with the sentence fixed, is the only model that clearly lowers the scores of the deception and truthfulness monitors when asked to (by up to 14.5 points against random feedback), although no model lowers a monitor's score enough to change its decisions.

We thank the reviewer for pushing us on this front. In the revision, we show the results at 33% and 66% depth in the main text (new Figure 3, left).

References

[1] Ji-An Li, Hua-Dong Xiong, Robert C. Wilson, Marcelo G. Mattar, and Marcus K. Benna. Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations. NeurIPS 2025.

[2] Marek Mateusz Kowalski, Joshua Fonseca Rivera, Uzay Macar, and David Demitri Africa. Measuring Activation Control in Large Language Models. arXiv:2608.21664, 2026.

Dear reviewer, we are following up before the discussion period closes on Nov 18. We would be grateful to know whether our reply addresses your concerns. If it does, we would appreciate it if you would consider updating your score.

I appreciate the discussion. I will adjust my score accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.