- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Prior work applied the Jacobian lens (J-lens) to transformer language models to identify representations used for answer output, and argued that the resulting J-space functions as a global-workspace-like space. This paper extends the J-lens to the recurrent state of Mamba models. This is a well-motivated question, since the recurrent state carries information across tokens through a mechanism that transformers lack. The authors introduce two readout variants, full-path and readout-only, which cleanly separate what the state contributes at the current step from what it carries forward. They find that state and joint readouts recover known intermediate concepts more reliably than the residual lens across Mamba checkpoints. In Mamba-2, this early content also persists: it remains readable in the state even when it is no longer visible in the residual stream. Finally, the authors propose sign-guarded steering, which improves verbal report relative to coordinate exchange.
Critical strengths
I appreciate that the authors treat the global workspace as a functional analogy rather than as the neuroscientific construct itself. In neuroscience, the workspace involves broadcasting information back to specialized modules through recurrent loops, which may not correspond to the recurrence in these language models. The paper instead tests specific functional properties and avoids claims about consciousness, which I think is the right framing. The recovery results are consistent across checkpoints and are supported by reasonable controls and baselines. The choice of comparison models are comprehensive. Pythia provides a transformer control at a similar scale, and Nemotron tests whether the recurrent state still matters when attention is also available to carry information across tokens.
Critical weaknesses
The Qwen3.6-27B reference values are quoted from prior work rather than reproduced, and, as the authors note, the protocol differs from the one used here. It would help to either reproduce these values under the current protocol or more clearly qualify the comparison. The persistence claim currently rests on selected examples and lens-based word counts. Population-level statistics (for example, persistence rates across a larger set of prompts) would make this claim considerably stronger. The different operators have different native edit magnitudes and trajectories, so sign-guarded steering and coordinate exchange are not dose-matched. In addition, the improvement from sign-guarded steering reflects a local property, and it does not necessarily translate into improved downstream task performance. A dose-matched comparison and a downstream evaluation would help clarify the practical significance of this result.
Questions for authors
Since sign-guarded steering and coordinate exchange are not dose-matched, how much of the steering improvement might be attributable to larger perturbations? Is there direct causal evidence that the model actually uses the content held in the state?
5 comments
W3 and Q1. Dose matching: how much of the steering improvement comes from larger perturbations?
In verbal report, none of it. The revision measures every edit in units of the exchange's edit: an edit of size α has α times the norm of the exchange's edit at the same block and token (Section 2.3). At α = 1 the two operators make edits of the same norm at every site, along the same line, and differ only in sign.
With both operators at the exchange's original size, steering is ahead in verbal report in all 14 cells of model and edited stream (Table 3). Its success rate is higher by 36 percentage points in Pythia and by 49 to 80 points in every cell of the four recurrent models. On the residual stream, for example, it succeeds on 441 against 87 of 446 trials in Mamba-1 and on 602 against 121 of 621 in Nemotron.
On the downstream tasks, your question led to a further result: steering needs a smaller edit than the exchange and then does better. We ran both operators over a grid of sizes (Appendix I, Figure 8). With the residual stream edited, success on two-hop reasoning and flexible generalisation rises with size and then falls. In the five smaller models steering peaks at 1/8 to 1/2 of the exchange's size, and the exchange at 1 to 4 times its own size. Steering's peak is the higher of the two in every cell, for both tasks. Beyond its peak, steering makes the model output the edited concept itself, so a larger perturbation works against steering on these tasks. Table 19 gives the counts at every size the two share: steering is ahead at small sizes, and the exchange is ahead in most cells from the exchange's original size upwards.
One matched size therefore cannot settle the comparison, so we also give each operator its own best size. The size is cross-fitted: it is chosen on one half of the trials and scored on the other, and the halves then swap, so no trial is scored at a size it helped to choose. Both operators go through the same procedure on the same trials (Section 4.3, Figure 5, Table 13). The counts on the residual stream, steering against the exchange:
| Model | Verbal report | Two-hop (of 90) | Flexible gen. (of 192) |
|---|---|---|---|
| Pythia | 399 vs 242 | 9 vs 3 | 33 vs 14 |
| Mamba-1 | 441 vs 191 | 18 vs 6 | 71 vs 53 |
| Mamba-2 | 330 vs 150 | 15 vs 9 | 37 vs 20 |
| Mamba-3 | 727 vs 296 | 22 vs 2 | 59 vs 40 |
| Nemotron | 602 vs 250 | 30 vs 16 | 69 vs 44 |
| Qwen3.6-27B | 696 vs 346 | 40 vs 12 | 69 vs 20 |
Over every edited stream, steering makes the target reportable on 80% to 100% of verbal-report trials and the exchange on 15% to 61%. On the five smaller models the difference is significant in each of the 13 cells after Bonferroni correction. Counting the 26 cells of the two downstream tasks on the five smaller models and Qwen's three tasks, steering is ahead or level in all 29 comparisons and significantly ahead in 19. The pattern holds on Pythia, a transformer with no recurrent state, so it does not depend on one.
A size can also be chosen from each operator's verbal-report results alone, so that no downstream outcome enters the choice. At those sizes steering is ahead in all 18 cells with the residual stream edited in the five smaller models, and significantly so in 14 (Appendix I).
W3, second part. Does the improvement carry over to downstream tasks?
Two-hop reasoning and flexible generalisation are the downstream evaluation. Both score the model's greedy output, and an output that names the edited concept itself counts as a failure. In two-hop reasoning the edit changes an unwritten step and the model has to give the answer that follows from the new step. In flexible generalisation the model has to apply the function named in the prompt to the new argument.
The table in the previous comment gives the counts on the residual stream. With the state edited alone, at the exchange's original size, steering gives the requested two-hop answer on 7% to 17% of the items and the requested flexible-generalisation answer on 6% to 20% of the trials in the four recurrent models. The exchange, the unedited model and random edits of the same norm all stay at or below about 1% and 2% (Table 12). The improvement therefore carries over to both tasks once the edit's size is controlled, although the rates stay below those of verbal report.
The revision also adds a third downstream test, one edit followed by several questions (Section 4.4), which the answer to Q2 describes. There a single steering edit redirects 49%, 84%, 93% and 49% of the scored questions in Mamba-1, Mamba-2, Mamba-3 and Nemotron. The exchange redirects about as many in Mamba-1 and Mamba-3 (49% and 92%), and fewer in Mamba-2 and Nemotron (42% and 19%; Tables 20 and 21).
Q2. Is there direct causal evidence that the model uses the content held in the state?
Yes. The evidence comes from edits to the state alone, with the residual stream left untouched. A state edit is written inside the scan, so the block itself carries it to later tokens.
The two relational tasks show it. There the model has to compute with the written concept, and naming the concept itself counts as a failure. The edit changes only the state, and the model's answer changes to the one that follows from the new concept: after Brazil is turned into Mexico, it answers Spanish. With steering on the state alone, at the exchange's original size, the four recurrent models give the requested two-hop answer on 7% to 17% of the items and the requested flexible-generalisation answer on 6% to 20% of the trials. The unedited model and random edits of the same norm at the same sites stay at or below about 1% and 2%, so a disturbance of that size cannot account for the change. Table 12 gives the counts for each model, and Table 13 those with each operator at its own best size.
The new test asks whether content in the state serves several later questions. The prompt begins "Fact: My friend lives in A." and then asks for the capital, the language, the currency or the continent of "that country". We edit country A towards country B in the state only, and only at the tokens of the first sentence. No token of the question is edited, so the edit can reach the answer only if the state carries it forward. A question is scored only if the unedited model answers it correctly for both countries and the two answers differ. Steering redirects most of the scored questions to country B's answer in Mamba-2 and Mamba-3 and about half of them in Mamba-1 and Nemotron, and random edits of the same norm redirect none (Table 20). In Mamba-2 and Mamba-3 a single edit changes the answer to every scored question about the country in most swaps.
These edits show that the model uses content written into the state. They do not test whether the state is a necessary route for the intermediates the model computes itself, and the revision says so in Sections 4.2 and 6.
We thank the reviewer for the careful reading. Your question about dose matching (W3, Q1) was an excellent one. It led us to control the size of every edit, and the result is stronger than what the submission reported: steering's advantage does not come from a larger perturbation, and at a well-chosen size it extends from verbal report to the downstream tasks. The revision now runs both operators at controlled sizes throughout. This comment covers W1 and W2, the second covers dose matching, and the third covers the downstream evaluation and Q2.
W1. The Qwen3.6-27B reference values.
We have reproduced them, as you suggest. Every Qwen3.6-27B result in Sections 3.3 and 4 is now measured in this study, in the same environment as the other models, and Section 3.3 says so. The two relational tasks reproduce: 47 of 90 two-hop answers against the quoted 47, and 81 of 192 flexible-generalisation answers against the quoted 80. The only real difference is in verbal report, 81% against the quoted 69%, and there the prompts are not the same. The quoted rate is measured on 14 prompts, one per category. That is too few to compare models, so we use ten prompts per category for every model (Section 4.1), which gives 136 eligible prompts on Qwen. Appendix E reports this comparison.
W2. Population-level statistics for persistence.
Section 3.2 now measures persistence over all 551 prompts of the six recovery families, with 893 to 926 scored intermediates per model. An intermediate is readable at a token if some block of the band ranks it in the top ten, and its persistence is the share of later tokens at which it is still readable.
Once the state lens reads an intermediate, it reads it again at 81% to 89% of the later tokens in the four recurrent models, against 31% to 37% for the residual lens (Figure 3b, with bootstrap intervals over prompts). At a single fixed block, 70% to 83% of the intermediates that become readable in the state stay in the top ten until the readout token, against 10% to 18% for the residual stream. The state's persistence is the higher of the two in every family of every model (Table 8; poetry has few scored words), and it lies between 0.70 and 0.90 with the top 1, 5 or 25 as the threshold. As a control, we score on each prompt the intermediates of a different prompt. These become readable in the state 2% to 11% of the time, against 29% to 55% for the prompt's own intermediates. The two example prompts remain as illustrations in Figure 3a.
The word-list count is revised as well. The original study released its word lists but no scoring code, so we wrote the count from its description. The submission counted a word if it was in the top five, averaged over the last quarter of the list. The revision's main count uses the study's threshold: a word is held if some block of the band ranks it in the top 25 at a comma, averaged over commas 21 to 80, with the median of 100 lists and a chance level from words that never appear in the prompt (Section 3.3). The state lens holds 9.83, 13.45, 7.20 and 2.08 words in Mamba-1, Mamba-2, Mamba-3 and Nemotron, and the residual lens 5.46, 0.34, 2.05 and 0.28. Chance is at most 0.81. The top-five counts are kept as a second rule in Appendix E.1. The two rules count differently.