- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work is motivated from global workspace theory and aims to compare the workspace-like state for recursive reasoning models and autoregressive LLM. This work proposes to use multi-solution verifiable problems to probe the global workspace. With multi-solution problems, it is able to observe whether the interupted part influence a local solution or the global answer direction. Following this method, GRAM and Qwen3-32B are used for experiments.
N-queens and edge-coloring are used as the multi-choice task. GRAM reaches relatively high performance and converages on these tasks while Qwen3-8B thinking is relatively bad performances. GRAM is relatively robust to the removal of J-space features. Qwen3-32B mainly be influenced on last few depth.
Critical strengths
The work provides some valuable probing methods and interesting observations. The multi-choice tasks provide a way to distinguish local thinking and global thinking. We might observe more clear reasoning process of models. The comparision of recursive models and autoregressive models is an interesting and important topic. I think this work tries to give some insight from the inner state point of view.
Critical weaknesses
I have some concerns for this work.
- First of all, the motivation comes from global workspace theory, and the main work is not closely related to this. Jacobian lenes seem to provide low dimension spaces and these spaces are not neccessarily connected to GWT. Is there some better connection between the analysis and GWT.
- Secondly, The comparison of recurrent models and LLM are not quite fair. The GRAM is a much smaller model and is specifically trained for the eval tasks, while Qwen3-32B is a large model trained on general tasks. For example, I might wonder whether the difference is caused by the difference of specifical task training and general NTP training. The main question is that how should we know that the performance is mainly caused by the model architecture.
- Thirdly, the multi-choice problems for detecting change of reasoning might not be solid enough, since stochastic generation itself has high converage for solutions (see Fig. 3). Even we do not conduct a removal process, the generation process might change the answer. Fixing the random seed seems not to solve the issue completely. I think a better way might to compare the distribution shift of answers.
- Finally, I am willing to see a clear and useful formal mechanism is not provided for Interpretation. If it could be done, much more insightful conclusion we could gain from this work.
Questions for authors
See questions in weaknesses.
2 comments
Dear Reviewer CyJH,
Thank you again for your constructive feedback. If you feel that our additional experiments and our responses have addressed your concerns, we would greatly appreciate it if you would consider updating your rating. If any questions remain, please feel free to let us know, and we will respond right away.
You noted that comparing recursive and autoregressive models from the point of view of their inner state is an interesting and important topic. We agree, and we hope that multi-solution verifiable tasks, with the fairer comparison your comments led to, give the community a clearer way to look at how these models reason.
Sincerely,
The authors
Dear Reviewer CyJH,
Thank you for reading our work and for your constructive feedback. To address your questions and concerns, we ran several additional experiments, including a task-specific autoregressive baseline that isolates the effect of model architecture, and a comparison of the distributional shift of answers on multi-solution problems. We respond to each of your points below.
W1 The connection between the Jacobian-lens analysis and global workspace theory
We agree that the low-dimensional subspaces given by a Jacobian lens are not, by construction, necessarily connected to GWT. The connection is empirical: recent work from Anthropic [1], which largely motivated our study, showed in language models that the space identified by such a lens has several workspace properties, including availability for verbal report, use in internal reasoning, broadcast to downstream computation and limited capacity. For Qwen, we use the lens construction and ablation protocol of [1], and our result is consistent with one of these properties: removing the lens-selected directions impairs reasoning while the model still produces an answer. In our work, we further extend this analysis to a recursive reasoning model, GRAM, where a similar small subspace carries the information the answer depends on. We leave the other properties, such as broadcast, to future work, and we will state this connection explicitly in the revised introduction.
W2 Fairness of the GRAM–Qwen comparison: separating architecture from model size and task-specific training
This concern is well founded and deserves further analysis. To address it, we added a task-specific autoregressive model as a baseline, to isolate the effect of architecture (Appendix B.8, Figure 15). The training set and the intervention method are now both matched: GRAM and this baseline are trained on the same task, the J-lenses are fitted on the same subset, and the J-space construction and removal follow the same schedule, except that the autoregressive model is edited by layer where GRAM is edited by step. This follows from the architecture, since steps in GRAM correspond to layers in an autoregressive model.
Under J-space removal, this task-trained baseline loses validity as Qwen does, while the matched random control stays at baseline. The observation is therefore consistent across the two autoregressive models, one task-trained and small, the other generally trained and large, which indicates that the difference from GRAM is not explained by task-specific training or model size alone.
W3 Whether changes of solution reflect the intervention or sampling randomness, and comparing answer distributions
Thank you for this very helpful suggestion. Accordingly, we added a comparison of answer distributions (Appendix B.5, Figure 12). For each multi-solution puzzle, we compare the distribution of final answers over its branches before and after the removal, using the total variation distance. We control the sampling randomness directly. Each branch is replayed with its stored noise, which reproduces the original answer exactly when nothing is removed, and the matched random control keeps the original solution in nearly all branches. J-space removal moves about half of the probability mass, while a random removal matched in rank and norm moves almost none. So in GRAM the change of solution is caused by the removal and not by sampling.
W4 The lack of a clear formal mechanism for interpretation
We agree. The paper identifies which part of the state matters for the answer, but not yet what that part encodes or what computation the model performs on it. The mechanism we hypothesize is the one suggested by global workspace theory: a small subspace, the J-space, holds the information the answer depends on, and later steps of the recursion read it and rewrite it. The new experiments support parts of this account. The relevant content is low-dimensional (Figure 9), it is rewritten after being removed (Appendices B.6 and B.7), and the choice between valid solutions can be changed by writing one or two directions (Appendix B.4). A formal account would specify what the J-space directions encode and test the remaining workspace properties, broadcast and a capacity limit, which we leave to future work.
[1] Verbalizable representations form a global workspace in language models. Transformer Circuits Thread. https://transformer-circuits.pub/2026/workspace/index.html
Thank you again for your time! We hope we have addressed all your concerns. Should you have any further questions, please feel free to let us know.