acceptodds
Under review as a conference paper at ICLR 2027

Where Does the Verbalizable Workspace Band Begin? An Audit of the Jacobian-Lens Instruments Across Open-Weight Families

Abstract

Claims about where verbalizable representations occupy a model’s depth depend on an instrument that can locate the region and distinguish known capacities. We audit the Jacobian lens (J-lens), a map from intermediate states into vocabulary space, used by Gurnee et al. (2026) to place a mid-depth workspace band at roughly 38% of depth. We test five measurements, four band signatures and one occupancy estimator; one signature does not pass our validation conditions and one estimator depends on an unstated convention. One band signature, excess kurtosis, changes sign between two fits of one model differing only in fit corpus. Whether occupancy, under every operator we test, can separate 25 from 40 planted directions depends on an unembedding convention the original leaves unstated: it cannot on five of six open-model J-lens dictionaries uncentered, and can on all six centered. What survives is a J-lens effective-dimensionality transition, not a fixed workspace-band location: with reference-free averaged-Jacobian lenses and one rule fixed in advance, its onset ranges from 28.6% to 78.7% across seven families on 22 screened model–lens artifacts. Its location varies across model families and model–lens artifacts and depends on threshold and locator choice; open families begin both earlier and later than 38%, so comparison against that frontier-scale figure is uninformative.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, Jacobian lens, language model interpretability, measurement validation, dictionary learning, effective-dimensionality, open-weight models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_nG12OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper audits Jacobian-lens measurements used to characterize a verbalizable workspace. It examines depth signatures, an operational onset rule, and occupancy estimators across open-weight model–lens artifacts. It finds strong dependence on model, locator, and unembedding-centering convention, and distinguishes an effective-dimensionality transition from a functionally established workspace boundary.

Critical strengths
  1. The reference-free checks, corpus-sensitivity analysis, and planted-count tests identify concrete measurement dependencies rather than interpreting every numerical difference as a model property.
  2. The paper carefully separates supported and unsupported conclusions. In particular, the centering controls and the causal patching probe prevent the spectral onset from being presented as a universally established functional boundary.
Critical weaknesses
  1. The broad cross-family onset range pools artifacts with unequal fit sizes, provenance, and validation coverage; the two largest self-fitted lenses lack retained gate artifacts. The range is descriptive rather than an isolation of family effects, and the incomplete validation limits the strength of that central cross-artifact result.
  2. The functional evidence remains narrow: the capital-city patching test uses one model and one intervention site, and its onset is later than the spectral transition. This supports the paper's caution, but leaves the positive scientific meaning of the surviving onset measure limited.
Questions for authors
  1. How does the cross-artifact range change under a common fit size and complete reference-free validation on a matched subset?
  2. Does the gap between spectral and functional onset persist when intervention sites include earlier token positions or another relation?
1 comment

Thank you for your patience. Both checks are done.

We ran a second relation, "X is a city in the country of," and it lands in the same place as the capitals one: onset at 80.0% of depth, same 20-point gap. We tried a third, "the currency of X is," and couldn't make it work. The model's top answer is " the" on all 69 prompts, so there's nothing for the probe to measure. That's our template rather than the model, and we report it that way.

On earlier positions, patching at the subject tokens transfers from 0% of depth and at all positions from 8%, both well before the onset, which inverts what we'd predicted. The whole-state controls explain why. The answer moves from the subject tokens to the final position between 68% and 72% of depth, and that is exactly where swapping the whole state at the final position starts working. So the 20-point gap is roughly 12 points before the answer has even arrived at the last position, and 8 after it's there but before the J-space part is what carries it. One model, one relation, and we don't claim more than that.

On fit size, your question turned up something in our paper. The Gemma-2-9B lens was fitted at SEQ=40 while the paper states SEQ=128. Refitting at the stated setting moves the onset from 61.0% to 65.9%. The two curves correlate at 0.9998, so it’s a threshold crossing rather than a different result, and the 19.7-point spread doesn’t change. Both numbers are in the revision.

On the two largest fits, we have them now. They were on a co-author’s storage rather than lost, and the paper was wrong to say otherwise. Gemma-2-27B passes the identity check. Qwen3-32B doesn’t, and neither has a positive control, since that needs weights we don’t hold.

The 32B failure isn’t ours alone. The published checkpoint fails the same way at a different precision and prompt count, and the two curves agree at 0.993. So the check doesn’t hold for that architecture either way. Dropping Qwen3-32B would make our spread 13.3 points rather than 19.7, so we report both numbers and keep the model in. On the matched subset itself, the three fits that carry both per-lens gates span 6.7 points, though those three are small models from two families, so validation quality, family, and scale aren’t separable there.

The revision is up. Thank you again for your time!

Reviewer_dvydOpenReview ↗
Rating
3(+2)
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2(+1)
Read review
Paper summary

Gurnee et al. (2026) propose that language models have a mid-depth "global workspace" of verbalizable representations. Using a Jacobian perspective and four depth-profile signatures on a closed frontier model, they estimate the depth at around 38% and the capacity at around 25 directions. These numbers come from a single closed model and instruments that have never been independently validated. It is not clear whether the onset depth is a fixed rule across model families or whether the signatures and the capacity estimator are in any way reliable. The authors fit and validated averaged Jacobian lenses on five open-weight models and then added in published lenses to give them 22 model/lens pairs across seven families. They identified the onset using a pre-specified threshold on the effective dimensionality signature and checked that the signature was consistent across fits. They also tested the capacity estimator on synthetic data with a known number of active directions under both unembedding conventions.

Critical strengths
  • The paper has concrete and actionable findings. The planted-count gate tests whether a capacity estimator is present. The paper also notes that transformer_lens silently omits unembedding centering for soft-capped models such as Gemma-2. Table 8, which lists the underdetermined options when reimplementing the estimator, is a very useful template for anyone doing replication work.
Critical weaknesses
  • The presentation is really poor in terms of readability. It's hard to follow because of the extremely dense, hedging sentences and constant cross-referencing. The evidence is scattered across 12 appendices, and the main text often makes claims that are only substantiated in the appendices.
  • The connection between the located quantity and the workspace claim is not clear. The effective dimensionality signature is a spectral statistic of the lens dictionary, computed without any activations. It is therefore a property of the lens rather than a property of what the model represents on those inputs.
Questions for authors

N/A

1 comment

Thank you for your patience. The revised PDF is up.

On readability, the causal patching probe is now a main-text section with its table rather than an appendix, and the planted-count result sits in Section 5.2 with its numbers instead of a cross-reference, so the two results that most needed their evidence nearby now have it. We also went through the main text, breaking up the long sentences so the qualifications stop landing mid-clause. The appendices are still there, but the main text no longer leans on them for the central claims.

One correction to what we said before: we tried bringing Table 8 up too, and it pushed us to eleven pages, so it stayed where it was. Moving the probe shifted every table number by one, so the table you pointed out is now Table 9.

On the connection between signature (d) and the workspace claim, you’re right that it’s a spectral statistic of the dictionary with no activations in it. That is why we ran a causal probe that does use them. A second relation gives the same onset and the same gap. Patching at earlier token positions shows the answer moving from the subject tokens to the final position between 68% and 72% of depth, which is exactly where swapping the whole state at the final position starts working. So the gap splits into the stretch before the answer has even arrived at the last position, and a shorter one after it’s there but before the J-space part carries it.

Thank you again for your time!

Comments on the paper

2 comments
Area_Chair_G5SWOpenReview ↗

Review #3

Summary: In previous work (Gurnee et al. 2026), there is this idea that a new interpretability technique, the Jacobian lens, can be used to identify the representations a model will verbalize in its processing. Those representations (the J-space) have certain properties and the claim was that these properties give a view into the model's non-verbalized internal reasoning. Gurnee et al. 2026 claimed that in alignment testing, these representations show strategic deliberation, models being aware of evaluation, and more. This is important context because the present paper builds directly on this work. In this submission, the paper claims that actually models have a mid-depth band of layers that can be measured with a tool called the Jacobian lens on a frontier closed model. The main question they ask is what happens to the interpretability measurements when applied to open weight models under a fixed rule. Plain, they find that those measurements from (Gurnee et al. 2026) may not hold up in this open weight setting.

Strengths:

  • If a reader is familiar with (Gurnee et al. 2026), then this audit is quite interesting. Especially for researchers in interpretability.
  • They point to a real reliability concern that interpretability research would benefit from at least paying attention to and being more thoughtful about
  • The appendix and overall paper together is quite thorough and looks like the authors took care to really be rigorous about main claims. though that means there is so much hedging i.e. this is this not this, etc that it can be really hard to read this paper all the way through and follow (see weaknesses)

Weaknesses:

  • The paper is not self-contained. as written, it assumes a great deal of familiarity with (Gurnee et al. 2026), which does not even appear to be a peer reviewed paper. Its so recent which is fine, but the current draft needs to be greatly improved for exposition and clarity. What is the motivation behind this audit just for the interpretability community? This is not at all clear as written. Its as if you have to go read the (Gurnee et al. 2026) paper in full just to understand what this paper is trying to say.
  • The main novelty seems to be applying the J-lens to open models. Even that is not quite properly articulated. It would be good to get more insight into what the novelty claim is meant to be. maybe there is something I am missing about what makes this particular audit novel and what the main takeaway is?
  • There are some organizational issues with the paper. There are a few claims that are in the appendix like overlap with related work Wang 2026 that would have been super helpful in the main text. It would be good to edit the paper to improve overall flow and readability and make sure the most important components arent delegated to the appendix.

Questions: (see weaknesses)

Rating: 3. weak accept

Thank you for the review! Another reviewer raised readability too, and we have fixed it.

You're right that the paper isn't self-contained. We added a short passage early in Section 1 on why this matters outside interpretability: the same representations get read as evidence of strategic deliberation and evaluation awareness in alignment audits, and claims like that inherit whatever reliability the instrument behind them has. We also added one plain sentence on what the J-lens actually does, so readers don't have to read the entire original paper first.

You're also right that "we applied the J-lens to open models" undersells it, and that was our wording's fault. What's new isn't that the lens can be fitted on open weights. It's that the five measurements don't survive the move: one signature fails our validation conditions, one estimator's answer depends on a convention the original leaves unstated, and the one that does survive locates a transition whose depth isn't portable across families. All three are about the estimators rather than the models. The revision says that in the abstract and again in Section 1.

On organisation, we moved the Wang and Reid comparison from the appendix into Section 2, and the patching probe is now a main-text section with its table.

The revised PDF is up. Thanks again!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.