acceptodds
Under review as a conference paper at ICLR 2027

Readout Scaling Shapes Sparse Feature Recovery

Abstract

Sparse autoencoders (SAEs) can learn different features from representations that contain the same information. We study this effect at a neural network’s linear readout, whose singular-value gains change the reconstruction weights seen by an SAE. Readout-canonicalized decision-space (RC-DS) coordinates remove these gains while preserving retained decision information. Controlled synthetic studies show that the generating distribution determines which coordinates recover known factors best. When factors are generated before readout scaling, removing gains at condition number 100 raises mean recovery from 0.5063 to 0.9134. Changing the generating coordinates reverses the preferred method. An exact wrapper reproduces RC-DS training to numerical precision, isolating training geometry rather than additional information. In bargaining and Goofspiel policies, RC-DS improves mean logit reconstruction under fixed budgets. On three frozen language-model checkpoints, it improves gain-free reconstruction in all 18 tested task–sparsity setting means, while logit training better reconstructs output logits in 17. Longer training and separate tuning reduce gaps on small label heads. Label-detection and steering controls show that reconstruction gains do not establish more useful features. Together, these studies identify how readout scaling, data geometry, and training determine the effect of an information-preserving coordinate change.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
sparse autoencoders, representation learning, mechanistic interpretability, readout geometry, sparse feature recovery, preconditioning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_K6hvOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how scaling introduced by a model's linear readout affects sparse autoencoder feature learning. The authors introduce readout-canonicalized decision-space coordinates, which remove readout singular-value gains while preserving the retained decision information. The paper provides a mathematical analysis of the equivalence between scaled and gain-free representations, together with an exact training wrapper. Synthetic experiments demonstrate substantial improvements in sparse factor recovery when scaling is removed, while also showing that the benefit can reverse depending on the generating distribution.

Critical strengths
  1. The paper investigates an important aspect of SAE training that is often overlooked: how coordinate scaling affects learned features despite preserving the underlying information. The mathematical formulation and controlled synthetic experiments provide useful insights, particularly the observation that the preferred coordinate system depends on the generating geometry rather than the readout condition number alone.

  2. The authors evaluate multiple coordinate choices, reconstruction metrics, training budgets, and downstream applications. The exact-wrapper experiment helps distinguish information preservation from optimization effects, while the hidden-state, whitening, and raw-coordinate controls strengthen the analysis. I also appreciate the transparent reporting of negative results and limitations.

Critical weaknesses
  1. Although RC-DS improves synthetic factor recovery and gain-free reconstruction in several settings, its advantages do not consistently extend to semantic label detection or steering. In the full-vocabulary experiments, logit-trained SAEs generally provide better logit reconstruction, while raw singular-coordinate probes outperform the mean trained RC-DS detectors across the evaluated settings. These findings limit the demonstrated practical benefits of the proposed coordinate transformation.

  2. The empirical advantages depend on training budgets, coordinate choices, and evaluation metrics. Longer training and separate hyperparameter tuning substantially reduce some reconstruction gaps, and several fits do not satisfy the reported stability criteria. Moreover, the real-model evaluations cover a relatively small set of frozen language models and policies. These limitations make it difficult to determine how broadly the observed advantages generalize.

Questions for authors
  1. To what extent do the observed RC-DS advantages persist when different coordinate methods are trained to comparable convergence or matched reconstruction quality, rather than for fixed training budgets?
  2. Given the mixed results for label detection and steering, what practical use cases do the authors consider most promising for RC-DS features beyond gain-free reconstruction?
  3. How well do the observations extend to larger language models, different SAE architectures, or representations with substantially different readout spectra?
Reviewer_Ur28OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper is aimed at improving how SAE are trained, by focusing on the scale of individual features. The validation involves synthetic toy data as well as small LLMs.

Critical strengths

None.

Critical weaknesses

The manuscript’s contribution is difficult to identify, starting from the definition of the method in 3.1, which could use definition of some symbols before they are used (e.g. L(x), SLS_L, as well as direct comparison of the method with PCA whitening. The experimental section is broad, but lacks depth, with each section/experiment lacking motivation as well as clear conclusion.

Questions for authors

See weaknesses.

Reviewer_BgkKOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I have found this paper incredibly hard to read, due to the sloppy writing. However, my best guess for the paper is that it is looking at how an SAE should learn different hidden activations, when the readout matrix at this layer may weight different activations differently. That is, it is looking at the reconstruction error in logit-space. I believe that the paper is trying to understand how to take this logit-space into account, in order to train the SAE more efficiently / better. There seem to be both some small synthetic experiments, and some experiments for some bigger LLMs. I apologize that I cannot provide a better summary.

Critical strengths

There may be merit to this paper, however I cannot really understand it currently.

Critical weaknesses

I admit that logit-space SAEs are not my area of expertise, however I found this paper to be unnecessarily difficult to understand. The writing in this manuscript is incredibly poor in my opinion (examples to come). In the current state, I don't think it will be (easily) understood by most people. There is a huge amount of confusing writing, seemingly disconnected ideas and sentences, jargon that is not canon, or hasn't been defined, very very poor notation, or undefined notation, a large amount of experimental jargon that makes it hard to see the insight or appreciate result, a lack of motivation. The abstract has numerical results that mean nothing, as no context is given. Contributions are very specific. Very little intuition given.

Here are a few examples, out of many that I could choose:

  • L15-16: Haven't defined factors.
  • L18-19: What is generating space?
  • L50: What do you mean by gains?
  • L51: "Allowed-action set" hasn't been defined, seems specific enough that it should have been.
  • L94-95: The same lack of definition of "action set" means lines like "For allowed actions L(x), the matrix SL keeps their rows" is very confusing to understand.
  • L127: What is D? What is the point of defining s^\hat{s}?
  • Section 4.1: The first paragraph is very hard to understand what is happening, for example what is an 'arm'? What is the point of this experiment?
  • Figure 1: What is the 'base signal'?
  • L82: Poor connection to previous discussion.
Questions for authors

Could the authors provide a revised document that resolves my concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.