acceptodds
Under review as a conference paper at ICLR 2027

The Optimizer is an Uncontrolled Variable in Sparse Autoencoder Evaluation

Abstract

Sparse autoencoder (SAE) reconstruction quality is routinely compared across differently trained language models. We show it depends on the optimizer that trained the model, and that the sign of this dependence reverses with the depth at which the SAE is attached. Holding all else fixed, we train 13.9M to 210M-parameter transformers on two corpora under Euclidean (SGD), sign-like (Adam) and spectral (Muon) geometries, with per-geometry learning-rate sweeps and each comparison read separately at matched validation loss and matched training compute (3 to 8 seeds, seed-paired intervals). At the final residual stream and fixed sparsity (TopK, L0 = 32), SAEs fitted to the spectral arm reconstruct worse in every run, by 0.03 to 0.07 explained variance, without attenuation to 210M and surviving four times the training; varying only the update norm reproduces it. Refitting the same SAE at every depth reverses the sign: mid-stack, the spectral arm is reconstructed better, by up to 0.087. The model with the best validation loss is reconstructed worst in every run; identical JumpReLU settings yield a 4.08x spread in achieved sparsity. Cross-model SAE comparisons should report the base optimizer and rate, attachment depth and achieved sparsity.

Primary area:
interpretability and explainable AI
Keywords:
sparse autoencoders, mechanistic interpretability, optimizers, evaluation methodology, language models

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_ytEcOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how base-model optimizers affect SAE reconstruction quality. Controlled experiments show a consistent reconstruction deficit for spectral updates at the final residual stream, while comparisons at intermediate layers can reverse the ordering.

Critical strengths
  1. Extensive controls make the final-layer finding convincing. Learning-rate sweeps, isolated update rules, exact-SVD updates, and alternative SAE architectures address several plausible explanations.
  2. Depth-dependent reconstruction and the JumpReLU sparsity mismatch offer useful lessons for cross-model SAE evaluation. Reporting unsuccessful hypotheses also helps delimit the conclusions.
Critical weaknesses
  1. Attribution of the depth reversal remains incomplete. The isolated update family reproduces the final-layer deficit but fails to establish a reversal above the paper’s effect-size threshold at usable sites (Appendix J). Thus, the reversal observed with the named optimizers cannot yet be attributed specifically to update geometry.

  2. SAE fitting failures leave important comparisons unresolved. Most intermediate sites become inadmissible at larger scales under the matched-compute protocol, and longer training removes the usable intermediate sites at 60.5M. These failures limit the depth-dependent conclusion. Testing whether centering or separately tuning the SAE restores reliable comparisons would strengthen the evidence.

  3. Relevance to standard pretrained models remains uncertain. Most scale runs use only 1–5 tokens per parameter, and Appendix A shows substantial differences between their activation statistics and those of GPT-2 and Pythia. Longer training changes the gap substantially at 60.5M, leaving its behavior in mature representations unresolved.

  4. Readability needs substantial improvement. Dense numerical passages, repeated accounts of superseded analyses, and frequent appendix references obscure the central argument. Separating the robust final-layer result from the more conditional depth and mechanism findings would make the contribution easier to assess.

Questions for authors

Typos

  • Section 5.2: “too. the number to report” → “too. The number to report”.
  • Appendix E states that all three runs miss the 1.5% loss-parity threshold, although Table 7 reports 1.0145 for 60.5M, which falls within it. The corresponding statement in Section 3 also needs correction.
1 comment

Thank you for the constructive review, and for engaging so closely with the controls. We accept your assessment.

Both corrections are right. At matched loss, 60.5M does meet the 1.5% parity threshold, and the text in Section 3 and Appendix E was wrong to say otherwise.

You're also right that the depth reversal can't yet be put down to update geometry. When we looked again, it turned out to be mostly a difference from the SGD arm. Against Adam, the spectral arm only does better at one site in three of the four runs, by at most 0.019, and not at all at 60.5M. So your caution was, if anything, too mild. The SAE fitting failures and the open question about mature models are real limits as well.

Your suggestion to lead with the final-layer result and treat the depth and mechanism findings as conditional is the right way to read this work. Thank you.

Reviewer_SBL9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper presents an analysis of different models trained with different optimizers and datasets, and SAEs fitted to their activations. Results suggest that SAE reconstruction is worse for models trained with spectral-based optimizers at the final residual stream. The difference in EV depends on the SAE position and reverses at intermediate layers.

Critical strengths

Experiments cover different datasets and models.

The design of the experiments is clear. The controls help isolate the effect of the base model optimizer, including per-optimizer learning-rate sweeps and comparisons that vary only the update geometry.

Critical weaknesses

W1. My main concern is the practical motivation of the paper. The connection between the experiments and their claimed practical consequences remains unclear to me. The results show that the base model optimizer affects SAE reconstruction, but I am not convinced that this compromises existing evaluation practice. The cited pruning and compression studies share the same base optimizer within their main comparisons, while FAST compares SAE training methods within each base model. Can the authors demonstrate a realistic cross-model evaluation where accounting for the optimizer changes the interpretation?

W2. The scale of the pretrained transformers, both in dataset and model size, is rather anecdotal, and I am not sure whether the conclusions will transfer to models at a realistic scale. The models range from 13.9M to 210M parameters, and the main scale runs use roughly 1 to 5 training tokens per parameter. Although longer-training controls are included, the 44.8M and 210M models remain undertrained. I would suggest using an already pretrained model, for example, Kimi, which uses Muon modification.

W3. The presentation feels unpolished. For example:

Section 2 states the protocol, §3 and §4 the effect and its reversal, and §5 ...

Both "Section" and "§" are used in one sentence. Figures are black and white in the main text, while some appear in color in the Appendix. Speaking of the appendix, there are so many sections that they go beyond single-letter labels, with references like Appendix AE. There are a lot of ";" symbols, and "not" appears in almost every paragraph. As a reader, the paper gives me the impression of an automated AI-generated submission. The repetitive phrasing and the accumulation of experiments, with little critical synthesis of what they establish together, reinforce this impression.

W4. Explained variance is the main reported metric. Although the paper also includes reconstruction-induced loss changes and linear probes, I would like to see downstream task performance. I suggest adding metrics from SAEBench.

W5. Table 1 reports per-arm EV values without standard deviations.

Questions for authors

See Weaknesses

1 comment

Thank you for a careful and fair review. We accept it.

W1 is the point that matters most, and you're right. The pruning and compression papers share a base optimizer, and FAST compares SAE training methods within each model. We overstated how much current practice is exposed, and we don't have a real-world comparison where accounting for the optimizer changes a conclusion. We should have said that plainly in the paper.

You're right on the other points too. The 44.8M and 210M models are undertrained. And Table 1 should have had standard deviations.

Thanks for taking the time to spell all of this out.

Reviewer_QJNcOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the effect of the base model optimizer when training SAEs and investigates the resulting reconstruction loss, tracking these quantities across training and ablating across various factors like residual stream sites and learning rate. The main takeaway is that SAE reconstruction across base models should state these parameters as they have a non-negligable effect on reconstruction quality, and a reconstruction gap at the final layer relative to a random seed variation baseline.

Critical strengths
  • The paper identifies a clear experimental setting with per-optimizer learning-rate sweeps and separate SAE learning-rate checks to address tuning concerns, but there are other more significant concerns regarding training setup in weaknesses below.
  • The depth ablation is useful to caution against selecting one layer in determining trends based on base model optimizer choice.
Critical weaknesses
  • The training setup is limited, with pretraining settings running only to 210M parameters and moreover results are reported on prematurely trained models; with the exception of their smallest scale, the main runs train for roughly 1-2 tokens per parameter which is much below compute-optimal training heuristics. Authors themselves note that budget has an effect in conjunction with model scale, where at 60.5M parameters, extending training approximately halves the gap even though their ‘extended run’ is still only at roughly 5 tokens per parameter. Thus whether the findings from the experimental setting in this paper extend to adequately trained models used in practical SAE studies is highly questionable.
  • To be frank, I believe this paper needs significant rewriting in terms of writing organization and results reporting. The manuscript is 73 pages and repeatedly mixes up current findings, previous-but-since-overwritten analyses, and failed predictions. It honestly reads as an entirely AI-generated report rather than a polished paper, and while this doesn’t disqualify the paper’s findings, it makes the paper incredibly hard to read and makes the final contribution difficult to identify, with excessive Appendices that are not integrated well in the main paper. In particular, the appendices have many discussions of withdrawn claims and corrections, which makes it feel like the paper is an editorial conversation to develop the manuscript.
Questions for authors
  • Could the authors validate their main results in a more representative training regime (compute-optimal training) at greater model scales, even on a subset of datasets, learning rates and optimizers tried? I believe the reconstruction gap will diminish with extended training and increased model scale.
  • Can the authors test whether the reconstruction loss gap corresponds to differences in downstream interpretability-centric properties like feature recovery or semantic disentanglement which are the main motivations for using SAEs in practical settings?
  • Could the authors clarify their main supported conclusions and in their own words, separate its scope from the additional, more tentative observations?
1 comment

Thank you for the honest review. We accept it.

You're right about the training regime. Our scale runs stop well short of compute-optimal, and at 60.5M longer training already halves the gap, so these models can't tell us much about the ones people actually study. You're also right about the writing. We kept too much of the paper's history in it, and the main result got buried.

On your third question, here is what we actually believe holds. At the final residual stream, with the same SAE recipe, SAEs fitted to the Muon-trained models reconstructed worse than SAEs fitted to the SGD- and Adam-trained versions of the same models. That held at matched loss and at matched compute, but only for small, undertrained models. The depth story, the mechanism, and what happens with proper training are all still open.

Thanks again. This was useful to read.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.