acceptodds
Under review as a conference paper at ICLR 2027

When Coordinates Change the Features: A Symmetry Test for Sparse Autoencoders

Abstract

An orthogonal change of coordinates preserves the information in language-model activations, but it can change the features learned by a sparse autoencoder (SAE). We study this dependence with a coupled symmetry test. For TopK SAEs, a rotation of the inputs has an exact counterpart in parameter space that leaves sparse codes and reconstruction loss unchanged. We initialize two SAEs at corresponding parameters and train them on paired minibatches to test whether this correspondence survives training. With Adam, the resulting dictionaries differ substantially, even after latent matching, although their reconstruction quality is similar. The discrepancy begins with Adam's coordinatewise second-moment normalization. We derive a symmetry criterion for adaptive updates and show that group-RMS updates preserve the correspondence in exact arithmetic. In our experiments, these updates reduce dictionary disagreement at comparable reconstruction quality. Thus, differences between SAE dictionaries can come from the coordinates used during training even when the language model and its activation data are fixed. Reconstruction metrics alone do not reveal this dependence. The coupled test measures it against a known correspondence and identifies optimizer choices that respect that correspondence.

Primary area:
interpretability and explainable AI
Keywords:
sparse autoencoders, mechanistic interpretability, feature consistency, optimization, rotation equivariance, Adam

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_RobNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Problem: This paper identifies a challenge of TopK SAE training where the sparse features learned from such a model are not rotation invariant. The paper identifies the parameter-wise preconditioning of optimizers such (e.g. AdamW) as the cause for breaking this invariance. The paper begins by discussing simple cases such as signed permutations which TopK SAE learning is invariant to before expanding to dense rotations and non-orthogonal transformations.

Theory: Definition 1 introduces the invariance condition that the paper argues is desirable. The propositions in the paper highlight the main discussion surrounding when an update rule / feature transform is invariant against the other. They build cleanly towards Proposition 5 which introduces a variant of Adam satisfying the invariance described in Definition 1.

Experiment: Using a frozen dataset of 55,000 activations from GPT-2, a set of transformations ranging from signed permutation to the DCT-II transform.

Takeaway:

Critical strengths

Well-written paper, with that introduces and works on a minimal version of a problem. Many machine learning papers are written in a manner that makes it hard to read from beginning to end. I enjoyed that I did not have to strategically read sections out of order to understand the main idea of the paper. Further, I appreciate that focus of the paper was reduced to studying the geometry of the updates on the TopK SAEs without attempting to also build a full interpretability suite on top - this made it easy to focus on what was important. That being said, in the questions section below, I do request some more discussion about what the Author imagines the impact of this work on existing interpretability pipelines may be.

An principled angle to improving a simple challenge of TopK SAEs. From my cursory read of the field, it seems that invariance under simple (orthogonal) transformations of the activations is clearly something you would want out of an interpretability method - otherwise it is hard to argue that your features are a only property of the model being interpreted and not the SAE. That being said I am not sure about the related work, novelty of the problem, and other approaches that may already exist in the literature.

Critical weaknesses

I am not an expert in this field, but the results in this paper are quite simple, and it is not clear what their impact is. It is also not clear from reading the main body alone that this problem is unaddressed in the literature. Interpretability is topical both because of the (supposed) importance for AI safety and the funding being allocated to it. It would be valuable to connect the insight from this paper to the broader state of interpretability research here. It would also be nice to see a larger result suite with a more intensive tweezing apart of when the feature / decoder / reconstruction error misalignment impacts common interpretability performance metrics.

Questions for authors
  1. What is the empirical takeaway having run experiments that is not obvious from the propositions alone? It would be nice to highlight this. In this same question, I'd request that you clean up the Results section or even create a separate Experimental Setup and Results section, so that it is clear to the reader what information you are seeking with the experiments and why they may meaningfully impacts the takeaway of this paper.
    • The lack of communication of the significance of the claim / relationship to the rest of the field leads me to lean weak reject, but I am happy to amend if I see more justification for why this is an important result is presented.
    • I think it is fine if this problem has been introduced / approached in the literature before, but I'd then want to know what makes the result here important to publish.
  2. I understand that this paper is focused on the impact of update geometry on the learned sparse features, but have you run anything to see if features align beyond the Hungarian matching? What do you hypothesize the effect would be? As a reader not familiar with the area deeply, this would be good to know.
Reviewer_h8SFOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper investigates whether TopK sparse autoencoders preserve feature correspondence under orthogonal changes of coordinates, using coupled initialization and paired minibatches. The authors show that Adam’s coordinatewise updates can break this correspondence, while group-RMS updates preserve equivariance in exact arithmetic. Experiments demonstrate dictionary disagreement despite similar reconstruction quality and show that the proposed modification substantially reduces this disagreement.

Critical strengths
  1. The coupled experimental design clearly controls initialization and minibatch order, helping isolate the effects of coordinate changes.
  2. Latent matching, numerical precision controls, and longer-training comparisons provide useful evidence supporting the proposed optimizer mechanism.
Critical weaknesses
  1. The practical significance has not been established. The paper does not demonstrate that rotation sensitivity harms feature semantics, causal explanations, or practical interpretability analyses, nor that restoring equivariance improves these outcomes. Different dictionaries may provide equally useful decompositions. Moreover, the authors’ own controls show nearly identical dictionary similarities for repeated training in the original basis and comparisons across coordinate systems. Consequently, the paper has not established why coordinate sensitivity should be a priority to address in practical SAE applications.

  2. The evaluation is limited in scale and independence. The experiments use only GPT-2 Small and 50,000 training activation vectors. Training and validation positions share contexts, and the same validation set is used for hyperparameter selection, without a separate test set. The more comprehensive optimizer comparisons are also concentrated on layer 6.

Questions for authors
  1. Could you provide experiments demonstrating that rotation sensitivity affects practical interpretability quality and that the proposed modification improves it? Comparisons of feature semantics, concept detection, or causal intervention outcomes would help establish the practical significance of the work.

  2. Could you repeat the key comparisons on a larger dataset with document-disjoint training, validation, and test splits? Results across multiple random seeds would further strengthen the evidence.

  3. Could you revise Figure 1 and bring the key appendix controls into the main text? The presentation needs improvement. Figure 1 provides limited intuition, while important controls concerning ground-truth recovery, independent initialization, and longer training are primarily placed in the appendix. The main text should foreground these results and explain how they constrain the conclusions. Please explicitly discuss how these results support or limit the practical value of the proposed approach.

Reviewer_gwiSOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how the choice of coordinates affects sparse autoencoder (SAE) training. The authors first show that when the input activations are rotated by an orthogonal matrix, the SAE parameters can be transformed accordingly so that the reconstruction loss is unchanged (Proposition 1). This correspondence is also preserved during training by SGD and momentum SGD (Proposition 2). However, Adam does not preserve it, since it normalizes each coordinate separately (Proposition 3). To address this, the authors consider group-RMS updates, which use a single second-moment scalar for each parameter group, and show that they preserve this correspondence (Proposition 5). Experiments show that Adam learns quite different features under different coordinates, while group-RMS updates reduce this difference with similar reconstruction quality.

Critical strengths

The paper raises an interesting question of whether the features learned by SAE depend on the choice of coordinates. The proposed test, which trains two SAEs on the original and rotated activations, is simple and easy to understand. The experiments are also carefully designed to show that the difference comes from the optimizer (Adam), not from numerical errors or simple relabeling of features.

Critical weaknesses
  • Overall, I find the novelty and significance of the paper limited. Although the experiments are carefully designed, the main mechanism (i.e., Adam's coordinatewise normalization) and its fix have already been studied in prior work. The paper mainly studies this known effect empirically in the SAE setting.
  • While group-RMS updates preserve equivariance, it is unclear why this property is important in practice. For example, the experiments do not show whether group-RMS updates lead to more interpretable features across training runs. As a result, the paper mainly points out that SAE features can depend on the choice of coordinates without providing intuition on how SAEs should be trained or used differently.
Questions for authors

Additional questions are as follows:

  • The paper is difficult to follow. The main text is dense and frequently refers to the appendix, and it uses many terms that are unfamiliar (e.g., seed-reference rails, twins, and parameter transport, etc). This makes it difficult to clearly see the main message of the paper, such as why the coordinate dependence is important.
  • Propositions 3 and 5 seem to be similar to the results in VectorAdam (Ling et al., VectorAdam for Rotation Equivariant Geometry Optimization, NeurIPS 2022), which already showed that Adam is not rotation equivariant and proposed a norm-based second-moment update to fix this. Could the authors clarify what is new in this paper compared to VectorAdam? It would also be helpful to include VectorAdam as a baseline in the experiments.
  • Could the authors clarify the practical advantages of the group-RMS updates? The experiments mainly report the decoder mismatch and reconstruction quality, which do not clearly show why preserving equivariance is important in practice.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.