acceptodds
Under review as a conference paper at ICLR 2027

Bilinear autoencoders find interpretable manifolds

Abstract

Sparse autoencoders have become a standard tool for uncovering interpretable latent representations in neural networks. Yet salient concepts often span manifolds that current linear methods cannot capture without post hoc analysis. This paper uses quadratic latents to close this gap: we implement these with bilinear autoencoders, which decompose activations into low-rank quadratic forms, compose linearly in weight space, and admit input-independent geometric analysis. This qualitative difference in what concepts quadratic latents can detect challenges the standard linear representation hypothesis. Our experiments and visualisations show that multi-dimensional geometries are highly prevalent and that composite latents capture them well, systematically improving reconstruction error in language models. Furthermore, we show that autoencoders with varying geometric priors recover the same input subspace despite their dictionary entries being distinct. Practically, these models serve as an unsupervised tool for manifold discovery, which we demonstrate through an interactive online visualizer for Qwen 3.5. This is a step toward nonlinear but mathematically tractable latent representations whose composition is expressive and interpretable by design.

Primary area:
interpretability and explainable AI
Keywords:
Mechanistic Interpretability, Neural Geometry, Tensor Networks, Compositionality, Sparse Autoencoders

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_Rm3QOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Bilinear autoencoders are introduced to find manifold structure in neural activations. The resulting non-linear representation is demonstrated to achieve improved reconstruction and interpretable geometric structure in language model representations compared to representations restricted to linear features.

Critical strengths
  1. An interesting and explicit architecture for capturing nonlinear structure that goes beyond standard linear representation is introduced. The quadratic forms are not only more expressive, they are designed to be analyzable through geometric tools and matrix decompositions. This can be a impactful step towards better interpretable nonlinear representations.

  2. The evaluation of the methods includes real LLM activations and not only toy examples.

Critical weaknesses
  1. The evaluation of the interpretability remains somewhat quantitative. Compelling visual examples are shown, but it is not clear to me whether the discovered manifolds correspond to the structures underlying the model and can be used to gain a deeper understanding of the model.

  2. Recent work, (Bhalla et al., 2026), suggests ordinary sparse autoencoders can already contain manifold information distributed across sets of features. It is not clear to me whether bilinear autoencoders discover genuinely new structure or just extract structure that existing SAEs already encode.

  3. It is not clear to me how well the approach scales to more large and complex data. The quadratic representations can become expensive to compute and more difficult to analyze when latent dimensions grows.

Questions for authors

For every manifold detected by the bilinear autoencoder, can the same manifold also be reconstructed from a standard SAE using grouped feature analysis, clustering, or a low-dimensional embedding of the feature activations? If not, can you provide examples where the manifold is identifiable only in the bilinear representation?

2 comments

We thank the reviewer for their efforts and address their remarks in order. We are currently running additional experiments and will upload the revised paper soon.

Weaknesses

W1 Bilinear latents match TopK under autointerpretability (App. H) and sparse probing (App. G). The viewer already characterises each latent by effective rank, support, importance and the variance captured in 3D. To further characterise their geometry, we will report the distributions of these statistics, plus definiteness, across all latents compared with an untrained dictionary. Whether the model uses these representational properties downstream requires causal validation, which we leave to future work. Quadratic interventions act on energy landscapes instead of fixed directions. This makes them fundamentally different and potentially less prone to overshooting. We believe they deserve a study of their own.

W2 SAE latents may span the same subspaces as ours. Sec. 4.3 shows that different decompositions can span the same subspace, and concurrent work finds that SAE subspaces are reproducible across seeds even when individual latents are not (Gerasimov et al., 2026). However, the manifold structure itself is not in the SAE. Bhalla et al. (2026) recover it post hoc from activation statistics, which they call "possible but unreliable … a workaround, not a solution", and they call for featurisers that "treat geometric objects as the primitive". Our contribution is such a featuriser. A composite captures a quadric manifold in the weights of a single latent. To compare, we added an experiment that measures, for each composite, how many TopK latents are needed to span its subspace. A few TopK latents capture over 50% of it, but capturing 90% typically takes more than 128. This suggests that bilinear autoencoders find coherent structure that TopK SAEs fragment.

W3 For input dimension dd, k≥dk \ge d latents and n≥kn \ge k tokens per batch, a training step of the composite model costs:

Non-zero parametersComputeMemory
TopKO(dk)\mathcal{O}(dk)O(ndk)\mathcal{O}(ndk)O(nk)\mathcal{O}(nk)
CompositeO(dk)\mathcal{O}(dk)O(nk2)\mathcal{O}(nk^2)O(nk)\mathcal{O}(nk)

At a fixed expansion factor, both methods scale as O(nd2)\mathcal{O}(nd^2) with model size. The extra cost is quadratic in the dictionary size kk, and all our models train in about 15 minutes on one RTX 4080 (App. B). Analysis cost depends on a latent's rank, not on dd. Composites stay low-rank (Fig. 5) and remain competitive in reconstruction and autointerpretability. Therefore, we see dictionary size, not rank, as the main scaling factor. We will add this to Sec. 2.2.

Questions

Q1 Not necessarily (see W2). Post-hoc grouping is only as good as the underlying tiling (Bhalla et al., 2026). Fig. 3 gives an example. The difference of two rank-1 atoms yields a hyperboloid that fires positively on w1w_1 and negatively on w2w_2. These regions are disjoint, so the SAE latents tiling them rarely co-activate and are unlikely to be grouped. The experiment in W2 quantifies how many TopK latents such a composite requires.

We hope this addresses the reviewer's concerns and would be grateful if they would consider raising their score. We welcome any further suggestions.

Thank you for the detailed response. The new experiments concerning W2 strengthen the paper. They indicate that bilinear latents capture manifold structure more efficiently than standard SAEs. Concerning W1, the additional analysis is interesting but I still see the weakness remaining. Concerning W3, the provided analysis and training times provide more insight into the performs at the scale consider. However, it is still unclear how well the approach scales to substantially larger model and dictionary sizes. Overall, the rebuttal strengthens the paper in particular with respect to W2 but W1 and W3 remain. So my recommendation is still a weak accept.

Reviewer_CR7GOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3(+1)
Soundness
3
Read review
Paper summary

This paper proposes a new architecture for interpreting neural representations. Instead of the linear nature of sparse autoencoders, the authors use a bilinear autoencoder that tries to reconstruct quadratic features. They describe their model in detail and explain its expressivity and interpretability. A sparsity prior on the rank of the learned weights distinguishes between three different representation hypotheses (atomic, composite, quadratic; in decreasing order of sparsity). Finally, they analyze the reconstructions performance and stability of these three sparsity modes and inspect the learned concepts qualitatively on Qwen 3.5.

Critical strengths
  • S1 The paper describes a, to my mind, novel architecture for interpreting neural representations, an important and currently popular research direction.

  • S2 The authors describe their method in detail and explain its relation to the commonly used sparse autoencoders theoretically in terms the representation hypotheses each model assumes. Overall, the paper is well-written and accessible even to non-domain-experts.

Critical weaknesses

I do not really see critical weaknesses. Below are a medium and minor one.

  • W1 Structure (medium): First, the paper has no dedicated related work section. Much related work is discussed in the intro and up to Sec 2.1. But it is not 100% clear where exactly the novel contribution starts (I assume Sec 2.2). Second, symmetry of WiW_i is stated in line 214 already, but the argument for it is only given line 321. Third, the kernel trick was listed as a core contribution in line 69, but is not described in the main paper at all.

  • W2 Code review (minor): The authors state that they a) did not use AI for coding and b) that they did not review the AI-generated code. Was AI-used for code generation? If so, why was the code not reviewed?

Questions for authors

Q1: How exactly were the plots in Fig 1 generated? I did not get the 3D projection. Is it the projection on the eigenspace spanned by the three leading eigenvectors of each dictionary element? The 3D nature is not visible in the scatter plots in Fig 1, only in the interactive viewer.

Q2: How is W\mathcal{W} parametrized? Via the L,RL, R matrices? Or directly as symmetric matrix?

Q3: Why is a direct materialization of XX not possible? At typical values of dd in the low thousands, a square matrix of size d×dd\times d seems feasible.

Q4: I assume that using the constant feature makes the bilinear autoencoder strictly more expressive than a sparse autoencoder. Is this true? If so, it might be worse clarifying in the paper.

Q5: In line 245 the authors describe that their approach is equivalent to ICA on the Veronese embedding. In what way is their approach novel compared to ICA?

Q6: Why do the encoder and decoder need to be tied in the proposed architecture?

Q7: How high is the effective K for the composite sparsity regime? How does it compare to the fact that seemingly only about 5 ranks are used in Fig 5 left, but many more in fig 5 middle?

2 comments

We thank the reviewer for their positive review and address their remarks in order. We are currently running additional experiments and will upload the revised paper soon.

Weaknesses

W1 We will make the structure of the first sections more explicit:

  1. Due to page limits, related work is discussed in Sec. 2.1. We will rename Sec. 2.1 to reflect this and state explicitly that our contribution begins in Sec. 2.2.
  2. Good catch. WiW_i need not be symmetric, but only its symmetric part affects the latent. We will give this argument (footnote 3) before Def. 2.
  3. The kernel trick is essential for efficient training but not for the paper's main message, so we moved it to App. D. We will add the main identity (Eq. 7) to Sec. 2.2.

W2 Our wording was ambiguous. AI tools only helped write the visualisation (plotting and viewer), whose correctness was visually verified. All 'core' code such as the methods, training and evaluation was written by the authors. We will clarify the statement.

Questions

Q1 The reviewer is correct. Each panel projects activations onto the top-3 eigenvectors of WiW_i (App. J). We will state this in the caption.

Q2 W\mathcal{W} is never stored directly. It is factored through LL, RR and CC as Wi=∑jCijljrj⊤W_i = \sum_j C_{ij} l_j r_j^\top, which needs O(dk)\mathcal{O}(dk) parameters instead of O(kd2)\mathcal{O}(kd^2) for the dense tensor. Only the symmetric part matters, so we symmetrise only for analysis.

Q3 A single XX is indeed cheap, but a batch requires O(nd2)\mathcal{O}(nd^2) memory (which easily exceeds consumer GPU memory at typical sizes). A dense encoder/decoder over the lifted space, which the naive loss needs, would also require O(kd2)\mathcal{O}(kd^2) parameters. The kernel trick (App. D) avoids both.

Q4 With the constant feature x~=[x;1]\tilde{x} = [x; 1], each latent computes x⊤Ax+2b⊤x+cx^\top A x + 2b^\top x + c. This contains every pre-activation of an SAE, but not its post-ReLU activation, so it is not strictly more expressive. They also reconstruct different targets (x~x~⊤\tilde{x}\tilde{x}^\top vs. xx). We will clarify this near footnote 2.

Q5 Standard ICA has a different objective and does not scale to the d2d^2-dimensional product space. Our objective relates to its relaxation, reconstruction ICA (Le et al., 2011). Bilinear autoencoders are an efficient low-rank formulation of it, trainable by gradient descent via the kernel trick. We give them a geometric reading and compare them against SAEs. We will replace "equivalent to" with "related to".

Q6 Tying is not required, but untying did not improve reconstruction (App. B). Without activation functions (unlike SAEs), the encoder and decoder are equally expressive. Tying makes each latent's receptive field equal to its dictionary atom, so the geometry describes both what it detects and what it writes. It also simplifies the kernel trick. We will clarify this in the text.

Q7 Apologies if we misread the question. We interpret "effective K" as the sparsity of CC, which keeps 0.1% of entries, or about 16 symmetrised rank-≤2\le 2 atoms per latent. Composites could therefore reach rank around 32 on average, but most are captured by at most 5 dimensions (Fig. 5, left). The quadratic prior (middle) is unconstrained and much higher-rank. We will clarify the panel labels.

We hope to have resolved the reviewer's main weakness and addressed their questions. If the reviewer agrees, we would be grateful for a score increase. We welcome any further suggestions or questions.

Dear authors,

many thanks for the helpful and succinct rebuttal! My concerns are resolved and I am happy to increase my score to 4.

Reviewer_D5wwOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The authors introduce bilinear autoencoders, a family of autoencoders that are linear in the lifted product space of pairwise interaction, mainly for the purpose of dictionary learning in interpretability. They address a well-known limitation that most mainstream autoencoder methods suffer in interpretability: the assumption of linear representations. The paper still assumes the necessity for representational assumptions in dictionary learning and elegantly formulate the next-simplest non-linear one within the polynomial family, which they coin the quadratic representation hypothesis. This enriches the geometric landscape in the representational space to include quadratic surfaces. The authors define a hierarchy of increasingly constrained versions of the hypothesis: the quadratic, the composite, and the atomic. As an empirical test, they first compare instatiations of these three versions through reconstruction error. They also briefly analyze the spectrum of each latent’s encoder and show that the composite version captures higher-dimensional structure, and provide a qualitative assessment of the geometries in their data, whose significance is still somewhat unclear to me. They finally study the stability of their results across different sparsity penalties using two different similarity functions.

Critical strengths
  • The paper uses a principled approach to extend the linear representation hypothesis (LRH). After identifying concrete, relevant examples in the literature where the LRH does not hold, they propose their quadratic assumption as the simplest extension in the polynomial domain, where the encoder and decoder remain linear in a lifted space. They also provide two techniques that fit their framework and make it feasible: the kernel trick and block decomposition.
  • The compositionality of their method preserves interpretability. While the LRH seems too rigid for deep neural networks, it has proven to hold to a surprising extent, and linearity provides the advantage of interpretability. The encoder/decoder’s compositional property gives more expressivity without sacrificing this interpretability. I think this is a very strong feature of the method.
  • The theoretical presentation is clearly written and sound. Decisions about the framework are well motivated in relation to the literature and arguments are coherent.
Critical weaknesses
  • The comparison with other techniques is insufficient. In order to make a compelling case for the bilinear autoencoder, a more thorough comparison is warranted. Firstly, while TopK makes a decent baseline for reconstruction, a single value of K=32K = 32 is used, and this value was not justified. Moreover, I think more time should be spent comparing how both methods capture interpretable concepts, and show when the concrete advantages mentioned in the previous section materialize in their dataset. Perhaps most importantly, a cost-benefit assessment is missing. It is still unclear to me how the compute budget compares to that of TopK, let alone how that scales with model size, and this information is crucial to evaluate practical usefulness.
  • The significance of the quantitative results is unkown. Reconstruction error is computed for a single run only, and as far as I understand, running more seeds wouldn’t be significantly more expensive, which could provide Figure 4 with error bars to assess significance. While there is a cross-sparsity-penalty comparison later on, and global subspaces are fairly similar, reconstruction spread isn’t evaluated.
  • Qualitative evidence could be made more quantitative. The geometric patterns seem interesting, but they are not very clearly characterized, and are not always immediately evident from the interactive viewer. I believe a more quantitative characterization of these or other patterns should be possible, and it would make the case for the interpretability of the method much stronger, especially in contrast to an LRH-based method. While this may also be the focus of a follow-up work, since that might have complexities of its own, the assessment as written does not constitute a very meaningful piece of evidence in my view.
Questions for authors
  • You mentioned in lines 250-253 that the bilinear autoencoder can be good to detect features stored in antipodal superposition. But isn’t it the other way around? When two features are in antipodal superposition, you actually want the encoder to disentangle them, and a bilinear autoencoder conflates the two signs because of the antipodal symmetry of the quadratic form. Please clarify.
  • Could you either do the TopK comparison with different values of KK or justify K=32K = 32?
  • Can you provide an analysis of the time and memory costs of your method in comparison with TopK?
  • Can you run more seeds to incorporate error bars in your histogram of Figure 4?
  • Is there a reason for α=0.3\alpha = 0.3?
  • In Theorem 1, w\mathrm{w} and x\mathrm{x} are suddenly defined on the unit sphere. Is this really necessary? Could you provide a heuristic argument for why the result with unit vectors is still valid?
3 comments

We thank the reviewer for their constructive review and address their remarks in order. We are currently running additional experiments and will upload the revised paper soon.

Summary

We agree that our work still makes a representational assumption, but this is unavoidable. Dictionary or disentangled representation learning is impossible without inductive biases (Hindupur et al. 2025; Locatello et al. 2019). We therefore make these assumptions explicit and choose the simplest non-linear one whose latents can be analysed in closed form.

Weaknesses

W1 We will extend the comparison with TopK autoencoders:

  1. K = 32 is a common setting in the literature (Gao et al., 2024). We will add a sweep over K ∈ {16, 32, 64, 128} to the appendices.

  2. Appendices G and H already compare probing and autointerpretability against TopK. We will summarise them in Sec. 4.1. We also added an experiment that measures how many TopK latents are needed to span the subspace of a single composite latent: a few capture over 50%, but 90% typically takes more than 128.

  3. For input dimension dd, k≥dk \ge d latents and n≥kn \ge k tokens per batch, the cost of a single training step is displayed in the table below.

    The extra cost comes from the kernel trick, which is quadratic in kk. At a fixed expansion factor, both methods scale as O(nd2)\mathcal{O}(nd^2) with model size, and all our models train in about 15 minutes on one RTX 4080 (App. B).

Non-zero parametersComputeMemory
TopKO(dk)\mathcal{O}(dk)O(ndk)\mathcal{O}(ndk)O(nk)\mathcal{O}(nk)
CompositeO(dk)\mathcal{O}(dk)O(nk2)\mathcal{O}(nk^2)O(nk)\mathcal{O}(nk)

W2 Reconstruction varied negligibly across seeds in preliminary runs (NMSE differences of roughly 10−410^{-4}), too little for the error bars to be visible. We will report this in the caption of Fig. 4 and add NMSE across the sparsity sweep of Fig. 6. In addition, App. I already reproduces the ordering on Gemma-3-1B and Llama-3.2-1B.

W3 Fig. 5 already quantifies how common higher-rank latents are. The viewer also characterises each latent by effective rank, support, importance and the variance captured in 3D. We will report the distributions of these statistics, plus latent definiteness, across all latents compared with an untrained dictionary. W1.2 will add a contrast with linear TopK latents. We leave a full geometric taxonomy to future work.

Questions

Q1 The reviewer is right. We will correct lines 250–253. A quadratic latent captures an entire direction, both signs included. This only suits features that span the full axis, which a sparse autoencoder would have to split into two latents.

Q2-Q4 Addressed in W1.1, W1.3 and W2.

Q5 α=0.3\alpha = 0.3 gave a good trade-off between sparsity and reconstruction in our experiments. Fig. 6 shows that varying α\alpha barely changes the recovered subspace. The NMSE sweep discussed in W2 will show this trade-off.

Q6 The quadratic encoder shrinks activations ≤1\le 1, so receptive fields shrink in high dimensions, which promotes sparsity. Inputs are unit-normalised in training (the NMSE loss is invariant to this), and the norm of ww can be absorbed into τ\tau, so the unit sphere is without loss of generality. We will state this more clearly in Thm. 1.

We thank the reviewer again for their feedback and hope this addresses their concerns. As we believe the revision resolves the weaknesses they raised, we would be grateful if they would consider raising their score. We also welcome any further suggestions.

We thank the authors for their reply. Most of our questions were answered satisfactorily.

It is true that appendices G and H provide meaningful comparison with TopK. Extending the results' current summary in section 4.1 should be useful.

On W3, we did not intend to dismiss the quantitative analyses of Section 4.2, which do give important information; however, these are not strictly interpretability results. Rather, it is the structure of the latent space that is being quantified there, without asking how interpretable it is. A multidimensional representation may allow for interesting, non-trivial structure, which is qualitatively shown in the different geometric patterns, but not characterized further. Since the title refers to "interpretable manifolds", there is burden of evidence in this respect. Nonetheless, Appendix H does provide evidence of interpretability (which should be more emphasized in the main text), and although we still believe that a more thorough analysis of geometric interpretability could considerably strengthen the paper, we understand that its scope, especially with the incorporated additions, is sufficient. We encourage the authors to study the geometric taxonomy in future work.

The method's motivation is very clean and correct, and its results indicate that the method works just as well as TopK SAEs, or sometimes marginally better, thus providing a valuable ground for the community to further explore the quadratic representation hypothesis and derived methods. We therefore decide to raise the soundness score to 3, and generally keep our positive evaluation.

However, SAEs are already very expensive to compute, and this method incurs significantly higher costs (scaling quadratically in the number of features k, so O(nk^2) vs. O(ndk), a factor of (k/d). We believe that the evidence for a valuable trade-off for interpretability is not yet entirely tangible given the current results. This prevents us from raising the overall score further, but we maintain a weak recommendation for acceptance: the research is valuable and should be considered for publication.

We thank the reviewer for their reply and positive recommendation.

we still believe that a more thorough analysis of geometric interpretability could considerably strengthen the paper

There is a delicate balance between doing quantitative analyses like in the results section (and appendix G, H) and qualitative one as in section 4.2 and the interactive viewer (along withfigure 1 and appendices). What additional analyses does the reviewer have in mind? We want to avoid indiscrimately adding overly many observations.

As for the last concern, the reviewer is correct that step-wise, this autoencoder is slower than a top-k counterpart. That said, ordinary sparse autoencoders take long to train, despite reconstruction quickly improving, because they only become interpretable after a lot of steps (a common issue in sparse optimisation). As Appendix B states, all bilinear autoencoders were trained in minutes on a single consumer GPU and yield clean results almost immediately. This is because the hard sparsity is not required in our training (and hence gradients are more stable).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revised paper

We thank all reviewers for their constructive feedback. We have uploaded the revised paper with changes highlighted in magenta. The main changes are:

  • Comparison with TopK autoencoders (D5ww, Rm3Q):
    • A new experiment measuring how many TopK latents are needed to span the subspace of a single composite latent.
    • Additional robustness checks for the reconstruction results.
    • A discussion of computational cost and scaling.
  • A study of latent quality, comparing the distributions of geometric properties with an untrained dictionary (D5ww, Rm3Q).
  • Theory of bilinear autoencoders (CR7G, D5ww):
    • Clarifications on the kernel trick, weight tying, receptive fields and the relation to sparse autoencoders.
    • An improved rate for Theorem 1, with proof.

We believe these changes have substantially improved our argument and welcome any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.