acceptodds
Under review as a conference paper at ICLR 2027

Ambient Dimension Governs Whether Frozen Single-Cell Embeddings Can Be Edited and Interpreted

Abstract

Frozen single-cell foundation models are increasingly used as off-the-shelf embeddings, and linear-probe benchmarks report that perturbation phenotypes are well “contained” in them. Yet the same embeddings rarely help unsupervised analysis, where perturbed and control cells fail to separate. We resolve this tension by distinguishing two axes the field conflates: recoverability, whether a supervised probe can read a label, and surfacing, whether the label occupies embedding variance without supervision. Across seven representations and six datasets, recoverability is high everywhere and, perturbation by perturbation, scales with the true transcriptional effect, whereas surfacing is low in every one of them, and whether the two axes come apart is governed by one geometric quantity, the ambient dimension of the representation. Ambient dimension thus leaves recoverability unchanged but gates whether a frozen embedding can be edited and interpreted, and this single quantity predicts three otherwise-separate phenomena. In high-ambient embeddings surfacing rather than recoverability predicts unsupervised usability (Spearman 0.77 vs 0.27), while at low ambient dimension the two axes coincide. Surfaced nuisances are removable at zero parameters everywhere, whereas aggressive edits and selective steering are safe only when ambient room exceeds the number of entangled directions: a compressed latent fails this criterion, a high-ambient one satisfies it, and an intermediate one sits on the boundary. And sparse-autoencoder features become perturbation-specific and recover the perturbed regulatory program only as ambient dimension grows (win rate 34% to 92%; regulon recovery 5% to 22%). One axis thus tells a practitioner in advance which operations a frozen embedding will support.

Primary area:
interpretability and explainable AI
Keywords:
single-cell foundation models, representation geometry, mechanistic interpretability, sparse autoencoders, model editing

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_jJNmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors study why frozen single-cell foundation-model embeddings score well on linear probes yet underperform on unsupervised analyses. They distinguish recoverability, whether a supervised probe can read a label, from surfacing, whether the label occupies a substantial fraction of embedding variance, and argue that the ambient dimension of the representation governs whether the two diverge. Across seven representations (scVI, PCA, scGPT, Geneformer, Tahoe, TranscriptFormer, C2S) and three Perturb-seq datasets, perturbation identity is recoverable and calibrated to the true transcriptional effect, but surfaces only a few percent of variance. In high-dimensional embeddings, surfacing rather than probe accuracy predicts unsupervised clustering performance. The authors further evaluate retraining-free editing (mean-centering, subspace projection, additive steering) and sparse-autoencoder interpretability, reporting that safe aggressive edits, selective steering, and perturbation-specific SAE features become available as ambient dimension grows. Finally, structureless controls (random projections and random dimension padding) are used to argue that ambient dimension itself, rather than learned structure, drives the editing and interpretability trends.

Critical strengths

I think the paper offers some valid perspectives. Overall, the study of feature interpretability in perturbation studies is quite important to prompt an informed embedding selection. I haven't seen many studies investigating feature redundancy and the role this has in perturbation discrimination, but I find it quite important, as low-dimensional representations are crucial in ML for single-cell data.

In general, I think the study is well designed. Beside addressing the main message, the authors include some crucial controls, like randomization and sanity checks. The visuals are informative and the results tested across many different datasets.

Critical weaknesses

Despite the positives, I am leaning towards a weak rejection. I leave my technical notes below. More generally, I personally think that communication could be improved. Paragraphs are really clogged and very wordy. In my opinion, studies of this type require more schematization and summarization, like bullet points and bolding. I think a rewrite with more technical adjustments and more accessible structure would give more justice to the message.

Abstract and introduction

  1. Minor: the abstract is a little bit heavy. Some concepts are not clear without reading the paper, such as usability. Also, the sentence starting with “And” on line 30 is a bit abrupt.
  2. L42-43. I personally find this sentence slightly overstated. While it’s true that some of the perturbation effect gets lost, linear probe discrimination is not as good across all main perturbation datasets as this sentence hints at. I would rephrase it to make it lighter. Something suggesting this can happen in some perturbation datasets.

Related works

  1. This section is a bit dense. I recommend splitting it into paragraphs.

Results

  1. I personally believe the paper should use language that suggests that many perturbations are actually not expected to be distinguished by controls necessarily. Currently, it reads as if the expected behaviour would be to be able to disentangle perturbation effects from controls.
  2. The math is not very rigorously presented. Variables remain undefined. What are SbS_b, StS_t and μ\mu? What does the ⋅\cdot stand for in the subscript of zz? Some variables are defined later in the Notation paragraph. I find it a bit odd to place it after the first methodological section. Also, gPg_P and gCg_C are undefined.
  3. I agree with keeping foundation models at their original size, but I feel using PCA and scVI as proxies for “low dimensionality” is not quite rigorous. Since both of them can be trained across dimensions, I think it’s important to validate them beyond 10- and 5-dimensional embeddings. For example, PCA is potentially quite interesting, being a linear encoding.
  4. The technical controls for recoverability cover only sequencing depth (Table 4) and cell count (App. B). Other technical factors, such as lane/GEM-group composition, cell quality (mitochondrial fraction, detected genes), cell-cycle or stress states, or guide-related effects, could also separate perturbed from control cells and inflate probe AUC. Since the effect size used for calibration is itself a pseudobulk log-fold change, it would absorb such differences too, so the effect calibration (C2) does not rule them out.
  5. 273-281. This is an ok sanity check for me, but maybe we could use less space, as this is pretty intuitive.
  6. The surfacing diagnostic (C4) compares ρ(ARI, ve) with ρ(ARI, probe) only among perturbations with AUC > 0.8. Beyond the range restriction acknowledged by the authors, AUC saturates once groups are separable, whereas ARI keeps varying with how dominant the label is in the variance. A low ρ(ARI, probe) in high-ambient models may therefore reflect a ceiling effect rather than a property of ambient dimension. The authors should report the AUC distributions within the selected sets per group, and ideally repeat the analysis without the threshold or with a non-saturating separability measure (e.g., probe margin or log-odds).
  7. L324-330. Similar to some points I made earlier, I think this paragraph should add a clause suggesting that this is valid only on some datasets and scenarios. It is not always enough to act on the cluster means to correct batch correction, especially when such a technical effect is nested, non-linear, or subtle.
  8. The editing results rely on macro one-vs-rest batch AUC, which can mask poorly corrected batches: a single batch with residual signal is diluted by the average, and pro- and anti-predictive batches can cancel to “chance”. I suggest reporting per-batch AUCs or a multiclass measure (balanced accuracy, or held-out log-loss relative to the batch-prior entropy), complemented by a nonlinear probe and standard mixing metrics (iLISI, kBET).
  9. The off-target metric (E3) treats any overlap between a perturbation’s edit direction and the span of the others as leakage, measured only against a random-direction baseline (k−1)/d. However, many perturbations are expected to have similar effects (e.g., subunits of the same complex in the essential-gene screens), and weak perturbations yield mostly noise directions from 40-vs-40 cell estimates. High off-target may therefore reflect true biological redundancy rather than representation entanglement. A gene-space reference. And restricting the analysis to perturbations with a clear effect would show whether the embeddings add entanglement beyond the biology.
  10. L376-401. A style note: This paragraph is really dense and, in my opinion, hard to parse. It would be great to split it into paragraphs and make it more schematic.
  11. L376-401. It would be helpful to clarify whether the specificity metric (I1) captures biological rather than technical specificity. Since both the feature signatures and the pseudobulk targets would absorb technical differences specific to a perturbation’s cells (e.g., lane or quality), and larger dictionaries at high d offer more candidates, a check that the selected features are not driven by QC covariates (e.g., by regressing them out) would strengthen this result.
  12. The causal argument relies on structureless constructions (random projection, signal-free padding), while the real low-dimensional models are only evaluated at a single d (scVI ≈ 10, PCA = 50). As mentioned above, retraining scVI and recomputing PCA at several latent dimensions (e.g., 10, 50, 128, 512) would directly vary d while keeping learned structure and model family fixed, and would more convincingly separate ambient dimension from the VAE confound.
  13. Regarding structureless controls. That the trends appear without learned structure suggests they may partly reflect dimensional arithmetic built into the metrics rather than properties of the representations. Moreover, the d=10 control is not as fragile as scVI (0.195 vs 0.496 identity loss).
Questions for authors
  1. Regarding the effect size metric. Isn't pooling across log fold changes from all genes potentially confounded and biased by the noise of non-differentially expressed genes? Is this metric run only on, at least, highly variable genes?
  2. It's quite odd to consider a batch correction model like scVI in the context of an editing operation for batch correction. Was the model trained without batch labels?
  3. L212: Why compared to the random perturbation and not another compared to the other perturbations?
  4. The PCA over-separation (S1) is left unexplained. A plausible alternative is that PCA's top-variance directions retain technical variation (cell quality, residual depth, lane composition) that a probe exploits independently of the true effect size, which would explain high AUC with low calibration. Only depth is controlled (Table 4), and the pattern holds clearly only on norman (Table 3). Does S1 persist after regressing out technical covariates?
  5. L330-336: If I understand correctly, the take-home message is that removing directions in less redundant representation spaces harms identity quite a bit. This seems to be associated with dimensionality, as scVI-10 appears to be significantly affected. Isn't this quite an obvious behavior? You are dealing with non-linear encoders creating highly entangled latents. Is this just a sanity check?
Reviewer_NUk4OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies “recoverability” and “surfacing” in single cell representation methods applied to perturbation datasets. Based on seven frozen representations, of “low ambient dimensions” (scVI and PCA) and “high ambient dimensions” (scGPT, Geneformer, Tahoe, TranscriptForner, C2S), they suggest that the separability between recoverability and surfacing as well as the safety of embedding editing and interpretability is governed by the magnitude of ambient dimensions. They claim that differences in the results are not confounded by the fact that all the high ambient dimension representations are foundation models whereas scVI and PCA are not.

Critical strengths

The paper provides a complete framework to study the impact of ambient dimension in single-cell representations (formalizes the geometric ideas and aligns with concrete tasks and metrics).

Critical weaknesses
  1. Overall presentation: The paper is not easy to read, full of prose and novel definitions that obscure the underlying ideas. Instead of relaying out the content it is covered with abstractions and unclear sentences. Figures are of very poor quality; both in terms of quality and readability.

  2. Practical contribution / novelty / claims vs results: Considering information theory basics it is unclear what is the major finding of the paper. Moreover many of the claims are not grounded / supported by the actual evidence (e.g., the authors say that selective steering is "achievable" when the ambient dimension is high enough but leakage stays high. The authors claim that ambient dimension causes the results but there are results showing that training also has a large effect when dd is fixed).

Questions for authors
  1. Methods description; The evaluation setup is described only in words only – dense with prose and jargon. Providing equations/algorithms/diagrams would help follow the evaluation setup. Can the authors provide some of this formulation for a more methodological understanding?

  2. Claims vs results; Following on the weaknesses it would be great to better justify the ambient dimension contribution vs model architecture.

Reviewer_b2ETOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates how perturbation information is encoded in frozen single-cell embeddings. The paper proposes two properties for characterizing embeddings: recoverability, measured by supervised linear-probe performance, and surfacing, measured by the fraction of embedding variance associated with a label. These properties capture different aspects of the representation and can conflict with each other.They evaluate seven representations across three perturbation datasets and three integration benchmarks, using clustering, batch removal, perturbation steering, sparse-autoencoder feature specificity, and regulon recovery. The results show that perturbation information can remain recoverable despite occupying little variance, while higher-dimensional representations generally support less destructive subspace projection and more selective SAE-derived features. Random-projection and dimension-padding controls are presented as evidence that dimension contributes to these trends beyond model architecture or pretraining.

Critical strengths
  1. The paper makes the gap between supervised recoverability and geometric prominence explicit and measurable. Its finding that perturbation information can be readily decoded while explaining little embedding variance highlights a consequential limitation of linear-probe benchmarks. This framework helps assess whether embeddings are suitable for unsupervised biological analysis, an important practical use of frozen representations.

  2. The paper evaluates embedding-space edits for removing batch effects and steering perturbation responses while preserving other information. Its experiments identify conditions under which these operations succeed or cause collateral damage, providing useful guidance for adapting existing representations without retraining the backbone.

Critical weaknesses
  1. Section 4.2 mainly evaluates editing through cell-identity probe performance and overlap between perturbation directions. These measurements do not establish that the edits improve downstream biological analyses. Moreover, perturbations affecting the same pathway may legitimately have overlapping directions, so reduced overlap does not necessarily indicate a better biological edit. This limits the practical interpretation of the reported editing improvements.

  2. The geometric explanation for the dimensional effect needs stronger controls. In Section 4.4, higher-dimensional random projections also improve recoverability, so better editing could partly reflect better preservation of useful information. This leaves the contribution of additional geometric room uncertain.

Questions for authors
  1. Does higher dimension help primarily by providing additional geometric room or by preserving more useful information? Could you clarify how the experiments distinguish these explanations?

  2. When different perturbations affect the same biological pathway, their directions may legitimately overlap. How does the evaluation separate this shared biological response from undesirable off-target leakage?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.