acceptodds
Under review as a conference paper at ICLR 2027

Interpreting Sparse Autoencoder Features in Cytometry Foundation Models through In Silico Perturbations

Abstract

Cytometry foundation models learn sample-level representations for patient-level prediction, yet how they encode immune cell composition remains unclear. Sparse autoencoders (SAEs) expose individual features in these representations, but do not by themselves explain which composition changes those features respond to. We introduce a perturbation-based framework that interprets SAE features through controlled exchanges of cell populations. Resampling a sample’s own cells varies a target population within a specified group while holding cells outside that group fixed. Each annotation specifies the populations exchanged and the direction of the feature’s response, making its interpretation explicit and testable. Across GPCT and MAESTRO, more than 98% of annotations retain their response direction with corrected significance on held-out samples. Annotations of case-enriched features recover 82% and 65% of expert-expected population changes, respectively, across 20 case–control comparisons, while MAESTRO decoder steering reproduces 93.2% of annotated directions. The annotations also support classification from feature descriptions alone. Beyond individual annotations, comparing perturbation responses between cases and controls reveals shared-feature programs linking populations through state-dependent model responses. Together, this framework connects learned features to interpretable composition changes and provides a way to examine how cytometry models represent immune variation across samples and disease states.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Cytometry, Sparse Autoencoders, Automated interpretation, Cytometry Foundation Models, Agentic AI

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_iz2hOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how cytometry foundation models represent immune cell composition. SAEs expose features in sample embeddings, but associations with population abundances do not isolate which composition changes affect those features. The paper proposes controlled in silico perturbations that resample a sample's cells to exchange a target population with the remainder of a specified scope, keeping cells outside that scope fixed. Consistent feature responses are converted into annotations naming the target, scope, and response direction.

Critical strengths
  • The target–scope formulation makes composition interpretations explicit and testable. Specifying the displaced cells distinguishes whole-sample abundance changes from changes within an enclosing population. Resampling a sample's own measured cells provides a controlled way to implement these exchanges.
  • The experiments provide substantial evidence of repeatable composition responses in the frozen models. More than 98% of annotations reproduce their slope direction with corrected significance on held-out samples, and MAESTRO decoder steering agrees with 93.2% across all six signed doses.
Critical weaknesses
  • The paper shows that SAE features can be interpreted reliably, but it's unclear why these interpretations are useful in practice. The authors don't demonstrate what new insights or benefits they provide beyond simply using the foundation model or standard cytometry analysis.
  • The added value over simpler interpretations is insufficiently established. Correlation-derived annotations and PCA or random Top-k features annotated by the same procedure are not evaluated. The existing random steering controls do not establish whether the proposed annotations provide more informative interpretations than these alternatives.
Questions for authors
  1. How do perturbation-based annotations compare with simpler correlation-based annotations under the same evaluation protocol?
  2. Are the results robust to donor-level clustering or using only one sample per donor?
  3. Do the shared-feature programs remain consistent when cases and controls come from the same study?
4 comments

W1. Practical value

The practical value of the interpretations was not highlighted sufficiently in the paper, and we clarify it here. Compared with using the foundation model directly, the benefit lies in identifying what the model responds to, with greater accuracy than a correlation-based analysis provides. The interpretations further reveal which populations the model treats as moving together in disease, an aspect that standard cytometry analysis does not capture.

Beyond using the foundation model directly. The model alone gives embeddings and predictions but not what drives them. The simplest way to find out is to correlate its features with population frequencies across samples, but a correlation can point to the wrong population. GPCT feature 1251 is an example. It is higher in samples rich in myeloid dendritic cells (ρ = 0.48), so a correlation would label it a dendritic cell feature. Yet increasing these cells directly lowers it (in 239 of 310 held-out samples), because the feature responds to monocytes, which are more abundant in the same samples (ρ = 0.64). Our perturbation identifies this because it changes one population within a sample and measures the response of the feature.

Feature 1251 is not an isolated case. For every feature and every population, we compared the Spearman correlation between the two across discovery samples with the response of the feature when the population is changed directly.

Feature–population pairs with a significant correlation across samplesGPCTMAESTRO
Number of pairs332,415400,085
Response in the same direction (%)61.429.9
Response in the opposite direction (%)20.15.4
No significant response (%)18.564.7

Of the pairs with a significant correlation, 20.1% (GPCT) and 5.4% (MAESTRO) respond in the opposite direction, as feature 1251 does, and a further 18.5% and 64.7% show no significant response. A correlation therefore agrees with the response of the model in only 61.4% and 29.9% of the pairs. The annotations also turn the disease signal of the model into statements about populations. Without using clinical labels, they recover 81.8% (GPCT) and 64.6% (MAESTRO) of 99 expert statements about which populations rise or fall in disease (Section 5.4).

Beyond standard cytometry analysis. Standard analysis compares populations one at a time and finds which of them differ between cases and controls. Shared-feature programs (Section 6) add which of these populations the model treats as moving together. For example, one MAESTRO program for ANCA vasculitis links CD4 T-cell subsets including Th1 and Th1Th17 cells. The programs remain consistent when cases and controls come from the same study (Q3).

W2 & Q1. Comparison with correlation, PCA and random annotations under the same protocol

We agree with this assessment. The paper did not compare against these simpler alternatives, and we have now done so. The comment raises two separate questions, whether the perturbation is needed and whether the SAE is needed. We address each below. Perturbation versus correlation (same SAE features). A simple correlation might give the same annotations, in which case the perturbation would not be needed. We find that the two are not interchangeable. Populations rise and fall together across samples, so a feature can correlate with a population that it does not respond to. Changing one population directly separates the two cases.

To test this, we annotated the same SAE features by correlation on the discovery samples. For each condition, we took the features with the largest absolute Spearman correlation between their activation and the target's share across samples, as many as the perturbation method annotated, with the sign of the correlation as the direction. Both kinds were then scored with the evaluations of the paper (Sections 5.1, 5.2, and 5.4).

GPCT, perturbationGPCT, correlationMAESTRO, perturbationMAESTRO, correlation
Held-out Stage 2, perturbation response replicated (%)99.9890.498.378.5
Held-out Stage 3, firing difference in annotated direction (%)93.891.794.393.7
Expert statements recovered (%)81.862.664.677.8
Zero-shot AUROC, held-out samples.847.792.770.806

When the composition of held-out samples is changed directly (Stage 2), about one in ten correlation annotations for GPCT and one in five for MAESTRO do not replicate, whereas nearly all perturbation annotations do. The two methods also pick different features, agreeing on the feature for a condition in only 19% (GPCT) and 23% (MAESTRO) of annotations. One example is GPCT feature 1251 (see W1), which correlates positively with myeloid dendritic cells yet falls when these cells are increased directly.

In expert-statement recovery, perturbation annotations are 19.2 pp higher for GPCT, whereas correlation annotations are higher for MAESTRO. Zero-shot AUROC follows the same pattern, and Stage 3 is close. A correlation annotation states in which samples a feature is high, whereas a perturbation annotation states how the model responds when one population changes, which is what our framework claims.

SAE versus PCA and random features (same annotation procedure). PCA or random features annotated by the perturbation procedure of the paper might be equally informative, in which case the SAE would not be needed. We find that SAE annotations are the most informative of the three. This procedure yields reproducible annotations whichever features it is applied to, so reproducibility does not distinguish them. What distinguishes them is agreement with expert knowledge, where the SAE leads in both models.

PCA uses the principal components of the same sample embedding. Random features are Gaussian projections of it with the dictionary size and Top-k firing (k = 64) of the SAE. Both were annotated exactly as the SAE features were.

GPCT, SAEGPCT, PCAGPCT, randomMAESTRO, SAEMAESTRO, PCAMAESTRO, random
Annotations12,26693916,3347,44053114,509
Held-out Stage 2, perturbation response replicated (%)99.9899.899.9798.3100.099.9
Held-out Stage 3, firing difference in annotated direction (%)93.894.195.494.397.296.3
Expert statements recovered (%)81.855.670.564.640.439.8
Zero-shot AUROC, held-out samples.847.644.776.770.766.791

SAE annotations recover the most expert statements in both models, by 26.3 pp over PCA and 11.3 pp over random features for GPCT, and by 24.2 pp and 24.8 pp for MAESTRO. For GPCT, SAE annotations give higher zero-shot AUROC as well, while for MAESTRO the three are similar. Held-out replication is high for all three, and random features receive more annotations than SAE features, so neither separates them.

For random features, annotation counts, held-out replication and expert recovery are means over five seeds. We will add both comparisons to the appendix.

Q2. One sample per donor

Repeated samples from the same donor are a plausible source of bias in our results. One donor can have several samples in our data, and samples from the same donor resemble each other. The tests of the paper nevertheless count each one as a separate observation, so a result could look more significant than it is. To check this, we repeated the two statistical tests of the annotation procedure with one sample per donor. Our analysis suggests that the results are robust to repeated donors.

We used the samples with known donor identity, excluding batch controls. These are 1,214 samples from 583 donors, of which the perturbed samples are 448 from 309 donors for GPCT and 445 from 297 for MAESTRO. We drew one sample at random from each donor and ran both tests, the perturbation response (Stage 2) and the firing comparison (Stage 3), over all features and conditions with the multiple-testing corrections of the paper. One sample per donor leaves fewer samples, so for a fair comparison we also drew two independent random sets of the same number of samples without regard to donor and analysed them in the same way.

A feature and a condition that pass a test form a significant pair. We measured how many significant pairs of one analysis are also found by another, and we did this twice. The first measurement compares the two random sets with each other. The two random sets have the same number of samples and are drawn in the same way, so they differ only in which samples were drawn. Their agreement therefore shows how much the results change when only the drawn samples differ. We use this value as the reference. The second compares one sample per donor with a random set. If repeated donors mattered, the second value would be clearly lower than the first. All values are means over 100 repetitions, each with a new draw of one sample per donor and of the two random sets.

Significant pairs also found in a random set of the same number of samples (%)Random set (reference)One sample per donor
Perturbation response (Stage 2), GPCT96.895.8
Perturbation response (Stage 2), MAESTRO83.083.2
Firing comparison (Stage 3), GPCT51.752.3
Firing comparison (Stage 3), MAESTRO48.349.6

In every row, the two values differ by at most 1.3 percentage points. One sample per donor therefore agrees with a random set as closely as two random sets agree with each other. For the perturbation response of GPCT, about 96% of the pairs are found again in both columns.

Why both analyses use the same number of samples. One sample per donor retains 583 of the 1,214 samples (309 of 448 and 297 of 445 perturbed samples). Fewer samples alone reduce the number of significant results, so a decrease relative to the paper could reflect either the smaller sample or the removal of repeated donors, and the two could not be distinguished. We therefore fixed the number of samples in both analyses, so that they differ only in whether repeated donors are kept. A difference between the two columns can then be attributed to repeated donors. This design also indicates how much agreement can be expected at this sample size. With fewer samples, results near the significance threshold are less stable across random sets, and two random sets share about half of their significant pairs in the firing comparison. One sample per donor reaches the same level.

Removing repeated samples from the same donor therefore changes the findings no more than drawing a different random set of samples does. We will add this analysis to the appendix.

Q3. Same-study case–control comparisons

In several disease comparisons of the paper, cases and controls originate from different studies, so the programs might not remain consistent when both come from the same study. To assess this, we used acute COVID-19, for which several separate cohorts are available, and derived the programs independently within each cohort, using only its own cases and controls. Our analysis suggests that the programs remain consistent for this disease.

Five cohorts were analysed. Three are acquisition studies within our data (A, B and C), and two are external cohorts, one of hospitalized and one of intensive-care patients, which share no sample, donor or study with our data. The procedure of Section 6 was applied with GPCT to each cohort separately, and additionally to the three studies pooled.

In Section 6, conditions that share features are grouped into communities, and a program is one of these communities. We therefore compared the cohorts by the conditions (a population changed within a parent population) that each assigns to a community. The agreement between two cohorts is the Jaccard index of these conditions, that is, the number of conditions common to both cohorts divided by the number present in either. If the programs were not consistent across cohorts, few conditions would be shared and the index would approach zero.

Cohort (case / control samples)Study AStudy BStudy CExternal, hospitalizedExternal, intensive careConditions in a community
Study A (47 / 6)–.66.06.59.4850
Study B (30 / 30).66–.07.58.4856
Study C (12 / 41).06.07–.06.066
External, hospitalized (77 / 15).59.58.06–.4561
External, intensive care (116 / 8).48.48.06.45–49
Studies A + B + C pooled (89 / 77).66.59.07.65.5173

In four of the five cohorts, the procedure identifies largely the same set of conditions. Studies A and B and the two external cohorts overlap with one another, with Jaccard indices between .45 and .66, and the direction of the effects is fully concordant, since all 266 case–control differences detected in both cohorts of a pair have the same sign. The pooled analysis yields 73 conditions, which overlap with both external cohorts (.65 and .51), and all 171 shared differences again have the same sign. These 73 conditions include all 9 conditions of the analysis reported in the paper.

Study C is the exception. It comprises 12 case samples from only 3 donors and yields 6 conditions, compared with 49 to 61 in the other cohorts. With so few conditions, its Jaccard index cannot exceed .12 even if all of them were shared, and 3 or 4 of the 6 are indeed present in each of the other cohorts. The small number of conditions most likely reflects the small number of cases. We therefore consider Study C inconclusive and do not count it as a replication.

Beyond acute COVID-19, re-estimating the case–control differences within the same study retains their direction for 93.9% (GPCT) and 100% (MAESTRO) of the differences, across the 8 and 5 disease comparisons that have same-study controls.

Programs derived within a single study are therefore in agreement with those derived in other studies, including two cohorts that are entirely independent of our data. We will add these analyses to the appendix.

We would like to thank the reviewer once more for the detailed review.

We hope the revision and these responses address the reviewer's concerns. If any point remains unclear, or if a further analysis would be helpful, we would be glad to provide it within the discussion period, and we would greatly value the reviewer's view of the revised paper.

Reviewer_Y8DwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework for interpreting sparse autoencoder (SAE) features in cytometry foundation models through in silico cell composition perturbations. The method resamples cells within predefined populations to measure how SAE features respond to changes in cell composition, and uses these responses to generate interpretable feature annotations. The authors evaluate the framework on two cytometry foundation models, GPCT and MAESTRO, using held-out reproducibility, zero-shot classification, agreement with expert knowledge, and decoder steering. They further analyze differences in feature responses between cases and controls to identify state-dependent shared-feature programs.

Critical strengths

S1. The paper addresses an interesting problem in understanding the representations learned by cytometry foundation models. Rather than relying only on correlations between feature activations and cell population, the authors introduce controlled cell composition perturbations to examine how individual SAE features respond. The distinction between target populations and exchange scopes is particularly useful, as changes in cell proportions necessarily affect other populations. This provides a more explicit way to interpret the learned features.

S2. The experimental evaluation is relatively comprehensive. The authors validate their annotations on held-out samples, compare them with expert immunological knowledge, and use decoder steering to examine whether manipulating SAE features produces the expected changes in generated cell compositions. The high reproducibility of annotation directions across held-out samples is encouraging. Evaluating the framework on two foundation models and additional external cohorts also strengthens the empirical study.

Critical weaknesses

W1. My main concern is the limited methodological novelty and the unclear added value of the proposed framework over existing interpretability approaches. Previous work, such as Pedrocchi et al. (2025), has already explored SAE-based interpretation of biological foundation models and the manipulation of learned features. While the proposed cell composition perturbations provide a different approach to feature annotation, the framework mainly combines existing SAE techniques, cell population resampling, and statistical analysis. More importantly, the resulting annotations largely describe changes in predefined cell populations, which are already available from conventional cytometry analysis. It is therefore unclear what additional information about the foundation model representations is revealed by this framework beyond what could be obtained from simpler analyses of cell population proportions and their relationships with latent features. A more direct comparison with such approaches would help establish the value of the proposed method.

W2. I am not fully convinced that the evaluation establish the biological meaningfulness of the identified SAE features. The high agreement of annotation directions on held-out samples is encouraging, but it primarily demonstrates the reproducibility of model responses to the same predefined perturbations. Similarly, decoder steering evaluates consistency between the encoder-derived annotations and the decoder, rather than providing independent biological validation. The zero-shot classification results are also difficult to interpret because they depend on an LLM, and the authors acknowledge that disease labels are confounded with study differences in several comparisons. Although the agreement with expert knowledge provides additional support, the evaluation mainly confirms known changes in immune cell composition. It remains unclear whether the identified features capture meaningful biological information beyond these predefined population changes or provide new insights into disease-related immune states.

Questions for authors

Q1. How does the proposed perturbation-based annotation approach compare with simpler methods that directly associate SAE feature activations with cell population proportions? What additional information does the controlled perturbation procedure reveal?

Q2. Given that the annotations are defined using a fixed hierarchy of known cell populations, can the framework identify meaningful features that are not captured by these predefined populations, such as changes in cell activation states or other continuous cellular phenotypes?

Q3. The zero-shot classification results show different trends for GPCT and MAESTRO, with GPCT outperforming cell type proportions on average while MAESTRO performs worse. Can the authors explain this difference? To what extent the results depend on the selected SAE features and the LLM-based evaluation protocol?

7 comments

W1. Novelty

We thank the reviewer for this comment. We acknowledge that the paper did not sufficiently clarify how our work differs from Pedrocchi et al. (2025), from conventional cytometry analysis and from correlation. The novelty of our approach lies in three differences.

  1. The model we interpret takes the cells of a patient sample as input, and not the gene expression of an individual cell.
  2. Conventional cytometry analysis describes the sample, whereas our annotations describe the model.
  3. A perturbation identifies the population to which a feature responds, which a correlation does not ensure. We address each below.

Relation to Pedrocchi et al. (2025). Pedrocchi et al. train SAEs on the gene-token representations of single-cell RNA foundation models, whose input is the gene expression profile of an individual cell. They annotate features from observed associations with cell metadata and gene sets, and they intervene on the features by clamping them, which corresponds to our decoder steering (Section 5.5). The models we study are set encoders, whose input is the set of cells measured in a patient sample. Our features therefore describe which cell populations are expanded or depleted in a sample, as immune disease is commonly described in cytometry, and we annotate them by perturbing the cell composition of the sample and measuring their response. How a model responds when the proportion of monocytes rises is a question with no counterpart in a single-cell model, where input perturbations alter genes within a cell (Claye et al., 2025). The two studies therefore differ in the object of interpretation, an individual cell or a patient sample, and in the way features are annotated, by observed association or by perturbing the cell composition. To our knowledge, this is the first work to apply SAEs to a cytometry foundation model and the first to annotate SAE features by perturbing the cell composition of a sample.

Relation to conventional cytometry analysis. Population proportions describe the sample. They do not indicate what a foundation model has learned to respond to, which is needed to interpret its embeddings and predictions. The predefined populations provide the terms of this explanation, and the finding is which of them each feature responds to, and in which direction.

Relation to correlation. This difference is the subject of Q1. We address it in our response to Q1, where we compare the two methods directly.

Q1. Comparison with correlation

Correlating feature activations with population proportions is the natural simpler alternative. We compared the two and found that a correlation cannot replace the perturbation, because the two often lead to different annotations. A correlation shows which populations vary with a feature across samples, whereas a perturbation shows which population the feature responds to.

Why a correlation and a response can differ. Populations do not vary independently across samples. Some rise and fall together, and because proportions sum to one, others move against each other. A feature can therefore correlate with a population that it does not respond to. For example, GPCT feature 1251 correlates positively with the proportion of myeloid dendritic cells across samples (ρ = 0.48), yet increasing these cells directly lowers it (239 of 310 held-out samples). Samples rich in these cells are also rich in monocytes (ρ = 0.64), and the feature rises as monocytes increase.

Comparison over all feature–population pairs. For every SAE feature and every population, we computed the Spearman correlation between its activation and the proportion of the population across the discovery samples, and the response of the feature when this proportion is changed directly. We then asked how the feature responds in every pair with a significant correlation, both tested after correction for multiple testing.

Feature–population pairs with a significant correlation across samplesGPCTMAESTRO
Number of pairs332,415400,085
Response in the same direction (%)61.429.9
Response in the opposite direction (%)20.15.4
No significant response (%)18.564.7

The rows sort these pairs by the response of the feature when the population is increased. With a positive correlation, a rise is the same direction and a fall, as for feature 1251, is the opposite direction, and conversely for a negative one. A significant correlation is matched by a response in the same direction in only 61.4% (GPCT) and 29.9% (MAESTRO) of pairs. In the rest, it responds in the opposite direction or shows no significant response.

Comparison as annotation methods. The comparison above includes all significant correlations, whereas an annotation would use only the strongest, so we annotated the same SAE features by correlation on the discovery samples. For each condition, that is, a target population within a parent population, we selected the features with the largest absolute Spearman correlation with the proportion of the target, as many as the perturbation method annotated, and took the sign of the correlation as the direction.

Both were scored with the paper's evaluations. Stage 2 (Section 5.1) repeats each annotated change on the 314 held-out samples and asks whether the feature responds significantly in the annotated direction. Stage 3 compares the proportion of the population between held-out samples where the feature fires and where it does not. Expert-statement recovery (Section 5.4) counts how many of 99 expert statements on disease the annotations reproduce. Zero-shot AUROC (Section 5.2) measures how well an LLM predicts the clinical label of a held-out sample from the annotations.

GPCT, perturbationGPCT, correlationMAESTRO, perturbationMAESTRO, correlation
Held-out Stage 2, perturbation response replicated (%)99.9890.498.378.5
Held-out Stage 3, firing difference in annotated direction (%)93.891.794.393.7
Expert statements recovered (%)81.862.664.677.8
Zero-shot AUROC, held-out samples.847.792.770.806

Stage 2 tests what an annotation states, that the feature responds when the population changes. It uses the same kind of intervention as the perturbation method, so the informative values are those for correlation. Although these are the strongest correlations, 9.6% (GPCT) and 21.5% (MAESTRO) do not replicate. The two methods agree on the feature for a condition in only 19% (GPCT) and 23% (MAESTRO) of annotations.

On the other evaluations, Stage 3 is close, and expert-statement recovery and zero-shot AUROC are higher with perturbation for GPCT and with correlation for MAESTRO.

In both comparisons the reference is the response to a direct change, so they show how far a correlation supports a statement about the response of the model, not a general ranking of the two methods. The perturbation adds a direct test of the response, the cases in which correlation and response disagree, and the comparison of responses to the same change between cases and controls (Section 6). We will add both comparisons to the appendix.

W2. From annotations to biological insight (1/3)

We agree with the reviewer's reading of the evaluations. Held-out replication, decoder steering and zero-shot classification test whether the annotations describe the model correctly. Expert agreement shows that the annotations are consistent with established immunology. None of these evaluations shows that the SAE yields biological insight.

The shared-feature programs of Section 6 are how we use the SAE for insight. A program groups conditions (a population changed within its parent population) into communities. Each community is a set of populations that move together in cases relative to controls.

We ran three new analyses about the shared-feature programs. Together they show that communities are reproducible across independent cohorts, contain the populations that experts name, and relate diseases in a plausible way.

1. Communities reproduce in independent cohorts. For acute COVID-19 (GPCT), we searched for programs separately in each of the three discovery studies and in two external cohorts. The external cohorts share no donor with our data. The table gives the Jaccard index between cohorts, computed on the sets of conditions each cohort places in communities. The last row pools the three discovery studies.

Cohort (case / control samples)Study AStudy BStudy CExternal, hospitalizedExternal, intensive careConditions in a community
Study A (47 / 6)–.66.06.59.4850
Study B (30 / 30).66–.07.58.4856
Study C (12 / 41).06.07–.06.066
External, hospitalized (77 / 15).59.58.06–.4561
External, intensive care (116 / 8).48.48.06.45–49
Studies A + B + C pooled (89 / 77).66.59.07.65.5173
  • Overlap is well above chance. Studies A and B and the two external cohorts place largely the same conditions in communities (Jaccard .45–.66). Random condition sets of the same sizes give .14–.16. The shared conditions include CD4 and CD8 memory subsets, Th1 and Th1Th17 cells, classical monocytes and eosinophils.
  • Directions agree. For every pair of cohorts, all 266 differences retained in both cohorts of the pair have the same sign.
  • Limitation. Study C, with three case donors, does not reproduce the communities.

2. Communities contain the populations that experts name. This test uses the expert statements of the paper. Each statement names a population that the expert expects to change in cases. Of the 99 scoreable statements, 88 belong to the 12 comparisons for which we built shared-feature communities. The other 11 belong to three comparisons without communities and are not scored here.

For each comparison, we take the conditions in its communities and count the statements whose population is the target of one of these conditions. We score random conditions in the same way. The random sets have the same number of conditions per comparison, and we report the mean over 2,000 draws.

ConditionsStatements whose population is included
In communities78% (69/88)
Random (mean)54%

Communities contain the population of 78% of the statements, against 54% for random conditions (p < .001).

We score the population only, not the direction. A statement gives the direction of an abundance change, and a community condition has no direction of this kind.

Reviewer_nSBzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies cytometry foundation models (GPCT and MAESTRO) by interpreting controlled changes in input cell populations using sparse autoencoders. The paper uses fixed marker thresholds to evaluate changes in identified cell type populations. Each input data includes 30 markers to define cell types. Generally, GPCT performs better than MAESTRO in held-out and external cohort datasets. The paper also compares direction of change to six expert annotators' estimated directions. GPCT performs clearly better on the expert statement recovery task.

Critical strengths
  • The paper studies very recent foundation models for cytometry by interpreting the outcomes from these models during interaction with the input data sampling. The study gives additional information on the performance of these models.
  • The paper evaluates foundation models in various tests using zero-shot metadata classification, cell type proportion classification in external datasets, expert annotation retrieval, and feature manipulation prediction direction testing.
Critical weaknesses
  • The paper is slightly hard to follow. Many important technical details are put into the appendix and skipped in the main article text. Simplistic cell type prediction tests are missing. It could be included how manipulation of specific features are linked to specific cell types.
  • LLM is used for some population annotations. However, there is no validation that LLM performs accurately nor testing of different LLMs.
Questions for authors
  • See the weak points above.
  • How was the expert annotation consensus reached or did all experts annotate separate data partitions?

Minor comments:

  • "Pretrained cytometry models such as MAESTRO and GPCT take this set as input and learn sample-level representations that perform well on downstream patient-level prediction tasks": Instead of learning, these frozen models extract representations.
4 comments

W1 — Clarity and details in the appendix

We appreciate this comment, and to make the paper easier to follow we summarize the method below.

Summary

Cytometry foundation models such as GPCT and MAESTRO are deep neural networks that take the full set of cells in a sample as input and map it to a single sample-level representation. Despite their strong performance on patient-level prediction tasks, they remain black boxes. Cell type proportions, by contrast, are directly interpretable. However, they characterize the sample and not the model, and thus reveal nothing about what the model has learned. Our goal is to interpret the model in the same terms, by describing each of its internal features with respect to cell populations.

A straightforward approach would be to examine which populations are abundant in the samples where a feature is active. However, many populations vary together across samples, so such correlations can be spurious. We therefore perturb one population at a time. Specifically, we resample a sample's own measured cells so that one population, the target, occupies a larger or smaller share of an enclosing population, the scope. All cells outside the scope remain unchanged. For example, the target may be CD8 T cells and the scope all T cells. We then measure the response of each feature to this perturbation. Each feature is thus described by the population changes to which it responds, and we refer to this description as an annotation.

The procedure consists of three steps.

  1. We pass each of the 1,380 discovery samples through the frozen GPCT or MAESTRO encoder to obtain its sample-level representation.
  2. We train a Top-k sparse autoencoder to reconstruct these embeddings, so that each embedding is expressed as a small number of active features out of several thousand. These are the features we interpret, and both the encoder and the sparse autoencoder are kept fixed from this point on.
  3. We pass the perturbed versions of each sample through the same two models and regress the change in each feature on the target's share. We retain a response as an annotation only if it is statistically significant, consistent in direction across samples, and concordant with the original, unperturbed samples.

A separate set of 314 held-out samples is reserved for the evaluations described below.

Revised main text

We agree that the main text relied too much on the appendix. We will move the details a reader needs to follow the method into the main text. Three changes address this directly, and a fourth is described under Q2.

  • Perturbation doses (Section 4, Stage 2). The main text states that each perturbation is measured at seven levels of the target's share, called doses. They are the 5th, 20th, 35th, 50th, 65th, 80th and 95th percentiles of the target's share within its scope across discovery samples. Where two percentiles coincide, there are six. Every perturbed input therefore stays within the range of compositions observed in real samples.
  • How the population hierarchy was defined (Section 4, Stage 1). The previous text said only that the gates follow standard conventions. The revised text states the procedure. The structure follows the standard gating tree of the 30-marker panel (Bagwell et al., 2020). Each cutoff was fitted once, before any perturbation was measured, on cells pooled from the 1,380 discovery samples. Of the 54 cutoffs, 45 lie at a density valley of the marker within its parent population. The remaining 9 are for markers without a valley, such as CCR7, and follow gating convention. The cutoffs are listed in Appendix Table 6.
  • What the LLM receives in the zero-shot evaluation (Section 5.2). The main text describes the input. In each call, Claude Opus 5.5 receives the two candidate labels and the evidence for one sample. It receives no conversation history, study identity or label. For the annotation input, the evidence is the annotations of the sample's 16 most active annotated features. Each annotation gives the target, scope, direction and gate definitions. It does not give activation values, effect sizes or population proportions.

"Simplistic cell type prediction tests are missing."

Our framework does not predict cell types. Cell types are defined by the fixed gating hierarchy, which is the standard way to define them in cytometry. Each annotation names one of its 53 populations. The zero-shot evaluations also include a baseline that uses only the proportions of the same 53 populations (Sections 5.2 and 5.3).

"It could be included how manipulation of specific features are linked to specific cell types."

This is tested by decoder steering (Section 5.5). We add a feature's decoder direction to the embedding of each held-out sample and decode the result into cells with MAESTRO. We gate these cells with the same hierarchy and check whether the annotated population moves in the annotated direction.

W2 — Use of the LLM

No LLM is used to produce SAE feature annotations. The LLM only reads the finished annotations in the zero-shot metadata classification, as described under W1. Its answers are scored against recorded diagnoses over three runs.

Q2 — Expert statements

The six experts answered separately, and no consensus step was applied. Every statement from every expert is scored on its own. Of the 122 directional statements they wrote, 99 name a population that exists in the hierarchy. These 99 are scored, and per-expert results are in Table 18 (Appendix E.7).

Minor comment

We agree. Both models are frozen throughout our analysis. We will revise the sentence to read "Pretrained cytometry models such as MAESTRO and GPCT take this set as input and extract sample-level representations that perform well on downstream patient-level prediction tasks."

We would like to thank the reviewer once more. We hope these responses address the reviewer's concerns. If any point remains unclear, or if a further analysis would be helpful, we would be glad to provide it within the discussion period.

I thank the authors for clear response that clarifies some of my concerns. However, it seems that these changes have not been made in the current version of the paper.

We thank the reviewer for the follow-up, and we apologize that our earlier reply described the changes without including them in the PDF. All of them are now in the revised PDF, marked in blue.

We noted that the review rates the paper’s soundness as having “minor or fixable concerns”, and that the main weaknesses raised concern presentation and the role of the LLM, which we have now addressed directly in the manuscript. We would therefore be grateful if the reviewer could reconsider the overall rating in light of the revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.