- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks whether a frozen, self-supervised cytometry set encoder can transfer across institutions without target labels. It identifies a sample-context bias in masked set encoders such as MAESTRO: each cell token is computed in the context of its sample's other cells, but only the pooled sample embedding is supervised, so tokens become sample-specific. Without the proposed term, .93–.98 of token variance lies between samples, versus .10–.19 for the raw markers, and re-encoding the same cells within another sample moves their tokens by about 6–9 times their cell-to-cell spread; the bias is already present at random initialization. SIGCell adds cell-SIGReg on top of masked-set reconstruction and sample-level SIGReg. Cell-SIGReg applies the SIGReg isotropic-Gaussian (Epps–Pulley) statistic to a batch-normalized 64-d projection of cell tokens pooled across the 12 samples of a mini-batch. Proposition 1 argues that, at its target, this caps the between-sample share of variance at about without using sample identities. Validation uses one 1,872-sample pretraining corpus and two external COVID-19 cohorts (92 and 124 samples), all on the same 30-marker panel. SIGCell matches MAESTRO in distribution (13-class BA .789 vs .791) and gives the best external severity grading among frozen representations (BA .501, against at most .441). Its source-fitted COVID-19 detection rule transfers with the threshold unchanged at AUROC .949 and .995, first among ten models. A controlled 2×2 ablation shows that the cell term lowers the between-sample share to .12–.24 and raises cell-type read-out on unseen cohorts from .13–.19 to .64–.69 BA, while its benefit to the sample-level read-outs appears only together with sample-SIGReg.
Critical strengths
The paper turns a vague trust in frozen embeddings into a concrete, measurable failure mode, and then shows that a label-free term removes it. It is not only that a new regularizer improves a benchmark: the authors first establish that masked set encoders write the sample into every cell token, demonstrate this both with a variance decomposition and with an intervention that holds the probed cells fixed and changes only their context, show that the bias is already present at initialization, and show that it appears in an independently designed encoder but not in one that attends across cells while being trained differently. That chain of evidence is what makes the diagnosis credible, and the diagnosis should matter beyond cytometry, since the same structural problem applies to any encoder that supervises only a pooled representation of tokens produced in context. The monotone response of the token geometry to the weight of the new term then ties the diagnosis to the proposed fix. The transfer protocol is careful in ways that are unusual in this literature. All baselines are retrained on the same corpus, panel and budget, splits are donor-grouped, the decision threshold is fixed before the external data are opened, and no target label is used for fitting or recalibration. Each baseline is adapted in its own regime and every deviation from its published recipe is documented, while healthy-only and within-gate analyses rule out case mix as the source of the institutional shift. Negative results are also reported rather than buried. Frozen cross-institution evaluation is scarce here, so a protocol this explicit is itself a contribution: it is what allows Table 2 to be read as a comparison of whole models rather than of adaptation tricks.
Critical weaknesses
The sample-level claims rest on a single pretraining run per model and on very small external cohorts. Every model, including all four ablation conditions, is pretrained once at a single seed, so the reported standard deviations describe probe and split variation only and say nothing about variation in the learned representation. The headline severity margin over the next-best frozen representation (.501±.032 against .441±.046) is of the same order as those split SDs, and no paired interval is reported for it. The interaction term of +.141 is computed from four single runs. The cell-weight sweep is notably non-monotone at the sample level: Geanon source-fitted balanced accuracy is .742 / .553 / .710 / .871 / .875 for / .25 / .5 / 1 / 2, so the intermediate weights fall below the no-cell-term model while the default and twice-default exceed it, which is the pattern one expects when run-to-run variance is substantial. With only 15 and 8 healthy samples in the two external cohorts, as the authors note, a single healthy error moves balanced accuracy by .033 and .063. The cell-level effect is large enough to survive this uncertainty, but the sample-level conclusions that the abstract leads with are not established at a level I would consider robust. The path from token-level bias to sample-level transfer is asserted rather than demonstrated, and simpler alternatives are not ruled out. The sample-level gain appears only in the presence of sample-SIGReg: cell-SIGReg alone lowers external severity balanced accuracy from .459 to .422 and moves source-fitted AUROC by about +.01 on Geanon and +.005 on Penttilä. At the embedding level the paper reports that 10-nearest-neighbour label purity on the sample embeddings is the same for every model on both external cohorts (all between .39 and .48), and that cohort identity stays linearly decodable from every representation; so the mechanism by which a shared cell frame improves a source-fitted rule on the pooled embedding is left open, and the "associated with" hedge in the abstract is doing real work. The "no sample labels" framing is also somewhat overstated: sample membership within a mini-batch is available for free, so a direct penalty on the between-sample share of pooled-token variance, or a VICReg- or whitening-style penalty on pooled tokens, would be equally label-free and equally cheap. Neither is compared, and no label-free target-side normalization baseline is reported either. It therefore remains unclear whether the SIGReg statistic specifically, or any constraint on pooled token geometry, is what produces the gain, or whether part of it could be recovered by simple test-time normalization applied to a much simpler encoder.
Questions for authors
- Since each model is pretrained once, could you repeat the 2×2 conditions with additional pretraining seeds and report severity balanced accuracy, source-fitted AUROC and the interaction term with intervals over those seeds, together with paired intervals over external donors rather than over source splits only? The cell-weight sweep already varies at the sample level, so this is the analysis most likely to change my assessment of the sample-level claims.
- How does cell-SIGReg compare against simpler label-free constraints on the same pooled tokens at a matched weight, for example a direct penalty on the between-sample share of pooled-token variance using the within-batch sample index, or a VICReg- or whitening-style penalty? Since sample membership in a mini-batch is free, the absence of this comparison leaves open whether the SIGReg statistic specifically, or any constraint on pooled-token geometry, produces the gain. In the same spirit, would label-free target-side normalization, such as per-cohort marker standardization or per-sample token centering before pooling, close part of the Table 2 gap when applied to a reconstruction-only or MAESTRO encoder? The centering analysis in the appendix suggests this baseline is informative.
- What measurable property of accounts for the difference in source-fitted transfer? On the external cohorts, embedding-level label purity is indistinguishable across models, and cohort identity remains linearly decodable from every representation, so can you instead report how itself moves when disjoint cell subsets of the same donor are encoded, in units of between-donor spread, with and without cell-SIGReg?
- MAESTRO is retrained in its released-code configuration rather than its published one, with five encoder blocks and a 256-d embedding instead of three blocks and 1,024 dimensions. Does the comparison hold under the published configuration?
4 comments
Q3
-
z is stable to which cells of a sample are encoded, in every model, so the requested measurement does not account for the difference. The context does: when the cells stay and the cells they are encoded with change, .53–.68 of the between-donor spread of z remains with cell-SIGReg and .23–.32 without it. Along the frozen source rule, the external class separation of the two groups of models differs by the part that a sample's own cells give.
-
All measurements use real cells and the 15 models (five training conditions, three pretraining seeds). D is the between-donor spread of z on the source set.
- Which cells (requested). Two disjoint 10,000-cell subsets of one sample move z by .010–.014 D without the cell term and .020–.022 D with it at the source, and by .028–.040 and .043–.049 on the external COVID-19 dataset. Across subset draws the source-fitted AUROC varies with an SD of at most .004.
- Which context. We keep 5,000 cells of a sample fixed, encode them in chunks of 1,000 together with 9,000 cells of healthy donors of the source institution (the same cells for every sample), and pool z from the fixed cells only. The between-donor spread that remains is .53–.68 D with the term (SIGCell, cell-SIGReg only) and .23–.32 D without it (reconstruction only, sample-SIGReg only, MAESTRO). Three experiments with independently drawn cells, different donors in the context and four or sixteen donors per context give .51–.69 and .20–.35.
- Along the source rule. The score of each sample splits exactly into what its cells give in the common context and what its own context adds. On the external COVID-19 dataset the class separation, in units of the source class gap, is .54–.72 with the term and .36–.53 without. The cells' part is .16–.35 against −.01–.09; the context's part is similar in the two groups (.23–.52 and .32–.47). Of the difference in separation, +.16, the cells give +.21 and the context −.04. The split was fixed before the data of the third experiment were read.
- In AUROC. The context's part alone transfers at .83 with the term and .86 without (source-fitted AUROC, means over models). Adding the cells' part raises this by .11 with the term (.03 to .16 per model) and by .01 without (−.03 to .04).
- A context that carries only the institution. We repeat the reading with the context taken from healthy donors of each sample's own institution, so that the context differs between institutions and says nothing about the sample. In that reading .69–.80 of the token variance of an encoder without the term lies between institutions, against .06–.08 with the term: without the term, the tokens of the same cells follow the context. The source-fitted AUROC of the frozen rule is .91 with the term and .62 without (means over models; +.25 and +.31 for the two sets of context donors), against .93 and .87 in the samples' own context.
-
The revision reports these measurements in Section 5.4 and in a new Appendix L with a per-model table (Table 22), and Appendix M names the share of z written by a sample's own cells as the measurable property.
-
Scope:
- With five training conditions the property separates the two groups of models; it is a between-group property rather than a ranking within a group. In AUROC the accounting holds for the condition means, not model by model (one model with the term gains .03 from its cells, as much as the largest gain without it).
- The term reduces the dependence of z on the context, and the context still contributes. Replacing the context moves z by .57–.66 D with the term (.70–.82 D without), the source-fitted AUROC of the models with the term falls from .93 to .78 in the common context, and within each of these models the cells give .28–.61 of its own external separation.
- The cells' part is a share of the score. Read alone, it ranks the external donors equally well in the two groups (source-fitted AUROC .78 and .78); without the term it is small (its within-class spread is .10–.34 of that of the whole score, against .55–1.16 with the term), so it adds little to the whole.
- The ranges of the external separation are close (.537 with the term, .534 without) and overlap under other heads; the two quantities that stay separated are the retained spread and the cells' part.
- One set of four context donors can move a model's AUROC by up to .46, which is why each context is averaged over two sets. In the reading with the institution's context the two ranges touch (.83–.96 with the term, .49–.83 without), and in the common context the difference between the groups is −.00 with one set of donors and +.10 with the other.
- The split is defined against a reference context of healthy source donors.
Q4
- The configuration we used is a published one: that of the MAESTRO authors' 2026 manuscript (Lee et al., bioRxiv 2026, doi:10.64898/2026.07.15.724605), which the released code implements. Its Methods (Sec. 3.3) specify:
- five induced-point attention blocks with 16 learned points each, one attention head;
- hidden size 384 and a 256-d latent;
- EMA momentum 0.99, batch size 3 and 500 epochs.
Our MAESTRO run has each of these values.
-
The ICLR 2025 paper describes an earlier configuration (three ISABs with 2,500 inducing points, a 1,024-d latent). The 2026 manuscript reports all of its results with the newer one.
-
We have not run the ICLR 2025 configuration, so the comparison is established for the 2026 configuration only; Appendix N states both configurations.
Thank you for the thorough rebuttal. The three pretraining seeds with donor-level paired intervals, the three-seed interaction, the comparison with simpler pooled-token constraints, and the revised wording address most of my concerns, and the cell-level effect, which holds at every seed, is the robust core of the paper. Some points remain open: the sample-level comparisons for the alternative constraints and for MAESTRO + cell-SIGReg rest on single runs, the external benchmark was changed during the rebuttal, and the link from token geometry to sample-level transfer is still only associative, so I would keep the "associated with" hedge. I therefore keep my rating of 3.
Thank you for the review; the analyses you asked for in Q1 are now in the paper.
Weakness paragraph (single run, severity margin, interaction), Q1 → Blocks G1, G2 and G3. In brief:
Cell-weight sweep
The non-monotone sample-level rows of the sweep come from one pretraining seed, and we no longer read them as a dose response. The two ends of the sweep, which exist at three seeds, give a consistent sign.
The sweep and the seed-by-seed comparison:
Section 5.5 and Tables 12–13 say so; the token-geometry rows, which change monotonically, are unchanged.
The intermediate sweep points remain at one pretraining seed.
Path from token-level bias to sample-level transfer → Block G6.
"No sample labels": we agree; sample membership within a mini-batch is free, and an explicit penalty that uses it is in Block G4. The revision replaces "without sample labels" by "without using sample membership".
Q2, first part (simpler constraints) → Block G4.
Q2, second part (target-side normalization)
Label-free target-side normalization does not close the gap; in almost every model and seed it lowers source-fitted transfer.
Source-fitted AUROC, base → per-cohort marker standardization / per-sample token centering before pooling, three pretraining seeds, Table 2 protocol:
New Appendix Table 11.
The base values come from a re-embedding of the external samples for this analysis and differ from Table 2 by up to .05.