- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The others present a method that jointly learns cell and patient representations and offers insights into learning patient representations through downstream applications and interpretable analysis. However, I believe the method does not draw a clear conclusion, and the interpretability claims seem overstated.
Critical strengths
The others present a method which can jointly learning the representations of cells and patients, and offer insights in learning the patient representation through downstream applications and interpretable analysis.
Critical weaknesses
The others present a method which can jointly learning the representations of cells and patients, and offer insights in learning the patient representation through downstream applications and interpretable analysis. However, I believe the method does not present a clear conclusion and the interpretability seems overstated. Here are my questions:
- I think the learning of cells and patients is not a novel idea as it is very natural to learn the cell information before the patient level, The authors should discuss more in the introduction section to justify their novelty.
- The presentation method is also limited. If the data do not have cell type labels, we cannot use this method based on the visualization. Many singles of data do not have precise cell type labels The authors should consider this issue.
- The benchmark study is also limited and the authors ignore many related base lines such as MultiMIL and scDiffusion in the patient level. The authors should include them in the benchmark, and justify that they make a fair comparison by tuning other methods to their best performance.
- Did the authors split training/testing data through study or batch? If they split through batch, there could be data leakage.
- I cannot see the contributions of interpretability, at least the authors need to show how they can use it to interpret the prediction results.
- The testing of recent virtual cell foundation models, such as X-atlas and STATE, should considered.
Questions for authors
Please see my comments above.
5 comments
Response to Reviewer 2MkJ (2/3)
2 — Dependence on cell-type labels.
CoState can construct and visualize donor profiles for new samples that lack manual cell-type annotations. We infer cell types with a 15-nearest-neighbour classifier on CoState’s frozen cell embeddings, using annotated cells from training donors as references. The inferred labels define the cell-type groups in the visualization and are used to build the donor profile for disease prediction. On the Test split, the inferred cell types matched the manual annotations with 91.8% accuracy. We describe this option in §3.1, lines 160–161 and the label-transfer procedure in Appendix E.1, lines 1160–1165.
To assess the effect on donor-level disease prediction, we rebuilt the donor profiles with inferred cell types using the same trained CoState checkpoints and repeated the evaluations in Table 1. The table reports mean disease prediction macro-F1 across three seeds (Appendix E.1, Table 4).
| Cell-type labels used to build donor profiles | Test (kNN / Proto. / Linear) | Val. (kNN / Proto. / Linear) | Ext. (kNN / Proto. / Linear) |
|---|---|---|---|
| Atlas labels (manual for Test, scANVI label transfer for Val. and Ext.) | 0.891 / 0.900 / 0.849 | 0.860 / 0.888 / 0.867 | 0.840 / 0.887 / 0.823 |
| Labels inferred from CoState embeddings | 0.884 / 0.910 / 0.842 | 0.857 / 0.884 / 0.864 | 0.832 / 0.891 / 0.825 |
Using inferred cell types changed mean donor-level disease prediction macro-F1 by at most about 0.010 across all nine comparisons.
3 — MultiMIL, scDiffusion and fair tuning.
We added MultiMIL and scDiffusion to the donor-level disease prediction benchmark (Table 1). Bold indicates the best score for each readout among the methods shown.
| Method | Test (kNN / Proto. / Linear) | Val. (kNN / Proto. / Linear) | Ext. (kNN / Proto. / Linear) |
|---|---|---|---|
| CoState | 0.891 / 0.900 / 0.849 | 0.860 / 0.888 / 0.867 | 0.840 / 0.887 / 0.823 |
| MultiMIL | 0.923 / 0.860 / 0.844 | 0.676 / 0.704 / 0.686 | 0.760 / 0.744 / 0.677 |
| scDiffusion | 0.695 / 0.709 / 0.617 | 0.652 / 0.505 / 0.496 | 0.838 / 0.769 / 0.666 |
We used the official MultiMIL recipe with scVI latents and selected its checkpoint by validation accuracy. For scDiffusion, we used the upstream autoencoder defaults and mean-pooled cell latents to obtain donor embeddings. For ProtoCell4P, we chose the number of prototypes on a 15% holdout of training donors using its validation criterion, donor AUC plus one tenth of cell-type accuracy. We describe these protocols in Appendix E.4, lines 1439–1450.
4 — Data split.
We held out donors for Test and Validation, and entire studies for External evaluation. Test and Validation donors did not enter CoState training and came from studies represented in training. All seven External studies were held out from CoState training. We split the main cohort by donor and sequencing-library component, keeping each donor’s cells in a single split. We held out 10% of cells within the training donors for checkpoint selection under a fixed training budget. We describe the splits in Appendix E.1, lines 1147–1154 and checkpoint selection in Appendix E.4, lines 1564–1565. Cross-study retrieval excludes all candidates from the query donor’s own study (§4.2, lines 326–331).
Response to Reviewer 2MkJ (3/3)
5 — Interpretability.
We added a practical example in §4.5, lines 471–477 to show how CoState explains a prediction. We compared one held-out donor from COMBAT [1] with two reference profiles, built by averaging the profiles of 24 other COVID-19 donors and eight donors with non-COVID sepsis from the same study. Each profile describes how common each cell type is and which gene-expression patterns its cells show. The donor’s profile was closer to the COVID-19 reference (squared Hellinger distance 0.08325 vs. 0.26367 for the sepsis reference). Within-type monocyte states accounted for 75.2% of the distance margin favoring the COVID-19 reference (Appendix H.4, lines 2335–2340).
Of the 250 cells sampled from this donor, 67 were monocytes (26.8%). Monocytes made up 31.6% of the COVID-19 reference and 38.8% of the sepsis reference. CoState assigns each monocyte weights across 16 learned expression patterns. One pattern accounted for 85.1% of the donor’s average monocyte assignment weight, compared with 82.5% in the COVID-19 reference and 10.0% in the sepsis reference (Appendix H.4, lines 2341–2344).
CoState’s monocyte state analysis highlighted IFI27 in the comparison between COVID-19 and sepsis (q = 0.023 after correction across 2,000 genes, see §4.5, lines 473–475 and Appendix H.4, lines 2353–2358). Saichi et al. independently reported elevated IFI27 RNA in CD14⁺ monocytes from patients with moderate COVID-19 [2]. This example shows how CoState can trace a donor match to a cell state and highlight genes that help interpret that state.
Reference
- COMBAT Consortium. A blood atlas of COVID-19 defines hallmarks of disease severity and specificity. Cell, 2022. DOI.
- Saichi M et al. Single-cell RNA sequencing of blood antigen-presenting cells in severe COVID-19 reveals multi-process defects in antiviral immunity. Nature Cell Biology, 2021. DOI.
6 — X-Atlas and STATE
We agree that STATE is a relevant addition to our benchmark alongside UCE, scGPT, Geneformer, and Nicheformer. We averaged frozen STATE (SE-600M) cell embeddings within each donor view and evaluated the resulting donor embeddings on disease prediction (Table 1) and cross-study retrieval (Table 12). The revised manuscript describes the protocol in Appendix E.4, lines 1546–1548.
| Method | Test (kNN / Proto. / Linear) | Val. (kNN / Proto. / Linear) | Ext. (kNN / Proto. / Linear) | Cross-study retrieval (Test / Val. / Ext.) |
|---|---|---|---|---|
| CoState | 0.891 / 0.900 / 0.849 | 0.860 / 0.888 / 0.867 | 0.840 / 0.887 / 0.823 | 0.969 / 0.984 / 0.639 |
| STATE (SE-600M) | 0.725 / 0.703 / 0.716 | 0.659 / 0.561 / 0.539 | 0.807 / 0.783 / 0.756 | 0.234 / 0.310 / 0.368 |
We also evaluated the individual STATE cell embeddings on held-out Test donors and added these results to Table 2.
| Method | Leiden NMI | Leiden ARI | k-means NMI | k-means ARI | kNN macro-F1 | Type first | Transfer F1 |
|---|---|---|---|---|---|---|---|
| CoState | 0.796 | 0.762 | 0.703 | 0.520 | 0.846 | 0.883 | 0.901 |
| STATE (SE-600M) | 0.490 | 0.239 | 0.444 | 0.238 | 0.731 | 0.254 | 0.723 |
X-Atlas/Orion is a Perturb-seq dataset for studying gene perturbations in HCT116 and HEK293T cell lines. Its labels describe perturbations, while our benchmark requires disease labels for individual donors. X-Atlas/Orion dataset card
Response to Reviewer 2MkJ (1/3)
Thank you for the careful review. We have revised the manuscript in response to your comments. In the revised version, we clarify how CoState uses distance decomposition to explain disease predictions, report results for the requested baselines, and include a worked example showing why a donor was matched to one disease reference.
1 — Novelty
Prior methods such as MrVI, MultiMIL, ProtoCell4P, and PaSCient already connect cell-level information to donor-level analyses. CoState makes this connection explicit by representing each donor as a joint distribution over cell types and learned cell states. This representation supports disease prediction and lets us explain the distances underlying kNN and prototype predictions (§1, lines 59–70, and §2, lines 128–135).
Interpretable donor comparisons. Proposition 1 decomposes the squared Hellinger distance exactly into a term for differences in cell-type proportions and one term per cell type for differences in its state distribution. Squared Hellinger is symmetric and bounded, remains finite when a cell type is absent from one donor, and equals half the squared Euclidean distance between the square-root probability vectors. Across the three cohorts, within-type state differences accounted for 74–78% of the distance between donors with different diseases from different studies (§3.3, lines 195–214, and Appendix G.4, lines 1989–1995 and Figure 9).
We kept the learned donor distributions fixed and varied the distance used for kNN disease prediction (Appendix H.3, Table 16).
Cell-level representations. CoState scored highest on the five cell-type structure metrics shown below. The comparison includes joint-label scANVI and ProtoCell4P, both of which use disease and cell-type labels. Removing CoState’s donor-level losses lowered the mean score on all five metrics across three seeds (§4.3, lines 380–383 and Table 2, and §4.5, lines 434–435 and Table 15).
Donor-level performance. Among the configurations shown, CoState had the highest point estimate on 10 of 12 readouts across disease prediction (Table 1) and cross-study retrieval (Figure 2 and Tables 8 and 12). Both report disease macro-F1. In cross-study retrieval, each query is matched to its nearest donor from a different study (§4.2, lines 326–331).
We clarified how the donor representation connects prediction and attribution in the revised introduction (§1, lines 59–66) and related work (§2, lines 128–135).