- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces GREmLN, a foundation model for single-cell transcriptomics that uses gene interaction graphs to guide attention. The main idea is to apply a graph diffusion kernel to the query embeddings. The passthrough method preserves full-graph kernel entries when selecting genes, and a Chebyshev approximation reduces the computational cost. The authors pre-train the model on 11 million cell profiles. They evaluate cell type annotation, masked-edge recovery, and reverse perturbation prediction, reporting improvements over the tested foundation models with a 14.7M-parameter model.
Critical strengths
- Using molecular interaction graphs is a sensible choice for this problem. Diffusing the queries incorporates these relationships while retaining the original key and value representations. I also find the passthrough idea useful: selecting expressed genes should not necessarily discard paths through genes that were omitted (Sections 3.2-3.3, pp. 4-5).
- The frozen-embedding results are the strongest empirical part of the paper. In Table 3, GREmLN reaches 0.435 perturbation classification accuracy, compared with 0.162 for Geneformer and 0.281 for lasso logistic regression. Table 1 also shows an improvement in immune-cell macro F1 over scGPT, from 0.924 to 0.939. The tests with different graph sources and model depths are useful additions (pp. 7-9).
Critical weaknesses
- My main concern is that the equations do not give a consistent definition of GDKA. Section 3.2 (p. 4) treats positive semidefiniteness as sufficient for an orthogonal projection, whereas exp(-βL) is generally not idempotent. The attention formula uses ({\varPhi }{L}Q)K^{T}, but the pairwise score below it is {q{i}}^{T}{{\varPhi }{L}}^{1/2}k{j}. Since q_{i} and k_{j} are in {\mathbb{R}}^{d} while {\varPhi }{L} is in {\mathbb{R}}^{G\ \times \ G}, the pairwise score is dimensionally incompatible with the stated definitions. It is not equivalent to the preceding attention formula. There is a similar problem with the Chebyshev approximation. Appendix A.3 maps the spectrum to [-1,1], yet the recurrence uses L instead of L - I. The sum in Section 3.5 also omits the constant term, which should be \frac{c{0}Q}{2} under the stated coefficient convention. These are errors in the written formulation of the central operator. What remains uncertain is whether the implementation contains the same errors, since code is not included in the submission (p. 10).
- I am not yet convinced that the experiments establish the benefit of GDKA itself. For cell type annotation, the graph depends on a k-NN posterior learned from labeled training cells (Eqs. 6-8, pp. 6-7). This does not establish test-label leakage, but the contribution of that supervision needs to be measured. I would like to see k-NN-only and graph-only controls, along with the same backbone without diffusion or with a simpler use of the same graph. The zero-shot claim also conflicts with the described evaluation protocol. Section 4.2 explicitly trains a linear classifier on cell representations using cross-entropy and cell-type labels (p. 7). As described, this is supervised linear probing / representation transfer, not zero-shot annotation. Excluding cell types from foundation-model pre-training does not make downstream evaluation with their labels zero-shot. If Table 1 uses a separate protocol without using held-out-type labels for training or model selection, that protocol must be specified; none is described. For reverse perturbation prediction, only GREmLN is fine-tuned, so that comparison does not support superiority over fine-tuned baselines. Table 3 provides no uncertainty estimates or separate Norman results, despite describing that dataset in Appendix A.8. Finally, its random Top-10 AUC is listed as 0.50. Following the caption, the uniform-random expectation for 20 candidates is \mathrm{mean}(\frac{k}{20}), for k = 1,…,10, which is 0.275. This discrepancy needs to be resolved before interpreting the reported values.
Questions for authors
- Could you show the equations actually used in the implementation, including tensor dimensions and the complete Chebyshev expansion? A small-graph check against exact exp(-βL)Q would help. Do these corrections change any results? Please also clarify how the sparse recurrence in Appendix A.4 relates to the cached kernels in Appendix A.6.
- Which cells and labels are used for GRN inference, the k-NN graph selector, and the final classifier in Table 1? Please identify the classes used to train the linear classifier with cross-entropy. For the held-out types, describe a separate protocol that excludes their labels from training and model selection, or relabel the results as linear probing / representation transfer. Please specify the splits and compare k-NN alone and the same backbone without diffusion under the same data and supervision.
- Are the 15% held-out edges removed before computing the evaluation kernels and cached embeddings? Please explain negative sampling and the edge splits used to train and evaluate the decoder. A graph-only predictor using the same incomplete graph would help show what the expression embeddings add.
- How was Top-10 AUC calculated in Table 3, and why is its chance value 0.50? Please identify the dataset and report Adamson and Norman results separately, with uncertainty. Were the 20 perturbations selected from training data only, and how were cells split? A comparable fine-tuning baseline is needed for the fine-tuned superiority claim; otherwise, please limit that claim to the frozen comparison.
- Could you provide code, graph-construction settings, dataset and split identifiers, and downstream training details? For Tables 1-2, please specify the confidence level, interval calculation, and resampling unit. For Table 3, I would like to see repeated-run results to assess the stability of the gains.
4 comments
Response (2/2)
-
Edge recovery (Table 2) For each held-out graph, we randomly remove 15% of the true edges; these masked edges are the positives. We then add an equal number of false edges, sampled uniformly at random from gene pairs not connected in the full graph; these are the negatives. The model receives the corrupted graph (the remaining 85% of true edges plus the false edges), so it cannot recover the masked edges, or reject the false ones, from the input graph structure alone. Kernels and embeddings are computed from this corrupted graph, so the masked edges are never seen by the encoder.
-
Reverse perturbation (Table 3)
Chance level. The reviewer is correct. For 20 candidates, the chance Top-10 AUC is , not 0.50. We will correct Table 3. All reported values are unaffected; only the chance row was wrong.
Datasets and uncertainty. Table 3 reports results on the Adamson et al. dataset only, and we will state this in the caption. Although Section 4.1 and Appendix A.8 describe the Norman et al. dataset, the submission did not include results on it; we apologize for the confusion. We now report Norman results on held-out dual-perturbation combinations, with repeated runs, in our General Response, and will add them to the revised paper. Within the discussion period, we were not able to repeat the benchmarking for every model in Table 3. In the revised paper, we will report mean ± standard error for all models over repeated downstream runs (multiple seeds and data splits), using the public pre-trained checkpoints for the baseline foundation models.
Perturbation selection and splits. The 20 perturbations were selected as those with the strongest transcriptional effect relative to control cells. Cells from these perturbations were then split into training, validation and test sets, and the same splits were used for all models.
Fine-tuning claim. We agree with the reviewer. Since the foundation-model baselines were not fine-tuned, we will limit the claim of superiority over baselines to the frozen-embedding comparison. The fine-tuned results in our General Response compare GDKA only against the no-diffusion Transformer.
- Reproducibility We are unable to release the code at this time. To support reproducibility, the revised paper will give the full implementation details described in this response, including the exact GDKA equations with tensor dimensions, the complete Chebyshev expansion, the kernel computation pipeline, graph-construction settings and dataset splits. In Table 1, ± denotes the standard error across 3 random train/test splits. In Table 2, ± denotes the standard error of AUROC across 3 cross-validation folds; AP is computed on the pooled predictions. Repeated-run results for perturbation prediction are reported in the General Response. We will state these definitions explicitly in the table captions.
We thank the reviewer again. Their detailed reading has substantially improved the correctness of the paper.
Thank you for the detailed response and additional experiments. After considering the reviewers' comments and the rebuttal, I will maintain my score.
We thank the reviewer for their careful consideration of our response and for their detailed feedback throughout. We will incorporate their suggestions in the revised version of the paper.
Response (1/2)
We thank the reviewer for an exceptionally careful and detailed review. The reviewer correctly identified several errors in the written formulation, and we address each point below. Shared items are covered in our General Response. We will make all corrections listed here in the revised paper.
Projection claim. e−βL has eigenvalues e−βλℓ∈(0,1] and is not idempotent, so it is not an orthogonal projection. It is symmetric positive definite, and is therefore a valid reproducing kernel on the gene set: the heat kernel of Kondor & Lafferty (2002). We will correct the statement. No part of the method relies on the projection claim.
Complete Chebyshev expansion. With L~=L−I, whose spectrum lies in [−1,1], and the coefficients of Appendix A.3, we use a degree-K expansion with K=8, i.e. nine terms including T0:
e−βL≈2c0I+∑k=1KckTk(L~),T0=I,T1=L~,Tk=2L~Tk−1−Tk−2
Our implementation includes both the constant term and the L−I rescaling; the omissions were transcription errors in the paper, so no results change.
Small-graph check. We verified the approximation on a GRN derived from the largest cell-type network in our corpus (neurons; 6.78M edges), keeping the 500 highest-degree genes (degree counted over the full network, as either regulator or target). On the resulting 500-node graph, we compared the Chebyshev approximation with the exact e−βL (β = 0.1), computed by eigendecomposition, across expansion orders:
The approximation converges by K = 4. Beyond that point, the error stays at float32 machine precision, and its slight growth at larger K reflects rounding accumulated in the three-term recurrence rather than truncation error. At K = 8, the order we use, the maximum absolute error is 1.19 × 10⁻⁷, so the kernel is effectively exact.
Appendix A.4 vs A.6. In the implementation used for our experiments, we evaluate the matrix recurrence once per cell type per training job to construct a sparse master kernel Gram matrix (KGM) over the full gene set. For each cell, we extract the principal submatrix indexed by its input genes. We refer to this as the passthrough method. KGMs are not cached to disk. We report construction and extraction costs in the General Response.
Controlled ablation of GDKA Please see our General Response for the controlled GDKA ablation.
Cell-type annotation (Table 1) GRN inference is performed using train cells from each cell type. For example, the NK cells’ ARACNe graph is based on NK Cells within the training set. Likewise, the KNN classifier uses train cells to guess the cell type of each test cell, and then uses the graph of the guessed cell type during diffusion in the GDKA. All immune cell classes are used to fit the immune cell linear classifier, and all non-immune cell types are used to fit the non-immune cell linear classifier. We would like to further clarify that “held-out cell types” refers to cell types held out during pre-training. Such cells and their cell type labels are used to train the linear classifier whose purpose is to discriminate them. We apologize for this ambiguity and will rename this subset of data as “Cell types held-out during pre-training”.