- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
scRFP is an untrained propagation method for sparse scRNA-seq data. It uses reciprocal neighbors to adjust cell similarities, adaptively stops hard propagation, and prunes the neighborhood before soft propagation. The method is evaluated on seven public datasets.
Critical strengths
The method has a simple design and does not require neural network training. Its three components target different parts of propagation: neighbor selection, propagation depth, and neighborhood size. It achieves the best clustering results on six of the seven datasets, and the ablations support the usefulness of these components.
Critical weaknesses
The hard-propagation stage uses adaptive stopping, but the soft-propagation stage always runs for 80 steps. The paper does not clearly explain this difference or show how sensitive the results are to this setting. The method is described as label-free, but the stopping threshold is selected using ARI on the other six datasets. This means that labels are still used during parameter selection, and the claim should be stated more precisely.
Questions for authors
Why is soft propagation fixed at 80 steps after the adaptive hard-propagation stage? Please provide a brief justification or sensitivity result.
Which labels are used during parameter selection, and which part of the method is truly label-free?
2 comments
Kind Follow-Up from Authors
Dear Reviewer 3uBR,
As the rebuttal period draws to a close, we wanted to follow up on our response to your questions. Thank you again for your thoughtful comments.
Regarding the choice of 80 steps, we clarified that adaptive stopping determines when to end hard propagation before graph reconstruction. Soft propagation then runs on the reconstructed graph for a fixed number of steps shared across datasets. We selected 80 steps empirically. Across the seven datasets, 80 steps gave a mean clustering ARI of .830, compared with .763 for 20 steps and .754 for 40 steps.
We also clarified how we used cell-type labels. For each held-out dataset, we selected the stopping threshold using clustering ARI on the other six datasets. All seven splits selected the same value. We selected the soft-propagation depth using clustering ARI on all seven datasets, including the target dataset. Once these parameters are set, scRFP performs imputation without cell-type labels.
We would appreciate it if you could take a look at our response. We hope these clarifications address your concerns, and we would be happy to answer any further questions.
We sincerely thank the reviewer for the positive assessment and the thoughtful questions, which helped us state the scope of our claims more precisely.
[W1] Biological zeros and expression leakage
We agree with the reviewer that masking nonzero entries tests the recovery of technical dropout rather than the preservation of biological zeros. As an additional analysis, we call a cell type negative for a gene if fewer than 1% of its cells have a nonzero raw count, and positive if at least 50% do. Negative types must contain at least 30 cells. Genes with both a negative and a positive type give 1,122 gene–dataset combinations. These populations are defined from detection rates and are not independently validated biological negatives. AUROC measures how well a method ranks positive cells above negative cells. Leak is the percentage of negative cells whose value exceeds the lower quartile of positive-cell values for the same gene and method. Both metrics are computed per gene, averaged within each dataset, and then averaged equally across datasets. Cell-type labels are used only for this evaluation. We compare scRFP with the three FP methods, which share its propagation framework.
scRFP has a Leak of 0.67%, compared with 5.27% for scFP, 3.00% for scBFP, and 0.33% for scCR. This analysis measures relative separation in empirically defined low-detection populations rather than preservation of genuine biological zeros.
[W2/Q1] Label use in parameter selection and baseline settings
We thank the reviewer for this comment and agree that our label-free description requires qualification. scRFP uses no cell-type labels during imputation, but labels are used for ARI-based parameter selection.
Appendix F.2 reports leave-one-dataset-out selection of the stopping threshold τ. For each held-out dataset, τ is chosen from the five tested values by maximizing mean ARI on the other six datasets. The held-out labels are used only to evaluate the selected τ. All seven splits select τ = 0.00745, the shared default used in Table 1. This exclusion of target labels applies only to τ. L = 80 was selected using clustering performance on all seven datasets, including each target dataset ([W2/Q2]). We will distinguish label use in parameter selection from the absence of label use during imputation.
For the baselines, we used the authors' released implementations with their default hyperparameters and no additional tuning, under the same preprocessing and evaluation protocol. We will state explicitly that this protocol differs from our development-stage parameter selection. Our comparison covers nine baselines including the recent propagation-based methods scFP, scBFP, and scCR.
[W2] Reciprocal edges without the global transformation
We thank the reviewer for this suggestion. We ran this comparison in response to Reviewer BE6y [W2/Q2], where the full per-dataset table is given. Using reciprocal edges directly as the initial graph (mutual kNN), with all other steps of scRFP unchanged, gives an average ARI of .602. Adding each cell’s nearest neighbor raises this to .660, compared with .830 for scRFP. These results support using reciprocal pairs to estimate a shared transformation over directly using them as graph edges within the same propagation pipeline.
[W2/Q2] Choice of L = 80 and sensitivity to L and the initial neighbor count
The adaptive stopping rule controls hard propagation before graph reconstruction. Soft propagation then operates on the resulting fixed graph for a shared number of steps L. We set k = 15 and α = 0.99 following scFP and scCR, and selected L = 80 empirically based on clustering performance across the seven benchmark datasets. The table below reports clustering ARI when varying the number of soft-propagation steps L and the initial neighbor count k. We vary one parameter at a time and keep all other settings at their defaults.
L = 80 and k = 15 achieve the highest average ARI among the tested settings, although individual datasets favor different values. We will add these sensitivity results and clarify the empirical choice of L = 80 in the paper.
We hope these responses address the reviewer's questions, and we would be happy to provide any further clarification.