- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
The paper studies cell type annotation when the reference misses some target populations. scOLAR combines ontology-based prototype learning with a calibrated rejection rule, clusters rejected cells without a supplied class count, and adds broader lineage descriptions afterward. This separates fine label acceptance from interpretation of unresolved populations. Experiments across several single-cell datasets show strong novelty detection and accurate labels among retained known cells. Comparisons with scBOL show mixed advantages across recovery metrics. Ontology controls suggest that its effects are mainly on routing and lineage context rather than novelty ranking or clustering.
Critical strengths
- Separating fine label acceptance, rejected-cell grouping, and lineage interpretation is sensible. In particular, adding a coarse description without silently converting a rejection into a known label addresses a real annotation problem.
- The experiments distinguish the stages of the pipeline and report mixed results clearly. Figure 3 and the common-evaluator comparison are useful for understanding what the ontology changes and where the method still loses to scBOL.
Critical weaknesses
- The motivation focuses on unseen types being confused with related known types, but the results do not show whether the hierarchy losses reduce that particular error. In Figure 3, real ontology has little effect on novelty ranking or grouping, and the star control remains competitive on correct-known routing. The specific benefit of learning biological hierarchy is still unclear.
- Figure 5 illustrates a rejected astrocyte population whose type is already in the reference. This is a useful fallback after a false rejection. The benefit of this stage for genuinely unseen populations is still unclear.
Questions for authors
- Do the existing predictions show fewer false known assignments for unseen types that have close represented relatives? This would help connect Figure 3 to the main motivation.
- Can the existing LCC results separate genuinely unseen cells from falsely rejected known cells, and explain what useful lineage information is recovered for the former?
- Does the clustering algorithm affect final algorithm outcomes?
4 comments
Response to Q3: Clustering Algorithm and Resolution Sensitivity
Thank you for this thoughtful question. Our controlled experiments show that clustering affects downstream PREDNOVEL grouping and LCC, while upstream open-set decisions remain invariant. We evaluated 20 frozen episodes (five datasets × four seeds), holding checkpoints, embeddings, MLS thresholds, rejected cells, and the original 15-NN graph fixed. All 20/20 upstream invariance audits passed. The same 594,685 target-cell–seed records were evaluated under each of six clustering settings. Novelty scores, Known/PREDNOVEL decisions, and accepted known labels were identical.
On the same graph, Louvain yielded comparable grouping performance to the original Leiden (resolution 1.0). Mean AMI was 0.6620→0.6643 (paired Louvain−Leiden Δ=+0.0023), and ARI was 0.3840→0.3949 (Δ=+0.0108). AMI improved in 12/20 paired episodes and ARI in 14/20. Mean community counts were 26.45 versus 25.15, with average per-episode median community sizes of 483.9 versus 494.7. Both algorithms grouped all rejected cells without noise or communities smaller than five cells.
The prespecified Leiden resolution sweep (0.5/0.8/1.0/1.2/1.5) revealed a systematic granularity effect. Across the extremes, mean community counts increased from 19.75 to 31.35, while AMI changed from 0.6964 to 0.6378 and ARI from 0.4696 to 0.3015. Relative to the original resolution 1.0, AMI increased at 0.5 in 20/20 episodes and decreased at 1.5 in 20/20. The original resolution was fixed without target-label tuning.
Clustering choice can also improve downstream lineage interpretation. On Cao genuinely novel cells, Louvain increased compatible LCC coverage from 23.81% to 27.40% (+3.60 pp) while maintaining high conditional compatibility among assigned N (91.04% versus 90.79%). Compatible coverage among falsely rejected K remained similar (53.42% versus 53.20%). N rates use all genuinely novel cells as the denominator, while K rates use rejected known cells. Effects varied across datasets. In Wagner, N assignment changed from 30.53% to 27.54%, but compatible N coverage remained zero under both algorithms. Both Quake datasets remained unassigned under the native LCC gate across the tested settings.
These results demonstrate a practical benefit of scOLAR's modular design. Clustering can adjust novel-group granularity and improve compatible lineage coverage, as observed on Cao, without retraining or changing Known/PREDNOVEL decisions. We plan to summarize the stage-specific findings alongside the grouping/LCC discussion, with complete paired AMI/ARI, community-size, resolution, and N/K-stratified LCC analyses in the Appendix. New and modified content will be highlighted in blue.
I appreciate authors' effort on the rebuttal. My major concerns have been resolved. However, I do think clutering is an important component in the method design as it acts as a role that controls novel-group granularity. As authors mentioned in the response, they plan to add these discussions into appendix. It would be interesting to examine what is the consequence of the granularity difference if combining LLMs with different clustering algorithms. As an update, I changed my soundness score to this manuscript. In summary, I think I am leaning to accept this paper.
Thank you so much for revisiting our responses and for your encouraging update! We're glad that our additional analyses have addressed your main concerns. We agree that clustering granularity deserves further attention, and your suggestion about combining LLMs with different clustering algorithms is an interesting direction for future work. We also appreciate your thoughtful feedback throughout the discussion!
The Authors
Response to CW1/Q1 and CW2/Q2: Close-Relative Rejection and Genuine-Novel LCC
We are truly grateful for your careful review and thoughtful questions. We examined frozen predictions, ontology controls, ablations, and biological evidence for genuinely unseen populations.
CW1 / Q1. Close-relative false-known acceptance and hierarchy learning
HCL organizes source-prototype geometry, nSep reduces likely-novel affinity to mapped types, and LCC provides ancestry after rejection without changing the decision.
We define false-known acceptance (FKA) as the fraction of genuinely unseen cells with close represented relatives incorrectly accepted as mapped known. All ontology conditions use the same REAL-CL-defined stratum (shared-ancestor depth-gap ≤2), each with its own source-calibrated threshold. Four-seed rates are:
REAL lowers close-relative FKA against both controls on three datasets. Cao's reversal is concentrated in withheld visceral muscle (CL:0008007), which accounts for just 8.1% of close-novel cells but 82.4% of REAL's close FKA, mainly accepted as represented relative CL:0000187. Type-level FKA is 81.2% under REAL versus 76.6%/73.2% under shuffled/star; REAL sensory-neuron/neuron FKA is 0.20%/1.79%.
Independent geometry audits establish HCL's role. Full models achieve true-CL triplet ordering 0.996 versus 0.468 without HCL, and ontology–cosine Spearman 0.501 versus −0.027, with 20/20 paired improvements. Separately trained REAL/shuffled/star models achieve ordering 0.996/0.603/0.497 and near–far separation 0.140/0.019/0.001 on identical true-CL triplets; REAL leads 40/40 comparisons on both metrics. With nSep, close FKA is lower than without it in 19/20 paired runs (one tie), including Cao 7.99% versus 11.92% and Wagner 1.44% versus 3.52%. A post-hoc, target-label-assisted Known-retention-matched diagnostic retains 19/20 improvements; deployment thresholds remain source-calibrated.
These results directly address the close-relative false-known acceptance question. HCL learns biologically ordered prototype relationships, and nSep consistently reduces erroneous acceptance of unseen relatives. The Cao type-level audit identifies where remaining errors concentrate.
CW2 / Q2. Lineage context for genuinely unseen cells
We separated genuinely unseen N from falsely rejected represented K, using ground truth only after freezing routing, communities, and LCC. Four-seed rates below use all genuine-N cells as the denominator:
For Cao N, compatible coverage of 23.81% comprises 18.70% exact withheld-term and 5.10% strict-ancestor recovery; compatibility among assigned N is 91.04%. Assigned K attains 96.85% conditional compatibility. Among eligible rejected cells, LCC assigns 27.28% of N versus 55.20% of K. This asymmetry is consistent with selective source support. Represented K has stronger direct reference evidence, so LCC abstains more often on N while retaining high compatibility and allowing exact recovery when it assigns.
For withheld Cao sensory neurons, LCC returns neuron (CL:0000540) in 3/4 seeds (scores 0.422/0.424/0.408); the fourth abstains at 0.3999 under q=0.40. Community Jaccard is 0.833–0.922. A literature-curated marker panel fixed before expression scoring favors neuronal over muscle/epidermal modules in 18/18 neuron-assigned Cao communities (16 novel-majority), versus 3/10 other-term-assigned and 21/66 unassigned communities. The markers corroborate the neuronal program; LCC adds a standardized reference-linked lineage term without forcing a fine identity.
At q=0, Quake_10x reaches 16.69% compatible coverage, identifying selective gating as one contributor; Wagner selects incompatible candidates even without gating. Frozen ancestry replay reproduces 529/529 native outputs. Cao N compatible coverage is 23.81% under REAL versus 13.24% under shuffled ancestry (18.81% after assignment-mass matching).
We will clarify §1's motivation, summarize FKA, geometry, and component evidence in §4.4 while retaining original statistics, and strengthen §4.6/Figure 5 with genuine-N LCC, frozen replay, and Cao marker evidence, distinct from the existing astrocyte K fallback. Full paired data, post-hoc diagnostic details, marker sources and community scores, and null controls will appear in the Appendix. New and modified content will be highlighted in blue.