acceptodds
Under review as a conference paper at ICLR 2027

KA-MIL: Knowledge-Anchored Multiple Instance Learning for Patient Classification from Single-Cell Transcriptomics

Abstract

Patient classification from single-cell transcriptomics requires aggregating highly heterogeneous cell populations under limited patient-level supervision. Standard data-driven multiple instance learning (MIL) pooling lacks an explicit mechanism for organizing cellular contributions by biological concepts. We introduce Knowledge-Anchored Multiple Instance Learning (KA-MIL), an architecture that utilizes embeddings of biological descriptions as semantic queries for attention-based cell aggregation. Within this framework, a geometry regularizer aligns the pairwise similarities of projected semantic queries with those of the original text embeddings, while learnable residual queries provide complementary task-adaptive capacity. Using Gene Ontology (GO) descriptions with cohort-wide, label-free vocabulary construction, KA-MIL achieves the highest mean macro-F1 in all five primary evaluation settings: four clinical cohorts and a reduced-training setting. On the COVID-19 cohort, it improves macro-F1 over Hier-MIL by 7.04 percentage points. Beyond prediction, our COVID-19 analysis reveals a recurring relative humoral–myeloid attention contrast between plasmablast and CD14 monocyte summaries in 54 of 58 higher-severity test patients across five folds. Together, these findings demonstrate that knowledge-anchored aggregation effectively couples accurate patient classification with the structured, description-guided inspection of cellular summaries.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Multiple Instance Learning, Single-Cell Data, Knowledge Integration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_4u3YOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces KA-MIL, a multiple instance learning framework for patient-level classification using single-cell transcriptomics. It uses Gene Ontology text embeddings to guide attention, with regularisation to preserve semantic relationships and additional learnable queries to capture other relevant information. Built on scMILD, the method is evaluated on four clinical cohorts and shows improvements over several MIL baselines. Ablation studies examine the main components, while an analysis of the COVID-19 cohort provides some insight into the learned attention patterns.

Critical strengths
  1. Effective integration of biological knowledge into MIL.

The main strength of the method is its use of biological knowledge to guide attention without requiring cell-level annotations. The geometry regularisation helps preserve semantic relationships while allowing the queries to adapt to the task. The additional learnable queries are also useful, as predefined biological concepts may not capture all relevant information. Overall, this is an interesting extension of conventional attention-based MIL.

  1. Encouraging experimental results.

KA-MIL shows promising results, particularly on COVID-19 and nasal swabs, where it clearly outperforms the matched scMILD baseline. The ablation studies also support the contribution of the main components, including geometry regularisation and residual queries. The additional experiments with reduced training data and a more restrictive vocabulary are useful for assessing the robustness of the approach.

Critical weaknesses
  1. Dependence on transductive preprocessing.

My main concern is that the GO vocabulary is constructed using data from the entire cohort, including test patients. When restricted to training data, performance drops noticeably, sometimes below the baseline. Even this experiment retains cohort-wide preprocessing. A fully inductive evaluation is needed to establish how well the method generalises to new patients.

  1. Limited validation of biological interpretability.

The attention patterns are interesting, but they do not necessarily show that the learned queries capture the intended biological pathways. The similar performance of random and GO anchors on COVID also raises questions about the contribution of biological knowledge. Independent validation, such as pathway enrichment in highly attended cells, would strengthen the interpretability claims.

Questions for authors

Could the authors evaluate KA-MIL using a fully inductive pipeline, with all preprocessing and vocabulary construction based only on training data? This is particularly important given the performance drops on COVID and NS.

Could the authors validate whether the learned queries actually capture their intended biological processes, for example through gene enrichment analysis of highly attended cells?

Why does KAA provide less consistent improvements with frozen foundation-model features? Does the method depend on jointly training the encoder?

1 comment

We thank the reviewer for these questions. The additional experiments are complete. We will update the PDF shortly and post a consolidated comment identifying the relevant content.

Q1: Fully inductive evaluation.

A1: We evaluated a fully inductive pipeline on COVID, NS, UC, and Lupus10, using the same splits as the original experiments. HVGs and GO vocabularies are selected using training patients only, and the AE is retrained using training cells only. The training-derived gene set and fitted encoder are then applied to validation and test cells.

All three models were retrained under this pipeline, retaining the original validation-based checkpoint/threshold selection rule. Each dataset row reports test AUC / macro-F1 averaged over its five splits. The final row is the equal-weight mean of these four setting-level results.

DatasetKA-MILscMILD(-OPL)ABMIL
COVID0.7738 / 0.69290.6835 / 0.56840.6713 / 0.5806
NS0.8893 / 0.80650.9083 / 0.77710.8988 / 0.7726
UC0.9778 / 0.93140.9778 / 0.93140.9750 / 0.9657
Lupus100.9910 / 0.92740.9830 / 0.90110.9654 / 0.8894
Mean across four settings0.9080 / 0.83960.8881 / 0.79450.8776 / 0.8021

For the two cohorts highlighted by the reviewer:

  • COVID: KA-MIL leads both baselines in test AUC and macro-F1.
  • NS: KA-MIL achieves the highest macro-F1.

The mean across four settings is highest for KA-MIL on both test AUC (0.9080) and macro-F1 (0.8396). We will include validation results and hyperparameter configurations in the revised PDF.

Q2: Do queries capture their intended biological processes?

A2: Using the original COVID checkpoints, we contrasted pseudobulk RNA expression between the top and bottom 20% of attention-ranked cells within each patient and cell type. Genes were then ranked by a paired expression statistic and evaluated via GSEA-style running-sum enrichment, using patient-level sign-flip permutations and FDR correction.

In a joint analysis of 90 held-out patients, 13/50 queries showed positive enrichment (FDR < 0.05), including inflammatory response (NES = 1.643, q = 0.0234). Averaged across patients, its attention also showed stronger within-cell-type correlation with its RNA gene-set score than the mean correlation of other queries’ attention with that same score (FDR q = 0.0010). For the 14 hospitalized patients in original Fig. 2 (fold 4), humoral/myeloid queries showed positive enrichment (NES = 1.532/1.501; both case-study FDR q = 0.0028).

To address the random-anchor concern, we used three vocabulary-selection seeds to generate three sets of 50 real, low-cohort-relevance GO biological-process terms, approximately matched on ontology depth, gene-set size, detected-gene count and coverage, and description length. We retained the original cohort-wide protocol, training settings, and frozen BioLORD encoder. COVID test macro-F1 fell from 0.7075 to 0.6067 (three-vocabulary mean), a 10.07-percentage-point advantage supporting the value of cohort-relevant descriptions. Per-vocabulary results and matching details will be included in the revised appendix.

Q3: Does KAA depend on jointly training the encoder?

A3: KAA shows potential on fixed representations, while the full framework is designed to further realize this potential through joint adaptation.

On identical frozen scGPT and State Embedding (SE) features, with validation-based configuration selection, KAA achieves equal-weight mean test AUC / macro-F1 of 0.8607 / 0.7750 across eight settings (four datasets × two embeddings), compared with 0.8434 / 0.7529 for full-covariance MixMIL and 0.8572 / 0.7765 for attention MIL. KAA improves over MixMIL on average and remains competitive with attention MIL.

Mechanistically, the architecture supports task adaptation at two levels:

  • With a frozen encoder: KAA learns the query/key projections, the scorer for patient-specific summary weights (Eq. 3), and classifier, allowing aggregation to adapt to the clinical target over fixed cell features.
  • With joint encoder training: Patient-classification gradients propagate through KAA into the cell encoder, allowing cell features supplying attention keys and values to adapt alongside aggregation. BioLORD text embeddings remain fixed in both regimes.

This synergy is directly corroborated by our local-AE freezing ablation (Tab. 3), where freezing the encoder after reconstruction pretraining reduces macro-F1 across all four cohorts. Together, these results confirm KAA's effectiveness on fixed representations and the decisive benefit of joint adaptation. We will clarify these comparisons in the revised manuscript.

Reviewer_FhvCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes KA-MIL, a multiple instance learning method for patient classification from single-cell transcriptomics. The method uses embeddings of 50 Gene Ontology descriptions as attention queries, with each query producing a separate weighted summary of a patient’s cells. The model also includes a geometry regularizer that preserves relationships between the GO embeddings and residual learnable queries for additional flexibility. The method is evaluated on four clinical cohorts and a reduced-training Lupus setting, with comparisons to attention MIL, scMILD, and foundation-model baselines, together with component ablations and a COVID-19 attention analysis. KA-MIL achieves the highest mean macro-F1 in the five primary settings under the cohort-wide GO construction protocol.

Critical strengths

The main strength is the core idea. The paper uses embeddings of 50 Gene Ontology descriptions as attention queries, so that each query produces a separate summary of a patient’s cells. This is a simple and interesting way to bring biological knowledge into MIL without requiring paired cell-text data.

The results are also encouraging. KA-MIL achieves the highest mean macro-F1 in all five main settings. Compared with the matched scMILD(-OPL) baseline, the gain is 12.9 points on COVID and 8.6 points on NS. The paper also includes useful ablations for random anchors, geometry regularization, residual queries, and encoder adaptation.

Critical weaknesses

The main concern is that the current ablations do not fully establish that the biological relevance of the GO terms is what drives the improvement. The random-anchor control is useful, but it is still a limited control for this question. On COVID, GO anchors give a macro-F1 of 0.7075 versus 0.7020 with fixed random anchors, while on NS the difference is larger, 0.8679 versus 0.7817. A stronger test would use real GO terms that are not selected to be relevant to the cohort, while keeping the rest of the model unchanged.

A second concern is the uncertainty around the reported gains. The main results are based on only five splits, and several settings have fairly large standard deviations. The paper does not report paired statistical tests or confidence intervals, so it is difficult to assess how stable some of the improvements are across splits.

Questions for authors
  1. Can KA-MIL be evaluated using 50 real but cohort-irrelevant GO biological-process terms, while keeping the same BioLORD embeddings, query geometry regularization, KAA architecture, and training procedure? This would help determine whether the biological relevance of the selected GO terms is important.

  2. Can paired fold-level statistical comparisons and confidence intervals be reported for the main comparisons, especially against scMILD(-OPL), ABMIL, and the random-anchor controls?

  3. Could the comparison include a simple MIL method using fixed, precomputed cell embeddings, such as MixMIL? This would help isolate the effect of the aggregation method from representation learning.

1 comment

We thank the reviewer for these concrete suggestions. The requested additional comparisons have been completed. We will incorporate their methods and full results into a revised manuscript shortly and post a consolidated update with the corresponding section, table, and figure references.

Q1: Random anchors are limited. Can KA-MIL be evaluated using 50 real GO terms with low cohort relevance?

A1: For each of COVID, NS, UC, and Lupus10, we constructed three vocabularies using three vocabulary-selection seeds, each containing 50 real GO biological-process terms with low cohort relevance under a label-free screen for enrichment among cohort sample-variable genes (FDR ≥ 0.5 and enrichment ratio ≤ 1). The controls were approximately matched on ontology depth, gene-set size, detected-gene count and coverage, and description length. Each vocabulary was evaluated on the original five patient splits under the original cohort-wide protocol, retaining the same frozen BioLORD encoder, query-geometry regularization, KAA architecture, and training settings.

Averaging the three vocabularies within each dataset and then the four dataset settings gives mean test AUC / macro-F1 of:

  • Original GO vocabulary: 0.9229 / 0.8678.
  • Low-cohort-relevance GO: 0.8925 / 0.7958.

Both metrics decrease in all four settings after averaging the three control vocabularies. The original vocabulary's macro-F1 advantages are 10.07 percentage points on COVID and 12.79 points on NS. These matched real-GO controls support the contribution of cohort-relevant biological descriptions beyond generic text-anchor capacity. We will report each control vocabulary separately as well as their averages.

Q2: Uncertainty around reported gains. Can paired fold-level statistical comparisons and confidence intervals be reported?

A2: We computed paired differences in five-split mean test performance for the original cohort-wide models, using 10,000 paired sample-cluster bootstrap resamples to construct pointwise 95% confidence intervals while preserving repeated samples across overlapping splits. The comparison with the matched architectural baseline, scMILD(-OPL), is summarized below. Differences are KA-MIL minus scMILD(-OPL), in percentage points:

DatasetΔ AUC [95% CI], ppΔ macro-F1 [95% CI], pp
COVID+5.60 [-3.49, 15.04]+12.85 [1.39, 24.78]
NS+4.76 [-4.40, 15.48]+8.56 [0.00, 18.31]
UC0.00 [0.00, 0.00]+6.86 [0.00, 16.76]
Lupus+0.08 [0.00, 0.50]+0.73 [0.00, 2.35]
Lupus10+0.82 [-0.02, 1.81]+6.17 [3.64, 8.85]

Across all four comparators (scMILD(-OPL), ABMIL, and fixed/learnable random anchors), KA-MIL achieves a higher mean macro-F1 in all 20 comparisons. When considering both AUC and macro-F1 combined, 34 of 40 mean differences are positive, five are tied, and only one is negative (NS AUC versus ABMIL, −1.25 percentage points). Eleven pointwise 95% confidence intervals lie entirely above zero.

These bootstrap confidence intervals quantify patient-sampling uncertainty conditional on the fitted models; they do not include retraining variability. Split-level paired differences additionally describe variation across the evaluated splits. We will report all 40 AUC/F1 comparisons, together with full foldwise differences and exact paired-test results, in the revised appendix.

Q3: Could the comparison include a simple MIL method using fixed, precomputed cell embeddings (MixMIL) to isolate aggregation from representation learning?

A3: We evaluated KAA, attention MIL, and full-covariance MixMIL on identical frozen scGPT/SE features and patient splits. KAA and attention MIL share the teacher–student framework, with attention MIL using ordinary attention pooling; MixMIL retains its own modeling and training framework. The attention MIL comparator is distinct from standalone ABMIL. Each dataset/embedding/model receives 24 candidate configurations, with one configuration across five splits selected by mean validation classification loss minus mean validation AUC. Across eight settings (four dataset settings × two embeddings), mean test AUC / macro-F1 is:

  • KAA: 0.8607 / 0.7750.
  • Attention MIL: 0.8572 / 0.7765.
  • MixMIL (full covariance): 0.8434 / 0.7529.

These comparisons support the potential of knowledge-anchored aggregation within a fixed feature space. Our full KA-MIL framework is designed to further realize this potential by jointly adapting the cell encoder and aggregation module to the patient-classification task. The benefit of encoder adaptation is supported by the local-AE freezing ablation (Tab. 3). Full configurations and dataset-specific results will be included in the revised appendix.

Reviewer_TS58OpenReview ↗
Rating
1(−1)
Confidence
4(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes the KAMIL, a knowledge anchored approach for patient-level classification from single-cell RNA sequencing data. KA-MIL integrate biological knowledge into the aggregation process, by representing genomic ontology biological process descriptions as semantic queries. These text embeddings, along with learned cellular features, are projected into a shared attention space, allowing each query to aggregate a subset of cells that may overlap. Geometric regularization preserves pairwise similarity between the projected query and its original text embedding, while additional learnable residual queries provide task-specific flexibility.

This framework is integrated into the “teacher-student” training architecture of scMILD and evaluated in four clinical single-cell cohorts (COVID, NS, UC, and lupus) and a streamlined lupus scenario. The authors report competitive classification performance, supported by component ablation experiments and analysis of biologically plausible query-specific attention patterns in COVID-19 patients.

The source code is not available

Critical strengths

KA-MIL provides an intuitive way to bridge the domain knowledge into weakly supervised patient classification. Differ from the purely data driven attention pooling, GO descriptions provide a structured interface for examining cellular contributions. The authors claim that their approach does not require cell-level semantic annotations or paired cell–text data. The combination of semantic queries, learnable residual queries, and geometry regularization is technically reasonable and potentially applicable to other knowledge-guided MIL problems.

Beyond the interesting insight, KA-MIL also achieve the good classification performance on 4 datasets. The ablation studies examine semantic anchors, geometry regularization, residual queries, and encoder adaptation, helping to characterize the contribution of individual design choices.

Critical weaknesses
  1. A central component of KA-MIL is the geometry regularization in Eq. (4), which encourages the projected semantic queries to preserve the pairwise cosine similarities of the original text embeddings. The authors argue that this maintains semantic relationships during task adaptation, I have several concerns regarding its theoretical motivation and necessity.

The geometric constraint admits a substantially simpler interpretation. If the regularization loss is minimized to zero, the normalized projected queries and the original normalized text anchors have identical Gram matrices. Consequently, the two finite sets of vectors are related by an isometry on their respective spans. In the present setting, where the number of semantic anchors is smaller than the query embedding dimension, such preservation can be achieved through a suitable linear isometric mapping. As the query projection in Eq. (1) is already linear, the constraint may effectively restrict its adaptation to transformations that preserve the original anchor geometry. Why is this restricted adaptation preferable to a simpler fixed or learned isometric projection?

Further, preserving the full pairwise geometry may conflict with task-relevant representation learning. From the information-geometry perspective, an effective classifier should preserve information relevant to the target while allowing irrelevant information to be discarded. However, the original text embeddings may encode many semantic relationships that are unrelated to the specific classification task.

Lastly, the necessity of preserving the original text geometry is not sufficiently justified. The paper motivates Eq. (4) through the intuition that semantic relationships should be retained, but does not establish why pairwise cosine similarities in the pretrained BioLORD embedding space should remain invariant after adaptation to a specific clinical prediction task.

  1. The biological interpretation experiments do not sufficiently establish semantic correspondence or predictive relevance. Specifically, fig 2 uses attention enrichment to demonstrate biologically coherent query–cell associations, which measures relative attention allocation rather than biological contribution to the final prediction.
Questions for authors

Please see the weakness.

I would be happy to raise my rating if the authors address the concerns during the rebuttal.

6 comments

We thank the reviewer for the constructive questions. The additional experiments reported below have been completed. We will incorporate their methods and full results into a revised manuscript shortly and post a consolidated update with the corresponding section, table, and figure references.

Q1: Theoretical motivation of geometry regularization (Eq. 4). Why not use a simpler fixed or learned isometric projection?

A1: We agree that zero Gram-matrix error implies an exact isometry between the anchor spans. An exact isometry preserves pairwise query similarities even when its orientation is learnable. Eq. (4), however, is a finite soft penalty that permits task-driven changes in these relationships. We evaluated both suggested isometric alternatives under the original cohort-wide protocol:

Geometry constraintMean test AUCMean test macro-F1
None0.90350.7740
Fixed isometry0.89180.8037
Learnable isometry0.88430.7876
Soft regularization (KA-MIL)0.92290.8678

Across COVID, NS, UC, and Lupus10, each evaluated over five splits, soft regularization achieves higher mean test macro-F1 than fixed isometry (+6.42 percentage points), learnable isometry (+8.02 points), and no geometry regularization (+9.38 points). These results support allowing task-driven changes in query relationships while retaining a soft text-derived prior.

Q2: Why preserve potentially task-irrelevant BioLORD relationships? Does it conflict with task-relevant representation learning?

A2: We agree that some text-derived relationships may be irrelevant to the clinical task. Our objective does not require every relationship to remain invariant: its finite penalty allows the classification objective to alter query similarities. Moreover, query geometry does not fix predictive importance. The cell encoder and patient-specific summary weights (Eq. 3) remain trainable, and residual anchors are not directly constrained by the geometry penalty.

The soft model's nonzero mean Gram error (0.00234, compared with ∼10−18\sim 10^{-18} for strict isometries) confirms that the learned queries depart from exact preservation. Together with its higher mean performance than both isometric controls, this supports soft regularization as an effective structural prior that constrains changes in text-derived query relationships while permitting task adaptation, without assuming every BioLORD relationship is clinically relevant.

Q3: Biological interpretation does not sufficiently establish semantic correspondence or predictive relevance.

A3: We agree that attention allocation alone establishes neither transcriptional correspondence nor predictive dependence. We therefore conducted two complementary analyses:

  • RNA-based gene-set enrichment: We contrasted RNA expression between the highest- and lowest-attention 20% of cells within each patient and cell type, using a GSEA-style ranked enrichment test with patient-level sign-flip permutations. In a joint patient-level analysis of all 90 held-out COVID patients, 13 of 50 queries showed positive enrichment after FDR correction, including inflammatory response (NES = 1.643, q = 0.0234). Separately, among the 14 hospitalized patients in the fold-4 case study shown in Fig. 2 of the original submission, both highlighted queries—humoral immune response and myeloid leukocyte activation—showed positive within-cell-type RNA enrichment (NES = 1.532 and 1.501, respectively; both case-study FDR q = 0.0028).

  • Predictive dependence: With checkpoints and decision thresholds fixed, masking all semantic GO summaries at inference reduced mean test macro-F1 from 0.7075 to 0.5299, changing 19/90 patient decisions. This supports predictive dependence on the semantic branch, complementing the RNA-based evidence of biological correspondence.

Together, these analyses support biological correspondence for a subset of queries and predictive dependence on the semantic branch, without establishing individual-query causal exclusivity.

Q4: Code availability.

A4: The anonymized implementation was included in the original supplementary ZIP, as noted in the Reproducibility Statement. We apologize that its location was unclear.

Thank you for the rebuttal and the additional experimental results. However, after carefully reviewing your responses, I do not consider any of my main concerns to have been satisfactorily resolved. My original questions primarily concerned the theoretical motivation, necessity, and interpretability of the proposed method. I was expecting theoretically grounded explanations or strong justifications.

Q1. Theoretical motivation of geometry regularization:

My central question was why preserving the pairwise cosine similarities of the original BioLORD embeddings is desirable for downstream clinical classification. However, empirical evidence per se does not explain why the original BioLORD geometry constitutes an appropriate structural prior for the downstream task, as the observed performance gains may arise from other factors rather than the preservation of biologically meaningful semantic relationships.

Q2: Why preserve potentially task-irrelevant BioLORD relationships? Does it conflict with task-relevant representation learning?

The additional experiments provide useful empirical evidence for geometry regularization, my central concern remains unresolved. Specifically, the results do not establish why preserving the original BioLORD pairwise similarities is beneficial for task-specific prediction, rather than merely acting as a generic regularizer. Nor do you clarify how potentially task-irrelevant semantic relationships affect downstream adaptation.

Biological interpretation does not sufficiently establish semantic correspondence or predictive relevance.

The enrichment of relevant gene sets among high-attention cells does not establish that the model actually relies on the corresponding biological processes when making predictions. The distinction between semantic correspondence and predictive relevance remains unclear.

Q4: Code availability.

I did not request source code in my original review.

At present, I do not believe that the additional empirical results alone adequately resolve these fundamental questions. I therefore maintain my original concerns and assessment for now.

Thank you for clarifying your concerns. We explain below how the prior guides aggregation, how the model adapts it to the task, and what our biological analyses show.

Q1: Why preserve BioLORD cosine similarities for downstream classification?

Our premise is that related biological descriptions should lead to related functions for scoring cells. BioLORD provides biomedical relationships that guide aggregation when patient labels are scarce. Clinical supervision then determines how the resulting summaries support classification.

For queries qm,qnq_m, q_n and a cell key kk, each normalized to unit length, the cell-scoring function before softmax in Eq. (2) is fm(k)=sqm⊤kf_m(k) = s q_m^\top k. The Cauchy-Schwarz inequality gives

∣fm(k)−fn(k)∣≤s∥qm−qn∥2=s2(1−qm⊤qn).|f_m(k) - f_n(k)| \le s \|q_m - q_n\|_2 = s \sqrt{2(1 - q_m^\top q_n)}.

Thus, query geometry bounds how differently two queries can score the same cell. Eq. (4) uses the Gram matrix of biological descriptions to guide these relationships during task adaptation.

The choice of descriptions also matters empirically. We kept the BioLORD encoder, architecture, regularization objective, and cohort-wide protocol unchanged. We evaluated three approximately matched control vocabularies of 50 real GO terms with low relevance under our label-free cohort screen. Compared with our selected vocabulary, these controls lower macro-F1 by 10.07 percentage points on COVID and 12.79 points on NS, averaged over the three vocabularies. These results support using cohort-relevant descriptions to guide aggregation.

Q2: Can task-irrelevant relationships conflict with task adaptation?

KA-MIL jointly learns how to extract cellular summaries and how to use them for prediction:

  • Aggregation. Semantic queries extract cellular summaries. Geometry regularization guides the relationships among their scoring functions.
  • Prediction. Eq. (3) forms hi=∑mβimzimh_i=\sum_m\beta_{im}z_{im} for the MLP classifier. Learned weights can downweight uninformative summaries, while the classifier can learn to ignore irrelevant variations in hih_i. This allows information to be discarded from predictive use even when its text relationship remains in the reference matrix. Related queries therefore need not have equal predictive importance.

The geometry penalty constrains query relationships. It does not require patient representations to retain all information in the text embeddings. The cell encoder, key mapping, and query projection remain trainable. The shared projection and pairwise constraints couple the queries. A finite soft penalty balances this prior with the classification objective and allows their relationships to change during training.

The geometry controls in our first response support this balance between retaining the semantic reference and allowing task adaptation.

Q3: Do named processes correspond to the summaries used for prediction?

Semantic correspondence concerns alignment between a query's attention and its named RNA program. Predictive relevance concerns the summary's contribution to accurate patient prediction. We assess this contribution by masking the summary and measuring task loss against the unmasked model and matched query masks. Changes in output alone establish sensitivity. We connect these analyses for the same query in the same patients.

  • RNA correspondence. Our first response reported positive RNA enrichment within cell types for the humoral and myeloid queries in the 14 hospitalized patients in Fig. 2. The masking analysis below uses these same patients.

  • Sensitivity of the same myeloid summary. We masked this summary while keeping the model checkpoint and decision threshold fixed. Compared with other query masks matched in count and removed aggregation weight, the myeloid mask produced a greater log-loss change in 13 of these 14 patients (mean excess change 0.00101, p = 0.106). This provides directional evidence of sensitivity in this case study.

Masking a single query tests its marginal effect while the other queries remain active, and biological processes are not independent and have synergistic effects. Masking all semantic summaries tests their collective contribution and reduces mean test macro-F1 from 0.7075 to 0.5299, changing 19/90 patient decisions.

We will include these clarifications and the full analysis in the revised manuscript.

We also apologize for presenting code availability as a separate question. Our note referred to the code-availability statement in the original review summary.

Comments on the paper

2 comments
Area_Chair_iGKzOpenReview ↗

Please engage with author's rebuttal

Dear Reviewers,

We've reached the midpoint of the discussion period, and the authors' rebuttal is now posted. This is a key moment for the review process, so I'd like to ask for your active involvement.

Please take a moment to: i) go through the authors' rebuttal and their replies to the discussion; ii) evaluate whether the concerns and questions you raised have been resolved; iii) offer the authors any further feedback you think is warranted.

Your engagement during this window is critical to ensuring a fair and informed outcome. Thank you for your continued effort on this paper.

Best regards,

Your AC

Overview of Revisions

We thank all reviewers for their feedback. We have incorporated the additional experiments and clarifications into the revised manuscript, with new material highlighted in blue. The main additions are:

  1. Fully inductive evaluation: Training-only preprocessing and comparisons across four settings, with complete protocols and configurations. See Section 4.3, Table 3, and Appendices B.5, C.5, and D.4.

  2. Geometry and biological vocabulary: Clarified the rationale for geometry regularization and added isometric controls and low-relevance GO vocabularies. See Sections 3.3 and 4.4, Table 4, and Appendices E.2–E.3 and G.

  3. Biological evidence and predictive contribution: Added RNA analyses for all 50 queries and masking analyses with confusion matrices and matched-control comparisons. See Section 5.3, Figure 3, and Appendices H.1–H.2 (Tables 21–24).

  4. Frozen representations and encoder adaptation: Compared KAA, attention MIL, and MixMIL on frozen scGPT/SE features, alongside the separate encoder-freezing ablation. See Section 4.4, Table 4, and Appendix F (Tables 17–18).

  5. Statistical uncertainty: Added paired performance differences and confidence intervals, with their interpretation and scope. See Appendix D.6, Table 14.

We also clarified training and model selection and revised Discussion and Conclusion to discuss remaining questions and concrete future research directions. All references above use the revised manuscript’s numbering.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.