acceptodds
Withdrawn from ICLR 2027

MechMoE: Discovering Task-Aligned Perturbational Mechanisms for Out-of-Distribution Molecular Perturbation Prediction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Predicting transcriptional responses to molecular perturbations is a key challenge in computational drug discovery, particularly when generalizing to unseen compounds and cellular contexts. Although biological priors such as chemical similarity, annotated targets, and mechanism-of-action labels can provide useful structure, small-molecule responses often arise through indirect, context-dependent, and off-target effects that are only partially captured by predefined annotations. We introduce MechMoE, a mechanism-aware mixture model for out-of-distribution molecular perturbation prediction. MechMoE encodes molecular perturbations, basal cellular state, and expression-conditioned gene tokens, then uses learnable mechanism queries to extract candidate perturbational programs from contextualized gene representations. A sparse router selects a small subset of mechanism slots for each drug-cell pair, producing a perturbation-specific mechanism mixture used to predict the perturbation-induced expression change. We evaluate MechMoE on molecular perturbation data under strict held-out molecule and held-out cell-line settings. MechMoE improves transcriptome-wide response prediction across correlation, cosine similarity, reconstruction error, direction accuracy, and perturbation discrimination metrics, while remaining competitive on differential-expression recovery. Ablations show that sparse routing and the global perturbation decoder are both important for performance. These results suggest that small-molecule perturbation responses are effectively modeled as sparse combinations of learned, task-aligned latent mechanisms rather than fixed predefined biological categories.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
perturbation prediction, molecular perturbation, mechanism of action, mixture of expert, foundation models, drug discovery

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_WumhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper predicts pseudobulk responses (delta vs DMSO, 2,000 HVGs) to drugs on Tahoe-100M, for scaffold-held-out molecules and five fully held-out cell lines. Fig. 1C motivates it via Tanimoto similarity, shared targets and shared MoA correlate with response similarity at r <= 0.041. MechMoE runs a Transformer over gene tokens, a MolFormer drug token and a basal token. M learnable "mechanism queries" cross-attend over the output and a Top-K router mixes them into a vector c. A per-gene "local" decoder on [gene token, c] is summed with a "global" decoder that sees only the drug and basal tokens (Eq. 7-8). Training uses InfoNCE, regression and diversity losses. They test against scDCA, STATE, SAMS-VAE and BioLORD, with delta Pearson 0.397 vs 0.381 on cell lines and 0.580 vs 0.570 with PDS 0.901 vs 0.858 on molecules.

Critical strengths
  • The evaluation setting is the right one. Five test cell lines are fully excluded from training and molecules are split by scaffold (L354-358). That's the regime that matters for screening.
  • There's a real component ablation, and it does show the routed module helps. Without it the model falls to 0.558 Pearson and 0.851 PDS, below BioLORD. The authors also say where they lose (AUPRC, DE@100, L401-405).
Critical weaknesses
  • The headline gains are small, unreplicated and measured against the wrong trivial baseline. Everything is on one dataset, with no seeds, error bars or per-cell-line results, and the delta Pearson margins over the best baseline are 0.016 (Tab. 1) and 0.010 (Tab. 2). To be fair, no baseline is close on both splits, and every learned model beats Mean on delta Pearson. But "Mean" is the global training mean, and in Tab. 1 it still ties MechMoE on MSE (0.001) and MAE (0.009), while the "substantially higher" direction accuracy (L389-390) is 0.954 vs 0.950 (Mean) and 0.949 (scDCA). The test is a per-drug mean over training cell lines (Tab. 1) and a per-cell-line mean over training drugs (Tab. 2). Such averages have matched deep models on genetic screens (Ahlmann-Eltze et al., 2025). Query cross-attention, Top-K routing and InfoNCE are standard, so these numbers have to carry the paper.
  • The mechanism claims go beyond what's tested. The central hypothesis is that learned mechanisms explain responses better than predefined categories (L202-204), but no model using targets, MoA or pathways is compared and Fig. 1C is only a pairwise correlation. The module does help, but the biggest drop in Tab. 3 (0.072 Pearson, 0.115 PDS) comes from removing the global decoder, which never sees the mechanisms (Eq. 7, despite the Fig. 2 caption?). The 0.96 -0.98 Jaccard across drugs (L478-479) has no ref point; fixed queries plus gene-identity embeddings (Eq. 1, 4) may give it almost for free.
  • Minor: several reference entries have wrong author lists (scGPT, scFoundation, Squair et al.).

Overall a sensible setup, but I don't yet see the model beating stratified averages or the learned slots beating predefined categories.

Questions for authors
  1. (experiment) How do those two stratified means (per dose, if dose is an input) score on all metrics? If MechMoE clearly beats both by more than the seed spread in Q2, I'd raise my score.
  2. (experiment) What's the std of delta Pearson and PDS over 3-5 seeds (MechMoE, Tab. 3 variants, best baseline)?
  3. (experiment) How does the global decoder alone perform, and the full model without InfoNCE? You train with a retrieval loss, and PDS (distance and candidate set unspecified) reads like a retrieval metric.
  4. (clarification) How were scDCA, SAMS-VAE and BioLORD adapted and tuned?
  5. (clarification) Is Mean's MSE 0.013 / MAE 0.001 in Tab. 2 swapped? Do Tab. 1-3 use the "corrected model input vocabulary" (L463)?
Reviewer_JvNSOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MechMoE for predicting transcriptional responses to small-molecule perturbations. The model combines latent mechanism queries, Top-(K) sparse routing, and local/global decoders, and is evaluated on Tahoe-100M under unseen-molecule and fully unseen-cell-line settings.

Critical strengths

1.The paper addresses a practically relevant OOD problem: predicting responses for unseen molecules and cell lines.

2.The overall design is clear: the model predicts perturbation residuals and combines gene-level contextualization with sparse routing. The evaluation also covers response reconstruction, perturbation discrimination, and differential-expression recovery.

Critical weaknesses

1.The methodological novelty appears limited. MechMoE combines standard Transformer gene encoding, cross-attention queries, sparse MoE routing, and local/global decoders. The paper does not clearly establish why this combination is fundamentally different from a conventional Transformer+MoE model, or why it specifically enables mechanism discovery.

2.Experiments use only Tahoe-100M and report point estimates. The paper provides no multi-seed variation, confidence intervals, or per-cell-line results.

3.Several details of the objective and architecture are underspecified, limiting reproducibility.

Questions for authors

1.What is the key methodological difference between MechMoE and a parameter-matched Transformer+MoE baseline?

2.How does this design constitute an MoE, and what evidence shows that it improves prediction beyond added capacity and supports interpreting the selected slots as biological mechanisms?

3.Please provide the exact definitions of the retrieval, delta, global, and diversity losses, including the InfoNCE pairs, the diversity-loss computation, and all loss weights. In particular, the weights on the retrieval and delta losses are not specified.

4.Figure 2 applies cross-attention and pooling to the gene-token representations, whereas Equations (4)--(5) use the full contextualized representation. Do the summary tokens participate in mechanism extraction and routing?

5.How were the five mechanisms in Figure 3 selected? Please specify the criterion, thresholds, and whether held-out molecules informed this selection.

Reviewer_jECSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents MechMoE, a mechanism-aware mixture-of-experts framework for predicting transcriptional responses to molecular perturbations under out-of-distribution (OOD) settings. The proposed architecture jointly encodes molecular representations, basal cellular states, and gene-level features through a Transformer. Learnable mechanism queries are then used to extract candidate latent perturbational programs, while a sparse Top-K router selects and combines a subset of these programs for each drug–cell pair. The resulting representations are used by local and global decoders to predict perturbation-induced gene expression changes.

Critical strengths
  1. Biologically motivated modeling framework. The motivation for learning task-aligned latent perturbational programs, rather than relying exclusively on predefined biological annotations, is reasonable given the complexity of context-dependent and off-target drug effects.
  2. Competitive predictive performance with informative architectural ablations. The experiments demonstrate promising performance on several response-level metrics, particularly Pearson correlation, cosine similarity, and perturbation discrimination.
Critical weaknesses
  1. Limited methodological novelty and insufficient evidence supporting the claimed mechanism discovery. The main distinction of MechMoE appears to be the task-specific integration of learnable mechanism queries, gene-level cross-attention, and sparse Top-K routing. While this is a reasonable architectural design, the individual components and underlying modeling principles are largely established. More importantly, the evidence for the proposed mechanism-discovery interpretation remains limited.
  2. Incomplete experimental comparisons and insufficient validation of broad OOD generalization. First, several highly relevant perturbation prediction methods, particularly PRNet and chemCPA, are missing from the main experimental comparisons. Second, all experiments are conducted within a single perturbation atlas, Tahoe-100M. External validation on independent perturbation resources, such as LINCS L1000 or sci-Plex, would substantially improve the generalizability of the conclusions. Finally, the reported improvements are not uniformly substantial across evaluation metrics.
Questions for authors
  1. What is the essential methodological contribution of the proposed mechanism-learning architecture beyond existing latent perturbation modeling approaches?
  2. What evidence establishes that the learned mechanism slots capture biologically meaningful and functionally distinct perturbational programs?
  3. Can the authors strengthen the experimental comparisons and clarify the fairness of the evaluation protocol?
  4. How is the claimed OOD generalization verified, and how robust are the reported performance gains?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.