acceptodds
Under review as a conference paper at ICLR 2027

Learning Perturbation-Specific Representations through Conditional Energy Prior for Cellular Response Prediction

Abstract

AI-based cellular perturbation prediction is essential to accelerate drug discovery and therapeutic identification. However, perturbation-specific signals are generally obscured by cellular complexity, making it difficult for existing methods to correctly capture perturbation effects. To address this problem, we propose PerturbEnergy, which introduces a conditional energy-based prior over perturbation representations. It assigns lower energy (higher probability) to representations compatible with the given perturbation and higher energy to others, thereby explicitly tilting the latent space toward perturbation-preferred regions. Besides, PerturbEnergy can be integrated into autoencoder architectures as we find its Evidence Lower Bound (ELBO) is equivalent to the ELBO of the complete autoencoder. We further prove that optimizing PerturbEnergy increases the lower bound on the conditional mutual information between representation and perturbation. Extensive experiments on four benchmark datasets demonstrate the effectiveness of our PerturbEnergy: it improves perturbation recognition especially in latent space with improvement of up to 83.2%; It also achieves state-of-the-art performance for predicting top-50/100/200 differentially expressed gene(DEG) response and remaining competitive on highly variable gene(HVG) prediction. The code is provided anonymously: https://anonymous.4open.science/r/AnonymousPerturbEnergy-F5D2

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Cellular Perturbation Prediction, Cellular Response Prediction, Energy-based Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_82EUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PerturbEnergy introduces a conditional energy-based prior to encourage perturbation-aware latent representations for cellular response prediction. It trains with Langevin or mismatched negative samples and selects low-energy latent candidates during inference. Evaluation on four datasets reports improved perturbation classification and the best top-DEG RMSE on three datasets.

Critical strengths
  1. The evaluation covers genetic, chemical, and cytokine perturbations and includes competitive recent baselines. Improvements on response-relevant DEG metrics provide useful evidence beyond agreement on overall expression levels.

  2. The additional backbone experiments support the usefulness of the energy objective beyond the proposed VAE implementation. This makes the approach potentially useful as a reusable training component.

Critical weaknesses
  1. The claimed mutual-information guarantee is not established by the proof. In Appendix A.1.4, the Bayesian derivation drops normalization constants that depend on the perturbation and therefore cannot generally be cancelled. T

  2. The evidence does not clearly separate label encoding from faithful biological response modeling. The perturbation encoder already receives the perturbation label, so improved label classification can reflect retaining that input rather than learning its biological effects. Moreover, population-level fidelity can worsen despite better mean-expression prediction: Table 12 reports an energy distance of 8.65 for PerturbEnergy-Lang versus 0.70 for STATE on Replogle. These findings require a clearer explanation before interpreting latent separation as improved modeling of perturbation responses.

Questions for authors
  1. How are the perturbation-dependent normalization constants accounted for in Theorem 2, and what establishes that the implemented training updates increase the stated lower bound?

  2. Which representations from each model were classified in Table 3? How was direct access to perturbation labels accounted for when interpreting the classification gains?

  3. How should the substantially higher energy-distance values be reconciled with improved mean-expression metrics? What do existing diagnostics indicate about preservation of variation among cells under the same condition?

8 comments
AuthorsOpenReview ↗

Q1/W1 Response to Reviewer 82EU

Q1 / W1 Response

We sincerely thank you for the thoughtful questions. We provide further explanations and theoretical corrections/discussions have been updated in paper. (Theorem 2, Appendix A1.4, Appendix A1.5)

W1/Q1-(a): The Partition Function.

Thank you for pointing out this oversight. The Fα(zb,a)F_{\alpha}(z_b,a) must be included when applying Bayes' rule; we have corrected Eq. 17 and Appendix A.1.4. Importantly, the omission only affects the form of rα(a∣za,zb,c)r_{\alpha}(a|z_a,z_b,c) and the inequality in Theorem2(Eq. 16) remains valid: it is the Barber–Agakov variational bound, which holds for any normalized conditional rr.

 

W1/Q1-(b): The Implemented Updates and Mutual Information.

We have two variants (Langevin and Mismatch) for the implementation:  

Case 1: Langevin Langevin directly optimizes the Energy-ELBO, and we claimed that “this optimization increases the variational mutual information lower bound”. This describes the optimal case: when rα(a∣za,zb,c)r_\alpha(a\mid z_a,z_b,c) perfectly matches the posterior p(a∣za,zb,c)p(a\mid z_a,z_b,c), the bound is maximized and recovers I(A;Za∣Zb,C)I(A;Z_a\mid Z_b,C) of current dataset distribution. The original version only described this intuitively. Strictly, the relation is: Assuming the variational inference model qϕαq_{\phi_\alpha} accurately approximates the conditional latent landscape, this optimization reduces an upper bound on the gap between the true conditional mutual information and its variational lower bound. Proof below:

Step 1: Define the MI gap.

The mutual information gap is:

G=I(A;Za∣Zb,C)−H(A∣Zb,C)−Ep[log⁡rα(A∣Za,Zb,C)]=Ep[DKL(p(A∣Za,Zb,C) ∥ rα(A∣Za,Zb,C))]≥0.G = I(A;Z_a\mid Z_b,C)-H(A\mid Z_b,C)-\mathbb{E}_p\left[\log r_\alpha(A\mid Z_a,Z_b,C)\right]=\mathbb{E}_p\left[D_{\mathrm{KL}}\big(p(A\mid Z_a,Z_b,C)\thinspace\Vert\thinspace r_\alpha(A\mid Z_a,Z_b,C)\big)\right]\geq 0.

We define two conditional joint distributions:

P(a,z_a\mid z_b,c)=\pi(a\mid z_b,c)q_{\phi_\alpha}(z_a\mid z_b,c,a),$$$$Q_\alpha(a,z_a\mid z_b,c)=\pi(a\mid z_b,c)p_\alpha(z_a\mid z_b,a).

Step 2: Use KL chain rule under two factorizations.

Since the two distributions share the marginal π(a∣zb,c)\pi(a\mid z_b,c), the KL chain rule gives:

D(α)=EZb,C[DKL(P ∥ Qα)]=EZb,C,A[DKL(qϕα ∥ pα)].D(\alpha)=\mathbb{E}_{Z_b,C}\left[D_{\mathrm{KL}}(P\thinspace\Vert\thinspace Q_\alpha)\right]=\mathbb{E}_{Z_b,C,A}\left[D_{\mathrm{KL}}(q_{\phi_\alpha}\thinspace\Vert\thinspace p_\alpha)\right].

Alternatively, factorizing over ZaZ_a yields:

D(α)=EZb,C[DKL(P(Za∣Zb,C) ∥ Qα(Za∣Zb,C))]+EP[DKL(P(A∣Za,Zb,C) ∥ rα(A∣Za,Zb,C))].D(\alpha)=\mathbb{E}_{Z_b,C}\left[D_{\mathrm{KL}}\big(P(Z_a\mid Z_b,C)\thinspace\Vert\thinspace Q_\alpha(Z_a\mid Z_b,C)\big)\right]+\mathbb{E}_P\left[D_{\mathrm{KL}}\big(P(A\mid Z_a,Z_b,C)\thinspace\Vert\thinspace r_\alpha(A\mid Z_a,Z_b,C)\big)\right].

Step 3: Combine the two expressions of D(α)D(\alpha) to obtain the final relation

Since the first KL term is non-negative, we have:

D(α)=EZb,C,A[DKL(qϕα ∥ pα)]≥G≥0.D(\alpha)=\mathbb{E}_{Z_b,C,A}\left[D_{\mathrm{KL}}(q_{\phi_\alpha}\thinspace\Vert\thinspace p_\alpha)\right]\geq G\geq 0.

Therefore, optimizing DKL(qϕα∥pα)D_{\mathrm{KL}}(q_{\phi_\alpha}\Vert p_\alpha) reduces the mutual information gap GG upper bound.

 

Case 2: Mismatch Mismatch is an efficient surrogate that optimizes a lower bound on the conditional mutual information, described as Proposition 3.

Let S=(C,Xctr,Zb)S=(C,X_{ctr},Z_b). In training, a∼π(⋅∣c)a\sim\pi(\cdot|c) with positive za+∼qϕα(⋅∣zb,xctr,a)z_a^+\sim q_{\phi_\alpha}(\cdot|z_b,x_{ctr},a) the mismatched label a−∼π(⋅∣c, a−≠a)a^-\sim\pi(\cdot|c,\,a^-\neq a) gives negative za−∼qϕα(⋅∣zb,xctr,a−)z_a^-\sim q_{\phi_\alpha}(\cdot|z_b,x_{ctr},a^-). The negative shares the same (c,xctr,zb)(c,x_{ctr},z_b) with the positive and differs only in the perturbation. Define the energy gap:

G(α)=E[(1−π(a∣c))(Eα(za−,zb,a)−Eα(za+,zb,a))].G(\alpha)=\mathbb{E}\big[(1-\pi(a|c))\big(E_\alpha(z_a^-,z_b,a)-E_\alpha(z_a^+,z_b,a)\big)\big].

Since our energy is a scaled cosine similarity, ∣Eα∣≤τ|E_\alpha|\le\tau.

Proposition 3:   I(A;Za∣S) ≥ G(α)−τ2/2.\;I(A;Z_a | S)\ \ge\ G(\alpha)-\tau^2/2.

Proof below:

With uniform π\pi, (1−π(a∣c))=1−1/∣Ac∣(1-\pi(a|c))=1-1/|\mathcal{A}_c| is constant, so the bound is proportional to the Mismatch objective.

Step 1: MI as an expected KL. Since perturbations are assigned independently of the control cell within each context, A⊥(Xctr,Zb)∣CA \perp (X_{ctr},Z_b)|C. Hence I(A;Za∣S)=Es,a[KL(Ps,a ∥ Qs)]I(A;Z_a\mid S)=\mathbb{E}_{s,a}\left[\mathrm{KL}(P_{s,a}\thinspace\Vert\thinspace Q_s)\right], where Ps,a=q(za∣s,a)P_{s,a}=q(z_a|s,a) and Qs=∑a′π(a′∣c) q(za∣s,a′)Q_s=\sum_{a'}\pi(a'|c)\,q(z_a|s,a').

Step 2: Donsker–Varadhan bound. For any function TT, KL(P ∥ Q)≥EP[T]−log⁡EQ[eT]\mathrm{KL}(P\thinspace\Vert\thinspace Q)\ge\mathbb{E}_P[T]-\log\mathbb{E}_Q[e^{T}]. We take T=−Eα(⋅,zb,a)T=-E_\alpha(\cdot,z_b,a).

Step 3: Hoeffding’s lemma. Since T∈[−τ,τ]T\in[-\tau,\tau], log⁡EQ[eT]≤EQ[T]+τ2/2\log\mathbb{E}_Q[e^{T}]\le\mathbb{E}_Q[T]+\tau^2/2.

Step 4: Excluding the true label. Write Qs=π(a∣c)Ps,a+(1−π(a∣c))Q~s,aQ_s=\pi(a|c)P_{s,a}+(1-\pi(a|c))\tilde Q_{s,a}, where Q~s,a\tilde Q_{s,a} is exactly the distribution of mismatched negatives. Then EP[T]−EQ[T]=(1−π(a∣c))(EP[T]−EQ~[T])\mathbb{E}_P[T]-\mathbb{E}_Q[T]=(1-\pi(a\mid c))\big(\mathbb{E}_P[T]-\mathbb{E}_{\tilde Q}[T]\big).

Combining steps 1–4 and averaging over (s,a)(s,a) yields the claim.

AuthorsOpenReview ↗

Q2/W2 Response to Reviewer 82EU

Q2/W2 Response

Q2/W2-(a) Which representations are classified?

For all methods, we classify the representation fed into the decoder, i.e., the one finally used to predict the cellular response. We aim to measure how much perturbation information remains in this representation after training, since it also encodes the cell state, context, and potential heterogeneity-driven variation. The representations used for each method are summarized below:

MethodRepresentation
scGPTscGPT output zz
PerturbDiffdenoised latent fed to the decoder
STATEtransition output fed to the decoder
scGenz+δzz+\delta_z (δz\delta_z: perturbation embedding changes)
CPAzb+za+zcz_b+z_a+z_c (zcz_c: context embedding)
PerturbedVAE[za,zb][z_a,z_b]
PerturbEnergy[za,zb][z_a,z_b]

Q2/W2-(b) How is label access accounted for?

Direct access to the perturbation label is not unique to our method: all baselines above except scGPT also inject aa into the decoder input, yet their accuracy stays below 50% on Replogle, PBMC and Tahoe100M. Since hh also encodes context, basal state and cell-specific patterns, the perturbation signal can be overwhelmed when the model fits cellular heterogeneity-driven variation. The gain therefore reflects how well perturbation information is retained, not whether it is accessible.

To make it more clear, we conduct a simple experiment on Replogle, evaluating the checkpoints at 5k, 10k, 20k, 40k, and 60k training steps and track both perturbation discriminability and Top-50 DEG RMSE. Perturbation classification Acc (%, ↑) / Top-50 DEG RMSE (↓) is shown as:

Steps (Acc / DEG)PerturbedVAECPAPerturbEnergy-Mis
5k23.6 / 0.264218.1 / 0.378542.4 / 0.1941
10k26.8 / 0.229022.8 / 0.349148.5 / 0.1860
20k28.2 / 0.218822.6 / 0.343851.0 / 0.1780
40k27.3 / 0.226719.7 / 0.361850.6 / 0.1824
60k26.7 / 0.232520.2 / 0.360549.9 / 0.1843

For all methods, over-training leads to degraded Top50DEG RMSE, which means model learn non-generalizale patterns like cellular heterogeneity or noise. This also leads to lower discriminability for perturbation although label access is constant throughout training. Model forgets perturbation but remember heterogeneity patterns.

W2-(c) Improved Classification Relates to Biological Response

We can view label discriminability as a signal of how well a model captures perturbation responses, and our existing results support this (our Table 2 and Table 3). Compared with PerturbedVAE, PerturbEnergy-Mis substantially improves latent accuracy on Replogle (28.6 → 52.4), PBMC (48.2 → 61.8) and Tahoe100M (45.5 → 58.5), with correspondingly lower Top-50 DEG RMSE (0.2130 → 0.1780, 0.1071 → 0.0654, 0.3984 → 0.3123). The same pattern holds among baselines: from CPA to PerturbedVAE on Replogle, accuracy rises from 22.2 to 28.6 while RMSE drops from 0.3462 to 0.2130, and similarly on PBMC and Tahoe100M.

DatasetCPAPerturbedVAEPerturbEnergy-Mis
Replogle22.2 / 0.346228.6 / 0.213052.4 / 0.1780
PBMC43.0 / 0.158148.2 / 0.107161.8 / 0.0654
Tahoe100M33.3 / 0.707045.5 / 0.398458.5 / 0.3123

Latent Acc (%, ↑) / Top-50 DEG RMSE (↓), from Tables 2 and 3.

AuthorsOpenReview ↗

Q3/W2 Response to Reviewer 82EU

Q3/W2 Response

Q3/W2-(a) Energy Distance and Mean-expression Metrics

1.Metric-level explanation. We agree that the two types of metrics evaluate different dimensions of prediction like VC benchmark points[1]. Mean-expression metrics (RMSE/MSE/MAE) evaluate whether the predicted average response matches the observed one, whereas EDist compares full single-cell distributions over all cell pairs. By definition, EDist=2nN∑i,j∥x^i−xj∥2−1n2∑i,j∥x^i−x^j∥2−1N2∑i,j∥xi−xj∥2.\mathrm{EDist}=\frac{2}{nN}\sum_{i,j}\Vert\hat{x}_i-x_j\Vert_2-\frac{1}{n^2}\sum_{i,j}\Vert\hat{x}_i-\hat{x}_j\Vert_2-\frac{1}{N^2}\sum_{i,j}\Vert x_i-x_j\Vert_2. EDist aggregates distances between all pairs of cells on HVGs. Accurate mean response does not imply a low EDist.

For our perturbation prediction, we mainly focus on perturbation-induced mean responses rather than full single-cell distributions. Besides, perturbation focus more on DEGs, which are the genes most changed by the perturbation, than all 2,000 HVGs; Actually, much of the cell-to-cell variance is not caused by the perturbation: most of the 2,000 HVGs are unaffected by the perturbation, and their variation mainly comes from cell cycle and cellular heterogeneity. We report EDist for reference, following the VC Benchmark [1]. For HVG2000, in Table 12, mean expression metrics are more stable across methods than EDist, and our method outperform other baselines on them:

MethodRMSE ↓MSE ↓MAE ↓EDist ↓
CPA0.09530.01180.06124.8973
PerturbDiff0.08170.00790.05276.3453
PerturbedVAE0.07450.00630.05015.5135
STATE0.07250.00600.04910.7007
PerturbEnergy-Lang0.06910.00540.04658.6506
PerturbEnergy-Mis0.07040.00550.04739.0347

We further compute EDist on the Top-20/50 DEGs on Replogle:

MethodTop-20 EDist ↓Top-50 EDist ↓Top-50 RMSE ↓
CPA2.1732.8780.3462
PerturbDiff2.3982.6050.2612
PerturbedVAE1.9662.3550.2130
STATE1.5912.2330.2024
PerturbEnergy-Lang1.6632.4120.1870
PerturbEnergy-Mis1.5562.3100.1780

On DEGs, the influence of cell-to-cell variation is reduced and perturbation effect becomes significant. EDist becomes much more stable across methods. Our method achieves the best Top-20 EDist (1.556 vs. 1.591 for STATE) and is close to the best on Top-50 (2.310 vs. 2.233).

2.Method-level explanation. PerturbEnergy retains perturbation-specific information and selects the lowest-energy representation at inference. This mode-seeking selection reduces the diversity of generated cells, as the model focuses on stable perturbation effects rather than perturbation-irrelevant cell-to-cell variation. Moreover, our design does not explicitly model within-condition cell-to-cell variability.

Q3/W2-(b) Diagnostics for Within-condition Variation

Our existing results do not directly measure variation among cells under the same condition because it's not direct perturbation related target. We here provide supplementary experiments to illustrate this.

Dispersion ratio. It measures the ratio between the dispersion of predicted cells and that of real cells under the same condition (c,p)(c,p):

ρG(c,p)=DG({x^j}j=1m)DG({xi}i=1n),DG({xi}i=1n)=1n∑i=1n∥xi−xˉ∥22,\rho_{\mathcal{G}}(c,p)=\frac{D_{\mathcal{G}}(\lbrace\hat{\mathbf{x}}_j\rbrace_{j=1}^{m})}{D_{\mathcal{G}}(\lbrace\mathbf{x}_i\rbrace_{i=1}^{n})},\qquad D_{\mathcal{G}}(\lbrace\mathbf{x}_i\rbrace_{i=1}^{n})=\frac{1}{n}\sum_{i=1}^{n}\Vert\mathbf{x}_i-\bar{\mathbf{x}}\Vert_2^2,

To examine the effect of minimum-energy selection, we also evaluate a variant of PerturbEnergy that samples zaz_a randomly from qϕαq_{\phi_\alpha} instead of selecting the lowest-energy candidate.

MethodDispersion Ratio (HVG)EDist (HVG) ↓Top-50 DEG RMSE ↓
CPA0.1484.89730.3462
STATE0.5840.70070.2024
PerturbedVAE0.1065.51350.2130
PerturbEnergy-Mis0.0519.03470.1780
PerturbEnergy-Mis (No Minimum E)0.1155.73600.2221

Our method preserves less within-condition variance (dispersion ratio 0.051), as do other VAE-based methods (0.11–0.15), whereas STATE largely inherits control-cell variance (0.584). This explains our higher EDist. The reduced variance partly stems from minimum-energy selection: without it, dispersion doubles (0.115) and EDist drops to 5.74, but Top-50 DEG RMSE rises from 0.1780 to 0.2221.

 

[1] Mao, Xinjie, et al. "Benchmarking virtual cell models for in-the-wild perturbation response." arXiv preprint arXiv:2604.27646 (2026).

Reviewer_85k4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PerturbEnergy, a conditional energy-based prior designed to improve perturbation-specific representations for cellular response prediction. The method separates latent representations into basal and perturbation-related components and uses an energy function to assign lower energy to representations compatible with a given perturbation. The authors derive an ELBO-based training objective and provide a theoretical analysis relating the energy prior to conditional mutual information. The method is evaluated on four datasets covering genetic, cytokine, and chemical perturbations, with comparisons against several existing perturbation prediction models. The reported results show improvements in perturbation classification and prediction accuracy, particularly on top differentially expressed genes.

Critical strengths

S1. The paper aims to tackle an important problem in perturbation prediction, particularly how to preserve perturbation-specific information in learned cellular representations. Introducing a conditional energy-based prior is an interesting approach to this problem, and the method can be integrated into different autoencoder architectures. The paper also provides a theoretical analysis to motivate the proposed approach.

S2. The experiments cover four datasets with genetic, cytokine, and chemical perturbations, and include comparisons with several representative perturbation prediction methods, including recent diffusion-based and transformer-based models. PerturbEnergy achieves improvements on top-DEG prediction over most baselines, particularly on Replogle, PBMC, and Tahoe100M. The experiments integrating PerturbEnergy into scGen and PerturbedVAE further suggest that the proposed method can be applied to different architectures.

Critical weaknesses

W1. My main concern is the theoretical justification of the proposed method. In Eq. 4, the conditional energy prior includes a perturbation-dependent partition function, but this term is omitted when deriving the variational classifier in Eq. 17 and Appendix A.1.4. Unless the partition function is identical across perturbations, it cannot be canceled when applying Bayes' rule. More importantly, the proof does not establish that optimizing the proposed energy objective necessarily increases the stated conditional mutual information lower bound. Theorem 1 also seems to overstate the equivalence between optimizing the Energy-ELBO and the complete ELBO, since the latter includes additional reconstruction and regularization terms. These issues weaken the theoretical claims that motivate the proposed method.

W2. I am also concerned about the consistency between the theoretical objective and the actual optimization procedure. The negative phase in Eq. 12 requires samples from the conditional energy-based prior, but the mismatching strategy instead uses latent representations associated with other perturbation labels. While this may be a reasonable contrastive training strategy, it does not generally provide samples from the distribution required by the derived gradient. Therefore, it is unclear whether the mismatching variant optimizes the proposed energy-based objective or a different surrogate loss. This distinction is important because the mismatching variant achieves some of the strongest experimental results. In addition, the substantial gains in perturbation classification should be interpreted carefully, since the representations are explicitly conditioned on perturbation identity. Higher label recoverability does not necessarily imply more accurate recovery of perturbation-specific biological responses.

Questions for authors

Q1. In Eq. 17 and Appendix A.1.4, why is the perturbation-dependent partition function omitted when applying Bayes' rule? Can the authors clarify the assumptions under which this derivation is valid and explain how optimizing the proposed energy objective is guaranteed to improve the conditional mutual information lower bound?

Q2. In Eq. 12, the negative phase is defined as an expectation under the conditional energy-based prior. However, the mismatching strategy samples representations associated with different perturbations. How does this sampling procedure approximate the required distribution, and does it optimize the same objective as the Langevin-based variant?

Q3. Since the perturbation representations are explicitly conditioned on perturbation identity, how can the authors distinguish improved label recoverability from improved representation of actual transcriptional effects? Is there additional evidence that the enhanced discriminability corresponds to more accurate perturbation-specific biological responses?

5 comments
AuthorsOpenReview ↗

Q1/W1 Response to Reviewer 85k4

Q1/W1 Response

We sincerely thank you for the thoughtful comments, particularly on the theoretical analysis. We acknowledge several inaccuracies in the original presentation and have corrected them in the revised submission. Importantly, these corrections do not affect the PerturbEnergy formulation or experimental results.

 

W1/Q1-(a): The Partition Function in Eq. 17 / A.1.4.

Thank you for pointing out this oversight. The Fα(zb,a)F_{\alpha}(z_b,a) must be included when applying Bayes' rule. We have corrected Eq. 17 and Appendix A.1.4 accordingly. Importantly, omission only affects the formation of rα(a∣za,zb,c)r_{\alpha}(a|z_a,z_b,c) and the inequality in Theorem 2 (Eq. 16) remains valid: it is the Barber–Agakov variational bound, which holds for any normalized conditional distribution rr.

 

W1/Q1-(b): Energy-ELBO and Mutual Information.

We claimed that “optimizing PerturbEnergy leads to the increase of the variational mutual information lower bound”. This statement describes the optimal case of optimization: when rα(a∣za,zb,c)r_\alpha(a\mid z_a,z_b,c) perfectly matches the perturbation posterior p(a∣za,zb,c)p(a\mid z_a,z_b,c), the lower bound reaches its maximum, i.e., it fully recovers the mutual information I(A;Za∣Zb,C)I(A;Z_a\mid Z_b,C) under the current dataset and perturbation distribution. In the original version, this relationship was only described intuitively. If we strictly consider the optimization of the Energy-ELBO, the precise statement should be: Assuming the variational inference model qϕαq_{\phi_\alpha} accurately approximates the conditional latent landscape, optimizing PerturbEnergy reduces an upper bound on the gap between the true conditional mutual information and its variational lower bound.

We provide a mathematical proof of this relationship. It has been added to Appendix A.1.4, replacing the previous textual description, and Theorem 2 has been updated accordingly.

Proof:

Step 1: Define the mutual information gap.

The conditional mutual information gap is:

G=I(A;Za∣Zb,C)−H(A∣Zb,C)−Ep[log⁡rα(A∣Za,Zb,C)]=Ep[DKL(p(A∣Za,Zb,C) ∥ rα(A∣Za,Zb,C))]≥0.G = I(A;Z_a\mid Z_b,C)-H(A\mid Z_b,C)-\mathbb{E}_p\left[\log r_\alpha(A\mid Z_a,Z_b,C)\right]=\mathbb{E}_p\left[D_{\mathrm{KL}}\big(p(A\mid Z_a,Z_b,C)\thinspace\Vert\thinspace r_\alpha(A\mid Z_a,Z_b,C)\big)\right]\geq 0.

We define two conditional joint distributions:

P(a,z_a\mid z_b,c)=\pi(a\mid z_b,c)q_{\phi_\alpha}(z_a\mid z_b,c,a),$$$$Q_\alpha(a,z_a\mid z_b,c)=\pi(a\mid z_b,c)p_\alpha(z_a\mid z_b,a).

Step 2: Apply the KL chain rule twice under different factorizations.

Since the two distributions share the same perturbation marginal π(a∣zb,c)\pi(a\mid z_b,c), the KL chain rule gives:

D(α)=EZb,C[DKL(P ∥ Qα)]=EZb,C,A[DKL(qϕα ∥ pα)].D(\alpha)=\mathbb{E}_{Z_b,C}\left[D_{\mathrm{KL}}(P\thinspace\Vert\thinspace Q_\alpha)\right]=\mathbb{E}_{Z_b,C,A}\left[D_{\mathrm{KL}}(q_{\phi_\alpha}\thinspace\Vert\thinspace p_\alpha)\right].

Alternatively, factorizing over ZaZ_a yields:

D(α)=EZb,C[DKL(P(Za∣Zb,C) ∥ Qα(Za∣Zb,C))]+EP[DKL(P(A∣Za,Zb,C) ∥ rα(A∣Za,Zb,C))].D(\alpha)=\mathbb{E}_{Z_b,C}\left[D_{\mathrm{KL}}\big(P(Z_a\mid Z_b,C)\thinspace\Vert\thinspace Q_\alpha(Z_a\mid Z_b,C)\big)\right]+\mathbb{E}_P\left[D_{\mathrm{KL}}\big(P(A\mid Z_a,Z_b,C)\thinspace\Vert\thinspace r_\alpha(A\mid Z_a,Z_b,C)\big)\right].

Step 3: Combine the two expressions of D(α)D(\alpha) to obtain the final relationship.

Since the first KL term is non-negative, we have:

D(α)=EZb,C,A[DKL(qϕα ∥ pα)]≥G≥0.D(\alpha)=\mathbb{E}_{Z_b,C,A}\left[D_{\mathrm{KL}}(q_{\phi_\alpha}\thinspace\Vert\thinspace p_\alpha)\right]\geq G\geq 0.

Therefore, optimizing the Energy-ELBO term DKL(qϕα∥pα)D_{\mathrm{KL}}(q_{\phi_\alpha}\Vert p_\alpha) decreases an upper bound on the mutual information gap GG.

 

W1/Q1-(c) Theorem 1.

We clarify that "equivalent" refers to the equivalence of optimization gradient. Specifically, with the encoder and decoder parameters fixed, optimizing the Energy-ELBO and the complete ELBO yields the same gradients with respect to α\alpha. We have revised the statement to "optimizing the Energy-ELBO gives the same gradient as optimizing the VAE ELBO with respect to α\alpha."

AuthorsOpenReview ↗

Q2/W2 Response to Reviewer 85k4

Q2/W2 Response

We agree that the mismatching strategy is not an unbiased estimator of the Energy-ELBO gradient. It is, however, a principled and efficient surrogate. We prove below (Proposition 3) that the mismatching update is a stochastic gradient of a lower bound on the conditional mutual information between the perturbation and the perturbation representation, and it requires no MCMC sampling. This discussion has been incorporated into Appendix A1.5 of our paper. It is pointed out after Theorem 2. We now clarify why mismatch strategy can act as a proper surrogate and give some empirical analysis of why it achieves stronger performance than Langevin in some cases.

 

Why Mismatch is a valid and efficient surrogate: a mutual-information view.

Setup. Let S=(C,Xctr,Zb)S=(C,X_{ctr},Z_b). In each training block, a∼π(⋅∣c)a\sim\pi(\cdot|c) and the positive is za+∼qϕα(⋅∣zb,xctr,a)z_a^+\sim q_{\phi_\alpha}(\cdot|z_b,x_{ctr},a). The mismatched label is a−∼π(⋅∣c, a−≠a)a^-\sim\pi(\cdot|c,\,a^-\neq a) and the negative is za−∼qϕα(⋅∣zb,xctr,a−)z_a^-\sim q_{\phi_\alpha}(\cdot|z_b,x_{ctr},a^-). Crucially, the negative shares the same (c,xctr,zb)(c,x_{ctr},z_b) as the positive and differs only in the perturbation. Define the energy gap:

G(α)=E[(1−π(a∣c))(Eα(za−,zb,a)−Eα(za+,zb,a))].G(\alpha)=\mathbb{E}\big[(1-\pi(a|c))\big(E_\alpha(z_a^-,z_b,a)-E_\alpha(z_a^+,z_b,a)\big)\big].

Our energy is a scaled cosine similarity, so ∣Eα∣≤τ|E_\alpha|\le\tau.

Proposition 3:   I(A;Za∣S) ≥ G(α)−τ2/2.\;I(A;Z_a | S)\ \ge\ G(\alpha)-\tau^2/2.

In particular, with uniform π\pi over the ∣Ac∣|\mathcal{A}_c| perturbations of context cc, the factor (1−π(a∣c))=1−1/∣Ac∣(1-\pi(a|c))=1-1/|\mathcal{A}_c| is a constant, so the bound is proportional to the mismatching objective itself.

Proof.

There are four steps:

Step 1: MI as an expected KL. Since perturbations are assigned independently of the control cell within each context, A⊥(Xctr,Zb)∣CA \perp (X_{ctr},Z_b)|C. Hence I(A;Za∣S)=Es,a[KL(Ps,a ∥ Qs)]I(A;Z_a\mid S)=\mathbb{E}_{s,a}\left[\mathrm{KL}(P_{s,a}\thinspace\Vert\thinspace Q_s)\right], where Ps,a=q(za∣s,a)P_{s,a}=q(z_a|s,a) and Qs=∑a′π(a′∣c) q(za∣s,a′)Q_s=\sum_{a'}\pi(a'|c)\,q(z_a|s,a').

Step 2: Donsker–Varadhan bound. For any function TT, KL(P ∥ Q)≥EP[T]−log⁡EQ[eT]\mathrm{KL}(P\thinspace\Vert\thinspace Q)\ge\mathbb{E}_P[T]-\log\mathbb{E}_Q[e^{T}]. We take T=−Eα(⋅,zb,a)T=-E_\alpha(\cdot,z_b,a).

Step 3: Hoeffding’s lemma. Since T∈[−τ,τ]T\in[-\tau,\tau], log⁡EQ[eT]≤EQ[T]+τ2/2\log\mathbb{E}_Q[e^{T}]\le\mathbb{E}_Q[T]+\tau^2/2.

Step 4: Excluding the true label. Write Qs=π(a∣c)Ps,a+(1−π(a∣c))Q~s,aQ_s=\pi(a|c)P_{s,a}+(1-\pi(a|c))\tilde Q_{s,a}, where Q~s,a\tilde Q_{s,a} is exactly the distribution of mismatched negatives. Then EP[T]−EQ[T]=(1−π(a∣c))(EP[T]−EQ~[T])\mathbb{E}_P[T]-\mathbb{E}_Q[T]=(1-\pi(a\mid c))\big(\mathbb{E}_P[T]-\mathbb{E}_{\tilde Q}[T]\big).

Combining steps 1–4 and averaging over (s,a)(s,a) yields the claim.

Proof End.

Therefore:

  • Effectiveness. The mismatching update in Eq. 12, ∇α[Eα(za+)−Eα(za−)]\nabla_\alpha[E_\alpha(z_a^+)-E_\alpha(z_a^-)], is a stochastic estimate of −∇αG(α)-\nabla_\alpha G(\alpha), because the negatives are exact samples from Q~\tilde Q. Mismatch therefore optimizes an MI lower bound.
  • Efficiency. Mismatch needs one additional encoder pass and no MCMC: 16.4 ms/step, compared with 15.8 for the plain VAE, 28.4 for Langevin with K=20 and 73.1 for K=80 (Fig. 4d).

 

Comparision of Lang and Mismatch: Why can Mismatch outperform Langevin?

Empirically Mismatch can even beat Langevin. The main reason actually comes from the inherent instability of implementing Langevin sampling. Although theoretically consistent with the objective, it still depends on latent landscape complexity and hyperparameters.

Hyperparameter sensitivity. In Fig. 4b, Langevin's Top-50 RMSE on Replogle ranges from 0.1810 to 0.2213. More steps and smaller step sizes improve performance; the best setting (K=8K=8, η=10−5\eta=10^{-5}: 0.1810) nears Mismatch (0.1780) at 4.5× the cost (73.1 vs. 16.4 ms/step, Fig. 4d). Our Langevin makes compromise between the performance and efficiency.

Data-complexity sensitivity. We validate this on Reploge. We randomly select 100 gene perturbations, then we downsampling multiple subsets, 150 samples per perturbation and we select the hard one with Average Within-Group Mean Square (Average MS_W) 0.1604 (stronger cellular heterogeneity) and easy one with Average MS_W 0.1291 (weaker cellular heterogeneity). The results are as follows. The Mismatch–Langevin gap is larger on hard data (0.2199 vs. 0.2314) than easy data (0.1488 vs. 0.1505) and a larger KK is required to reduce this gap of hard setting while small KK suffices for easy data. This shows data complexity thus also limits Langevin.

AverageMS_WMismatchLang (K=20)Lang (K=40)Lang (K=80)
SubReploge (easy)0.12910.14880.15050.14920.1495
SubReploge (hard)0.16040.21990.23140.22650.2237
AuthorsOpenReview ↗

Q3/W2 Response to Reviewer 85k4

Q3 / W2 Response

We thank you for this insightful questions. We provide explanations and additional experimental evidence for these. These discussions will be incorporated into our paper accordingly.

 

Further Explanation and Analysis for Perturbation Classification

Although the perturbation aa is used when computing zaz_a, the representation we analyze is not zaz_a but the final representation for the decoder, h=[za,zb]h=[z_a,z_b]. For all methods, we use the representation that is finally used for decoding, because this is the one actually used for predicting cellular responses.Therefore, hh contains not only perturbation information but also information about the cellular context, the basal state, and even cell-specific patterns. The problem is: model can fit heterogeneity-driven variation when training, which overwhelms the perturbation signal and makes perturbations less distinguishable in hh.

We verify this with a simple experiment on Replogle, evaluating the checkpoints at 5k, 10k, 20k, 40k, and 60k training steps and track both perturbation discriminability and Top-50 DEG RMSE. Perturbation classification Acc (%, ↑) / Top-50 DEG RMSE (↓) is shown as:

Steps (Acc / DEG)PerturbedVAECPAPerturbEnergy-Mis
5k23.6 / 0.264218.1 / 0.378542.4 / 0.1941
10k26.8 / 0.229022.8 / 0.349148.5 / 0.1860
20k28.2 / 0.218822.6 / 0.343851.0 / 0.1780
40k27.3 / 0.226719.7 / 0.361850.6 / 0.1824
60k26.7 / 0.232520.2 / 0.360549.9 / 0.1843

We observe that when the model is over-trained, it captures patterns of cellular heterogeneity (overfitting), and its Top-50 DEG RMSE deteriorates (PerturbedVAE: 0.2188 → 0.2325; CPA: 0.3438 → 0.3605; PerturbEnergy: 0.1780 → 0.1843). At the same time, the discriminability of perturbation labels also decreases (PerturbedVAE: 28.2 → 26.7; CPA: 22.6 → 20.2; PerturbEnergy: 51.0 → 49.9).

 

Additional Evidence for Discriminability and Perturbation-specific Biological Responses

To test whether our representation captures the perturbation effect, rather than only the perturbation identity, we study perturbations with similar mechanisms, which should induce similar biological responses.

Experimental design. We use Tahoe-100M (50 cell lines) with the same preprocessed data (2,000 HVGs, log-normalized):

1. Perturbation effect For each cell line cc and perturbation aa, we average the expression of all cells to obtain a pseudobulk profile xˉa\bar{x}_a, which mitigates cellular heterogeneity. The perturbation effect is Δa=xˉa−xˉctr\Delta_a=\bar{x}_a-\bar{x}_{\mathrm{ctr}}.

2.Most similar pair Within each cell line, we select the perturbation pair (a0,a1)(a_0,a_1) whose effects Δa0\Delta_{a_0},Δa1\Delta_{a_1} have the highest cosine similarity. This yields 50 pairs, one per cell line.

3. Representation swap For each pair, we sample 300 control cells, obtain zbz_b from qϕβ(zb∣xctr,c)q_{\phi_\beta}(z_b\mid x_{ctr},c), and compute za0z_{a_0} and za1z_{a_1} from qϕα(za∣zb,xctr,a0)q_{\phi_\alpha}(z_a\mid z_b,x_{ctr},a_0) and qϕα(za∣zb,xctr,a1)q_{\phi_\alpha}(z_a\mid z_b,x_{ctr},a_1). We then decode [za0,zb][z_{a_0},z_b] to predict the response to a1a_1, and [za1,zb][z_{a_1},z_b] to predict the response to a0a_0. If the representation captures the perturbation effect, swapping in the representation of a perturbation with similar biological effect should still yield a similar prediction.

Evaluation. We report Top-50 DEG RMSE between the averaged prediction and the target's observation, averaged over 100 swaps (DEGs from real data, shared by all methods). For reference, we also report No swap, Random swap (a random instead of the most similar perturbation), and the accuracy of classifying a0a_0 vs. a1a_1.

MethodsSwap (similar)No Swap (optimal)Swap (random)Perturbation Acc
scGen0.7820.5850.9140.78
PerturbDiff0.7510.5520.8890.71
CPA0.8710.6280.9230.75
PerturbedVAE0.7160.5750.8810.81
PerturbEnergy-Lang0.6080.5410.8980.85
PerturbEnergy-Mis0.5990.5300.9070.89

Results. After swapping zaz_a between perturbations with similar responses, our method still partly predicts the true response: PerturbEnergy-Mis reaches 0.599 RMSE, the smallest gap to its no-swap optimum (0.530), vs. 0.716/0.575 for the best baseline. Thus PerturbEnergy's representations retain not only label information but also the biological response mechanism, so a similar perturbation's representation can partly recover the response. It also better identifies such similar perturbations (Acc 0.89 vs. 0.81). One key explanation is that other methods learn non-generalizable, cell-specific expression patterns that lose perturbation information, reducing perturbation discriminability and hindering perturbation-specific response prediction.

Reviewer_dCmKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors introduce PerturbEnergy, which addresses an interesting question in perturbation-effect prediction: how much information about the perturbation itself is captured in the learned latent space, as opposed to other information about the cell? The authors argue that isolating perturbation-specific effects from other sources of cellular variation, such as cell type and cellular context, is important for generalizable perturbation prediction.

To address this, they introduce an energy-based formulation that, in a manner similar to contrastive learning, assigns lower energy to latent representations compatible with a given perturbation and higher energy to incompatible representations. This reshapes the latent space to encourage the retention of perturbation-specific information. During inference, the model samples multiple candidate perturbation representations, selects the one with the lowest energy, and uses it to predict the perturbed gene-expression response. The method demonstrates strong performance across four commonly used benchmark datasets in this space.

Critical strengths

The main strength of the paper is its empirical performance. The authors evaluate the method across four datasets, and it generally outperforms existing methods, particularly on differentially expressed gene prediction. Another strength is the perspective of examining how much perturbation-specific information is actually captured in the latent spaces of existing models. Although this question has been explored in previous work, I think it remains an interesting and relatively underexplored aspect of this field.

Critical weaknesses

I would like to see whether the energy-based formulation actually outperforms a matched supervised contrastive objective with the same architecture, encoder, and inference budget. The standard-VAE ablation establishes that energy regularization helps, but does not establish that it is better than simpler contrastive alternatives.

Questions for authors

See weakness

3 comments
AuthorsOpenReview ↗

Response to Reviewer dCmK

We sincerely thank you for your valuable question. Comparing with a matched supervised contrastive objective is important for understanding our method. Below, we provide experimental results and explanation about these methods. We will provide the relevant discussions in the appendix of the subsequent version of the paper.

Response to Weakness: Comparison with a Matched Supervised Contrastive Objective

We compare PerturbEnergy with a VAE trained with a supervised contrastive objective (VAE+SupCon) on Replogle. For a fair comparison, SupCon is applied to the same representation as our energy, [za,zb][z_a,z_b], and all methods share exactly the same architecture, data setting and checkpoint selection strategy. Since SupCon is added as an extra loss term, we tune both its loss weight λ∈{0.1,0.5,1.0}\lambda\in\{0.1,0.5,1.0\} and temperature τ∈{0.1,1.0}\tau\in\{0.1,1.0\}, giving six configurations. We also report PerturbEnergy-Mis with M=4M=4 candidates for reference. In terms of cost, PerturbEnergy-Mis is almost as fast as the plain VAE (16.4 vs. 15.8 ms/step, Fig. 4d in our paper), as the MM candidates at inference are generated in parallel, so the extra cost of selection is negligible.

ModelTop-50 (best ckpt) ↓Top-100 (best ckpt) ↓Representation Classification ↑
VAE0.22180.185625.8
VAE+SupCon (λ=0.1, τ=0.1)0.21230.170244.5
VAE+SupCon (λ=0.1, τ=1.0)0.20180.168242.3
VAE+SupCon (λ=0.5, τ=0.1)0.19880.163946.9
VAE+SupCon (λ=0.5, τ=1.0)0.20500.169547.3
VAE+SupCon (λ=1.0, τ=0.1)0.19940.165949.4
VAE+SupCon (λ=1.0, τ=1.0)0.20020.167451.2
PerturbEnergy-Mis (M=4)0.19510.161450.6
PerturbEnergy-Mis0.17800.152252.4
PerturbEnergy-Lang0.18700.159150.5

Observations. (1) Adding SupCon to the VAE improves both perturbation response prediction and perturbation distinguisability, consistent with the observation of PerturbedVAE, which also uses a contrastive objective. (2) SupCon is sensitive to its hyperparameters: Top-50 RMSE ranges from 0.1988 to 0.2123 across configurations, as it is added to the VAE as a separate loss term where hyperparameter settings make different balance. (3) The best SupCon configuration gets Top-50 RMSE comparable to PerturbEnergy-Mis with M=4M=4 (0.1988 vs. 0.1951), but remains behind M=8M=8 (0.1780).

Why the energy formulation helps (difference from SupCon). Beyond the instability introduced by tuning an additional loss term, the most important difference is that the energy can measure and select the most suitable representation, whereas SupCon cannot. Compared with SupCon, the energy provides an explicit score of how much perturbation-specific information a representation carries: lower energy means higher compatibility with the perturbation. This enables PerturbEnergy to score multiple candidates and select the one with the lowest energy. Increasing MM enlarges the candidate pool and makes it easier to find a better representation, as shown by the gain from M=4M=4 to M=8M=8 (0.1951 → 0.1780). In contrast, SupCon only makes representations discriminative and it provides no such trained score to quantify how well a given representation matches the perturbation. As a result, the energy formulation can better sample the desired representation.

AuthorsOpenReview ↗

Follow-up on our rebuttal

Dear Reviewer dCmK,

We hope this message finds you well. As the discussion period ends in three days, we would like to kindly check whether our response has addressed your concern. Your feedback is very important to us, and we are happy to clarify anything further.

Thank you again for your time and effort that you have devoted to our paper.

Best regards,

The Authors

AuthorsOpenReview ↗

Follow-up on our rebuttal

Dear Reviewer dCmk,

We hope this message finds you well. As the discussion period is nearing its end, we would like to kindly check whether our responses have addressed your concerns. Your feedback is very important to us, and we are happy to clarify anything further.

Thank you again for your insightful comments, which have greatly improved our paper.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary from Authors

Summary for Our Paper and Rebuttal

We sincerely thank everyone for the time and effort devoted to our work. As the discussion period comes to an end, we would like to briefly summarize our work, the valuable feedback we received during the rebuttal and the corresponding response and revisions.

Paper Summary

Our paper aims to improve cellular perturbation response prediction. We identify an important but underexplored issue: how much perturbation-specific information is retained in the representations used for prediction. To enhance this information, we propose PerturbEnergy, a conditional energy-based model that can be plugged into autoencoder architectures to tilt the latent representation toward perturbation-preferred regions. Experiments show that it outperforms other baselines on perturbation response prediction (Top-DEG metrics) and validate the relationship between perturbation discriminability and response prediction.

Rebuttal Summary

The reviewers’ concerns are:

  • Theoretical rigor (Reviewer 85k4, 82EU): How the partition function is omitted in Eq. 17, and how optimizing the energy relates to the mutual-information lower bound.

  • Mismatch strategy (Reviewer 85k4): Whether the Mismatch strategy optimizes the same objective as the Langevin variant.

  • Label encoding vs. biological response (Reviewer 82EU, 85k4): Whether higher perturbation discriminability reflects biological responses rather than retaining the input label.

  • Energy distance (Reviewer 82EU): How the higher EDist can be reconciled with better mean-expression metrics.

  • Comparison with contrastive learning (Reviewer dCmK): Whether the energy formulation outperforms a matched supervised contrastive objective.

We addressed these questions as follows:

  • Theoretical analysis: We corrected Eq. 17, restated Theorem 1 precisely, and revised Theorem 2 to bound the gap between the mutual information and its lower bound, with full proofs.

  • Mismatch strategy: We proved that Mismatch optimizes a lower bound on the conditional mutual information, and explained its advantage through the sampling difficulty of Langevin dynamics with empirical support.

  • Label encoding vs. biological response: We clarified that all methods are evaluated on decoder inputs, and added training-dynamics to show the perturbation degradation and similar-perturbation swap experiments.

  • Energy distance: We showed that EDist and mean-level metrics capture different aspects, and added dispersion analyses revealing a trade-off between cell-level diversity and response accuracy.

  • Comparison with contrastive learning: A matched SupCon comparison shows that the key advantage of the energy lies in scoring and selecting representations.

Remarks

We are grateful that the reviewers’ comments helped us substantially improve the paper. We find the main concerns focus on the theoretical discussion of our method, where some statements are imprecise or too strong. We believe that these issues do not affect the main claims of the paper, including motivation, energy design, optimization, implementation, and experimental validity. All related statements have been corrected in the revised paper (highlighted in blue).

Thank you all again for your valuable feedback!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.