- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
First of all, I thank the authors for taking their time to write the paper. The paper tackles the problem of disentanglement in a regime with binary labels (the default regime), where one is provided observations with labels, and the task is to learn shared and condition-specific representations. Here, the former represents variation in the absence of the label, and the latter represents the portion that is correlated with changes caused by the label (e.g. cell types vs perturbations).
The paper is nicely written, and tackles the literature effectively by starting off with what most methods do: use a Variational Inference (VI) framework with additional regularizations (e.g. adversarial classifier on shared representations), prior specifications (e.g. an empirical prior) etc. In contrast to previous work, the paper extends upon the DGD framework and stands out in three aspects: MAP estimation for training, explicit masking of the conditional embedding when unperturbed, and using an empirical-like prior restricted only to the unperturbed class for the shared embeddings. Importantly, this considers both the supervised disentanglement problem in general and the specific setting it is in (i.e. perturbations), where one of the classes is a biological baseline.
In this sense, I found many parts of this manuscript particularly interesting which I outline below. Overall, I found the work well-motivated, and the approach justified both methodologically and on the application side. The remaining concerns are mostly on the empirical front; which, to be clear, is highly thorough and communicates the applicability of SplitDGD. The comments will mainly focus on specifics, especially considering the precise biological setting the model is framed in.
Critical strengths
-
The hard masking mechanism + MAP approach resolves the optimization ambiguity explicitly. Since the introduction of MultiGroupVI (Weinberger et al, 2022 ; which can be traced back to Abid & Zou, 2019), most methods operating over the VI framework used a similar idea for conditional embeddings, where the prior for unperturbed samples was set to be a delta at zero (see details of referenced paper) which was enforced throughout optimization with an additional KL term. Here, there is no encoder, and the masking gives no updates to the embedding, which essentially fixes this as part of the optimization
-
The prior term (i.e. alternate prior training) is very interesting, in that it follows recent previous ideas related to empirical priors but adapts it nicely to the perturbation setting. The prior terms (which for VI is also relevant for hard masking) are choices at the end of the day, and recent work by Slavutsky et al. 2025 (DisCoVR) has shown cases where hand-crafted priors can be harder to accurately specify in certain settings. They instead proposed an empirical prior that averaged shared embeddings per class for the conditional embeddings; and a standard normal prior for the shared embeddings (as this paper also notes). SplitDGD carries a similar idea, but with the biological context we know what the baseline is, hence it departs from the context in DisCoVR in the sense that the shared embedding is not what remains when one marginalizes out the label, but it is defined relative to what the expected baseline is (which for this context we know should be the unperturbed samples). This idea drives the optimization term for the prior, which I found quite fitting.
Critical weaknesses
-
Comparisons with competing methods are clear, but ablations on the different components would add to the story. Personally, I am curious as to which of the three ingredients (MAP estimation, and the other two mentioned in strengths) add to the gain here. I think this would also be helpful for future researchers to decide on what avenues of methodology they should pursue for similar questions. Ideally, one would have an ablation on every component starting from the base DGD framework (and also at the end maybe reverting partially to VI to see if MAP / DGD give the benefits), but given time constraints, I leave it to the authors’ judgement on what would be best.
-
For specific biological problems (e.g. counterfactual prediction) the baselines might be slightly misspecified, with some additional considerations required per method. For counterfactual prediction, it might be better to compare to a commonly used method (e.g. CPA, scGen etc.) as certain baselines (specifically DisCoVR, and as far as I can tell contrastiveVI) were neither originally designed nor evaluated for counterfactual prediction, although the procedure here has been previously used (starting with scGen, which used VAE + mean shifts in a similar way). Also, for VI based methods that use external components (e.g. adversarial classifiers for DisCoVR), it is quite often the case that hyperparameters are hard to tune. I am wondering if any hyperparameter tuning was done for these methods (outside of the parameters specified by authors themselves).
Questions for authors
-
Would it be possible to have some ablations regarding critical components of the model (as specified under weaknesses)? Particularly, finding out where the maximal gain is out of the three components would be very much appreciated.
-
Is it possible to compare with additional models that are specifically targeted for counterfactuals (Biolord / CPA)?
-
Would SplitDGD using a Gaussian likelihood vs baselines modeling counts directly change any of the results substantially?
-
(MINOR) Here are some typos I noticed: "arhe" (Sec. 2), "following the in protocol" (Sec. 4.2), "proovable" (Sec. 5), "gradients acts" (Fig. 1 caption), "an held-out", and a missing space in "Table 13.On". The image dataset is called "DF-MNIST" in the text and tables, "DigitFashion-MNIST" in Appendix B, and "Fashion-MNIST" in the Fig. 2 and Fig. 3 legends.
2 comments
Thanks for providing the missing tables (an edit to the original response might be needed to make the tables visible for others). I remain positive about the work and have updated my soundness evaluation accordingly.
We thank the reviewer for the careful reading and for the positive comments on the hard mask combined with MAP estimation and on alternate prior training.
W1/Q1 - Ablations. We ran the ablation on Haber, in both w-regimes, with the splits, the 10 seeds and the per-seed hyperparameters of the paper's runs (all six metrics are in the table of our answer to Reviewer RXQW). Following the reviewer's suggestion, the table below starts from a two-latent DGD with neither mechanism and adds them one at a time.
The largest gain comes from the hard mask, which lowers linear leakage by 0.05-0.13 and raises z-MCC by 0.23-0.32. The control-only prior adds little once the mask is in place (at most 0.008 on either metric) and more without it (z-MCC +0.03 and +0.10), so it acts as a second safeguard. Fidelity is unaffected: across the four variants RMSE stays within 0.001 of splitDGD and counterfactual R2 within 0.008 points.
To compare MAP estimation of free latents to amortised variational inference, this is not directly possible in the DGD framework. Thus, we contrast splitDGD to contrastiveVI, which is the closest counterpart to splitDGD as it applies the same zeroing of the condition block within a VAE (the contrastive-analysis lineage the reviewer mentions). The results are already in Table 2 of the manuscript. On Kang and Haber its z has the highest linear leakage of all methods (0.70 and 0.68, against 0.52 and 0.55 for splitDGD), and its reconstruction error and counterfactual R2 are worse than splitDGD's on all three single-cell datasets, while it recovers the perturbation subtypes on Haber better (w-MCC 0.84 against 0.60).
W2 / Q2 - Baselines comparison. For every method, splitDGD included, the counterfactual of a control cell is obtained by decoding its invariant latent with the mean condition-specific latent, the mean-shift scheme that, as the reviewer notes, goes back to scGen. Among our baselines, scDisInFact was designed for perturbation prediction, and splitDGD is ahead of it on every counterfactual entry (0.960 vs 0.871 on Kang, 0.926 vs 0.824 on Haber, 0.953 vs 0.872 on sci-Plex). As suggested, we have also run biolord (setup and full table in our answer to Reviewer wHDC, W1b): its counterfactual R2 on Haber is 0.64 (unsup-w) and 0.58 (sup-w), below every method in Table 2 (at least 0.82). We did not add CPA. Like biolord, it takes the perturbation label and the cell type as inputs and has no per-cell perturbation representation, so it cannot be scored on the decoupling criteria that are the subject of the paper, for which counterfactual prediction is one of four criteria and not the target task.
ContrastiveVI, DisCoVR and scDisInFact were tuned per seed on the single-cell datasets with the same multi-objective search as splitDGD (200 trials on Kang and Haber, 60 on sci-Plex), over spaces built around the authors' defaults. For DisCoVR the search includes the adversarial weight (1 to 100), the reconstruction weight, the two KL weights, the latent dimension and the learning rate. The DisCoVR paragraph of the Appendix described the authors' defaults; it was out of date and we will correct it.
Q3 - Modeling counts vs. Gaussian. We ran this comparison on Haber, training splitDGD with a negative-binomial likelihood on the raw counts and keeping everything else as in the paper's runs (splits, 10 seeds, and the per-seed hyperparameters, which were not re-tuned for the new likelihood). The table gives, for each w-regime, the paper's values and those obtained with counts (RMSE and counterfactual R2 in log-normalised space).
The results do not change substantially. Leakage and reconstruction are the same within seed variability, so splitDGD keeps the lowest linear leakage and the lowest RMSE of all methods in both regimes. z-MCC and counterfactual R2 are slightly worse with counts; w-MCC in unsup-w is also lower on average, within its seed variability. We have added this ablation in a new section of the Appendix.
Q4 - Typos.
Thank you for spotting them. We have applied those corrections in the revised version of the manuscript.