- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
First of all, I thank the authors for taking their time to write the paper. The core idea seems to be extending the additive decoder framework to incorporate cases when the data modality carries signals in the inputs that occlude one another. The authors provide occluded objects in images as a core example, but I can see how the concept can transfer across modalities (e.g. for scRNA-Seq data, having perturbations that might mask expression of genes belonging to cell identity). In this regard, I found the motivation clear.
The solution, as the authors suggest, lies in how the latent information is aggregated during decoding: one might assume averaging is a good choice, but for cases like occlusion, the object in the front of the view plane would completely obstruct signals of other objects. The authors therefore propose a family of aggregations that recapitulates averaging and max-pooling at opposite ends, controlled by the inverse temperature \tau. Importantly, their theory reportedly rests on milder assumptions (compared to previous work which require differentiable decoders and second-order conditions). The paper is also honest in that the theory provided is for \tau at infinity, and that this is an open question for future work. Alongside this, the authors also did their due diligence thoroughly in exploring most of the relevant literature (to which I have provided some suggestions below for extensions).
Overall, taken as a purely theoretical extension over a case in applying additive decoders, the paper seems to be a significant contribution. That being said, there are some parts of the paper that I found to be lacking, particularly for experimental applications, although I am inferring that the paper is mainly focusing on theory for that matter. In any case, I provide the authors with a list of strengths and weaknesses that can be acted upon, which I believe will make the paper overall stronger.
Critical strengths
-
The problem is clearly motivated, as to why existing theory excludes occlusion. Proposition 6 is short and general. It makes precise the informal claim that smooth decoders cannot represent visible occlusion boundaries, and it is likely to be useful independently of the rest of the paper. The authors provide a new identifiability result for decoders, with weaker smoothness assumptions, which seems particularly significant for those working on additive decoders. I also did not notice anything completely out of line with regards to proofs and assumptions during my reading (although the appendix was skimmed).
-
The design of the solution is quite intuitive and well approached. Focusing on the pooling side, and classifying the two ends as existing pieces of literature positions the paper and its contribution well.
Critical weaknesses
-
The experimental validation leaves much to be desired across all fronts. Although the authors have mainly focused on practically showing the results of their theoretical work, baselines are severely lacking. The only non-author baseline is IA for CLEVR6. Additionally, if the question is the general case of obtaining disentangled representations without any supervision, and the matter is finding a model that is good at this, I am sure there should be baselines the authors could include (\beta-VAEs, \beta TCVAEs, DIPVAE, FactorVAE etc. among others that the authors cite). Then again, the sections do establish what the theory proposed with regards to filling the gaps in the spectrum of “Cooperation - Winner Takes It All”. Similarly, for scRNA-Seq data, the experiments provided by the authors do not motivate why one should opt for annealing if the gains are as marginal as what Table 3 (similarly Tables 11-12) suggests. In my opinion, it would be much better to connect the representation quality with biological statements if that is the end goal (e.g. what would one have missed if one did not use annealed LSE)?. This further requires baselines that may not even be geared towards disentanglement (such as scVI which without batches is essentially a VAE with a Negative binomial / Poisson likelihood), or similarly adaptation of baselines listed above using a similar likelihood adaptation to compare if one gains anything by adopting annealing.
-
Overall, not really clear why one would go for the annealed members of the family, when one can choose ends of the spectrum. I am wondering if there are cases when one would prefer not being on the ends of the spectrum (i.e. full cooperation or the max regime). This would be made even stronger if this corresponds to some intuitive quantity / quality that is easy to understand (e.g. maybe there are certain scRNAseq settings where occlusion of expression occurs in a specific way etc.). If the answer to this is that the temperature is essentially only to allow max-pooling and is irrelevant to the setting, then one would definitely require comparing to baselines to see how this approach fares against others.
Questions for authors
-
For all datasets, it would be great to have baselines (doesn’t need to be exhaustive), especially for the authors’ setting where they find the current literature lacking. The synthetic experiment demonstrates this already (with the additive panel vs LSE), but it would make the experimental side a lot stronger if a similar analysis / intuition / qualitative message could be obtained for other datasets.
-
For scRNA-Seq data, cell types are the easiest properties of the cell to disentangle, as they are the dominant axis of expression. I am wondering if there’s a way to extend this experiment to see if this approach allows disentangling factors of variation that are more “interesting” compared to commonly employed representation learning methods such as scVI?
Annealed members of the family: the way to view them etc. (are they purely for approximating as \tau goes to infinity, or do the finite values have something interesting about them). I may have missed this in the paper, which in this case might mean that outlining this benefit in a more apparent way, perhaps through figures, would be better.
As the paper mainly evaluates on scRNA-seq, side information (“Employing an auxiliary variable”) is routinely used. Examples include temporal and spatial covariates structuring latent factors (MEFISTO, Velten et al., 2022; NSF, Townes & Engelhardt, 2023), paired modalities split into shared and modality-specific latents (APOLLO, Zhang et al., 2026), with identifiability results for the unpaired multi-domain case (Sturma et al., 2023). Condition labels are likewise used to separate invariant from condition-specific factors (contrastiveVI, Weinberger et al., 2023; scDisInFact, Zhang et al., 2024; DisCoVR, Slavutsky et al., 2026), with identifiability theory for this model class (Lopez et al., 2024) and sparse-mechanism-shift models for perturbation data (Lopez et al., 2023). The authors might consider using any of these references to better position the proposed pooling to clarify when a max-type inductive bias is needed and when auxiliary information already suffices.
2 comments
Reply to Reviewer hTzY (part 2)
For scRNA-Seq data, cell types are the easiest properties of the cell to disentangle, ... methods such as scVI?
We agree. We additionally evaluated the cell cycle (G2M and S phase scores), a well-characterized process that varies within cycling cell types, across all 11 atlases. Please see point 2 of our General reply to all reviewers.
Annealed members of the family: the way to view them etc. (are they purely ... figures, would be better.
As the reviewer suggests, we added new Figure 4, which shows how the annealed model evolves from low to high during training on slider. Please see point 1 of our General reply to all reviewers for an empirical comparison between annealing and constant temperatures, and our response to the third comment above for a discussion on the role of annealing.
As the paper mainly evaluates on scRNA-seq, side information ("Employing an auxiliary variable") is ... auxiliary information already suffices.
We thank the reviewer for these valuable references. We have expanded our extended literature review (Appendix J, under a new paragraph "Auxiliary and supervised information in single-cell omics") to discuss these works and position our approach in relation to them.
Briefly, as with vision datasets where supervised segmentation masks and annotations aid disentanglement, auxiliary information clearly benefits scRNA-seq analysis when reliable labels are available. Because auxiliary variables are typically incorporated through additional loss terms rather than architectural inductive biases, they can be directly combined with architectural biases such as max-mask decoders (our work). However, when the auxiliary variable is itself the target of analysis (such as the perturbation in genetic screens), models must employ it with care, as label noise or unmodeled confounders directly transfer into the latent representation. A detailed discussion is provided in Appendix J.
References:
[1] F. Locatello, S. Bauer, M. Lucic, G. Raetsch, S. Gelly, B. Scholkopf, and O. Bachem. Challenging common assumptions in the unsupervised learning of disentangled representations. Proceedings of the 36th International Conference on Machine Learning, 2019.
Reply to Reviewer hTzY (part 1)
We thank Reviewer hTzY for the thorough review, the constructive suggestions, and the references. Following the review, we added baselines in all three settings and an analysis of a biological program beyond cell types (the cell cycle). We answer each point below.
We thank the reviewer for this suggestion. On the synthetic data, we added β-VAE, β-TCVAE, FactorVAE and DIP-VAE-II with a joint decoder, tuning each by grid search on the validation block-MIG, which uses the true latents and thus favors them (Table 1, Table 15, Appendix L.1). None reaches annealed LSE or LSE at a constant τ between 1 and 8. The best, β-VAE, has a mean block-MIG of 0.77 on slider and 0.52 on clock, against 0.99 and 0.85 for annealed LSE (five seeds). This is aligned with the unidentifiability of nonlinear decoders without inductive biases or auxiliary information (Locatello et al., 2019). We describe new single-cell baselines in the next response point.
We agree that the LMS-SMI gain of annealing over τ=1 on single-cell data is modest, since most of the improvement over mean pooling (which scores 21 to 43% lower than τ=1 on every atlas) comes from max-like pooling itself. Annealing gives a much sharper gene-to-latent assignment than τ=1 (thus more interpretable and closer to our identifiability guarantees) while avoiding the degradation of a constant τ=8. Please see point 1 of our General reply to all reviewers.
To connect representation quality with biological statements, we evaluated the cell cycle and closely related cell-type pairs (Figure 10, Appendix K.3). Without max-like pooling, the cell-cycle signal is spread over several latents, and without annealing, some closely related cell types (myofibroblasts vs. pericytes in intestine, cDC1 vs. cDC2 in spleen) overlap in the matched latents. Please see point 2 of our General reply to all reviewers.
As the reviewer suggested, we also trained scVI (128 latents, same hidden dimensions) and an NB β-TCVAE based on it, with three seeds on all 11 atlases. Both score below the LSE models on every atlas (mean LMS-SMI 0.40 and 0.54 vs. 0.61 to 0.67, Tables 3 and 11).
We use a finite τ primarily to train the model toward the identifiable max end (τ→∞, Section 3.2). A small τ yields less sharp feature-to-slot assignment and poorer occlusion modeling, whereas a large τ is harder to optimize. Finding the best intermediate constant τ requires hyperparameter tuning, as the optimal value varies across datasets and initializations (for example, τ=8 performs well on slider but worse on clock, while τ=32 is best on clock but degrades on slider; Figure 3b, Tables 6 and 7). With annealing, slots cooperate early at low τ, and as τ rises, single slots win features with growing margins (new Figure 4). Please see point 1 of our General reply to all reviewers.
We also appreciate the comment on expression in scRNA-seq data. Biological processes in cells interact extensively. Inhibition is a type of interaction in which one process slows down or completely blocks another. For example, CDKN1A protein is known to inhibit the cell cycle (e.g., to prevent tumor growth). By analogy with occlusion in imaging data, such inhibition could be modeled as masking, in which one latent variable masks the activity of another. Whether masking decoders describe such systems better than additive ones is an open question that we have not tested, and modeling and analyzing such complex systems is beyond the scope of the current work. We now mention this as an interesting future direction in the conclusion (Section 5).
We agree and added baselines on both the synthetic data (first response above) and the 11 single-cell atlases (scVI and NB β-TCVAE). On single-cell data, joint-decoder VAEs and mean pooling fail to isolate individual biological programs or closely related cell types into single latents as cleanly as LSE (Tables 3, 11 and 13).
For intuition and a qualitative comparison across datasets, we added Figure 4 (slot specialization during annealing on synthetic data) and Figure 10 (Appendix K.3, latent separation of closely related cell-type pairs across models). Please also see point 2 of our General reply to all reviewers for the cell-cycle results.