- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CARMA combines contextual encoding of accessible candidate cis-regulatory elements (cCREs) with a neural topic model. A Transformer encodes sampled, TF–IDF-ranked cCRE tokens. Shared topic prototypes assign contextual tokens to topics, and assignment-weighted pooling of identity embeddings parameterizes a logistic-normal posterior over cell-level topic mixtures. A Bernoulli decoder reconstructs the full accessibility profile using the same topic prototypes and cCRE identity embeddings. This yields explicit contextual assignments, cell mixtures, and topic–cCRE log-odds coefficients.
The model is pretrained on 5.16 million human cells and evaluated on seven studies totaling 278,090 cells. Experiments assess cell-state separation, linear cell-type prediction, batch mixing, regulatory annotation recovery, motif associations, and input-hidden accessibility prediction. Matched pretraining ablations examine self-attention, parameter sharing, and posterior pooling. The results support useful representations and an inspectable predictive topic structure, with further questions about contextual biological interpretation and generalization.
Critical strengths
1. A coherent predictive connection between contextual inference and explicit topic axes, supported by matched ablations.
The same topic indices organize contextual assignments, posterior inference, and full-profile reconstruction. Consequently, the topic–cCRE coefficients have a defined interpretation in accessibility log-odds, rather than being solely post-hoc labels for an embedding. The from-scratch ablations are particularly useful: removing self-attention lowers mean AvgBIO from 0.5907 to 0.4922 and macro-F1 from 0.7580 to 0.6951; separating encoder and decoder parameters yields 0.5400 and 0.6972, respectively. Parameter sharing also reduces the trainable count by 47.0%. These controls support the proposed design beyond an architectural description.
2. Broad empirical evidence with meaningful biological and predictive controls.
Across the seven studies, CARMA's topic representation has mean macro-F1 0.7580 versus 0.7166 for EpiAgent and exceeds all three main ATAC-only pretrained encoder comparators on each study. Regulatory probes use common loci and chromosome-held-out label splits, with co-occurrence and genomic-covariate controls. Motif enrichment uses a covariate-stratified genomic background and correction over all 64×282 topic–motif-family tests. The input-hidden prediction analysis removes query loci before ranking and token selection; inferred mixtures reduce query NLL from 0.1769 to 0.1661 relative to mixtures shuffled within study, sample group, and depth strata. Together, these results establish predictive information and plausible global lineage-associated programs more convincingly than selected visualizations alone.
Critical weaknesses
1. The specifically contextual cCRE interpretation is less established than the usefulness of the cell representation and global decoder.
The distinctive object in Eq. (3) is the context-dependent assignment . However, the annotation probes evaluate static identity embeddings ; the motif and cell-type associations examine global decoder directions and cell mixtures; and masked reconstruction tests the predictive value of those mixtures. These are valuable results, but they do not directly show that the same cCRE's changed assignment across cellular contexts has a reproducible biological meaning. Parameter sharing establishes a common indexing and predictive structure, not by itself a validated token-level explanation.
There is also a distinction between the topic lists being assessed: Appendix I interprets cCREs ranked by , whereas Appendix J computes coherence and diversity from , which includes baseline accessibility. Both rankings have legitimate interpretations, but the latter does not directly validate the former. The small NPMI improvement over CARMA-Bag, −0.593 versus −0.613, is therefore limited evidence for more coherent biological signatures; sparse observations also limit this metric, as the authors acknowledge.
A same-locus, different-context analysis, with resampling stability and the existing ablation checkpoints, would address the distinguishing interpretability claim. I am not requesting a causal-regulatory theorem or wet-lab perturbation evidence: the manuscript appropriately describes candidate programs and does not claim causal identification.
2. Generalization and robustness are only partially characterized by the current evaluation protocol.
Appendix B's five seeds vary downstream evaluation while holding the pretrained checkpoint fixed. They therefore do not measure variability in pretraining or in the learned topic programs. The linear probes use cell-type-stratified folds within each study, rather than donor- or library-held-out folds. Evaluating frozen representations across seven studies is a real strength, but these probe results do not separately establish supervised transfer to new donors or experimental libraries.
The manuscript lists training-study names and evaluation-study summaries, but an accession-level mapping and explicit sample/cell-overlap audit would make the external-evaluation claim independently checkable. For cCRE annotation, the CATlas source atlas is explicitly included in pretraining, and chromosome holdout applies to probe labels rather than to the unsupervised learning of locus embeddings. This is a legitimate evaluation of annotation information in pretrained representations; it should not be interpreted as evidence of generalization to loci unseen during pretraining. I do not infer undisclosed label leakage from this overlap.
These limitations matter especially for small differences: the scETM-based model essentially matches CARMA's mean AvgBIO, 0.5906 versus 0.5907, although CARMA improves macro-F1 by 3.11 percentage points. Clearer provenance, donor/library-held-out probing, and evidence about pretraining variability would better delimit the architectural advantage.
Questions for authors
The requests below concern the current interpretation and evaluation claims. They do not require a new large-scale pretraining benchmark.
1. Contextual assignments — targeted validation or clarification. For cCREs observed in multiple cell types or states, can you show how their assignments change, whether those changes agree with the corresponding decoder programs, and how stable they are to input-token resampling? Comparing with CARMA-Bag and the independent-decoder checkpoint would help isolate the contribution of contextualization and tying. If such evidence is unavailable, please clarify the intended interpretation of these assignments as model intermediates.
2. Program definitions — additional analysis using existing checkpoints. How do coherence, diversity, and motif associations compare when using the centered ranking employed in the biological interpretation, including the relevant ablations? The current absolute-prediction ranking is meaningful, but evaluates a different list of regions.
3. Data separation and transfer — clarification and targeted evaluation. Please provide dataset accessions, exact training/evaluation memberships, and any sample/cell-overlap checks. On representative studies, does the macro-F1 advantage persist when probes hold out donors or libraries? For cCRE probes, please also specify whether PCA and all preprocessing are fitted on the training chromosomes and how the validation partition selects regularization.
4. Training robustness — clarification. How many independent pretraining runs support the main and ablation results? If replicated checkpoints are available, can you report variation in the main metrics and topic alignment across runs? Otherwise, please make clear that the reported seed uncertainty is conditional on one pretrained checkpoint and that topic stability across pretraining runs has not been established.