- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper presents a controlled empirical study of design choices for Joint-Embedding Predictive Architectures (JEPAs) applied to single-cell transcriptomics. The authors fix the encoder, view construction, training budget, and evaluation protocol, and vary only two axes: (1) the anti-collapse regularizer and (2) the observation-space preservation target. The paper reports three findings across zero-shot cell-type annotation, scIB multi-batch integration, and CITE-seq protein prediction: (1) latent-prediction-only JEPA variants underperform a plain masked autoencoder; (2) detection prediction outperforms magnitude reconstruction on all OOD cohorts; (3) adding latent prediction on top of PCG-BCE yields small, directionally consistent but statistically non-significant gains.
Critical strengths
- Existing single-cell JEPA adaptations each vary architecture, regularizer, and auxiliary loss simultaneously, making it impossible to attribute gains to any single component. By holding all confounds fixed and varying only the regularizer and preservation target, the paper provides the first systematic isolation of these two axes within a shared JEPA framework. This paper conclude the finding that collapse prevention alone is insufficient.
Critical weaknesses
- The methodological novelty is marginal. Each constituent element of DET-JEPA already exists. the paper's own results undermine the DET-JEPA contribution: PCG-BCE ONLY is nearly identical to DET-JEPA across all metrics. Therefore, the contribution of this paper should be better elaborated.
- The paper's central claims that detection prediction outperforms magnitude reconstruction, that collapse prevention alone is insufficient, and that the JEPA latent-prediction branch contributes little. All the findings are architectural claims, yet they are tested exclusively on a 5-layer MLP encoder. All major single-cell foundation models use Transformers, and the relative merit may differ substantially under self-attention. Without at least a Transformer validation, the paper cannot establish that its findings are general design principles rather than MLP-specific observations.
Questions for authors
Please refer to Critical Weaknesses
4 comments
Dear Reviewer cckd,
With less than two days left in the discussion phase, we wanted to gently check in to see if our responses have addressed your earlier concerns. We truly look forward to your feedback, as your insights are invaluable in helping us better articulate our paper's positioning and contributions, and further improve the manuscript. Please let us know if any questions remain.
Thank you again for your time and thoughtful review!
The authors.
Official Comment by Reviewer cckd
I have reviewed your rebuttal carefully, which address my major concerns substantially. However, this work represents an incremental advance within the established JEPA+auxiliary loss paradigm rather than a paradigm-shifting innovation. Based on the current rebuttal and the comments from other reviewers, I am adjusting my recommendation from weak reject to weak accept. Points to address in the final version:
- All core findings are validated exclusively on MLP encoders, and their generalizability to Transformer-based single-cell foundation models lacks direct experimental support.
We sincerely thank the reviewer for carefully evaluating our rebuttal, recognizing our clarifications, and raising the recommendation!
We fully take your point regarding the architectural scope and will thoroughly address it in the final version in two ways:
- Explicit Scope & Standalone Framing: Building on our current Discussion section, we will ensure our claims throughout the main text clearly reflect the MLP setting while contextualizing the standalone practical value of MLP backbones in single-cell workflows.
- Targeted Transformer Validation in the Final Version: While the rebuttal window did not permit training additional compute-intensive backbones, we commit to adding targeted controlled experiments with a Transformer encoder at the key decision nodes in our final revision (specifically verifying binary detection vs. continuous magnitude supervision and single- vs. dual-branch synergy). This will directly test cross-architecture generalizability and make the final paper substantially more complete and solid.
Thank you once again for your constructive feedback and support in helping us strengthen this work!
Value for single-cell JEPA development, component contributions, and MLP encoders
We thank the reviewer for the constructive feedback. Disentangling which design choices genuinely drive representation quality is critical for single-cell JEPA. Below, we clarify our novel contributions, demonstrate the statistical gains of our dual-branch design, and highlight the standalone value of our architectural scope.
Weakness 1. Novelty and the Contribution of DET-JEPA
We address this concern in two parts:
Rather than claiming a complex new architecture from scratch, our foundational contribution is establishing a unified single-cell JEPA design space—holding the encoder and protocol fixed to cleanly disentangle collapse prevention, preservation, and latent prediction. Grounded in this design space, we respectfully clarify that PCG-BCE is newly proposed in this work (not a pre-existing single-cell component) as a simple yet highly effective domain-tailored objective, positioning DET-JEPA as a principled, best-adapted configuration for noisy single-cell data. Moreover, our new weighting control (
PCG-MAE, Tables 7–9) confirms that PCG-BCE’s advantage stems directly from binary detection supervision filtering count noise, rather than the weighting scheme alone.Firstly, we believe that PCG-BCE’s strong standalone performance validates our proposed preservation objective rather than undermining our work, while extended experiments confirm consistent, statistically significant gains from the full dual-branch DET-JEPA:
PCG-BCE + VISRegablation (Tables 7–8), holding preservation and regularization fixed while adding the JEPA latent-prediction branch improves all 3 OOD transfer scores and all 3 scIB metrics.Weakness 2. Standalone Importance of MLPs vs. Transformer Foundation Models
1. Standalone prominence of MLPs in single-cell genomics & clean attribution: Unlike natural language, single-cell expression profiles are inherently high-dimensional, unordered tabular data, where MLPs remain a dominant SOTA workhorse rather than a simplified proxy. MLP encoders underpin the field's most widely adopted foundational and generative tools—including scVI (Nature Methods) [1] and Squidiff (Nature Methods) [2]—and serve as the primary backbone in authoritative single-cell SSL benchmarks such as Richter et al. (Nature Machine Intelligence) [3] and scSSL-Bench (ICML 2025 Spotlight) [4], routinely matching or outperforming heavy Transformer models on core integration and representation tasks at a fraction of the compute cost. Establishing principled SSL objectives for MLPs therefore delivers immediate, high-impact practical value in its own right. Methodologically, whereas single-cell Transformers depend on non-standardized heuristics (e.g., gene tokenization, expression binning, or rank-ordering) that act as strong confounders, an MLP backbone cleanly isolates objective-level mechanisms.
2. Cross-architecture alignment: Notably, our finding on multi-branch synergy aligns with recent Transformer-based single-cell JEPA work (Cell-JEPA [5]), which similarly demonstrates that dual-branch training outperforms single-branch variants under self-attention. Nevertheless, we agree that direct Transformer evaluation is an important direction. In our manuscript, we also explicitly calibrate our claims to reflect MLP encoders, and we plan to extend this controlled benchmark to compute-intensive Transformers as resources permit.
References
[1] Lopez et al. Deep generative modeling for single-cell transcriptomics. Nature Methods (2018).
[2] He et al. Squidiff: predicting cellular development and responses to perturbations using a diffusion model. Nature Methods (2026).
[3] Richter et al. Delineating the effective use of self-supervised learning in single-cell genomics. Nature Machine Intelligence (2025).
[4] Ovcharenko et al. scSSL-Bench: Benchmarking Self-Supervised Learning for Single-Cell Data. ICML (2025, Spotlight).
[5] ElSheikh et al. Cell-JEPA: Latent representation learning for single-cell transcriptomics. arXiv:2602.02093 (2026).
We hope these clarifications and new statistical evidence resolve your concerns and merit a positive reassessment of our work. Please feel free to reach out during the discussion phase if any questions remain!