acceptodds
Under review as a conference paper at ICLR 2027

Balancing Invariance and Information Preservation in Single-Cell JEPA

Abstract

Self-supervised learning for single-cell genomics must balance invariance to nuisance variation against preservation of biological information. Existing single-cell Joint-Embedding Predictive Architecture (JEPA) adaptations change backbones, anti-collapse mechanisms, and auxiliary objectives at the same time, so it is unclear which choice actually drives transfer. We therefore fix the encoder, view construction, training budget, and evaluation protocol, and vary only two axes: the anti-collapse regularizer and the observation-space preservation target. Within this setting we introduce PCG-BCE, a per-cell-balanced and gene-entropy-weighted binary cross-entropy loss that supervises gene detection rather than expressionmagnitude, and DET-JEPA (detection-aware JEPA), which couples PCG-BCE with projector-space latent prediction. Three findings emerge. First, preventing collapse is not sufficient: variants trained with latent prediction alone, including one built on a provably collapse-free regularizer, fall below both a plain masked autoencoder and PCA on zero-shot cell-type annotation and on scIB biological conservation. Second, which observation-space target is preserved matters more than whether one is used: detection prediction outperforms magnitude reconstruction on every out-of-distribution cohort, consistent with a sampling analysis showing that detection labels are insensitive to positive-count magnitude. Third, PCG- BCE alone is already a strong representation learner; adding latent prediction improves 11 of 12 downstream metrics, but each gain is small and not individually significant over five seeds. Across zero-shot cell-type annotation, multi-batch integration, and protein-abundance prediction, DET-JEPA is the best matched JEPA variant, with VISReg as the preferred regularizer. Our study localizes where the gains in single-cell JEPAs come from and gives concrete guidance for designing preservation objectives.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Single Cell Transcriptomics, Joint Embedding Predictive Architectures, Self-Supervised Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_cckdOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents a controlled empirical study of design choices for Joint-Embedding Predictive Architectures (JEPAs) applied to single-cell transcriptomics. The authors fix the encoder, view construction, training budget, and evaluation protocol, and vary only two axes: (1) the anti-collapse regularizer and (2) the observation-space preservation target. The paper reports three findings across zero-shot cell-type annotation, scIB multi-batch integration, and CITE-seq protein prediction: (1) latent-prediction-only JEPA variants underperform a plain masked autoencoder; (2) detection prediction outperforms magnitude reconstruction on all OOD cohorts; (3) adding latent prediction on top of PCG-BCE yields small, directionally consistent but statistically non-significant gains.

Critical strengths
  1. Existing single-cell JEPA adaptations each vary architecture, regularizer, and auxiliary loss simultaneously, making it impossible to attribute gains to any single component. By holding all confounds fixed and varying only the regularizer and preservation target, the paper provides the first systematic isolation of these two axes within a shared JEPA framework. This paper conclude the finding that collapse prevention alone is insufficient.
Critical weaknesses
  1. The methodological novelty is marginal. Each constituent element of DET-JEPA already exists. the paper's own results undermine the DET-JEPA contribution: PCG-BCE ONLY is nearly identical to DET-JEPA across all metrics. Therefore, the contribution of this paper should be better elaborated.
  2. The paper's central claims that detection prediction outperforms magnitude reconstruction, that collapse prevention alone is insufficient, and that the JEPA latent-prediction branch contributes little. All the findings are architectural claims, yet they are tested exclusively on a 5-layer MLP encoder. All major single-cell foundation models use Transformers, and the relative merit may differ substantially under self-attention. Without at least a Transformer validation, the paper cannot establish that its findings are general design principles rather than MLP-specific observations.
Questions for authors

Please refer to Critical Weaknesses

4 comments
AuthorsOpenReview ↗

Value for single-cell JEPA development, component contributions, and MLP encoders

We thank the reviewer for the constructive feedback. Disentangling which design choices genuinely drive representation quality is critical for single-cell JEPA. Below, we clarify our novel contributions, demonstrate the statistical gains of our dual-branch design, and highlight the standalone value of our architectural scope.


Weakness 1. Novelty and the Contribution of DET-JEPA

We address this concern in two parts:

  1. Positioning of DET-JEPA & "Each constituent element... already exists":

Rather than claiming a complex new architecture from scratch, our foundational contribution is establishing a unified single-cell JEPA design space—holding the encoder and protocol fixed to cleanly disentangle collapse prevention, preservation, and latent prediction. Grounded in this design space, we respectfully clarify that PCG-BCE is newly proposed in this work (not a pre-existing single-cell component) as a simple yet highly effective domain-tailored objective, positioning DET-JEPA as a principled, best-adapted configuration for noisy single-cell data. Moreover, our new weighting control (PCG-MAE, Tables 7–9) confirms that PCG-BCE’s advantage stems directly from binary detection supervision filtering count noise, rather than the weighting scheme alone.

  1. On "PCG-BCE ONLY is nearly identical... undermine the DET-JEPA contribution":

Firstly, we believe that PCG-BCE’s strong standalone performance validates our proposed preservation objective rather than undermining our work, while extended experiments confirm consistent, statistically significant gains from the full dual-branch DET-JEPA:

  • Consistent directional gains: Across 5 runs, DET-JEPA outperforms standalone PCG-BCE on 8 of 9 endpoints (classification, integration, and protein prediction).
  • Statistical significance with higher power: Expanding scIB integration to 9 runs (Appendix E.4, Fig. 7), two-sided Welch’s tt-tests with Holm correction confirm statistically significant improvements over PCG-BCE across Bio (p=0.0167p = 0.0167), Batch (p=0.0207p = 0.0207), and Total (p=0.0167p = 0.0167).
  • Isolated branch benefit: In our new PCG-BCE + VISReg ablation (Tables 7–8), holding preservation and regularization fixed while adding the JEPA latent-prediction branch improves all 3 OOD transfer scores and all 3 scIB metrics.

Weakness 2. Standalone Importance of MLPs vs. Transformer Foundation Models

1. Standalone prominence of MLPs in single-cell genomics & clean attribution: Unlike natural language, single-cell expression profiles are inherently high-dimensional, unordered tabular data, where MLPs remain a dominant SOTA workhorse rather than a simplified proxy. MLP encoders underpin the field's most widely adopted foundational and generative tools—including scVI (Nature Methods) [1] and Squidiff (Nature Methods) [2]—and serve as the primary backbone in authoritative single-cell SSL benchmarks such as Richter et al. (Nature Machine Intelligence) [3] and scSSL-Bench (ICML 2025 Spotlight) [4], routinely matching or outperforming heavy Transformer models on core integration and representation tasks at a fraction of the compute cost. Establishing principled SSL objectives for MLPs therefore delivers immediate, high-impact practical value in its own right. Methodologically, whereas single-cell Transformers depend on non-standardized heuristics (e.g., gene tokenization, expression binning, or rank-ordering) that act as strong confounders, an MLP backbone cleanly isolates objective-level mechanisms.

2. Cross-architecture alignment: Notably, our finding on multi-branch synergy aligns with recent Transformer-based single-cell JEPA work (Cell-JEPA [5]), which similarly demonstrates that dual-branch training outperforms single-branch variants under self-attention. Nevertheless, we agree that direct Transformer evaluation is an important direction. In our manuscript, we also explicitly calibrate our claims to reflect MLP encoders, and we plan to extend this controlled benchmark to compute-intensive Transformers as resources permit.

References

[1] Lopez et al. Deep generative modeling for single-cell transcriptomics. Nature Methods (2018).

[2] He et al. Squidiff: predicting cellular development and responses to perturbations using a diffusion model. Nature Methods (2026).

[3] Richter et al. Delineating the effective use of self-supervised learning in single-cell genomics. Nature Machine Intelligence (2025).

[4] Ovcharenko et al. scSSL-Bench: Benchmarking Self-Supervised Learning for Single-Cell Data. ICML (2025, Spotlight).

[5] ElSheikh et al. Cell-JEPA: Latent representation learning for single-cell transcriptomics. arXiv:2602.02093 (2026).


We hope these clarifications and new statistical evidence resolve your concerns and merit a positive reassessment of our work. Please feel free to reach out during the discussion phase if any questions remain!

Dear Reviewer cckd,

With less than two days left in the discussion phase, we wanted to gently check in to see if our responses have addressed your earlier concerns. We truly look forward to your feedback, as your insights are invaluable in helping us better articulate our paper's positioning and contributions, and further improve the manuscript. Please let us know if any questions remain.

Thank you again for your time and thoughtful review!

The authors.

Reviewer_cckdOpenReview ↗

Official Comment by Reviewer cckd

I have reviewed your rebuttal carefully, which address my major concerns substantially. However, this work represents an incremental advance within the established JEPA+auxiliary loss paradigm rather than a paradigm-shifting innovation. Based on the current rebuttal and the comments from other reviewers, I am adjusting my recommendation from weak reject to weak accept. Points to address in the final version:

  1. All core findings are validated exclusively on MLP encoders, and their generalizability to Transformer-based single-cell foundation models lacks direct experimental support.

We sincerely thank the reviewer for carefully evaluating our rebuttal, recognizing our clarifications, and raising the recommendation!

We fully take your point regarding the architectural scope and will thoroughly address it in the final version in two ways:

  1. Explicit Scope & Standalone Framing: Building on our current Discussion section, we will ensure our claims throughout the main text clearly reflect the MLP setting while contextualizing the standalone practical value of MLP backbones in single-cell workflows.
  2. Targeted Transformer Validation in the Final Version: While the rebuttal window did not permit training additional compute-intensive backbones, we commit to adding targeted controlled experiments with a Transformer encoder at the key decision nodes in our final revision (specifically verifying binary detection vs. continuous magnitude supervision and single- vs. dual-branch synergy). This will directly test cross-architecture generalizability and make the final paper substantially more complete and solid.

Thank you once again for your constructive feedback and support in helping us strengthen this work!

Reviewer_sfozOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work propose single-cell JEPA and DET-JEPA, which studies self-supervised learning for single-cell genomics, aiming to reduce technical noise while preserving biological signals. The paper compare VICReg, SIGReg and VISReg under a common encoder and evaluation setup, then introduce a gene-detection preservation loss, PCG-BCE, within the dual-branch DET-JEPA framework. Across cell-type transfer, multi-batch integration and protein abundance prediction, detection-aware preservation delivers consistent gains, while the latent JEPA prediction branch adds only modest improvements.

Critical strengths

PCG-BCE addresses two important challenges in single-cell transcriptomics: sparse detection and uneven gene detection frequencies. The proposed per-cell balancing and gene-entropy weighting help prevent common detection patterns from dominating the loss, and the progressive ablation from BCE to PC-BCE to PCG-BCE supports the contribution of both components. Although weighted BCE is well established, this adaptation is practically useful. Focusing on gene presence rather than expression magnitude offers a sensible way to preserve biological information while reducing sensitivity to technical variation, with consistent gains in the controlled experiments.

Critical weaknesses
  1. Insufficient evidence to establish a standalone methodological contribution for DET-JEPA DET-JEPA combines PCG-BCE with a latent JEPA prediction branch, yet nearly all performance gains appear to originate from the PCG-BCE-only baseline. The gap between DET-JEPA and this baseline is negligible across key metrics: HLCA Macro-F1, PBMC Macro-F1, Protein Pearson, and DET-JEPA even lags on the Great Apes cohort. These minor differences lack statistical significance across five seeds (Figure 3c). A critical confounder further clouds interpretation: switching to DET-JEPA introduces both JEPA prediction loss and VISReg regularization, so any marginal shift cannot be attributed cleanly to the JEPA branch. This undermines the core novelty of the proposed dual-branch framework.

  2. The analysis for the information-preservation target does not fully support the main mechanistic claim The paper argues binary gene detection outperforms expression reconstruction, but this conclusion is premature. MAE uses unweighted MSE while PCG-BCE has custom per-cell balancing and gene-entropy weighting; gains may arise from weighting rather than the binary target. Matching controls such as weighted MSE or negative binomial loss are missing. The invariance argument is also narrow: binary labels are stable for detected genes, but detection probability itself is sensitive to sequencing depth and capture noise. Without perturbation tests, the causal link between binary detection and improved biological preservation remains unproven.

Questions for authors
  1. Can you compare PCG-BCE+VISReg against PCG-BCE with latent prediction to separate regularization and prediction effects?
  2. Could PCG-BCE’s gains over MAE come from its weighting scheme? Have you tested a weighted MSE baseline?
  3. Can you test representation stability under downsampling or varying sequencing depth to validate detection-target robustness?.
5 comments
AuthorsOpenReview ↗

Separating prediction from regularization and controlling the preservation objective

We thank the reviewer for identifying the attribution confound and for recognizing the practical motivation behind PCG-BCE. Your insightful suggestions prompted additional ablations in Appendix E—specifically a regularization-only control, matched-weighting magnitude reconstruction, and depth-perturbed pretraining—that cleanly disentangle prediction, regularization, and preservation, substantially strengthening our empirical evidence.


Weakness 1 + Question 1. Separating prediction from regularization

Concern: Gains over PCG-BCE are minor, and prediction is confounded with VISReg; add a PCG-BCE + VISReg control.

Across the original five-run evaluation, DET-JEPA achieves higher means than standalone PCG-BCE on 8 of 9 reported endpoints spanning cell-type classification, scIB integration, and protein prediction—demonstrating a consistent pattern of gains. While individual five-run differences in classification and protein prediction do not reach statistical significance, statistical power at n=5n=5 is constrained by run-to-run variability and task discriminability. We substantiate the contribution of latent prediction through two targeted analyses:

1. Expanded 9-run scIB significance analysis (Appendix E.4, Figure 7): Adding four independent runs per method expands the comparison to 9 runs per method (Appendix E.4). Two-sided unpaired Welch’s tt-tests with Holm correction yield adjusted pp-values of 0.0167, 0.0207, and 0.0167 (with pooled-SD Cohen’s dd of 1.64, 1.28, and 1.62) for Bio, Batch, and Total, respectively—confirming that DET-JEPA significantly outperforms PCG-BCE only across all three integration metrics (p<0.05p < 0.05).

2. Disentangling prediction from regularization (Table 6): We added a PCG-BCE + VISReg control that retains the detection objective and regularizer while removing latent prediction:

MethodHiT Macro-F1NN Macro-F1Great Apes Macro-F1scIB Total
PCG-BCE only0.71490.19080.28530.5063
PCG-BCE + VISReg (no latent pred.)0.55040.14950.18520.5051
DET-JEPA (full)0.75250.19270.25300.5197

With PCG-BCE and VISReg held fixed, adding latent prediction improves all three OOD transfer scores and all three scIB integration metrics (full scIB breakdown in Table 8), whereas adding VISReg alone degrades or fails to reproduce these gains.


Weakness 2 + Questions 2–3. Weighting control and depth robustness

Concern: Could weighting explain PCG-BCE's gains? Add matched loss controls and test sensitivity to sequencing depth.

1. Matched-weighting control (PCG-MAE, Tables 7–9): To test whether the weighting scheme alone drives performance, we evaluated PCG-MAE, which reconstructs continuous expression via MSE using the exact same per-cell balancing and gene-entropy weights as PCG-BCE:

Preservation objectiveHiT Macro-F1NN Macro-F1Great Apes Macro-F1scIB Total
Unweighted MSE (MAE)0.59530.15360.24120.4869
PCG-weighted MSE (PCG-MAE)0.55510.15860.20180.4965
PCG-BCE only0.71490.19080.28530.5063

PCG-BCE outperforms PCG-MAE across all three OOD cohorts and all three scIB metrics (Tables 7–8), while also achieving higher mean protein-prediction scores (Table 9). Thus, weighting alone cannot account for the advantage of detection-based supervision.

2. Simulated sequencing-depth reduction (Appendix E.3, Table 10): To assess robustness to capture noise during pretraining, we draw a per-cell depth fraction ri∼Uniform(0.1,1.0)r_i \sim \mathrm{Uniform}(0.1, 1.0), Poisson-resample normalized pseudo-counts, and renormalize prior to forming pretraining views and targets (keeping fine-tuning and evaluation profiles unperturbed). Resulting metric shifts on protein prediction are:

Pretraining objectiveΔ\Delta Pearson (↑\uparrow)Δ\Delta Spearman (↑\uparrow)Δ\Delta MSE (↓\downarrow)
PCG-BCE−0.0002−0.0003+0.0010
PCG-MAE−0.0005−0.0008+0.0019

Detection-based pretraining exhibits smaller degradation across all three metrics. Intuitively, in noisy measurements, replacing continuous magnitudes with binary gene-presence targets filters out within-positive count noise and improves the signal-to-noise ratio of supervision. As formalized via a sampling model in Appendix D, binarized detection removes variance among positive counts and becomes increasingly stable as expected counts grow—though severe downsampling can still flip low-count genes to undetected.


We hope the expanded experiments and clarifications fully address your concerns and kindly invite you to reconsider your score, please let us know without hesitation if anything else remains unclear!

Thanks for adding the matched-weight PCG-MAE control. It addresses my concern that the gains over MSE reconstruction could come from the weighting scheme alone.

I have two remaining questions. Could you report the scIB comparison with PCG-BCE alone separately for each loss-weight setting, with per-seed scores? Also, how do PCG-BCE and PCG-MAE compare when the RNA inputs are downsampled at test time? The current experiment applies the perturbation during pretraining, so it leaves this question open.

A simpler Figure 1 and a larger legend in Figure 5 would help. A marked-up PDF or page and line references for the revisions would also make the changes easier to verify. I’m keeping my score at 2 for now.

Thank you for your prompt follow-up and for acknowledging that the matched-weight PCG-MAE control addresses your concern about the weighting scheme. We appreciate your further questions and suggestions.


1. Per-seed scIB results: DET-JEPA vs. standalone PCG-BCE

Using the order (latent prediction, VISReg, PCG-BCE), the loss weights are (1.0, 0.8, 1.0) for DET-JEPA and (0.0, 0.0, 1.0) for standalone PCG-BCE (which omits latent prediction and VISReg).

After checking our experimental records, we found that one of the five additional runs terminated abnormally, leaving nine completed runs (five original + four additional). Matching run IDs across the two methods correspond to the exact same random seed. Their correspondence with the paper is:

  • run1–run5: Original results reported in Figure 2 and Table 3.
  • run6–run9: Additional results added to the appendix during the rebuttal.
  • run1–run9: Combined results reported in Table 8 and Figure 7 of the supplementary experiments.
RunSplitDET-JEPA (Bio)DET-JEPA (Batch)DET-JEPA (Total)PCG-BCE (Bio)PCG-BCE (Batch)PCG-BCE (Total)
run1Original0.62740.37090.52480.61730.32490.5003
run2Original0.64280.38510.53970.61790.33940.5065
run3Original0.61890.34370.50880.62770.33880.5121
run4Original0.63860.33820.51840.61910.34330.5088
run5Original0.62170.34720.51190.61780.33090.5031
run6Additional0.63840.36750.53010.61660.32930.5017
run7Additional0.62990.34560.51620.62180.34360.5105
run8Additional0.62190.32920.50490.62100.33200.5054
run9Additional0.63510.35420.52270.61860.34350.5086
MeanAll (1–9)0.63050.35350.51970.61980.33620.5063

(Note: We merged the two tables side-by-side by seed and added the 9-run mean row to make paired comparisons easier to inspect.)


2. Rationale for pretraining downsampling and additional test-time evaluation

In the multiomics prediction task, downsampling can be introduced at three stages: pretraining, fine-tuning, or testing. Since our study focuses on pretraining algorithms, our original experiment asks which objective learns representations that are less affected when the quality of the pretraining corpus deteriorates. We therefore applied the downsampling perturbation during pretraining, consistent with the motivation in Appendix D.

To address the separate question regarding test-time robustness, we conducted an additional evaluation where models pretrained and fine-tuned on clean data are evaluated directly on downsampled test inputs (measuring performance degradation relative to clean test inputs):

MethodPearson Drop (50% Depth) ↓Pearson Drop (10% Depth) ↓MSE Increase (10% Depth) ↓
DET-JEPA0.00540.047673.64%
PCG-BCE0.00530.049575.32%
PCG-MAE0.00540.047374.32%
MAE0.00520.046675.38%

As shown above, all methods exhibit comparable sensitivity to test-time sequencing depth reduction, and DET-JEPA as well as PCG-BCE does not show a distinct advantage under test-time perturbation. Together with our original findings, this clarifies that PCG-BCE provides stronger robustness against pretraining data degradation rather than unseen test-time distribution shifts. This actually aligns with our original claim and expectation.


3. Presentation and revision tracking

Thank you for the constructive suggestions on presentation! We have created a simplified version of the framework diagram and placed it at the end of the appendix (Figure 8) to illustrate the design space more clearly. In addition, we have enlarged the legend in Figure 5 and highlighted all revisions in blue in the updated manuscript to make the changes easy to verify.

Reviewer_hRuaOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the roles of anti-collapse regularization and information preservation in single-cell JEPA. The authors propose PCG-BCE, a gene-detection loss with per-cell class balancing and gene-entropy weighting, and combine it with VISReg and latent prediction to develop DET-JEPA. The method is evaluated on cell-type annotation, batch integration, and protein prediction tasks.

Critical strengths
  1. The paper systematically compares different regularizers and preservation objectives under a shared experimental setup, providing useful insights into their effects on representation quality.
  2. The inclusion of standalone PCG-BCE and other ablation experiments helps identify which components contribute to performance improvements.
Critical weaknesses
  1. The paper does not clearly explain why JEPA is needed to balance invariance and biological information preservation. The connection between the motivation and the chosen methods needs to be better justified.
  2. The proposed framework largely combines existing techniques, and the methodological novelty of PCG-BCE appears limited. The authors should better distinguish their contributions from related work.
  3. Standalone PCG-BCE performs similarly to DET-JEPA, while the additional improvements from latent prediction are not individually statistically significant. This raises questions about the necessity of the JEPA branch.
  4. Figure 5 does not clearly identify the reported 40 cell types or visualize the distribution of the 9 batches. The authors should provide cell-type-colored and batch-colored UMAPs with complete legends to better demonstrate biological conservation and batch mixing.
  5. The claimed robustness of binary detection requires further validation under different sequencing depths or downsampling conditions. Additional controls with normalized loss weights are also needed to isolate the benefits of PCG-BCE.
  6. The authors should clarify the stop-gradient operation in Eq. (2) and whether ReLU is applied to the final detection-decoder layer. Figure 1 could also more clearly distinguish the proposed components from existing methods.
Questions for authors
  1. What benefit does the JEPA branch provide beyond standalone PCG-BCE, given the small and statistically inconclusive improvements?
  2. Does detection-aware preservation remain effective under different sequencing depths or controlled downsampling?
  3. Can the authors provide batch-colored UMAPs and complete cell-type annotations for the integration results?
4 comments
AuthorsOpenReview ↗

Clarifying objective design, component contributions, and robustness

Thank you for recognizing our controlled comparisons and for your constructive suggestions, which helped us make the experimental evaluation more complete and the method exposition clearer.

To address your comments, we have added new control experiments (Appendix E.1–E.3, Tables 6–10), supplementary scIB significance analysis (Appendix E.4, Figure 7), UMAPs with complete legends (Appendix C.2, Figures 5–6), and implementation clarifications (Sections 3.1–3.2, Appendix A.2). Our point-by-point responses are as follows:


Weakness 1. About JEPA vs. information preservation

Concern: Why is JEPA needed to balance invariance and biological information preservation?

Latent prediction encourages consistency across masked views, while detection prediction retains cell-specific gene-presence information. When view perturbations approximate technical noise, latent consistency helps suppress unwanted variation; however, excessive invariance risks discarding biological signal, and generic anti-collapse regularization alone does not specify which features to retain.

Consistent with this motivation, VISReg-only JEPA achieves slightly higher scIB Batch than DET-JEPA (0.3614 vs. 0.3570) at the expense of much weaker scIB Bio (0.5620 vs. 0.6299) and HLCA Macro-F1 (0.6013 vs. 0.7484). Below, we examine the convers: what the latent prediction branch adds when PCG-BCE is already present.


Weakness 2. Methodological contribution and related work

Concern: The novelty of PCG-BCE and its distinction from prior work need clarification.

Existing single-cell JEPAs vary simultaneously in regularizers and preservation targets, obscuring the true drivers of performance. We address this by establishing a much-needed controlled benchmark, with a fixed encoder and training protocol to isolate what truly drives performance. We believe that disentangling these confounding factors to understand which components genuinely work constitutes a valuable contribution to the field in its own right.

Crucially, this controlled attribution reveals the empirical necessity of maintaining a dual-branch paradigm in single-cell modeling. Grounded in this finding, we formulate PCG-BCE by adapting standard weighting components to the sparsity of gene detection. Rather than claiming novel weighting primitives, we demonstrate that this clean, domain-adapted formulation, when paired with latent prediction in DET-JEPA—achieves superior and reliable performance.


Weakness 3 + Question 1. Contribution of the JEPA branch

Concern: What does latent prediction add beyond standalone PCG-BCE, given the small, statistically inconclusive gains?

While differences between DET-JEPA and PCG-BCE only in cell-type classification and protein prediction do not reach individual significance over the initial five runs, mean performance broadly favors DET-JEPA (with Great Apes as the exception). Non-significance does not establish the absence of an effect, as statistical power depends on the run count (n=5n=5) and task discriminability. We substantiate the role of latent prediction via two targeted analyses:

1. Expanded scIB significance test (Appendix E.4, Figure 7): Adding four independent runs per method expands the comparison to 9 runs per method (Appendix E.4). Two-sided unpaired Welch’s tt-tests with Holm correction yield adjusted pp-values of 0.0167, 0.0207, and 0.0167 (with pooled-SD Cohen’s dd of 1.64, 1.28, and 1.62) for Bio, Batch, and Total, respectively—confirming that all three integration metrics significantly favor DET-JEPA (p<0.05p < 0.05) with large effect sizes.

2. Disentangling latent prediction from VISReg (Tables 7 & 8): Adding a PCG-BCE + VISReg control (removing latent prediction while retaining VISReg) confirms that VISReg alone fails to reproduce DET-JEPA's gains (full means and SDs in Table 8):

MethodscIB BioscIB BatchscIB Total
PCG-BCE only0.61980.33620.5063
PCG-BCE + VISReg (no latent pred.)0.61470.34080.5051
DET-JEPA (full)0.63050.35350.5197

Moreover, relative to PCG-BCE + VISReg, restoring latent prediction substantially boosts OOD Macro-F1 across HiT (0.5504→0.75250.5504 \rightarrow \mathbf{0.7525}), NN (0.1495→0.19270.1495 \rightarrow \mathbf{0.1927}), and Great Apes (0.1852→0.25300.1852 \rightarrow \mathbf{0.2530}) (Table 7), isolating latent prediction as the key driver of improvement.

Weakness 4 + Question 3. Complete cell-type and batch UMAPs

Request: Provide UMAPs with complete annotations for all 40 cell types and 9 batches.

Thank you for requesting these complementary views. In Appendix C.2, we have updated the cell-type-colored UMAP to include a complete legend for all 40 cell types (Figure 5) and added the corresponding batch-colored UMAP with a complete legend for all 9 batch IDs (Figure 6).


Weakness 5 + Question 2. Weighting controls and sequencing depth

Request: Add normalized-weight controls and test robustness under sequencing-depth variation or downsampling.

We added both a matched-weighting reconstruction control and a sequencing-depth perturbation experiment:

1. Weighting control (Tables 7 & 8): PCG-MAE applies the exact same per-cell detection balancing and gene-entropy weighting to continuous expression reconstruction with MSE. Standalone PCG-BCE still outperforms PCG-MAE by +0.1598, +0.0322, and +0.0835 Macro-F1 on HiT, NN, and Great Apes, respectively, and by +0.0098 on scIB Total. Thus, transferring the weighting scheme to magnitude reconstruction does not recover the performance of detection prediction.

2. Depth-perturbation robustness (Appendix E.3, Table 10): Prior to forming pretraining views and targets, we Poisson-resample normalized pseudo-counts with per-cell scaling factors ri∼Uniform(0.1,1.0)r_i \sim \mathrm{Uniform}(0.1, 1.0), while keeping fine-tuning and evaluation RNA unperturbed. Resulting metric shifts on multi-omic protein prediction are:

ObjectiveΔ\Delta Pearson (↑\uparrow)Δ\Delta Spearman (↑\uparrow)Δ\Delta MSE (↓\downarrow)
PCG-BCE−0.0002−0.0003+0.0010
PCG-MAE−0.0005−0.0008+0.0019

PCG-BCE exhibits smaller degradation across all three metrics. As analyzed in Appendix D, binarized detection eliminates within-positive magnitude variation, whereas sensitivity to the loss of gene detection depends on underlying expected counts.


Weakness 6. Stop-gradient, decoder activation, and Figure 1

Request: Clarify stop-gradient, the final decoder activation, and the proposed components in Figure 1.

  • Stop-gradient (Section 3.1, Eqs. 1–2): We updated Section 3.1 to explicitly define stop-gradient, which acts as an identity mapping in the forward pass with zero derivative in the backward pass. In latent prediction (Eq. 1), it detaches the global target zi0z_{i0} so that gradients flow solely through the local-view predictions—this does not freeze the shared encoder, but rather stabilizes training against representation collapse.
  • Decoder activation (Section 3.2, Appendix A.2): We clarified Section 3.2 and corrected the overly broad statement ("all layers use ReLU") in Appendix A.2. Specifically, the detection decoder applies ReLU only in its hidden layers; its final layer is strictly linear and outputs unconstrained logits, which are subsequently mapped via a sigmoid function to compute the BCE loss in Eq. (9).
  • Scope of Figure 1: Figure 1 illustrates the unified design space that we formalize for single-cell JEPAs—directly reflecting our first contribution. Because prior studies conflate architectural and objective choices without clearly delineating component roles, this conceptual framework anchors our controlled benchmark and guides future single-cell JEPA design, while our specific PCG-BCE formulation is detailed in the equations immediately following the overview.

We hope that these new experimental controls and manuscript revisions satisfactorily address your concerns, and we would deeply appreciate it if you would consider raising your evaluation in light of this strengthened evidence. Please do not hesitate to let us know if any points require further clarification or discussion!

Thank you for the additional experiments and clarifications. The new ablations and statistical analyses address most of my earlier concerns. However, I still have three points:

  1. Limited methodological novelty: PCG-BCE is a useful adaptation to single-cell data, but it mainly combines binary detection prediction with existing weighting strategies. I still find the methodological contribution somewhat incremental.

  2. Contribution of the JEPA branch: The new experiments provide stronger evidence for the benefit of latent prediction, particularly on scIB. However, PCG-BCE alone remains a strong baseline, and the additional gains from JEPA are relatively small and task-dependent. The results do not fully establish why the dual-branch design should be preferred more generally.

  3. Experimental consistency: The loss coefficients reported in the rebuttal appear inconsistent with those in the original manuscript. Please clarify which configurations were used for the nine runs, particularly if results from different settings were combined.

I appreciate the authors' efforts in addressing my concerns and have raised my score from 1 to 2. While the experimental evidence is now more convincing, my main concern about the limited methodological novelty remains, which prevents me from raising the score further.

We sincerely thank the reviewer for the prompt follow-up, for recognizing our new experiments, and for raising the score! We truly appreciate your thoughtful engagement and address your three remaining points below.


1. Positioning of Contributions & Scientific Novelty

We appreciate and understand your perspective on architectural novelty. Rather than positioning DET-JEPA as a complex, entirely new architecture that "solves" single-cell representation learning, our primary contribution is formalizing a unified design space and rigorous controlled attribution.

In modern AI research, assembling complex composite methods has become increasingly easy, yet bundled designs often obscure what genuinely drives performance. Stepping back to first principles, disentangling collapse prevention, feature preservation, and latent prediction is urgently needed to establish solid scientific takeaways for single-cell SSL (as highlighted in our General Response). Within this design space, we intentionally designed PCG-BCE to be a simple but effective adaptation tailored to noisy single-cell counts. As you noted, the fact that such a clean formulation stands as a strong baseline in its own right directly validates our core insight: coarse-grained binary detection filters out count-magnitude noise far better than standard reconstruction.


2. Why the Dual-Branch Design Should Be Generally Preferred

Beyond empirical numbers, there is a clear mechanistic complementarity explaining why the dual-branch design is generally preferable over standalone PCG-BCE:

  • Direction vs. Magnitude: Across benchmarks, adding the JEPA branch yields consistent directional gains; what is task-dependent is the magnitude of improvement rather than whether it helps, meaning the dual-branch design improves robustness without hurting single-modality tasks.
  • Complementary Mechanisms (Bio Conservation vs. Batch Mixing): Appendix Table 3 clearly illustrates why both branches are needed on multi-batch data. Standalone PCG-BCE ranks near the very top in Bio conservation, yet on Batch correction—while outperforming MAE—it lags behind every variant equipped with latent prediction + regularization (even single-branch JEPA without preservation). Mechanistically, explicit feature preservation retains fine-grained biological identity, whereas JEPA latent prediction enforces cross-view invariance to perturbations, which is crucial for mixing technical batches. Because real-world single-cell atlases inevitably suffer from batch effects, combining both branches is essential in practice.

3. Clarification on Experimental Consistency

Thank you for catching this detail! To clarify unequivocally: for each method, all 9 runs (the 5 original runs + 4 additional runs) were executed using the exact same training script and identical loss-coefficient configuration, we did not pool results from different hyperparameter settings. Specifically, because both methods share our unified codebase parameterized by the loss weights (latent prediction, VISReg, PCG-BCE), our earlier phrasing ("two loss-weight settings") referred to the two compared methods themselves—(1.0, 0.8, 1.0) for DET-JEPA and (0.0, 0.0, 1.0) for standalone PCG-BCE rather than two hyperparameter variants of DET-JEPA. We noticed the transcription error in Appendix Table 2 and have corrected it in the updated version. We apologize for any confusion and thank you for your careful reading!


Summary

We fully agree that DET-JEPA builds thoughtfully upon existing SSL components, and as discussed in Section 5, developing stronger domain-specific augmentations and preservation targets still remains an exciting open avenue. By demystifying the single-cell JEPA design space, introducing a simple, domain-tailored objective (PCG-BCE), and demonstrating how preservation and latent invariance complement one another, we hope our study provides a clear, actionable foundation that meaningfully benefits the single-cell SSL community. Thank you again for your constructive feedback throughout the review process!

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for their constructive feedback and for recognizing the value of our controlled study. Your suggestions directly guided targeted new experiments and manuscript revisions that substantially strengthen our empirical evidence and clarity. Below, we briefly summarize (1) our core positioning, (2) new shared experiments, and (3) manuscript revisions.


1. Core Positioning: Why Controlled Attribution Matters Today

In modern AI research, proposing composite methods has become increasingly easy, yet a proliferation of bundled designs often obscures fundamental mechanisms and masks where performance gains truly originate. Stepping back to first principles, rigorous controlled studies are essential to disentangle confounding factors and isolate what genuinely drives performance.

In single-cell JEPA specifically, prior works alter regularizers, preservation targets, and protocols simultaneously. By holding the encoder and protocol fixed, our study isolates these choices, motivates our streamlined PCG-BCE objective and DET-JEPA framework, and distills three key takeaways:

  1. A Unified Design Space: We formalize the single-cell JEPA design space, separating collapse prevention, feature preservation, and latent prediction—establishing a clean basis for benchmarking and future design.
  2. Necessity of the Dual-Branch Paradigm: Pairing latent prediction (promoting cross-view invariance and batch mixing) with an explicit preservation branch (retaining biological signal) is empirically essential for single-cell representations.
  3. Coarse-Grained Supervision for High-Noise Data: Rather than fitting noisy continuous expression counts, binary gene-detection targets filter out within-positive magnitude noise and boost the supervision signal-to-noise ratio—exemplified by our simple, effective PCG-BCE branch.

2. Resolution of Shared Concerns

To address shared questions on statistical reliability, objective confounding, weighting, and depth robustness, we conducted four sets of new experiments:

  • Expanded scIB Significance Analysis (Appendix E.4, Fig. 7): While 5-run differences over standalone PCG-BCE in classification and protein prediction are not individually significant—reflecting limited power at n=5n=5 and task sensitivity despite higher DET-JEPA means on 8 of 9 endpoints—expanding scIB to 9 independent runs per method confirms statistically significant gains across Bio, Batch, and Total (padj=0.0167,d=1.64p_{\text{adj}}=0.0167, d=1.64; padj=0.0207,d=1.28p_{\text{adj}}=0.0207, d=1.28; and padj=0.0167,d=1.62p_{\text{adj}}=0.0167, d=1.62) via two-sided unpaired Welch’s tt-tests with Holm correction.
  • Isolating Latent Prediction from Regularization (Appendix E.1–E.2, Tables 6–8): In a PCG-BCE + VISReg control without latent prediction, restoring latent prediction boosts OOD Macro-F1 on HiT (0.5504 to 0.7525), NN (0.1495 to 0.1927), and Great Apes (0.1852 to 0.2530), as well as scIB Total (0.5051 to 0.5197)—confirming that gains stem from latent prediction rather than VISReg alone.
  • Matched-Weighting Control (Appendix E.1–E.2, Tables 7–9): Applying the exact same per-cell and gene-entropy weights to continuous MSE (PCG-MAE), standalone PCG-BCE still outperforms PCG-MAE by +0.1598, +0.0322, and +0.0835 OOD Macro-F1 and +0.0098 scIB Total, with higher multi-omic means—confirming that binary detection supervision itself drives the gain.
  • Sequencing-Depth Robustness (Appendix E.3, Table 10): Under Poisson downsampling (ri∼Uniform(0.1,1.0)r_i \sim \mathrm{Uniform}(0.1, 1.0)) during pretraining, PCG-BCE degrades less than PCG-MAE across all protein-prediction metrics: Δ\DeltaPearson (−0.0002-0.0002 vs. −0.0005-0.0005), Δ\DeltaSpearman (−0.0003-0.0003 vs. −0.0008-0.0008), and Δ\DeltaMSE (+0.0010+0.0010 vs. +0.0019+0.0019), consistent with our sampling analysis in Appendix D.

3. Summary of Manuscript Revisions

  • New Controls & Statistical Tests (Appendix E.1–E.4, Tables 6–10, Fig. 7): Added PCG-BCE + VISReg, PCG-MAE, Poisson depth-perturbed pretraining, and 9-run scIB evaluations.
  • Complete-Legend UMAPs (Appendix C.2, Figs. 5–6): Updated 15,000-cell UMAPs with full legends for all 40 cell types and all 9 tech_sample batch IDs.
  • Method Clarifications (Secs. 3.1–3.2, Appendix A.2): Defined stop-gradient in Eqs. (1)–(2) and clarified that the detection decoder uses a linear final layer outputting unconstrained logits before sigmoid BCE.

For ease of review at this stage, we placed the new results primarily at the end of the Appendix (highlighted in blue), and will integrate the essential parts into the main text in the final manuscript.


We thank all reviewers again for their insightful critiques, which made our paper substantially stronger and clearer! Point-by-point responses follow in each thread.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.