Balancing Invariance and Information Preservation in Single-Cell JEPA

Self-supervised learning for single-cell genomics must balance invariance to nuisance variation against preservation of biological information. Existing single-cell Joint-Embedding Predictive Architecture (JEPA) adaptations change backbones, anti-collapse mechanisms, and auxiliary objectives at the same time, so it is unclear which choice actually drives transfer. We therefore fix the encoder, view construction, training budget, and evaluation protocol, and vary only two axes: the anti-collapse regularizer and the observation-space preservation target. Within this setting we introduce PCG-BCE, a per-cell-balanced and gene-entropy-weighted binary cross-entropy loss that supervises gene detection rather than expressionmagnitude, and DET-JEPA (detection-aware JEPA), which couples PCG-BCE with projector-space latent prediction. Three findings emerge. First, preventing collapse is not sufficient: variants trained with latent prediction alone, including one built on a provably collapse-free regularizer, fall below both a plain masked autoencoder and PCA on zero-shot cell-type annotation and on scIB biological conservation. Second, which observation-space target is preserved matters more than whether one is used: detection prediction outperforms magnitude reconstruction on every out-of-distribution cohort, consistent with a sampling analysis showing that detection labels are insensitive to positive-count magnitude. Third, PCG- BCE alone is already a strong representation learner; adding latent prediction improves 11 of 12 downstream metrics, but each gain is small and not individually significant over five seeds. Across zero-shot cell-type annotation, multi-batch integration, and protein-abundance prediction, DET-JEPA is the best matched JEPA variant, with VISReg as the preferred regularizer. Our study localizes where the gains in single-cell JEPAs come from and gives concrete guidance for designing preservation objectives.