TotiCell: Structured Masking over Full-Coverage Gene Blocks for Single-Cell Representation Learning

A masking ratio specifies how much expression is hidden during pretraining, but not which genes are hidden together. We introduce TotiCell, a compact single-cell representation model whose block tokenizer represents the full set of 21,912 genes—including genes with zero expression in a given cell—as a fixed sequence of 48 tokens, roughly 1/67 of the input length of the audited gene-level baselines. TotiCell is trained with GroupMask, a structured masking policy that groups the 48 blocks into three disjoint, exhaustive target sets and masks one entire set per cell. GroupMask preserves RandomMask's per-block masking probability and visible-token budget while replacing the combinatorial mask distribution with three recurring target sets. Trained on 50.5 million cells, GroupMask leads five of seven task means over RandomMask, most clearly in perturbation retrieval, where all three datasets favor it (mean MRR gain 0.0293). Scaling the recipe yields the highest annotation, perturbation-retrieval, and mapping means among the evaluated methods with complete task coverage, and exceeds CellVQ on five of seven task means with 55.5% fewer total parameters. The scaled model also achieves the best mean rank percentile among the nine complete-coverage methods. In a controlled single-RTX-3090 benchmark, the smaller architecture processes over 1,600 cells per second while using under 0.5 GiB of memory. Three-seed probes favor recurring mask sets and reduced zero-target loss weight, and characterize partition and block-count sensitivity.