Attribution-based diagnostics for single-cell foundation models

Single-cell RNA foundation models (scFMs) are increasingly used for _in silico_ biological discovery, yet predictive performance alone does not reveal whether their decisions rely on biologically meaningful signals or model-specific shortcuts. We present a systematic post-hoc attribution study of downstream predictions in two applications: cell-type annotation with pretrained Cell2Sentence and scGPT models, and a controlled scGPT+STATE perturbation-prediction case study. For cell annotation, our analyses connect predictions to established marker genes while examining how gene relevance varies with model architecture, expression rank, and the surrounding expression profile. In a synthetic perturbation prediction setting, we compare attribution patterns with known regulatory ground truth to assess what they reveal about the model's learned computations. Across these tasks, attribution exposes biological feature use, higher-order dependencies, pathway-specific computation, learning progression, prediction reliability, and potential failure modes that are not fully captured by aggregate output metrics. Experiments with complete and sparse regulatory knowledge further show that the conclusions drawn from attribution depend critically on the chosen biological reference, baseline, and null model. Together, these results establish attribution as a flexible post-hoc diagnostic for understanding, debugging, and guiding the development of scFMs.