Generalizable Bilevel Knowledge Distillation from Vision Foundation Models for 3D Medical Segmentation

Generalizable 3D medical image segmentation is increasingly vital for clinical deployment, given cross-site distribution shifts and restricted access to target-site data. However, cross-site distribution shifts and restricted access to target-site data during training make generalization to unseen sites challenging. Vision foundation models (VFMs) offer transferable segmentation priors, but their computational demands limit direct deployment. Existing distillation approaches transfer these priors to lightweight students through teacher adaptation (TA) and student distillation (SD). However, optimizing these tasks primarily for source-domain fitting does not explicitly align knowledge transfer with the student’s cross-site generalization. We propose generalizable bilevel knowledge distillation (BiLD), a framework that couples lower-level TA with upper-level SD. A dynamic response strategy incorporates the response of teacher adaptation into student updates using a Gauss–Newton approximation and conjugate-gradient iterations. To support this coupled optimization without target-site data, we introduce a pseudo-site generator that constructs diverse appearance variations from source-domain volumes through acquisition-motivated perturbations while preserving anatomical labels. These pseudo-sites expose the upper-level distillation objective to simulated distribution shifts, encouraging knowledge transfer beyond source-domain matching.Experiments on brain and cardiac segmentation across five external datasets demonstrate improved zero-shot cross-site generalization, with mean out-of-domain Dice exceeding the strongest baseline in each task by 3.23% and 5.99%, respectively. Two target-domain fine-tuning experiments further utperforms the strongest baseline in each task by 2.25% and 3.37%in Dice for brain and cardiac, respectively. Code will be released upon acceptance.