Source-Free Test-Time Adaptation for Brain Tumor Segmentation by Jointly Handling Covariate and Label Shifts

In real deployment scenarios, covariate and label shifts typically co-occur, yet most Test-Time Adaptation (TTA) methods account only for the covariate shift. This limitation arises primarily because covariate shift adaptation assumes a stationary label distribution, whereas label shift correction presumes a fixed class-conditional input distribution. Consequently, neither approach is directly applicable when both shifts are simultaneously present. Moreover, the label shift is inferred from model predictions, while the covariate shift distorts these predictions, making accurate label shift correction feasible only after the covariate shift has been effectively mitigated. We further observe that the label shift is particularly detrimental in brain tumor segmentation, where the evaluation regions are hierarchically nested and any deviation in a class propagates to every region containing that class, thereby amplifying the shift in the predictions toward the source-domain label distribution. We term this effect *hierarchical label shift*. Based on the above analysis, we propose **H**ierarchical-**R**estoring **A**daptation and **C**orrection (HRAC), a source-free TTA framework that jointly handles the covariate shift and the hierarchical label shift in an *adapt-then-correct* manner by explicitly structuring adaptation and correction around the nested evaluation regions. Our HRAC comprises two synergistic components: (i) **S**tructure-**C**onsistency **A**daptation (SCA), which mitigates the covariate shift by restoring the spatial structure it corrupts, with consistency losses imposed across the nested regions; and (ii) **H**ierarchical **L**abel **S**hift **C**orrection (HLSC), which corrects the hierarchical label shift by applying a bounded logit bias to the violated region ratios. To the best of our knowledge, HRAC is the first source-free method to jointly mitigate the covariate shift and correct the hierarchical label shift without estimating a target prior. Extensive experiments on three BraTS benchmarks show that HRAC outperforms seven state-of-the-art adaptation baselines in mean Dice, mean HD95, and mean IoU, demonstrating its effectiveness and strong generalization in brain tumor segmentation.