ReDi-TTA: Refinement-guided Distillation for Multi-Modal Test-Time Adaptation in 3D Semantic Segmentation

Multi-modal test-time adaptation (MM-TTA) enables 3D semantic segmentation models to adapt to deployment shifts using unlabeled target observations without access to source data. Existing methods exploit predictive reliability to construct adaptation supervision. However, reducing fused predictions to hard pseudo-labels can still preserve an incorrect class decision, even when the full distributions of base models and teachers retain complementary support for alternative classes. To address this shortcoming, we propose Refinement-Guided Distillation for Test-Time Adaptation (ReDi-TTA), which exploits class-wise evidence through probability refinement and role-decoupled student learning while retaining the base adaptation procedure. First, Cross-modal Class Probability Refinement (CCPR) reweights each base prediction using class-aligned probabilities from both exponential moving average teachers, producing branch-specific soft targets that preserve information beyond the highest-scoring class. Second, Role-Decoupled Bimodal Classification (RBC) introduces dedicated student heads operating on aligned image and point-cloud features. These heads are excluded from target construction and EMA teacher updates, preventing student optimization from altering the guidance generation pathway. Third, Proximal Soft-Target Distillation (PSD) addresses the evolving feature representations during base adaptation by fitting pre-update soft targets on detached features recomputed after the base update, while regularizing student-head changes with a proximal constraint. Together, these designs transform refined class-wise evidence into a persistent student learning signal, allowing future predictions to benefit from refinement rather than replacing current-batch outputs. Extensive experiments demonstrate that ReDi-TTA achieves state-of-the-art performance across multiple MM-TTA benchmarks.