- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies online test-time adaptation for 3D vision-language foundation models under distribution shift. The proposed MODE-TTA models each class-conditional feature distribution using a Gaussian Mixture Model and updates the mixture parameters online through a hierarchical EM procedure guided by soft zero-shot assignments.
The method also introduces residual text alignment to improve the consistency between class text embeddings and the learned target-domain feature distributions. Experiments on several corrupted and clean 3D datasets with Uni3D and ULIP-2 show consistent improvements over cache-based and Gaussian-based TTA baselines.
Critical strengths
-
The paper addresses a relevant problem and proposes a clear extension beyond prototype-based TTA by explicitly modeling multi-modal class-conditional feature distributions.
-
The empirical evaluation is broad, covering several corruption benchmarks, clean datasets, and two different 3D vision-language backbones. The ablation studies also show that the GMM modeling is the main source of improvement.
Critical weaknesses
-
The methodological novelty is somewhat limited. The main idea is to replace prototype or single-Gaussian modeling with a GMM and combine it with residual text adaptation. While reasonable and effective, the overall framework feels more like a natural extension of prior density-based TTA methods than a fundamentally new formulation.
-
The method relies on zero-shot predictions as soft assignments when updating the GMM. Under severe domain shift, these assignments may themselves be inaccurate, which could lead to error accumulation in the online density estimates. The paper would benefit from a clearer analysis of robustness to noisy pseudo-labels.
Questions for authors
-
How robust is MODE-TTA when the initial zero-shot predictions are unreliable? Is there any analysis showing how errors in the soft class assignments affect the online GMM updates?
-
Could the authors clarify the main conceptual difference from prior density-based TTA methods such as DOTA and FreeTTA, beyond using multiple Gaussian components per class?
-
How sensitive is the method to the number of mixture components M across different datasets and backbones?
Methodological Novelty, Pseudo-Label Robustness, and Mixture Sensitivity
Thank you for the constructive review. We appreciate the recognition of MODE-TTA's multimodal density modeling and empirical improvements. We address the concerns with additional experiments using Uni3D-L, severity 5, batch size 1, and M=8 unless explicitly varied for sensitivity analysis. Results are mean ± std over 3 stream orders × 15 corruptions, with matched features and streams.
W1/Q2 — Methodological novelty beyond DOTA and FreeTTA. The novelty of MODE-TTA goes beyond replacing a single Gaussian with a GMM. We propose a coupled probabilistic adaptation framework in which streaming density estimation supports both target-domain classification and cross-modal alignment:
The central contribution is the formulation of multimodal density estimation as a shared foundation for online classification and text adaptation, rather than simply combining existing adaptation components. Crucially, the learned distributions serve two complementary roles: probabilistic classification and supervision for cross-modal alignment. This formulation distinguishes MODE-TTA from simply combining GMM estimation with existing residual adaptation techniques.
W2/Q1 — Robustness to unreliable zero-shot assignments.
(a) Natural corruption. On SONN-C, zero-shot predictions are incorrect for approximately 62% of samples, yet MODE-TTA improves accuracy by 10.1 points in the additional evaluation. In Tables 1–3, MODE-TTA improves over Source-Only across all 45 corruption settings, including severe shifts (SONN-C Gaussian: 25.99→40.27; MN40-C LiDAR: 22.93→40.31).
W2/Q1 — Robustness to unreliable zero-shot assignments.
(a) Natural corruption. On SONN-C, zero-shot predictions are incorrect for approximately 62% of samples, yet MODE-TTA improves accuracy by 10.1 points. In Tables 1–3, MODE-TTA improves over Source-Only across all 45 corruption settings, including severe shifts (SONN-C Gaussian: 25.99→40.27; MN40-C LiDAR: 22.93→40.31).
(b) Injected assignment noise. We replace zero-shot responsibilities for a fraction p of samples with one-hot assignments to randomly selected incorrect classes:
Despite injected errors, MODE-TTA remains above zero-shot through 30% noise on SONN-C and 50% on MN40-C. Entropy-guided fusion mitigates degradation relative to the density branch alone.
(c) Soft vs. hard responsibilities.
Soft responsibilities improve accuracy by 7.74 / 2.58 points over hard assignments, reducing the impact of incorrect confident predictions. Oracle results quantify the remaining improvement potential. We will add these robustness analyses.
Q3 — Sensitivity to mixture components M. The main experiments use M=8. We additionally vary M∈{1,2,4,8,12} across Uni3D and ULIP-2 to assess sensitivity.
The results show that multimodal modeling is substantially better than a single Gaussian in the full Uni3D pipeline. For M≥2, accuracy remains relatively stable: within 1.5 points on Uni3D SONN-C, 0.8 on Uni3D MN40-C, and 1.2 on ULIP-2 MN40-C. The largest degradation at M=1 appears related to its interaction with residual alignment and progressive density weighting.
We will expand Fig. 5 with sensitivity results across datasets and backbones, clarifying the robustness of the selected M=8 configuration.