acceptodds
Under review as a conference paper at ICLR 2027

Class Topology Survives Corruption: Prior, Monitor, and Repair for 3D Continual Test-Time Adaptation

Abstract

Continual test-time adaptation (CTTA) aims to adapt a source-trained model to continuously evolving target distributions without access to source data, target labels, or domain-boundary information. Most existing CTTA methods rely on computationally expensive mean-teacher frameworks. Moreover, they often require explicit distribution-shift detection and treat each target distribution independently, limiting their ability to exploit information shared across domains. In this work, we show that, despite substantial accuracy degradation, the relational topology among classes remains remarkably stable across diverse 3D corruptions. Motivated by this observation, we propose **To**pology-Anchored **Co**ntinual **T**est-**T**ime **A**daptation (ToCoTTA), which leverages this class topology as a stable structural anchor throughout continual adaptation. Specifically, ToCoTTA derives a reference class topology from the source classifier and uses it in three complementary ways: as a **prior** to refine noisy online class prototypes, as a **monitor** to quantify the reliability of ongoing adaptation, and as a **repair** signal to regulate updates and recover from accumulated parameter drift. By providing a unified structural signal for both prediction and adaptation, class topology enables ToCoTTA to remain robust under continually evolving target distributions. Extensive experiments on multiple 3D point cloud benchmarks demonstrate consistent improvements over existing CTTA methods with negligible computational overhead.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Continual Test-Time Adaptation, Point Cloud Recognition

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_6wM7OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ToCoTTA, a framework for continual test-time adaptation of 3D point-cloud classifiers. The core contribution is the use of a classifier-derived class topology as a prior for online prototype refinement, a monitor of adaptation reliability, and a signal for parameter restoration. The refined prototypes support prediction and entropy minimization, while the topology-based trust score regulates adaptation. Experiments on ModelNet40-C, ScanObjectNN-C, and ShapeNet-C report improved mean accuracy across three backbones and stable performance over ten corruption rounds.

Critical strengths
  1. Using the same class-affinity reference for prototype refinement, monitoring, and restoration provides a coherent design for addressing error accumulation. Table 5 shows that preserving relative class relationships performs better than directly anchoring prototypes to source classifier weights, supporting this design choice.
  2. The experiments cover both synthetic and scanned objects, with the highest reported mean accuracy in all nine dataset–backbone combinations in Tables 11–13. The long-horizon ablations in Table 7 further demonstrate the practical value of the repair components, particularly on ShapeNet-C.
Critical weaknesses

1: The monitor needs stronger validation: Figure 2 uses true-label prototypes, while Figure 4’s domain-level correlation does not establish whether online estimates detect harmful updates within a domain. The authors should test limited class coverage and compare against entropy gating and matched fixed restoration. 2: The throttling mechanism needs clarification under AdamW, which can offset loss scaling and retain momentum-driven updates at zero trust. The authors should report actual update magnitudes and explain optimizer-state handling during restoration. 3: The small SONN-C gains (0.93 and 0.50 percentage points) lack paired uncertainty estimates; Table 10 only reports the variability of ToCoTTA. The authors should compare with the strongest baseline values and report the paired gains along with their uncertainty intervals.

Questions for authors
  1. What additional functionality does ToCoTTA provide beyond TCA's inter-class topology constraints? Comparing reference construction and the three-role design, together with a matched 3D adaptation of TCA, would make this distinction clearer.
  2. Can online topology estimates reliably identify harmful updates when class coverage is incomplete? Comparing them with true-label topology and testing within-domain update quality against entropy gating and matched fixed restoration would provide more direct evidence for the monitor's reliability.
  3. Under AdamW, how does multiplying the entropy loss by g affect the actual update magnitude as g decreases? Do the authors skip optimizer steps or modify its accumulated moments at g=0 or during parameter restoration, and could they provide an analysis of the resulting parameter drift?
6 comments
AuthorsOpenReview ↗

Response to Reviewer 6wM7 (1/4)

We thank the reviewer for recognizing the coherent design of our topology-based framework and its consistent performance across datasets and backbones. We also appreciate the constructive suggestions on validating the topology-based monitor and repair mechanisms, which have motivated us to conduct additional experiments and strengthen our empirical analysis.

W1-1. Ground-truth prototypes in Figure 2

Figure 2 uses ground-truth labels solely to demonstrate the stability of class topology under corruption, motivating its use as a reference. During adaptation, we approximate this reference using source classifier weights and monitor reliability by comparing it with online prototype topology, without using ground-truth labels.

W1-2 and Q2. Within-Domain Reliability of the Topology Monitor

Our topology-based monitor is designed to assess the structural reliability of the evolving online representation, rather than to identify harmful individual gradient updates.

While Fig. 5 (left) already presents batch-level traces of the trust score and accuracy, we agree that the domain-level correlation in Fig. 5 (right) does not establish within-domain reliability. To address this, we conduct an additional within-domain topology reliability analysis, included in Appendix A.3 of the revised manuscript.

Specifically, we construct an oracle prototype bank using ground-truth labels solely for evaluation and define topology estimation quality as qt=corr(Aμt,AGT,t)q_t = \text{corr}(A_{\mu_t}, A_{\text{GT},t}). We compare this with the raw monitor score rt=corr(Aμt,Aref)r_t = \text{corr}(A_{\mu_t}, A_{\text{ref}}). To isolate within-domain variations, we subtract the domain-wise mean from each score and compute correlations over pooled batches. On ShapeNet-C, we observe consistent positive Pearson correlations of 0.61, 0.59, and 0.63 for Point-MAE, DGCNN, and PointNet, respectively. These results suggest that the proposed monitor is informative of within-domain variations in online topology estimation quality, beyond domain-level differences. Furthermore, Tables 6 and 8 demonstrate the practical utility of this reliability signal in controlling adaptation through throttling and restoration.

Table 12: Within-domain correlation between the raw monitor score rtr_t and the online topology quality qtq_t on SN-C.

DatasetModelPearson
SN-CPointMAE0.61
DGCNN0.59
PointNet0.63

W1-3. Limited Class Coverage

To evaluate the effectiveness of topology-based monitoring under limited class coverage, we conduct additional experiments on non-IID test streams using Dirichlet sampling (δ=0.1\delta=0.1), following RoTTA [1]. As shown in Tab. 13, removing topology-based throttling (gTg^T) or restoration control (gRg^R) consistently reduces accuracy across all three datasets, with additional degradation when both are disabled. Although the overall performance of ToCoTTA degrades substantially under non-IID streams, as acknowledged in our Limitations section (Sec. 6), these results indicate that topology-based monitoring remains beneficial for regulating adaptation even under limited class coverage.

[1] Robust Test-Time Adaptation in Dynamic Scenarios, CVPR23

Table 13: Effect of gg under non-IID streams.

MethodMN-CSONN-CSN-C
ToCoTTA59.6735.9560.22
−gT−g^T58.9834.9759.16
−gR−g^R58.7534.8060.01
−gT−gR−g^T−g^R58.2734.4958.99

W1-4 and Q2. Entropy Gating and Matched Fixed Restoration

We compare our topology-based monitoring and adaptive restoration against entropy gating and matched fixed restoration, respectively. For entropy gating, we replace the topology-based trust score gg in both throttling and restoration with normalized entropy-based confidence. For matched fixed restoration, we use a constant kˉ\bar{k} equal to the average restoration coefficient from a full ToCoTTA run on the same test stream. This oracle setting matches the overall restoration strength while removing temporal adaptivity, although the oracle coefficient is unavailable at test time.

ToCoTTA consistently outperforms both alternatives across all three datasets. The improvement over entropy gating supports the effectiveness of topology-based reliability monitoring, while the improvement over matched fixed restoration highlights the benefit of temporally adaptive restoration.

We have included these additional analyses and comparisons in the revised manuscript (Tables 9, 12, and 13).

Table 9: Comparison with alternative monitoring and restoration strategies.

MethodMN-CSONN-CSN-C
Entropy gating72.5649.6068.16
Matched fixed kˉ\bar{k}73.3248.9467.44
ToCoTTA74.3850.2468.56
AuthorsOpenReview ↗

Response to Reviewer 6wM7 (2/4)

W2-1 and Q3. Effect of loss scaling under Adam.

We acknowledge that Adam's adaptive normalization can partially offset gradient scaling, particularly when the scaling factor remains constant. However, as illustrated in Fig. 5 (left), our trust score gg varies throughout adaptation, causing the current gradient scaling to differ from the historical scaling accumulated in Adam's moment estimates. Consequently, the effect of trust-based gradient scaling is not necessarily canceled by Adam's adaptive normalization. To quantify the actual effect, we measure the parameter change induced by each optimizer step on SN-C, which has the longest test stream among our benchmarks. As shown in Tab. 14, trust-based throttling reduces the actual optimizer update magnitude by 51.1% when g<0.2g < 0.2, while having little effect when g≥0.8g \geq 0.8. Furthermore, the ablations in Tables 6 and 8 show that removing throttling degrades adaptation performance, supporting its practical effectiveness despite Adam's adaptive normalization.

Table 14: Mean ∥Δθ∥2\Vert \Delta \theta \Vert_2 (×10−2\times 10^{-2}) per optimizer step, with and without trust-based throttling, excluding parameter restoration. Results are averaged over batches within each trust range.

ggLent\mathcal{L}_{ent}g⋅Lentg \cdot \mathcal{L}_{ent}
<0.2< 0.29.44.6
0.2−0.80.2-0.85.44.6
≥0.8\geq 0.83.93.8

W2-2 and Q3. Momentum-driven updates at zero trust.

We acknowledge that Adam can still produce momentum-driven updates when g=0g=0 due to its accumulated optimizer state. We do not skip optimizer steps or reset the accumulated moments when g=0g=0. In our implementation, we use Adam with zero weight decay, eliminating weight-decay-induced parameter updates. Importantly, when gg is low, the current loss gradient is scaled down, reducing its contribution to the optimizer's momentum estimate, although previously accumulated momentum may still induce parameter updates. Meanwhile, the restoration coefficient increases as gg decreases, reaching its maximum value (k=αk=\alpha) when g=0g=0, thereby pulling the adapted parameters toward their source values to counteract accumulated drift.

W2-3 and Q3. Optimizer-state handling during restoration.

In our implementation, we keep the optimizer state unchanged during parameter restoration. To examine whether retaining the optimizer state affects adaptation performance, we compare our approach (Keep) with two alternatives: Scale, which scales the first-moment estimate by (1−k)(1-k) during restoration (m←(1−k)mm \leftarrow (1-k)m), and Reset, which resets it to zero (m←0m \leftarrow 0). As shown in Tab. 15, Keep achieves the highest accuracy (68.56%), outperforming Scale (67.86%) and Reset (66.56%). Keep and Scale exhibit similar parameter drift, whereas Reset substantially reduces drift but yields lower accuracy, indicating that minimizing parameter drift alone does not necessarily improve adaptation performance.

We have added these results to the revised manuscript (Tables 14 and 15).

Table 15: Effect of optimizer-state handling during restoration on SN-C. Drift ∥θt−θ0∥2\Vert \theta_t - \theta_0 \Vert_2 is measured at each batch after restoration (mean / max / final over the stream).

MethodAcc (%)$\Vert$$\theta_t - \theta_0 \Vert_2$ (mean/max/final)
Reset66.562.36 / 6.80 / 0.22
Scale67.865.34 / 8.35 / 2.02
Keep (Ours)68.565.48 / 8.53 / 2.31
AuthorsOpenReview ↗

Response to Reviewer 6wM7 (3/4)

W3. Statistical Significance of Paired Improvements on SONN-C

To assess the statistical reliability of the improvements on SONN-C, we conducted additional experiments across 10 random seeds and 10 random domain orders, comparing ToCoTTA against the strongest baseline for each backbone. We report the mean accuracy, standard deviation, and paired accuracy gains with 95% confidence intervals computed using paired tt-tests.

As shown in Tables 20 and 21, ToCoTTA consistently outperforms the strongest baselines across all three backbones under both evaluation settings. For random seeds, the paired gains are +0.93, +1.52, and +0.57 percentage points for Point-MAE, DGCNN, and PointNet, respectively. For random domain orders, the corresponding gains are +1.21, +0.77, and +0.57 percentage points. Importantly, all 95% confidence intervals exclude zero, indicating that the improvements are statistically significant even when the absolute gains are relatively modest.

These results demonstrate that ToCoTTA consistently outperforms the strongest baselines on SONN-C across random seeds and domain orders, with statistically significant paired improvements.

We have added these results to the revised manuscript (Tables 20 and 21).

Table 20: Comparison of ToCoTTA with the strongest baseline on SONN-C across 10 random seeds (mean ±\pm std). Δ\Delta denotes the mean accuracy improvement over the baseline, with 95% paired-tt confidence intervals (n=10n=10 seeds).

ModelBaselineBaseline Acc (%)ToCoTTA Acc (%)Δ\Delta95% CI
Point-MAELN-TTA49.32 ±\pm 0.350.26 ±\pm 0.1+0.94[0.67, 1.20]
DGCNNCoTTA49.02 ±\pm 0.450.54 ±\pm 0.3+1.52[1.19, 1.84]
PointNetCoTTA39.96 ±\pm 0.340.53 ±\pm 0.2+0.57[0.35, 0.80]

Table 21: Comparison of ToCoTTA with the strongest baseline on SONN-C across 10 random domain orders (mean ±\pm std). Δ\Delta denotes the mean accuracy improvement over the baseline, with 95% paired-tt confidence intervals (n=10n=10 orders).

ModelBaselineBaseline Acc (%)ToCoTTA Acc (%)Δ\Delta95% CI
Point-MAELN-TTA49.13 ±\pm 0.250.34 ±\pm 0.3+1.21[0.98, 1.43]
DGCNNCoTTA49.66 ±\pm 0.350.43 ±\pm 0.3+0.77[0.41, 1.13]
PointNetCoTTA40.06 ±\pm 0.340.63 ±\pm 0.3+0.57[0.17, 0.97]
Reviewer_cuc4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes ToCoTTA, a method for 3D continual test-time adaptation motivated by the observation that inter-class topology remains relatively stable even under corruptions of 3D point clouds. It uses a fixed reference topology derived from the source classifier weights for prototype refinement, adaptation reliability monitoring, and parameter restoration. Across three 3D corruption benchmarks, the method demonstrates higher accuracy and computational efficiency than existing approaches.

Critical strengths
  1. An interesting observation and a reasonable methodological motivation. The observation that relative relationships among classes remain preserved even when corruptions substantially degrade classification accuracy is meaningful. Building on this observation without requiring additional source data or a separate teacher model is a reasonable design choice for source-free CTTA.

  2. Consistent performance improvements across a broad range of experiments. The method improves upon existing approaches across three point cloud datasets and multiple backbones. In addition to ablation studies of individual components, the paper provides analyses of long-horizon adaptation, domain order, and hyperparameter sensitivity, offering a reasonably thorough evaluation of the proposed method’s effectiveness.

  3. Strong computational efficiency. The method performs adaptation using a single model, without a separate teacher model or explicit distribution-shift detection. Achieving higher accuracy while maintaining throughput comparable to TENT is a practical advantage.

Critical weaknesses
  1. The novelty relative to prior work and the contributions specific to 3D data are unclear. TCA [1] has already addressed inter-class topology consistency in CTTA, while FedPAT [2] has explored the stability of prototype affinity topology and the use of structural priors. Simply applying the proposed approach to 3D data is unlikely to provide sufficient novelty and contribution.

  2. The actual effectiveness of topology preservation is insufficiently analyzed. Table 5 demonstrates the contribution of the topology prior to performance, but it does not directly examine how effectively the proposed regularization preserves inter-class topology during adaptation.

  3. The design rationale for topology regularization and reliability monitoring is insufficient. Using Spearman correlation, the authors observe that the rank ordering of inter-class affinities is preserved. However, the proposed method uses KL divergence and Pearson correlation. These choices do not follow directly from the observation and therefore require further justification. The ablation study in Table 5 likewise demonstrates the effectiveness of the topology prior but does not establish the validity of these specific design choices.

[1] Ni, Chenggong, et al. "Maintaining consistent inter-class topology in continual test-time adaptation." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2025.

[2] Guo, Shunxin, et al. "FedPAT: Federated Test-Time Adaptation via Prototype Affinity Topology." Forty-third International Conference on Machine Learning. 2026.

Questions for authors

Please address the weaknesses.

8 comments
AuthorsOpenReview ↗

Response to Reviewer cuc4 (1/4)

We thank the reviewer for recognizing the significance of our core observation, the consistent experimental improvements, and the computational efficiency of ToCoTTA. We also appreciate the insightful comments on novelty, topology preservation, and metric selection, which have motivated us to clarify our contributions and provide additional analyses.

W1. Clarification of Novelty and Distinctions from TCA and FedPAT

We acknowledge that inter-class relationships have been explored in prior work, and our contribution is not the introduction of class topology itself. Rather, we investigate its stability under diverse 3D point cloud corruptions and demonstrate how this preserved structure can serve as a unified signal for prediction and continual adaptation. In particular, we show that class topology remains highly correlated with its clean counterpart even when classification accuracy degrades substantially (Fig. 2), motivating its use as a structural anchor in challenging 3D CTTA settings.

Distinctions from TCA

While both TCA and ToCoTTA leverage inter-class relationships, they differ fundamentally in what they aim to preserve.

TCA employs an inter-class uniformity loss log⁡∑i<je−t∥pi−pj∥22\log\sum_{i<j} e^{-t\Vert \mathbf{p}_i-\mathbf{p}_j \Vert_2^2} that encourages class centroids to spread apart by minimizing a Gaussian potential over pairwise distances. However, this objective does not use a reference topology or explicitly constrain the relative relationships among classes.

In contrast, ToCoTTA explicitly preserves source-specific relational topology through a KL-based regularizer that aligns the refined prototype affinities with a fixed reference derived from the source classifier weights. Unlike direct anchoring to source prototypes, this regularization constrains inter-class relations while allowing prototypes to follow the evolving target distribution.

To empirically examine this distinction, we add a comparison with TCA's inter-class uniformity loss in Tab.5 of the revised manuscript. Specifically, we replace our KL-based topology prior with TCA's separation objective within the same prototype refinement framework. Across MN-C, SONN-C, and SN-C, the TCA-based regularizer achieves 73.19%, 49.39%, and 67.50%, respectively, whereas our topology prior achieves 74.38%, 50.24%, and 68.56%.

These consistent improvements demonstrate the practical benefit of preserving source-specific inter-class relations over merely encouraging centroid separation.

Table 5: Comparison of topology prior designs.

R(P)R(\mathbf{P})SFMN-CSONN-CSN-C
None✓72.2948.9166.42
∥P−W^∥F2\Vert \mathbf{P}-\hat{W} \Vert_F^2✓72.1948.4266.74
log⁡∑i<je−t∥pi−pj∥22\log\sum_{i<j} e^{-t\Vert \mathbf{p}_i-\mathbf{p}_j \Vert_2^2} (TCA)✓73.1949.3967.50
1−Pearson(Aref,AP)1 - \text{Pearson}(\mathbf{A}_{\text{ref}}, \mathbf{A}_{\mathbf{P}})✓74.3050.0468.24
KL(APS∥AP)\text{KL}(\mathbf{A}_{\mathbf{P}_S} \Vert \mathbf{A}_{\mathbf{P}})✗74.6150.4268.65
KL(Aref∥AP)\text{KL}(\mathbf{A}_{\text{ref}} \Vert \mathbf{A}_{\mathbf{P}}) (Ours)✓74.3850.2468.56
AuthorsOpenReview ↗

Response to Reviewer cuc4 (2/4)

Distinctions from FedPAT

While both FedPAT and ToCoTTA exploit stable inter-class topology, they differ in how it is obtained and utilized. Unlike FedPAT, ToCoTTA leverages topology not only as a structural prior but also to monitor reliability and regulate continual adaptation.

1. Source-free topology extraction vs. federated prototype aggregation. FedPAT constructs a global topology by aggregating labeled source-client prototypes and performing topology-guided validation refinement. In contrast, ToCoTTA extracts its reference topology directly from pretrained classifier weights, without requiring source data or additional training. Despite this simplicity, the classifier-derived topology closely approximates the source prototype topology (Spearman ρ=0.90\rho=0.90), providing a practical structural anchor for source-free continual adaptation.

2. Online topology-guided prototype refinement vs. posterior diffusion. FedPAT employs topology-guided validation refinement and uses fixed global prototypes with posterior diffusion for test-time inference. In contrast, ToCoTTA continuously refines online target prototypes using the reference topology, preserving inter-class relationships while allowing prototypes to follow evolving target distributions. These refined prototypes guide both prediction and adaptation, rather than solely refining prediction posteriors.

3. Structural prior vs. reliability-aware continual adaptation control. Most importantly, ToCoTTA leverages domain-shared topology not only as a structural prior but also as actionable feedback to monitor and regulate continual adaptation. Unlike FedPAT, which performs entropy minimization without topology-based reliability monitoring, ToCoTTA measures the agreement between evolving target and reference topologies to assess adaptation reliability. This signal suppresses unreliable updates and adaptively restores drifting parameters toward their source initialization, mitigating error accumulation and catastrophic forgetting. Our long-horizon experiments (Tab. 4) and component ablations (Tabs. 6 and 8) further demonstrate the effectiveness of this reliability-aware adaptation mechanism.

In summary, the key distinction lies not in the use of class topology itself, but in its role throughout continual adaptation. While FedPAT leverages cross-client structural consistency for federated adaptation, ToCoTTA exploits domain-shared topology as a unified Prior–Monitor–Repair mechanism for prototype refinement, reliability monitoring, and adaptive error correction.

We have revised Sec. 2.2 to discuss TCA and FedPAT and clarify their connections to our approach.

AuthorsOpenReview ↗

Response to Reviewer cuc4 (3/4)

W2. Effectiveness of topology preservation

Since the topology prior directly regularizes the refined prototypes P∗\textbf{P}^*, we evaluate how effectively it preserves their inter-class structure throughout continual adaptation. Specifically, we measure the Spearman correlation between the off-diagonal affinities of P⋆\textbf{P}^{\star} and those of a clean source oracle constructed from ground-truth class feature means of the source model. The oracle is used only for evaluation, and we compare refinement with (λ=10\lambda{=}10) and without (λ=0\lambda{=}0) the prior.

Table 7 shows that the prior consistently improves topology alignment with the clean source oracle across all three datasets. To isolate the prior's contribution, we additionally disable throttling (gTg^T) and restoration. Under this setting, the improvement becomes more pronounced, particularly on SN-C, where the correlation increases from 0.596 to 0.784 (+0.188). This demonstrates that the prior remains effective even when other mechanisms for limiting adaptation drift are removed. This table is added as Tab.7 in the revised manuscript.

Figure 4 of the revised manuscript illustrates how the prior preserves topology throughout the continual stream under the same setting. As adaptation proceeds, the correlation of the online prototype bank μ\mu drops sharply from 0.86 to 0.27, and the refined prototypes without the prior similarly degrade to 0.34. In contrast, applying the prior maintains a correlation above 0.72 across all 15 corruptions, despite substantial degradation in the input topology. Importantly, the prior is constructed solely from classifier weights Aref\textbf{A}_{\text{ref}}, yet the resulting prototypes remain highly aligned with the independently computed clean source oracle. Thus, the observed preservation is not merely agreement with the regularization target, but also reflects alignment with the inter-class structure independently measured from clean source data.

We have added Tab. 7 and Fig. 4 to the revised manuscript.

Table 7: Topology preservation of P∗\textbf{P}^* relative to the clean source oracle APS\textbf{A}_{\textbf{P}_S}.

SettingMN-CSONN-CSN-C
Full ToCoTTA
w/o Prior0.8930.6570.816
w/ Prior0.9430.7460.869
Δ\Delta+0.050+0.089+0.053
w/o Throttling & Restoration
w/o Prior0.8730.6790.596
w/ Prior0.9430.7470.784
Δ\Delta+0.070+0.068+0.188
Reviewer_eNCoOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates whether class topology can help stabilize continual test-time adaptation for 3D point cloud classification. The authors define “class topology” as pairwise cosine similarities between class prototypes and observe that these relationships remain relatively stable under corruption. Based on this observation, the authors propose ToCoTTA, which estimates a reference class structure from the pretrained classifier weights. (1, Prior) This reference is used to refine prototypes computed from unlabeled test data, which guide model updates through entropy minimization. (2, Monitor) The model measures how closely the class relationships in the test data match the reference. (3, Repair) When agreement is low, the method reduces adaptation updates and increases restoration toward pretrained parameters. The method uses a single model and updates only normalization parameters, without explicitly detecting domain changes. Across three corrupted point cloud datasets, the authors report accuracy improvements over the evaluated baselines.

Critical strengths
  1. The paper is very well written and structured, making the methodology easy to understand and follow.
  2. The method is simple and effective, with the reported results supporting its design. The reference class structure provides a shared signal for guiding prototype-based updates, assessing adaptation reliability, and triggering parameter restoration when needed.
Critical weaknesses
  1. The motivation does not appear sufficiently distinct from prior CTTA research, which has already considered (1) maintaining a teacher model updated through an exponential moving average, (2) using a domain detector, and (3) introducing modules that retain knowledge for different domains. Regarding (1), existing methods already address the memory overhead of maintaining additional model copies, including EcoTTA and BECoTTA [1, 3]. Regarding (2), explicit domain-transition detection is not required by methods such as NOTE [2]. Regarding (3), CTTA assumes that diverse, previously unseen domains may arrive during deployment. Under this assumption, a single shared model may not always be preferable to retaining domain-specific knowledge, as explored in BECoTTA [3]. It would be helpful for the authors to clarify which limitations of these approaches motivate their design.

  2. The reviewer has a similar concern about the methodological contribution. While the proposed combination is effective, prototype-based adaptation and weight regularization have already been explored in domain adaptation and TTA [4, 5]. Structural information has also been used to guide adaptation [6], although that work adopts a different definition of topology. The reported gains on 3D point cloud classification benchmarks are acknowledged, but a clearer explanation of the new insight beyond combining these established ideas would strengthen the paper.

  3. The effectiveness of prototypes in 3D point cloud classification is not particularly surprising. With many models already achieving high accuracy on standard classification benchmarks, further gains on this task alone may offer limited additional value. Prototype-based methods have also demonstrated their usefulness in image classification and domain adaptation [4, 5]. The reviewer would therefore find it more compelling to address challenges in practical settings, such as 3D object detection in scenes containing multiple objects. Developing and evaluating the method in such settings would help clarify its contribution beyond applying established ideas to point cloud classification.

[1] EcoTTA: Memory-Efficient Continual Test-Time Adaptation via Self-Distilled Regularization. CVPR, 2023.
[2] NOTE: Robust Continual Test-time Adaptation Against Temporal Correlation. NeurIPS, 2022.
[3] BECoTTA: Input-dependent Online Blending of Experts for Continual Test-time Adaptation. ICML, 2024.
[4] Improving Test-Time Adaptation via Shift-agnostic Weight Regularization and Nearest Source Prototypes. ECCV, 2022.
[5] Transferrable Prototypical Networks for Unsupervised Domain Adaptation. CVPR, 2019.
[6] Learning Topology-Aware Representations via Test-Time Adaptation for Anomaly Segmentation. TPAMI, 2026.

Questions for authors
  1. Could the authors clarify how the trust-score thresholds t0t_0 and t1t_1 were selected, including whether any labeled target data were used?
  2. During entropy minimization, are the refined prototypes treated as fixed targets, or do gradients propagate through the prototype-refinement step?
6 comments
AuthorsOpenReview ↗

Response to Reviewer eNCo (1/3)

We thank the reviewer for recognizing the clarity of our presentation and the simplicity and effectiveness of our approach. We also appreciate the constructive feedback on our motivation and methodological novelty, which has helped us better clarify the key distinctions between ToCoTTA and prior work.

W1-1. Distinction from Efficient CTTA Methods

We acknowledge that our emphasis on efficiency may have obscured our primary motivation, and we agree that prior methods, including EcoTTA and BECoTTA, have already addressed computational and memory overhead. Our motivation, however, stems from a different question: what structural knowledge remains stable across continually changing target distributions, and how can it be explicitly leveraged to guide adaptation? ToCoTTA addresses this question by identifying and exploiting the largely preserved class topology across diverse corruptions, rather than primarily focusing on reducing computational overhead.

While EcoTTA effectively reduces adaptation memory through lightweight meta networks and self-distilled regularization, it does not explicitly characterize the inter-class structure that remains stable across changing target domains. In contrast, ToCoTTA extracts a structural reference directly from the pretrained classifier without requiring additional source-data-dependent initialization and leverages it to guide both prediction and adaptation.

BECoTTA, on the other hand, improves efficiency through domain-adaptive low-rank experts and routing mechanisms. Although this design enables adaptive knowledge reuse, it still relies on additional expert modules and input-dependent routing. Conversely, ToCoTTA leverages a fixed, classifier-derived structural reference to guide continual adaptation, rather than relying on expert-based knowledge reuse.

The key distinction is that ToCoTTA identifies class topology that remains largely preserved across diverse corruptions and exploits it as a unified structural anchor for prototype refinement (Prior), adaptation reliability estimation (Monitor), and error mitigation through throttling and restoration (Repair). By leveraging this shared structural reference, ToCoTTA improves prediction and regulates continual adaptation without requiring domain-boundary detection or teacher–student frameworks.

W1-2. Distinction from domain-detection-free CTTA methods

We agree that explicit domain-transition detection is not necessary for all CTTA methods, as demonstrated by NOTE. Our contribution, however, is not simply avoiding domain detection, but identifying and explicitly exploiting structural knowledge shared across changing target distributions.

NOTE addresses temporal correlation in non-i.i.d. test streams using Instance-Aware Batch Normalization (IABN) and Prediction-Balanced Reservoir Sampling (PBRS). Although NOTE operates without explicit domain-transition detection, it does not explicitly identify or leverage stable inter-class relationships across domains. To provide a direct empirical comparison, we have included NOTE as an additional baseline in Tables 1–3. ToCoTTA consistently outperforms NOTE across all three 3D benchmarks.

ToCoTTA instead introduces a topology-based monitor that measures the alignment between the evolving target topology and the reference topology as a proxy for adaptation reliability. The resulting reliability signal controls throttling and restoration, enabling continual adaptation without requiring domain-boundary information.

Table: Comparison with NOTE

DatasetModelNOTEToCoTTA
MN-CPoint-MAE68.6874.38
DGCNN65.3275.72
PointNet50.2262.91
SONN-CPoint-MAE46.1850.24
DGCNN38.3950.65
PointNet24.3640.43
SN-CPoint-MAE64.6168.56
DGCNN62.2666.37
PointNet51.6256.88
AuthorsOpenReview ↗

Response to Reviewer eNCo (2/3)

W1-3.Shared Structural Knowledge vs. Domain-Specific Knowledge

Retaining domain-specific knowledge, as explored in BECoTTA, can indeed be beneficial when diverse unseen domains arrive sequentially. Our goal, however, is not to claim that a single shared model is universally superior to domain-specific expert architectures, but to identify and exploit structural knowledge that remains stable across domains. Importantly, we do not merely assume that such shared knowledge exists; we empirically identify it. As shown in Sec. 3 and Fig. 2, class topology remains largely preserved across diverse corruptions, with an average Spearman correlation of 0.85 even at the highest severity.

Preserving shared topology does not prevent adaptation to individual target distributions. Our topology prior (Eq. 4) preserves relative inter-class relationships without fixing prototypes to their source positions, allowing adaptation to changing target distributions while maintaining a stable structural reference.

To directly address this concern, we additionally evaluated BECoTTA on the same 3D CTTA benchmarks (Tables 1–3). Despite using a single model without domain-specific experts, ToCoTTA consistently outperforms BECoTTA across all three datasets and models, with average improvements of +5.58, +2.47, and +3.24 percentage points on MN-C, SONN-C, and SN-C, respectively. These results demonstrate that explicitly exploiting shared structural knowledge provides an effective alternative to domain-specific expert architectures on the evaluated 3D corruption benchmarks.

We have revised the Introduction to emphasize the identification and exploitation of stable cross-domain class topology as our primary motivation, while presenting computational efficiency and domain-detection-free adaptation as additional benefits rather than the central novelty.

Table: Comparison with BECoTTA

DatasetModelBECoTTAToCoTTA
MN-CPoint-MAE68.9674.38
DGCNN70.8175.72
PointNet56.4962.91
SONN-CPoint-MAE47.2350.24
DGCNN48.6450.65
PointNet38.0340.43
SN-CPoint-MAE65.5368.56
DGCNN64.5666.37
PointNet52.0056.88

W2.Concern about the methodological contribution

We agree that prototype-based adaptation, parameter regularization, and structural information have been explored in prior work [4–6]. However, the central insight of ToCoTTA is that inter-class relational topology remains relatively stable across diverse 3D corruptions, even when individual class representations shift and classification performance deteriorates. This suggests that the relationships among classes can serve as a more robust adaptation reference than their individual feature-space locations.

Building on this observation, ToCoTTA preserves the relative relationships among class prototypes rather than enforcing direct alignment to source prototypes, allowing online prototypes to follow evolving target distributions while retaining the underlying class structure. This differs from the prototype-level alignment objectives in [4, 5]. Moreover, unlike [4], which employs regularization weights precomputed from source-domain gradient sensitivity, ToCoTTA uses the agreement between online and reference class topologies as a dynamic indicator of adaptation reliability, which regulates self-training and parameter restoration to mitigate error accumulation.

While [6] also exploits topology for adaptation, it focuses on the spatial connectivity of anomaly maps through persistent homology to generate pseudo-labels for segmentation. In contrast, ToCoTTA models inter-class relationships in feature space and leverages their consistency as a shared signal for prototype refinement, adaptation reliability estimation, and recovery from accumulated errors.

Thus, our contribution is not merely the combination of existing adaptation components, but the observation that inter-class relational structure remains relatively stable under 3D corruptions, and the exploitation of this stability as a unified principle for guiding and stabilizing continual adaptation.

[4] Improving Test-Time Adaptation via Shift-agnostic Weight Regularization and Nearest Source Prototypes. ECCV22

[5] Transferrable Prototypical Networks for Unsupervised Domain Adaptation. CVPR19

[6] Learning Topology-Aware Representations via Test-Time Adaptation for Anomaly Segmentation. TPAMI26

AuthorsOpenReview ↗

Response to Reviewer eNCo (3/3)

W3. On the Practical Relevance of 3D Classification and Extension to Object Detection

We agree that prototype-based methods have proven effective in image classification and domain adaptation [4, 5].

As discussed in our response to W2, however, our contribution is not the use of prototypes itself, but the observation that class topology remains stable under diverse 3D corruptions and its use as a unified reference for prototype refinement, reliability monitoring, and restoration.

We also emphasize that 3D point cloud classification under distribution shift remains challenging. Although recent backbones achieve high accuracy on clean benchmarks, their accuracy drops sharply under corruptions. For example, the source-only Point-MAE drops from 89.22% on clean ModelNet40 to 57.13% on ModelNet40-C, whereas ToCoTTA achieves 74.38%. We follow the standard protocol of prior 3D TTA work on ModelNet40-C, and further evaluate on ScanObjectNN-C, which consists of real-world scans with noise and occlusion, and on ShapeNet-C.

Extending ToCoTTA to scene-level tasks such as 3D object detection is indeed an important direction. In principle, the reference topology can be derived from the classification head of a detector, but such an extension raises additional challenges. First, object-level features must be extracted under localization errors and in the presence of the background class. Second, each scene contains only a few object classes, which leaves most class prototypes unobserved for long periods. Third, harmful updates may primarily degrade box regression and objectness rather than the classification branch, which a monitor based solely on inter-class topology may not fully capture. Addressing these challenges requires a dedicated design and evaluation protocol, which we leave for future work.

We have added this discussion to the Limitations and Future Work section of the revised manuscript.

Q1. On the Selection of Trust-Score Thresholds and Target Label Usage

The thresholds t0t_0 and t1t_1 were selected once based on classification accuracy on ModelNet40-C with Point-MAE, using ground-truth labels for hyperparameter selection. They were then fixed before adaptation and applied unchanged to all other datasets and backbones, without further tuning. Importantly, no target labels are used during online adaptation. Furthermore, as shown in Fig. 6, ToCoTTA maintains stable performance across a broad range of threshold values, indicating low sensitivity to their selection.

Q2. Clarification on Gradient Flow During Prototype Refinement

During entropy minimization, the refined prototypes are treated as fixed targets, and gradients do not propagate through the prototype-refinement step. The prototypes are updated separately using a single refinement step per batch.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal and Additional Evidence

Dear Area Chair and Reviewers,

We sincerely thank the reviewers for their thoughtful feedback and constructive discussion.

Reviewer 6wM7 raised their rating from 2 (weak rejection) to 3 (weak acceptance) and subsequently stated that "The problem has been basically solved." Reviewer eNCo also confirmed that their concerns regarding comparisons with prior CTTA methods had been "fully resolved." We summarize below how we addressed the reviewers' concerns through additional experiments and clarifications.

Our paper investigates the stability of inter-class topology under diverse 3D corruptions and proposes ToCoTTA, which leverages this structural information as a Prior for prototype refinement, a Monitor for adaptation reliability, and a Repair signal for controlling updates and parameter restoration.

Reviewer 6wM7 questioned the reliability of topology monitoring, optimizer behavior, and statistical significance. We added within-domain reliability analyses (Table 12), comparisons with entropy-based gating and fixed restoration (Table 9), optimizer-level verification (Tables 14–15), and paired statistical tests (Tables 20–21). These additional results addressed the reviewer's concerns, as acknowledged in their follow-up.

Reviewer eNCo questioned our distinction from existing CTTA approaches. We clarified our motivation and added comparisons with NOTE and BECoTTA across three datasets and backbones (Tables 1–3). ToCoTTA consistently outperforms both methods, and the reviewer subsequently acknowledged that their concerns regarding these comparisons had been fully resolved.

Reviewer cuc4 raised concerns about novelty relative to TCA and FedPAT, the effectiveness of topology preservation, and the contribution of the 3D setting. In our subsequent responses, we further addressed these concerns with additional experiments and clarifications.

First, we evaluated the complete TCA method, including its intra-class objective and imbalance-aware weighting, under matched 3D CTTA settings. ToCoTTA outperforms TCA across all nine dataset–backbone combinations (Tables 1–3) and in 10-round continual adaptation (Table 4). This complements our controlled regularizer comparison (Table 5), which isolates the benefit of preserving source-specific class relationships.

Second, regarding FedPAT and the significance of our contribution, we clarified that our contribution lies not in topology preservation itself, but in extending topology from a structural prior to a reliability signal for continual adaptation. In CTTA, errors from unreliable self-training can accumulate across successive distribution shifts, motivating the need to monitor and regulate the adaptation process rather than solely guiding predictions. Unlike FedPAT, ToCoTTA uses topology agreement to assess adaptation reliability, suppress unreliable updates, and adaptively restore drifting parameters. The long-horizon ablations (Table 8) and comparisons against entropy-based gating and matched fixed restoration (Table 9) provide empirical support for this mechanism.

We appreciate the reviewers' suggestions, which have helped clarify our contributions and strengthen the revised manuscript. Our work shows that the inter-class topology preserved under 3D corruptions can serve as a shared structural reference for both prediction and reliability-aware continual adaptation, helping mitigate error accumulation across successive distribution shifts.

Thank you for your consideration.

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.