- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents ToCoTTA, a framework for continual test-time adaptation of 3D point-cloud classifiers. The core contribution is the use of a classifier-derived class topology as a prior for online prototype refinement, a monitor of adaptation reliability, and a signal for parameter restoration. The refined prototypes support prediction and entropy minimization, while the topology-based trust score regulates adaptation. Experiments on ModelNet40-C, ScanObjectNN-C, and ShapeNet-C report improved mean accuracy across three backbones and stable performance over ten corruption rounds.
Critical strengths
- Using the same class-affinity reference for prototype refinement, monitoring, and restoration provides a coherent design for addressing error accumulation. Table 5 shows that preserving relative class relationships performs better than directly anchoring prototypes to source classifier weights, supporting this design choice.
- The experiments cover both synthetic and scanned objects, with the highest reported mean accuracy in all nine dataset–backbone combinations in Tables 11–13. The long-horizon ablations in Table 7 further demonstrate the practical value of the repair components, particularly on ShapeNet-C.
Critical weaknesses
1: The monitor needs stronger validation: Figure 2 uses true-label prototypes, while Figure 4’s domain-level correlation does not establish whether online estimates detect harmful updates within a domain. The authors should test limited class coverage and compare against entropy gating and matched fixed restoration. 2: The throttling mechanism needs clarification under AdamW, which can offset loss scaling and retain momentum-driven updates at zero trust. The authors should report actual update magnitudes and explain optimizer-state handling during restoration. 3: The small SONN-C gains (0.93 and 0.50 percentage points) lack paired uncertainty estimates; Table 10 only reports the variability of ToCoTTA. The authors should compare with the strongest baseline values and report the paired gains along with their uncertainty intervals.
Questions for authors
- What additional functionality does ToCoTTA provide beyond TCA's inter-class topology constraints? Comparing reference construction and the three-role design, together with a matched 3D adaptation of TCA, would make this distinction clearer.
- Can online topology estimates reliably identify harmful updates when class coverage is incomplete? Comparing them with true-label topology and testing within-domain update quality against entropy gating and matched fixed restoration would provide more direct evidence for the monitor's reliability.
- Under AdamW, how does multiplying the entropy loss by g affect the actual update magnitude as g decreases? Do the authors skip optimizer steps or modify its accumulated moments at g=0 or during parameter restoration, and could they provide an analysis of the resulting parameter drift?
6 comments
Response to Reviewer 6wM7 (2/4)
W2-1 and Q3. Effect of loss scaling under Adam.
We acknowledge that Adam's adaptive normalization can partially offset gradient scaling, particularly when the scaling factor remains constant. However, as illustrated in Fig. 5 (left), our trust score varies throughout adaptation, causing the current gradient scaling to differ from the historical scaling accumulated in Adam's moment estimates. Consequently, the effect of trust-based gradient scaling is not necessarily canceled by Adam's adaptive normalization. To quantify the actual effect, we measure the parameter change induced by each optimizer step on SN-C, which has the longest test stream among our benchmarks. As shown in Tab. 14, trust-based throttling reduces the actual optimizer update magnitude by 51.1% when , while having little effect when . Furthermore, the ablations in Tables 6 and 8 show that removing throttling degrades adaptation performance, supporting its practical effectiveness despite Adam's adaptive normalization.
Table 14: Mean () per optimizer step, with and without trust-based throttling, excluding parameter restoration. Results are averaged over batches within each trust range.
| 9.4 | 4.6 | |
| 5.4 | 4.6 | |
| 3.9 | 3.8 | |
W2-2 and Q3. Momentum-driven updates at zero trust.
We acknowledge that Adam can still produce momentum-driven updates when due to its accumulated optimizer state. We do not skip optimizer steps or reset the accumulated moments when . In our implementation, we use Adam with zero weight decay, eliminating weight-decay-induced parameter updates. Importantly, when is low, the current loss gradient is scaled down, reducing its contribution to the optimizer's momentum estimate, although previously accumulated momentum may still induce parameter updates. Meanwhile, the restoration coefficient increases as decreases, reaching its maximum value () when , thereby pulling the adapted parameters toward their source values to counteract accumulated drift.
W2-3 and Q3. Optimizer-state handling during restoration.
In our implementation, we keep the optimizer state unchanged during parameter restoration. To examine whether retaining the optimizer state affects adaptation performance, we compare our approach (Keep) with two alternatives: Scale, which scales the first-moment estimate by during restoration (), and Reset, which resets it to zero (). As shown in Tab. 15, Keep achieves the highest accuracy (68.56%), outperforming Scale (67.86%) and Reset (66.56%). Keep and Scale exhibit similar parameter drift, whereas Reset substantially reduces drift but yields lower accuracy, indicating that minimizing parameter drift alone does not necessarily improve adaptation performance.
We have added these results to the revised manuscript (Tables 14 and 15).
Table 15: Effect of optimizer-state handling during restoration on SN-C. Drift is measured at each batch after restoration (mean / max / final over the stream).
| Method | Acc (%) | $\Vert$$\theta_t - \theta_0 \Vert_2$ (mean/max/final) |
|---|---|---|
| Reset | 66.56 | 2.36 / 6.80 / 0.22 |
| Scale | 67.86 | 5.34 / 8.35 / 2.02 |
| Keep (Ours) | 68.56 | 5.48 / 8.53 / 2.31 |
Response to Reviewer 6wM7 (3/4)
W3. Statistical Significance of Paired Improvements on SONN-C
To assess the statistical reliability of the improvements on SONN-C, we conducted additional experiments across 10 random seeds and 10 random domain orders, comparing ToCoTTA against the strongest baseline for each backbone. We report the mean accuracy, standard deviation, and paired accuracy gains with 95% confidence intervals computed using paired -tests.
As shown in Tables 20 and 21, ToCoTTA consistently outperforms the strongest baselines across all three backbones under both evaluation settings. For random seeds, the paired gains are +0.93, +1.52, and +0.57 percentage points for Point-MAE, DGCNN, and PointNet, respectively. For random domain orders, the corresponding gains are +1.21, +0.77, and +0.57 percentage points. Importantly, all 95% confidence intervals exclude zero, indicating that the improvements are statistically significant even when the absolute gains are relatively modest.
These results demonstrate that ToCoTTA consistently outperforms the strongest baselines on SONN-C across random seeds and domain orders, with statistically significant paired improvements.
We have added these results to the revised manuscript (Tables 20 and 21).
Table 20: Comparison of ToCoTTA with the strongest baseline on SONN-C across 10 random seeds (mean std). denotes the mean accuracy improvement over the baseline, with 95% paired- confidence intervals ( seeds).
| Model | Baseline | Baseline Acc (%) | ToCoTTA Acc (%) | 95% CI | |
|---|---|---|---|---|---|
| Point-MAE | LN-TTA | 49.32 0.3 | 50.26 0.1 | +0.94 | [0.67, 1.20] |
| DGCNN | CoTTA | 49.02 0.4 | 50.54 0.3 | +1.52 | [1.19, 1.84] |
| PointNet | CoTTA | 39.96 0.3 | 40.53 0.2 | +0.57 | [0.35, 0.80] |
Table 21: Comparison of ToCoTTA with the strongest baseline on SONN-C across 10 random domain orders (mean std). denotes the mean accuracy improvement over the baseline, with 95% paired- confidence intervals ( orders).
| Model | Baseline | Baseline Acc (%) | ToCoTTA Acc (%) | 95% CI | |
|---|---|---|---|---|---|
| Point-MAE | LN-TTA | 49.13 0.2 | 50.34 0.3 | +1.21 | [0.98, 1.43] |
| DGCNN | CoTTA | 49.66 0.3 | 50.43 0.3 | +0.77 | [0.41, 1.13] |
| PointNet | CoTTA | 40.06 0.3 | 40.63 0.3 | +0.57 | [0.17, 0.97] |
Response to Reviewer 6wM7 (1/4)
We thank the reviewer for recognizing the coherent design of our topology-based framework and its consistent performance across datasets and backbones. We also appreciate the constructive suggestions on validating the topology-based monitor and repair mechanisms, which have motivated us to conduct additional experiments and strengthen our empirical analysis.
W1-1. Ground-truth prototypes in Figure 2
Figure 2 uses ground-truth labels solely to demonstrate the stability of class topology under corruption, motivating its use as a reference. During adaptation, we approximate this reference using source classifier weights and monitor reliability by comparing it with online prototype topology, without using ground-truth labels.
W1-2 and Q2. Within-Domain Reliability of the Topology Monitor
Our topology-based monitor is designed to assess the structural reliability of the evolving online representation, rather than to identify harmful individual gradient updates.
While Fig. 5 (left) already presents batch-level traces of the trust score and accuracy, we agree that the domain-level correlation in Fig. 5 (right) does not establish within-domain reliability. To address this, we conduct an additional within-domain topology reliability analysis, included in Appendix A.3 of the revised manuscript.
Specifically, we construct an oracle prototype bank using ground-truth labels solely for evaluation and define topology estimation quality as qt=corr(Aμt,AGT,t). We compare this with the raw monitor score rt=corr(Aμt,Aref). To isolate within-domain variations, we subtract the domain-wise mean from each score and compute correlations over pooled batches. On ShapeNet-C, we observe consistent positive Pearson correlations of 0.61, 0.59, and 0.63 for Point-MAE, DGCNN, and PointNet, respectively. These results suggest that the proposed monitor is informative of within-domain variations in online topology estimation quality, beyond domain-level differences. Furthermore, Tables 6 and 8 demonstrate the practical utility of this reliability signal in controlling adaptation through throttling and restoration.
Table 12: Within-domain correlation between the raw monitor score rt and the online topology quality qt on SN-C.
W1-3. Limited Class Coverage
To evaluate the effectiveness of topology-based monitoring under limited class coverage, we conduct additional experiments on non-IID test streams using Dirichlet sampling (δ=0.1), following RoTTA [1]. As shown in Tab. 13, removing topology-based throttling (gT) or restoration control (gR) consistently reduces accuracy across all three datasets, with additional degradation when both are disabled. Although the overall performance of ToCoTTA degrades substantially under non-IID streams, as acknowledged in our Limitations section (Sec. 6), these results indicate that topology-based monitoring remains beneficial for regulating adaptation even under limited class coverage.
[1] Robust Test-Time Adaptation in Dynamic Scenarios, CVPR23
Table 13: Effect of g under non-IID streams.
W1-4 and Q2. Entropy Gating and Matched Fixed Restoration
We compare our topology-based monitoring and adaptive restoration against entropy gating and matched fixed restoration, respectively. For entropy gating, we replace the topology-based trust score g in both throttling and restoration with normalized entropy-based confidence. For matched fixed restoration, we use a constant kˉ equal to the average restoration coefficient from a full ToCoTTA run on the same test stream. This oracle setting matches the overall restoration strength while removing temporal adaptivity, although the oracle coefficient is unavailable at test time.
ToCoTTA consistently outperforms both alternatives across all three datasets. The improvement over entropy gating supports the effectiveness of topology-based reliability monitoring, while the improvement over matched fixed restoration highlights the benefit of temporally adaptive restoration.
We have included these additional analyses and comparisons in the revised manuscript (Tables 9, 12, and 13).
Table 9: Comparison with alternative monitoring and restoration strategies.