acceptodds
Withdrawn from ICLR 2027

DESA-TTA: Dynamic EMA and Source Anchoring for Test-Time Adaptation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-language object detectors (VLODs) achieve strong zero-shot performance but remain vulnerable to distribution shifts during deployment. Mean-teacher methods for test-time adaptation (TTA) can improve robustness by updating a student model using teacher-generated pseudo-labels. However, mean-teacher TTA is highly sensitive to the choice of a fixed exponential moving average (EMA) coefficient for teacher updates, and repeated optimization with noisy pseudo-labels can cause cumulative student drift. We propose Dynamic EMA and Source Anchoring for TTA (DESA-TTA), a low-overhead method that jointly regulates teacher updates and student drift through dynamic temporal averaging and source anchoring. Dynamic temporal averaging estimates teacher uncertainty from pseudo-label confidence and box density and uses it to select a sample-wise EMA coefficient within bounds determined by teacher parameter drift. Source anchoring partially restores the updated student parameters toward their pretrained values, with the anchoring strength increasing according to student drift. Experiments across diverse distribution shifts and two VLOD architectures show consistent improvements over existing TTA methods. On VOC-C, DESA-TTA improves AP by 14.5 points over zero-shot inference while achieving 55% higher inference throughput than the previous state-of-the-art TTA method for YOLO-World. Our code is included in the supplementary material and will be made publicly available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
test-time adaptation, vision-language object detection, distribution shift, mean-teacher learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7EKGOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses test-time adaptation (TTA) for vision-language object detectors (VLODs) within a cumulative mean-teacher framework. The authors identify two primary failure modes in standard mean-teacher OD-TTA: extreme domain sensitivity to a fixed exponential moving average (EMA) teacher update coefficient, and long-term student parameter drift caused by repeated optimization on noisy pseudo-labels. To resolve these issues, the authors propose DESA-TTA, which combines two heuristics: (1) Dynamic Temporal Averaging, which computes a sample-wise teacher uncertainty score from mean pseudo-label confidence and normalized box density to interpolate the EMA coefficient within bounds controlled by ℓ1\ell_1 teacher parameter drift; and (2) Source Anchoring, which linearly interpolates gradient-updated student normalization parameters back toward their pretrained initialization based on an ℓ1\ell_1 student parameter drift metric. The method is validated across two VLOD architectures (YOLO-World and Grounding DINO) on style shifts (Watercolor, ClipArt, Comic), driving scenes (Foggy Cityscapes, BDD100K), low-light (ExDark), and common corruptions (VOC-C, COCO-C), demonstrating consistent AP50AP_{50} and mAP improvements alongside favorable inference throughput.

Critical strengths
  • Practical Efficiency and Empirical Gains in VLOD-TTA: Adapting open-vocabulary object detectors online is notoriously brittle and computationally expensive. By restricting updates to normalization layers and regulating the teacher-student trajectory via closed-form parameter interpolation and dynamic EMA scaling, DESA-TTA achieves substantial accuracy gains (+8.9 AP50AP_{50} over VLOD-TTA on VOC-C) while increasing FPS from 20 to 31 on YOLO-World and cutting memory consumption by 70.4% compared to foundation-model-assisted baselines like TTAOD-F on Grounding DINO.
  • Thorough Empirical Diagnostics: The paper does an admirable job of empirically motivating its design choices. The image-level Spearman correlation analysis (Table 7) and quintile analysis (Table 8) verifying that the proposed heuristic uncertainty score rtr_t negatively correlates with true pseudo-label precision and IoU—even when controlling for ground-truth object count—provides solid empirical grounding for an otherwise heuristic rule.
Critical weaknesses
  • Heavy Parameterization of Update Heuristics and Validation Leakage Concerns: While the high-level intuition of DESA-TTA is clean, its actual instantiation introduces 9 distinct hyperparameters (Mmax⁡,mmin⁡,mmax⁡,Δmin⁡,Δmax⁡,λmin⁡,λmax⁡,αmax⁡,λαM_{\max}, m_{\min}, m_{\max}, \Delta_{\min}, \Delta_{\max}, \lambda_{\min}, \lambda_{\max}, \alpha_{\max}, \lambda_{\alpha}, and pp) governed by ad-hoc rational transition functions (Equations 4–9). The authors state these were tuned on "six corruption types applied to the VOC training split" and kept fixed. However, tuning on synthetic corruptions of PASCAL VOC and then claiming strong performance on VOC-C (which shares the exact same underlying source imagery and object statistics, plus overlapping corruption families) represents a favorable domain overlap. Moreover, Figure 5 reveals that optimal hyperparameters diverge between short style-shift streams and long corruption streams (e.g., αmax⁡\alpha_{\max} and λmin⁡\lambda_{\min} have opposing trends after a threshold), suggesting the method's drift scales are implicitly coupled to stream length.
  • Counter-Intuitive Uncertainty-to-EMA Logic Under Severe Corruptions: The core premise of Dynamic Temporal Averaging is that high teacher uncertainty (rt→1r_t \to 1, meaning dense, low-confidence boxes) should yield a smaller EMA coefficient mtm_t, thereby incorporating the current student state more rapidly. However, the student was just updated using those exact same dense, low-confidence, noisy pseudo-labels from sample xtx_t. Accelerating the injection of a student step supervised by highly uncertain pseudo-labels seems risky. In Proposition 1 (Appendix A.1), this is only justified by assuming the student error variance VS(r)V_S(r) remains fixed while teacher error variance VT(r)V_T(r) grows—an assumption that breaks down in practice since etSe_t^S directly depends on the quality of the teacher's supervision at step tt.
Questions for authors
  1. Clarification on Student-Teacher Error Coupling: In Proposition 1, you assume VS(r)V_S(r) and C(r)C(r) are fixed with respect to VT(r)V_T(r). In practice, when teacher uncertainty rtr_t spikes on a heavily corrupted frame, the student gradient step is computed directly from those degraded pseudo-labels. Why does lowering mtm_t (faster student-to-teacher injection) on high-rtr_t samples not accelerate teacher corruption? Is this solely because the student has just been pulled back toward θ0\theta_0 via Source Anchoring?
  2. Clarification on Grounding DINO Hyperparameters: Were the exact same hyperparameters (such as λmin⁡=0.036,λmax⁡=0.021,λα=0.07\lambda_{\min}=0.036, \lambda_{\max}=0.021, \lambda_\alpha=0.07) tuned on YOLO-World (BatchNorm) directly transferred without modification to Grounding DINO (LayerNorm/GroupNorm) in Table 3? Given that δtT=1∣A∣∑l∈A∥θt,lT−θ0,l∥1max⁡(∥θ0,l∥1,ϵ)\delta_t^T = \frac{1}{|\mathcal{A}|} \sum_{l \in \mathcal{A}} \frac{\Vert{}\theta_{t,l}^T - \theta_{0,l}\|_1}{\max(\|\theta_{0,l}\|_1, \epsilon)} and δtS\delta_t^S measure relative ℓ1\ell_1 parameter drift from initialization, do affine normalization weights in Transformers drift at the same numerical scale as CNN BatchNorm statistics?
  3. Suggestion for Future Work / Continual TTA: Since the drift scales (λmin⁡,λmax⁡,λα\lambda_{\min}, \lambda_{\max}, \lambda_\alpha) are calibrated around a specific stream horizon, how would DESA-TTA behave in a continual TTA setting (e.g., sequenced corruptions without resetting to θ0\theta_0)? If δtT\delta_t^T saturates in Domain 1, mtm_t will be locked near mmax⁡m_{\max}, potentially preventing adaptation to Domain 2.
5 comments
AuthorsOpenReview ↗

Response to Reviewer 7EKG (1/3)

Thank you for your detailed and constructive review, and for recognizing the practical efficiency of DESA-TTA and the empirical diagnostics of the teacher-uncertainty score. Your questions about the hyperparameters, the tuning protocol, and the coupling between student and teacher errors helped us clarify the assumptions behind our design. We particularly appreciate your suggestion to study continual adaptation, which led to the new continual VOC-C experiment with shift detection below. Our modifications are highlighted in blue in the revised PDF.

W1. Hyperparameters, tuning protocol, and stream length.

Number of tuned hyperparameters.

We tune five hyperparameters, and the remaining quantities are fixed across all target domains:

  • Mmax⁡=30M_{\max}=30 normalizes the number of retained pseudo-labels in Eq. (4). The value of 300 stated in Sec. 4.1 of the original submission was incorrect and is corrected in Sec. 3.2, Sec. 4.1, and Appendix A.4 of the revised PDF.
  • The four quantities mmin⁡m_{\min}, mmax⁡m_{\max}, Δmin⁡\Delta_{\min}, and Δmax⁡\Delta_{\max} define the low- and high-drift EMA intervals. They are fixed based on the effective coefficient range observed in the fixed-EMA analysis.
  • The five hyperparameters αmax⁡\alpha_{\max}, λα\lambda_\alpha, λmin⁡\lambda_{\min}, λmax⁡\lambda_{\max}, and pp are selected once and shared across target domains.

All drift-dependent terms use the same monotone saturating map, δp/(δp+λp)\delta^p/(\delta^p+\lambda^p). Each scale λ\lambda has a direct interpretation: it is the drift at which the corresponding transition is half complete. A single exponent pp is shared by all three transitions.

Sensitivity.

Fig. 5 varies each tuned hyperparameter independently while keeping the others fixed. Across these sweeps, AP50 stays within approximately 1 point of the selected configuration on both benchmarks (VOC-C 47.5–48.6, Style 46.3–47.5). Every tested setting remains above MT (39.5 VOC-C, 44.1 Style) and VLOD-TTA (39.6 VOC-C, 44.9 Style). These ranges are now also stated in Appendix A.4 of the revised PDF.

Tuning data and overlap with VOC-C.

The hyperparameters were selected on six corruption types applied to the VOC training split, which shares no images with the VOC-C evaluation set. We acknowledge that the tuning and evaluation data share object categories and corruption families, so VOC-C results alone cannot establish generalization. However, the same configuration is also evaluated unchanged on six benchmarks with no image overlap with the tuning data. They cover artistic (Watercolor, ClipArt, Comic), adverse-weather (Foggy Cityscapes), real-world driving (BDD100K), and low-light (ExDark) shifts. DESA-TTA achieves the best AP50 on all six, improving over VLOD-TTA by +1.9, +2.9, +2.5, +2.2, +3.5, and +1.8 AP50, respectively (Table 1).

Stream length.

We agree that the best values of αmax⁡\alpha_{\max} and λmin⁡\lambda_{\min} differ between Style and VOC-C (Fig. 5a, c). However, the two benchmarks differ in both the type of shift and the stream length, so these differences cannot be attributed to stream length alone. Moreover, DESA-TTA sets source anchoring and the EMA bounds from the measured student and teacher drift rather than from the number of processed images.

Furthermore, a single configuration is effective for streams ranging from 500 images (ClipArt, Foggy Cityscapes) to 10,000 images (BDD100K). On the two longest streams, BDD100K and ExDark, neither of which was used for tuning, DESA-TTA improves over MT by +7.5 and +3.8 AP50. On BDD100K, MT itself falls below ZS.

AuthorsOpenReview ↗

Response to Reviewer 7EKG (2/3)

W2 / Q1. Uncertainty-to-EMA logic and student–teacher coupling.

We agree that the student update on xtx_t uses the same pseudo-labels whose uncertainty determines mtm_t, so VS(r)V_S(r) and C(r)C(r) may vary with rr. The expression for the unconstrained one-step optimum in Proposition 1 remains valid under this dependence, which is explicitly included through C(r)C(r). Writing A(r)=VS(r)−C(r)A(r)=V_S(r)-C(r) and B(r)=VT(r)−C(r)B(r)=V_T(r)-C(r), we obtain

m⋆(r)=A(r)A(r)+B(r)=11+B(r)/A(r).m^\star(r)=\frac{A(r)}{A(r)+B(r)}=\frac{1}{1+B(r)/A(r)}.

For A(r)>0A(r)>0 and A(r)+B(r)>0A(r)+B(r)>0, this coefficient decreases with rr whenever B(r)/A(r)B(r)/A(r) increases. The student error therefore need not remain constant. What matters is how the ratio of these conditional error terms changes with uncertainty. Holding VSV_S and CC fixed, with VS>CV_S>C and increasing VTV_T, is one sufficient special case. We have stated this general condition and clarified the conditional nature of the analysis in Proposition 1 (Appendix A.1) of the revised PDF.

Control of the student contribution.

Updating the teacher from a student trained on the teacher's own pseudo-labels is the standard mean-teacher mechanism. Because the teacher averages many past student states, it provides more stable supervision than the online student. This mechanism is widely used in semi-supervised object detection [4] and test-time adaptation for object detection [2]. In our experiments, MT improves over ZS on five of the six benchmarks in Table 1 and on VOC-C (Table 2). DESA-TTA retains this averaging and modulates the update rate only within bounded limits:

  • Bounded contribution. mtm_t is at least mmin⁡−Δmin⁡=0.993m_{\min}-\Delta_{\min}=0.993. Each teacher update therefore assigns at most 0.7% weight to the current student state.
  • Drift-dependent range. As teacher drift increases, both EMA bounds rise toward [mmin⁡,mmax⁡]=[0.999,0.9996][m_{\min},m_{\max}]=[0.999,0.9996]. The upper bound on the student contribution consequently approaches 0.1% in the high-drift limit.
  • Source anchoring. After each gradient step, the student is interpolated toward its source initialization before the teacher update. This contracts its post-gradient parameter distance from the source (Proposition 2).

Additional Benefit of Dynamic EMA Beyond Source Anchoring

With source anchoring disabled, D-EMA improves over fixed EMA:

MethodStyle AP50VOC-C AP50
Fixed EMA (MT, Table 5)44.139.5
Per-domain fixed-EMA oracle45.442.5
D-EMA (Table 5)46.643.1

D-EMA improves over MT by 2.5 AP50 on Style and 3.6 AP50 on VOC-C, and also exceeds the per-domain fixed-EMA oracle from our additional sweep. These comparisons show that the improvement does not depend solely on restoring the student toward the source.

Empirical test of the opposite rule.

Table 9 tests the reversed mapping, which increases the EMA coefficient on uncertain samples, using the same dynamic source-anchoring rule and drift-dependent EMA-bound functions. It performs worse than DESA-TTA on all 18 datasets and corruptions, by 2.6 AP50 on Style and 5.2 AP50 on VOC-C on average. This directly supports the proposed mapping direction under otherwise matched settings. Tables 7 and 8 further support the use of rtr_t as an indicator of pseudo-label quality.

Q2. Hyperparameters for Grounding DINO and drift scale.

YOLO-World and Grounding DINO are different architectures: YOLO-World is a CNN with batch normalization, whereas Grounding DINO is a Transformer with layer and group normalization. As the reviewer notes, their normalization parameters do not necessarily drift at the same numerical scale. For Grounding DINO, we use a learning rate of 5×10−45\times10^{-4} and set Mmax⁡=30M_{\max}=30, mmin⁡=0.993m_{\min}=0.993, mmax⁡=0.997m_{\max}=0.997, Δmin⁡=0.003\Delta_{\min}=0.003, Δmax⁡=0.004\Delta_{\max}=0.004, λmin⁡=0.02\lambda_{\min}=0.02, λmax⁡=0.03\lambda_{\max}=0.03, αmax⁡=5×10−4\alpha_{\max}=5\times10^{-4}, λα=0.04\lambda_\alpha=0.04, and p=3p=3. With these values, DESA-TTA improves the average COCO-C mAP from 20.6 to 27.6 and outperforms TTAOD-F and VLOD-TTA by 1.6 and 1.4 mAP, respectively (Table 3). We have added these values to Appendix A.2 of the revised PDF.

AuthorsOpenReview ↗

Response to Reviewer 7EKG (3/3)

Q3. Continual TTA and drift saturation.

We thank the reviewer for this suggestion. Our method focuses on single-domain TTA, following prior VLOD TTA work, VLOD-TTA [1] and TTAOD-F [2]. Continual adaptation across sequential domains is outside the scope of our main evaluation.

As teacher drift increases, the EMA bounds approach [0.999,0.9996][0.999, 0.9996]. In this regime, the teacher continues to update, with an approximate EMA memory scale of 1,000–2,500 updates, comparable to the best fixed EMA on VOC-C (m=0.9994m=0.9994). The student's gradient update is not directly scaled by mtm_t, although it remains coupled to the teacher through the pseudo-labels.

After an abrupt shift, however, this slower teacher response could delay adaptation to a new domain, because δtT\delta^T_t is measured relative to the source model and does not necessarily decrease when the new domain begins.

To examine this, we evaluated DESA-TTA on the 15 VOC-C corruptions as a single continual stream, without resetting between corruptions. We also combined it with a shift detector based on the feature-statistics criterion of CTTAOD [3] (τ2=1.08\tau_2=1.08), which resets the student and teacher to the source model when a shift is detected. Since DESA-TTA does not access source data, the training-feature statistics required by this criterion are taken from the batch-normalization running statistics stored in the pretrained YOLO-World model. These results are reported in Appendix A.3 (Sequential Domain Shifts, Table 15) of the revised PDF. The last column lists the image index within each corruption at which a reset occurred.

CorruptionSingle-domain (Table 2)Continual, no resetContinual, shift detectionResets (image index)
Gaussian noise34.434.434.4–
Shot noise35.838.238.2–
Impulse noise37.838.838.8–
Defocus blur39.634.439.211
Glass blur32.734.934.1–
Motion blur37.835.936.8–
Zoom blur36.136.338.0–
Snow54.551.952.4–
Frost57.858.657.72165, 4734, 5234
Fog73.672.372.874
Brightness74.273.974.241, 895, 1530, 3583, 4083
Contrast63.362.062.28
Elastic transform59.856.355.617, 4514, 5210
Pixelate37.917.633.114
JPEG compression52.652.153.0–
Average48.546.548.015 resets

Without resetting, DESA-TTA achieves 46.5 AP50, compared with 48.5 under the single-domain protocol, and remains well above MT (39.5) and VLOD-TTA (39.6) evaluated in the single-domain protocol. It exceeds its single-domain results on five corruptions, consistent with beneficial carry-over across some domains. However, the effect varies across corruptions, with Pixelate decreasing from 37.9 to 17.6 AP50. With shift detection, the average improves to 48.0 AP50, within 0.5 of the single-domain result, and Pixelate recovers to 33.1 AP50. The detector triggers within the first 75 images of 6 of the 14 new corruptions, with 9 additional resets occurring later within corruptions. These results show that source resets triggered by shift detection mitigate the performance loss in the tested continual sequence, and that more accurate shift detection is a natural direction for future work.

References

[1] A. Belal, H. R. Medeiros, M. Pedersoli, and E. Granger. VLOD-TTA: Test-time adaptation of vision-language object detectors. In ECCV, 2026.

[2] Y. Gao, Y. Zhang, Z. Cai, and D. Huang. Test-time adaptive object detection with foundation model. In NeurIPS, 2025.

[3] J. Yoo, D. Lee, I. Chung, D. Kim, and N. Kwak. What, how, and when should object detectors update in continually changing test domains? In CVPR, 2024.

[4] Y.-C. Liu, C.-Y. Ma, Z. He, C.-W. Kuo, K. Chen, P. Zhang, B. Wu, Z. Kira, and P. Vajda. Unbiased teacher for semi-supervised object detection. In ICLR, 2021.

Reviewer_sW6DOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DESA-TTA, a test-time adaptation method for vision-language object detectors that addresses fixed EMA sensitivity and parameter drift. It introduces uncertainty- and drift-aware dynamic EMA updates and drift-adaptive source anchoring. Experiments on YOLO-World and Grounding DINO demonstrate improved accuracy and efficiency, achieving 48.5 AP50 on VOC-C, an 8.9-point gain over VLOD-TTA.

Critical strengths
  1. The proposed mechanisms are lightweight and do not require an additional foundation model or expensive auxiliary optimization. It's efficient.
  2. The experimental evaluation is sufficient. Specifically, the experimental evaluation covers style shifts, adverse driving conditions, low-light conditions, and common corruptions.
  3. The motivation is well-founded, identifying two key failure modes in cumulative test-time adaptation.
Critical weaknesses
  1. The fixed EMA coefficient used by the MT baseline and the fixed-EMA comparisons is not clearly specified. It is unclear whether the fixed-EMA comparisons use the same source anchoring configuration. A more rigorous evaluation should compare DESA-TTA against both a globally tuned fixed EMA and a per-domain optimal fixed EMA (oracle), while keeping all other settings identical.
  2. The proposed components are closely related to existing tta related works (like CoTTA, SANTA), so it's not a novel idea.
  3. Eq. (4) defines rt=min⁡((Mt/300)(−log⁡sˉt),1)r_t=\min((M_t/300)(-\log\bar{s}_t),1), with pseudo-label confidence thresholded at 0.4. However, Figure 3 shows a median uncertainty of approximately 0.5 for ClipArt, which would require an unusually large number of retained detections per image. This raises concerns about the consistency between the stated formulation and the reported results.
  4. The experiments focus on cumulative adaptation within stationary target domains, without evaluating robustness to sequential distribution shifts in realistic deployment scenarios.
Questions for authors
  1. Proposition may not hold during pseudo-label adaptation. Can you further explain it?
  2. What exact EMA coefficient is used for the MT and fixed-EMA baselines?
  3. How sensitive is the method to image ordering and consecutive domain shifts?
4 comments
AuthorsOpenReview ↗

Response to Reviewer sW6D (1/3)

Thank you for your careful review and for recognizing the efficiency of DESA-TTA, the breadth of our evaluation, and the motivation behind the two failure modes we target. Your request for globally tuned and per-domain oracle fixed-EMA baselines helped us make the EMA comparison more rigorous, and your close check of Eq. (4) against Fig. 3 led us to correct an inconsistency in the paper. We also appreciate your questions on sequential domain shifts and image ordering, which prompted the new continual and permutation experiments below. Our modifications are highlighted in blue in the revised PDF.

W1 / Q2 - The fixed EMA coefficient and detailed comparison.

To provide a strong fixed-EMA baseline, we selected the coefficient that maximizes average AP50 within each benchmark while keeping it shared across all constituent domains. This gives m=0.998m=0.998 for the three Style datasets and m=0.9994m=0.9994 for the 15 VOC-C corruptions. We disable source anchoring for both fixed EMA and D-EMA in this comparison to isolate the effect of the EMA update rule. The complete fixed-EMA sweep is reported below.

Benchmark0.9940.9950.9960.9970.9980.9990.99940.9996
Style41.942.643.744.044.143.942.642.1
VOC-C20.822.724.428.132.337.139.539.2

We additionally evaluate the requested per-domain oracle by selecting the best fixed coefficient separately for each Style dataset and each VOC-C corruption, then averaging their AP50 values. This oracle uses target evaluation performance for coefficient selection. We also report a single fixed coefficient, m=0.9994m=0.9994, shared across both benchmarks.

EMA selectionStyle AP50VOC-C AP50
One shared coefficient, m=0.9994m=0.999442.639.5
Best coefficient per benchmark44.139.5
Per-domain fixed-EMA oracle45.442.5
D-EMA46.643.1

D-EMA improves over the best benchmark-level fixed EMA by 2.5 AP50 on Style and 3.6 AP50 on VOC-C. It also exceeds the per-domain oracle by 1.2 and 0.6 AP50, respectively. D-EMA adjusts the coefficient for each incoming image using teacher uncertainty and drift, outperforming even a separately selected fixed coefficient for each domain.

Regarding source anchoring, the MT and D-EMA variants in Table 5 both omit anchoring, isolating the EMA component. Table 5 also provides an anchored comparison between MT+D-SA and full DESA-TTA, which use the same dynamic source anchoring. DESA-TTA improves over this anchored fixed-EMA baseline from 46.3 to 47.3 on Style and from 42.2 to 48.5 on VOC-C. In the revised PDF, the fixed coefficients and their selection procedure are documented in Sec. 4.1 (Baselines), and the full sweep, the per-domain oracle, and the anchoring configuration are reported in Appendix A.3 (Fixed-EMA Selection, Tables 16 and 17).

W2. Relation to CoTTA and SANTA.

We agree that CoTTA, SANTA, and DESA-TTA share the broad objective of mitigating drift from the source model. However, they differ fundamentally in how drift is measured and controlled.

CoTTA restores each adapted parameter to its source value with a fixed probability at every iteration, independently of the current model state. Restoration is therefore stochastic and independent of drift: the probability of restoring a parameter is the same regardless of how far the model has moved from the source. CoTTA also relies on a fixed EMA coefficient for teacher updates.

SANTA anchors adaptation through a source-distillation loss computed with a source model, combined with a contrastive alignment objective based on stored source prototypes. Its anchoring strength is not explicitly controlled by measured parameter drift. Each update requires an additional forward pass through the source model, and the method is formulated for image classification.

In contrast, DESA-TTA deterministically interpolates the updated student toward its source parameters after each step, with an anchoring strength that increases with the measured student drift (Eqs. 7-9). Anchoring therefore remains weak while the student is close to the source and strengthens as student drift increases, without any auxiliary loss or additional forward pass. Furthermore, DESA-TTA combines this anchoring with a dynamic EMA teacher update, which neither CoTTA nor SANTA includes.

Our existing ablations isolate the contribution of this anchoring rule. With dynamic EMA held fixed, we replace drift-dependent anchoring with stochastic restoration, L2L_2 regularization toward the source, or static anchoring (Tables 12 and 13):

MethodStyle AP50VOC-C AP50
CoTTA44.644.4
D-EMA + stochastic restoration (CoTTA)46.047.1
D-EMA + L2L_2 regularization45.144.2
D-EMA + static anchoring46.445.8
DESA-TTA (D-EMA + drift-dependent anchoring)47.348.5
AuthorsOpenReview ↗

Response to Reviewer sW6D (2/3)

W2. Relation to CoTTA and SANTA (continued).

These results demonstrate the additional benefit of drift-dependent anchoring over stochastic restoration, source-centered L2L_2 regularization, and static anchoring. DESA-TTA also remains efficient, running at 31 FPS with 0.021M adapted parameters on YOLO-World (Table 4). We have clarified these distinctions in Sec. 2 of the revised PDF (Teacher Stability and Drift Control in TTA).

W3. Consistency between Eq. (4) and Fig. 3.

We thank the reviewer for this careful check. The value Mmax⁡=300M_{\max}=300 stated in Sec. 4.1 of the original submission was incorrect: all reported experiments use Mmax⁡=30M_{\max}=30. In addition, the scores in the original Fig. 3 did not correctly reflect Eq. (4), and we have regenerated the figure. With Mmax⁡=30M_{\max}=30, the corrected scores are consistent with Eq. (4) and do not require unusually large numbers of retained detections per image. Both are corrected in the revised PDF, with Mmax⁡M_{\max} updated in Sec. 3.2 and Sec. 4.1, and the regenerated figure shown as Fig. 3.

The conclusion drawn from Fig. 3 is unchanged. The ranking of median teacher uncertainty is the exact inverse of the ranking of m⋆m^\star: Fog has the lowest median and the largest m⋆m^\star, and ClipArt has the highest median and the smallest m⋆m^\star.

W4 / Q3. Sequential domain shifts and image ordering.

Our method focuses on single-domain TTA, following prior VLOD TTA work, VLOD-TTA [1] and TTAOD-F [2]. Continual adaptation across sequential domains is outside the scope of our main evaluation.

Nevertheless, we evaluated DESA-TTA on the 15 VOC-C corruptions as a single continual stream, without resetting between corruptions. We also combined it with a shift detector based on the feature-statistics criterion of CTTAOD [3] (τ2=1.08\tau_2=1.08), which resets the student and teacher to the source model when a shift is detected. Since DESA-TTA does not access source data, the training-feature statistics required by this criterion are taken from the batch-normalization running statistics stored in the pretrained YOLO-World model. These results are reported in Appendix A.3 (Sequential Domain Shifts, Table 15) of the revised PDF. The last column lists the image index within each corruption at which a reset occurred.

CorruptionSingle-domain (Table 2)Continual, no resetContinual, shift detectionResets (image index)
Gaussian noise34.434.434.4–
Shot noise35.838.238.2–
Impulse noise37.838.838.8–
Defocus blur39.634.439.211
Glass blur32.734.934.1–
Motion blur37.835.936.8–
Zoom blur36.136.338.0–
Snow54.551.952.4–
Frost57.858.657.72165, 4734, 5234
Fog73.672.372.874
Brightness74.273.974.241, 895, 1530, 3583, 4083
Contrast63.362.062.28
Elastic transform59.856.355.617, 4514, 5210
Pixelate37.917.633.114
JPEG compression52.652.153.0–
Average48.546.548.015 resets

Without resetting, DESA-TTA achieves 46.5 AP50, compared with 48.5 under the single-domain protocol, and remains well above MT (39.5) and VLOD-TTA (39.6) evaluated in the single-domain protocol. It exceeds its single-domain results on five corruptions, consistent with beneficial carry-over across some domains. However, the effect varies across corruptions, with Pixelate decreasing from 37.9 to 17.6 AP50. With shift detection, the average improves to 48.0 AP50, within 0.5 of the single-domain result, and Pixelate recovers to 33.1 AP50. The detector triggers within the first 75 images of 6 of the 14 new corruptions, with 9 additional resets occurring later within corruptions. These results show that source resets triggered by shift detection mitigate the performance loss in the tested continual sequence, and that more accurate shift detection is a natural direction for future work.

Regarding image ordering, we additionally evaluate DESA-TTA on three random permutations of each target stream, keeping all other adaptation settings fixed. We report mean AP50 and standard deviation below and in Table 14 of the revised PDF. The original-order results vary the augmentation seed, whereas the new experiments vary image order.

SettingStyle AP50VOC-C AP50
Original order, 3 augmentation seeds (Table 14)47.3 ± 0.2448.5 ± 0.15
Random orders, 3 permutations47.1 ± 0.3248.6 ± 0.10

Mean AP50 differs from the original-order results by only 0.2 on Style and 0.1 on VOC-C. The standard deviation across the three permutations is 0.32 and 0.10 AP50, respectively, supporting the stability of DESA-TTA across the tested random image orders.

AuthorsOpenReview ↗

Response to Reviewer sW6D (3/3)

Q1. Proposition 1 under pseudo-label adaptation.

We agree that VS(r)V_S(r) and C(r)C(r) may vary with rr, since the student is trained on teacher pseudo-labels. However, the expression for the optimal coefficient in Proposition 1 remains valid under this dependence: teacher–student error dependence is explicitly included through C(r)C(r), and no independence assumption is required. Writing A(r)=VS(r)−C(r)A(r)=V_S(r)-C(r) and B(r)=VT(r)−C(r)B(r)=V_T(r)-C(r), the unconstrained one-step optimum is

m⋆(r)=A(r)A(r)+B(r)=11+B(r)/A(r).m^\star(r)=\frac{A(r)}{A(r)+B(r)}=\frac{1}{1+B(r)/A(r)}.

For A(r)>0A(r)>0 and A(r)+B(r)>0A(r)+B(r)>0, this coefficient decreases with rr whenever B(r)/A(r)B(r)/A(r) increases. Thus, VS(r)V_S(r) and C(r)C(r) need not remain fixed for a decreasing relationship to hold. Holding them fixed, with VS>CV_S>C and increasing VT(r)V_T(r), is one sufficient special case.

Empirically, rtr_t correlates negatively with pseudo-label precision and IoU across all evaluated Style datasets and VOC-C corruptions (Table 7), and Table 8 further shows its association with pseudo-label quality. Each image contributes a single student gradient update, after which source anchoring contracts the student's parameter distance from the source initialization (Proposition 2). These observations are consistent with the condition above and motivate the proposed rule, although they do not formally establish it.

The empirical results are also consistent with the mapping direction: reversing the uncertainty-to-EMA mapping lowers AP50 by 2.6 on Style and 5.2 on VOC-C (Table 9). We have added the general condition to Proposition 1 in Appendix A.1 of the revised PDF and clarified the conditional nature of the analysis.

References

[1] A. Belal, H. R. Medeiros, M. Pedersoli, and E. Granger. VLOD-TTA: Test-time adaptation of vision-language object detectors. In ECCV, 2026.

[2] Y. Gao, Y. Zhang, Z. Cai, and D. Huang. Test-time adaptive object detection with foundation model. In NeurIPS, 2025.

[3] J. Yoo, D. Lee, I. Chung, D. Kim, and N. Kwak. What, how, and when should object detectors update in continually changing test domains? In CVPR, 2024.

AuthorsOpenReview ↗

Follow-up on our response

Thank you again for your review. With only three days remaining in the discussion period, we would appreciate your feedback on whether our response and revised manuscript have addressed your concerns, and whether any points require further clarification.

Reviewer_B4zTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DESA-TTA, a test-time adaptation method for vision-language object detectors that addresses the sensitivity of fixed EMA updates and cumulative student drift. The method dynamically adjusts the teacher's EMA coefficient based on prediction confidence, detection density, and parameter drift, while source anchoring constrains the student parameters toward their pretrained initialization to prevent excessive drift. Experiments across multiple distribution shifts and detector architectures demonstrate consistent accuracy improvements and computational efficiency.

Critical strengths
  1. The proposed method achieves consistent improvements across diverse distribution shifts and different detector architectures. It also outperforming existing methods on style-shift, adverse-driving, and low-light benchmarks. The improvements on Grounding DINO further demonstrate the applicability of the approach beyond a single detection architecture.
  2. This paper is clearly written and well organized, with an intuitive motivation and a clear presentation of the proposed method. The ablation studies are comprehensive and systematically evaluate the contributions of the individual components.
Critical weaknesses

The method novelty is relatively incremental, and the theoretical justification for the dynamic EMA rule remains limited. Prior works have already explored adaptive EMA updates, such as TTE [1], while source parameter restoration has been investigated in CoTTA [2]; DESA-TTA mainly introduces alternative uncertainty and drift dependent scheduling rules for these established mechanisms. Proposition 1 establishes that the optimal EMA coefficient decreases with teacher error only when the student error and teacher-student error correlation are held fixed, an assumption that may not hold when the student is trained using teacher-generated pseudo-labels. The analysis does not directly justify the specific uncertainty score defined in Eq. 4 or the linear uncertainty mapping adopted in Eq. 6.

[1] Test-Time Ensemble via Linear Mode Connectivity: A Path to Better Adaptation. ICLR 2025.

[2] Continual Test-Time Domain Adaptation. CVPR 2022.

Questions for authors

Please refer to the critical weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer B4zT (1/2)

Thank you for your positive assessment and for recognizing the consistent improvements of DESA-TTA across distribution shifts and detector architectures, as well as the clarity of the paper and the comprehensiveness of our ablations. Your questions about the relation to TTE and CoTTA and about the theoretical justification of the dynamic EMA rule helped us articulate more precisely what distinguishes our design and which parts rest on analysis versus empirical evidence. Our modifications are highlighted in blue in the revised PDF.

W1. Relation to TTE and CoTTA.

We agree that adaptive EMA and source parameter restoration have been explored in prior work. However, DESA-TTA differs from these methods in how each mechanism is controlled.

Adaptive EMA. TTE [1] adjusts its EMA coefficient using the reverse KL divergence between the adapter predictions and the bias-corrected ensemble predictions, without conditioning it on parameter drift. It combines this averaging with dropout and a distillation objective, and is developed for image classification. DESA-TTA instead uses teacher detection confidence and box density, defined directly on the variable-size box sets produced by detectors, to select a sample-wise EMA coefficient within bounds determined by teacher drift from the source model. The update rate therefore reflects both uncertainty in the current detections and the accumulated change in the teacher parameters.

Source anchoring. CoTTA [2] stochastically restores individual parameters to their source values using a fixed restoration probability, independently of measured parameter drift. DESA-TTA applies deterministic interpolation toward the source after each student update, with an anchoring strength that increases with student drift (Eqs. 7-9). The anchored student then updates the teacher through the uncertainty- and drift-aware EMA.

Ablations. With D-EMA held fixed, Tables 12 and 13 compare alternative ways of constraining the student. Drift-dependent anchoring outperforms stochastic restoration, source-centered regularization, and static interpolation on both benchmarks:

Student constraintStyle AP50VOC-C AP50
Stochastic restoration (CoTTA)46.047.1
Source-centered L2L_2 regularization45.144.2
Static source anchoring46.445.8
Drift-dependent source anchoring (DESA-TTA)47.348.5

Our ablations further show that the two components are complementary: drift-dependent anchoring with fixed EMA reaches 46.3 / 42.2 AP50 (Style / VOC-C), D-EMA without anchoring reaches 46.6 / 43.1, and their combination in DESA-TTA reaches 47.3 / 48.5 (Table 5). On VOC-C, the combined gain over MT (+9.0 AP50) exceeds the sum of the individual gains (+2.7 and +3.6). We have clarified these distinctions from TTE and CoTTA in Sec. 2 of the revised PDF (Teacher Stability and Drift Control in TTA).

AuthorsOpenReview ↗

Response to Reviewer B4zT (2/2)

W2. Proposition 1 and the design of Eqs. (4) and (6).

Proposition 1 under pseudo-label adaptation. We agree that VS(r)V_S(r) and C(r)C(r) may vary with rr when the student is trained on teacher pseudo-labels. The expression for the unconstrained one-step optimum in Proposition 1 remains valid under this dependence, which is explicitly included through C(r)C(r). Writing A(r)=VS(r)−C(r)A(r)=V_S(r)-C(r) and B(r)=VT(r)−C(r)B(r)=V_T(r)-C(r), we obtain

m⋆(r)=A(r)A(r)+B(r)=11+B(r)/A(r).m^\star(r)=\frac{A(r)}{A(r)+B(r)}=\frac{1}{1+B(r)/A(r)}.

For A(r)>0A(r)>0 and A(r)+B(r)>0A(r)+B(r)>0, this coefficient decreases with rr whenever B(r)/A(r)B(r)/A(r) increases. Thus, a decreasing relationship can hold even when both VS(r)V_S(r) and C(r)C(r) vary with uncertainty. Holding VSV_S and CC fixed, with VS>CV_S>C and increasing VTV_T, is one sufficient special case. We have stated this general condition and clarified the conditional nature of the analysis in Proposition 1 (Appendix A.1) of the revised PDF.

Uncertainty score (Eq. 4). Proposition 1 provides conditional motivation for the mapping direction but does not prescribe a specific uncertainty measure. Eq. (4) combines mean pseudo-label confidence with the number of retained detections. The term −log⁡sˉt-\log \bar s_t increases as mean confidence decreases, while dt=Mt/Mmax⁡d_t=M_t/M_{\max} weights this term by the normalized number of pseudo-labels used for adaptation.

Table 5 directly evaluates the contribution of these two cues. All three variants below use teacher-drift-dependent EMA bounds with source anchoring disabled:

Uncertainty signalStyle AP50VOC-C AP50
Confidence only46.142.7
Confidence and density46.643.1
Density only43.841.3

Combining both cues improves over either cue alone. Furthermore, rtr_t correlates negatively with pseudo-label precision and IoU on every Style dataset and VOC-C corruption, including after controlling for ground-truth object count (Table 7). Table 8 also shows lower precision and IoU in the highest uncertainty quintile than in the lowest on both benchmarks. These results support the score as an indicator of pseudo-label quality.

Linear mapping (Eq. 6). Under the condition above, Proposition 1 supports a decreasing relationship without prescribing its functional form. We use linear interpolation between the drift-dependent bounds as a simple monotone mapping that introduces no additional shape parameters and keeps mtm_t within [mmin⁡(δtT),mmax⁡(δtT)][m_{\min}(\delta^T_t),m_{\max}(\delta^T_t)] for every rt∈[0,1]r_t\in[0,1].

Table 9 provides empirical support for the mapping direction. Reversing it, while retaining the same drift-dependent bound functions and source-anchoring rule, lowers AP50 by 2.6 on Style and 5.2 on VOC-C. This comparison supports the decreasing direction, but does not establish that the linear form is optimal.

In the revised PDF, we have clarified in Sec. 3.2 and Appendix A.1 that Proposition 1 provides conditional motivation, while the uncertainty score and bounded linear interpolation are practical design choices evaluated through the reported analyses and ablations.

References

[1] B. Kim, C. Ahn, W. J. Baddar, K. Kim, H. Lee, S. Ahn, S. Han, S. Suh, and E. Yang. Test-time ensemble via linear mode connectivity: A path to better adaptation. In ICLR, 2025.

[2] Q. Wang, O. Fink, L. Van Gool, and D. Dai. Continual test-time domain adaptation. In CVPR, 2022.

Thank you for the response. While my concern regarding the incremental method novelty remains, I will maintain my original positive rating.

AuthorsOpenReview ↗

Response to Reviewer B4zT

Thank you for considering our response and for your constructive feedback and continued support for our work. Regarding novelty, neither TTE nor CoTTA jointly regulates teacher averaging and source anchoring using measured teacher and student drift. DESA-TTA introduces this coordinated control for online object detection, with matched ablations demonstrating complementary benefits from the interaction between the two mechanisms.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.