acceptodds
Under review as a conference paper at ICLR 2027

When and Why Does Test-Time Adaptation Improve Label Differentially Private Models?

Abstract

Label differential privacy (LabelDP) protects sensitive training labels but often degrades accuracy. We study when test-time adaptation (TTA) can recover part of this lost utility. Since TTA uses only the released model and unlabeled inputs, it incurs no additional training-label privacy cost. We first show that symmetric randomized-response (RR) noise contracts the population label posterior toward uniform without changing the Bayes decision, biasing training toward a uniform-label objective with a privacy-dependent excess clean risk. We then characterize empirically when adaptation improves a label-private model. On clean CIFAR-10, TENT lowers the error of every LabelDP model, with gains growing from 0.87 to 3.62 points as privacy strengthens, while it increases the error of a matched non-private model. These improvements come with lower calibration error and vanish when the source model is near chance. Under common corruptions, the robustness benefits of TTA transfer to LabelDP-trained models, mostly through re-estimating batch-normalization (BN) statistics: across four TTA methods, entropy-based updates help only when combined with target-batch BN statistics. We evaluate on CIFAR-10/100, their corruption benchmarks, and ImageNet64 subsets, and identify small batches and label-correlated streams as failure conditions. With sufficiently large test batches that are not strongly label-skewed, TTA is a practical way to keep label-private models useful with or without distribution shift.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
label differential privacy, test-time adaptation, differential privacy, randomized response, distribution shift, model calibration

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_1eCSOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies whether test-time adaptation (TTA) can recover utility lost when classifiers are trained under label differential privacy (LabelDP), without consuming additional privacy budget for the protected training labels. The paper combines existing LabelDP mechanisms, primarily LP-MST, with source-free TTA methods including batch-normalization adaptation and TENT.

On the theoretical side, the authors show that symmetric randomized response contracts the class posterior toward the uniform distribution while preserving the Bayes argmax, and that its population cross-entropy objective can be written as a mixture of the clean objective and a uniform-label objective. Under additional PL, quadratic-growth, and regularity assumptions, they derive bounds on clean-risk excess and changes in population BN statistics. They separately analyze convergence of entropy minimization under a fixed TENT objective. The paper carefully notes that these results do not directly guarantee that TTA improves classification accuracy.

Empirically, the paper reports that TENT improves clean CIFAR-10 performance at every evaluated finite privacy budget, with the improvement increasing from 0.87 percentage points at �0��0� to 3.62 points at �1��1�, while degrading a matched non-private model by 0.79 points. On corrupted data, TTA produces much larger gains, although the non-private model benefits at least as much, and the experiments suggest that most of this robustness gain comes from recomputing target-batch BN statistics rather than entropy minimization itself.

The paper further studies source-model quality, batch size, stream composition, calibration, prediction flips, pretrained representations, and several TTA variants. Notably, TTA fails when the source model is near chance, with very small batches, or under strongly label-correlated test batches.

Critical strengths
  1. The problem is interesting and practically relevant. Connecting LabelDP with deployment-time adaptation is a natural but underexplored question. The post-processing argument is clean: under the stated threat model, adaptation using only the released model and unlabeled inputs does not spend additional privacy budget for the original training labels.
  2. The empirical characterization is considerably stronger than a simple “TENT improves accuracy” result. I particularly appreciate the investigation of batch size, stream composition, source quality, arrival-order dependence, calibration, and individual prediction flips. The negative results are informative. For example, at �0��0�, a label-correlated stream increases error from 23.86% for source inference to 62.96% for BN adaptation and 66.07% for TENT.
  3. The main CIFAR-10 LabelDP results have reasonable seed analysis. The five-seed results support the main TENT improvements; e.g., clean errors at �1��1� move from �2��2� to �3��3�. Corruption gains are also much larger than the reported variability.
  4. The paper is unusually explicit about its limitations. In particular, it acknowledges the gap between symmetric-RR theory and LP-MST experiments, fixed-objective gradient-descent theory and streaming Adam-based TENT, and population BN statistics versus the actual target-statistics mismatch.
Critical weaknesses

See questions.

Questions for authors
  1. Could the authors provide a matched softened-supervision control? In particular, I would find a non-private model trained with label smoothing matched to �0��0�, ideally with the same mixup configuration and seeds, highly informative. If TENT exhibits the same clean-data improvement there, how would the authors revise the claimed mechanism?
  2. Could the non-private baseline be repeated across multiple seeds and reported as paired source-to-TTA changes? This would strengthen the central observation that TTA changes from beneficial under LabelDP to harmful without privatization.
  3. For Theorem 5, what guarantees existence of the finite entropy minimizer �1��1�? If this is not guaranteed for the parameterization considered, could the statement be reformulated in terms of an infimum or an explicitly restricted parameter domain?
3 comments

Thank you for the careful review and the precise questions. The control the reviewer proposed turned out to be very informative. We show that matched label smoothing does not achieve the same clean data gains, which points to label randomization as the source for the gains.

Q1. Matched softened supervision with Label Smoothing

We trained label-smoothing controls using clean labels smoothed by 1−ρ(ε)1-\rho(\varepsilon) = 0.853, 0.610 and 0.157 (ε\varepsilon = 1, 2, 4), with the same LP-MST pipeline and seeds as the private models and α=4\alpha=4 mixup. At ε=8\varepsilon=8 the coefficient is 0.003, so we use the non-private model with the same α=4\alpha=4 mixup as the ε=8\varepsilon=8 private model.

The label-smoothing controls on their own. CIFAR-10 test error (%) over 5 seeds. Change is TENT minus source, paired per seed:

Label smoothing (matched ε\varepsilon)Clean: sourceClean: TENTClean: changeSev-5: sourceSev-5: TENT
0.853 (ε=1\varepsilon=1)5.946.73+0.79 ±\pm 0.3841.1515.54
0.610 (ε=2\varepsilon=2)6.106.19+0.08 ±\pm 0.1640.2215.14
0.157 (ε=4\varepsilon=4)6.806.30−0.50 ±\pm 0.3240.0816.44

Under the corruption dataset, they benefit like every other model. On clean data, TENT does not show the gain it gives the matched LabelDP models:

ε\varepsilonLabelDP: sourceLabelDP: TENTLabelDP: changeLabel smoothing: changeExtra gain under LabelDP
145.8942.12−3.76 ±\pm 1.56+0.79 ±\pm 0.384.55
215.9513.18−2.78 ±\pm 0.81+0.08 ±\pm 0.162.86
48.947.50−1.43 ±\pm 0.64−0.50 ±\pm 0.320.93

To answer your question, since TTA does not reproduce the gain on label smoothing, the mechanism stands, and the control makes the argument sharper. Lemma 2's expected target is what private training aims at. Training on that target alone costs no accuracy and leaves nothing to recover. The recoverable loss comes from randomized response fixing one noisy label per example. Training on those labels leaves BN statistics that do not match clean inputs, and an underconfident but usable ranking. TTA fixes both, and the extra gain decreases as the amount of label randomization decreases, from 4.55 points at ε=1\varepsilon=1 near to zero at ε=8\varepsilon=8. Proposition 1 and Lemma 2 explain why the private target is underconfident and why its ranking is worth recovering.

Q2. Multi-seed control and paired changes

Five seeds each, paired per seed, test error:

Non-private controlSourceBNTENTBN − sourceTENT − source
CIFAR-10, no mixup8.5810.449.19+1.86 ±\pm 0.35+0.61 ±\pm 0.43
CIFAR-10, α=4\alpha=4 mixup7.777.966.96+0.20 ±\pm 0.14−0.81 ±\pm 0.23

Severity 5:

Non-private controlSourceBNTENTBN − sourceTENT − source
CIFAR-10, no mixup54.8828.2325.39−26.65 ±\pm 0.74−29.49 ±\pm 0.72
CIFAR-10, α=4\alpha=4 mixup41.4923.8518.36−17.64 ±\pm 1.19−23.13 ±\pm 1.18

Without mixup, as in the submission, TTA raises the clean error of the non-private control. With α=4\alpha=4 mixup, we observe a small gain for TTA on CIFAR-10. Our revision compares the LabelDP gains with this non-private control and with the label-smoothing control.

Table 1 and Table 15 (formerly Table 10) now report five-seed means: at ε=2,4,8\varepsilon=2, 4, 8, clean error goes from 15.95, 8.94 and 7.28 before TENT to 13.18, 7.50 and 6.52 after. The paired gains agree with our original seed within one standard deviation.

Q3. Theorem 5

Thank you for catching that. We restate the theorem with H∗:=inf⁡ψH(ψ)H^{\ast}:=\inf_{\psi}H(\psi) over the stated domain and Δt:=H(ψt)−H∗\Delta_t:=H(\psi_t)-H^{\ast}. The proof only uses H(ψ†)H(\psi^\dagger) as a lower bound, so it goes through unchanged:

H(ψT)−H∗≤(1−μHLH)T[H(ψ0)−H∗].H(\psi_T)-H^{\ast}\le\Big(1-\frac{\mu_H}{L_H}\Big)^T\big[H(\psi_0)-H^{\ast}\big].

The same assumptions then give the minimizer back. Each step has length ∥ψt+1−ψt∥=∥∇H(ψt)∥/LH≤2Δt/LH\|\psi_{t+1}-\psi_t\|=\|\nabla H(\psi_t)\|/L_H\le\sqrt{2\Delta_t/L_H}, which decays geometrically. So the path has finite length, and the iterates converge to some ψ∞\psi_\infty with H(ψ∞)=H∗H(\psi_\infty)=H^*. Defining ψ†:=ψ∞\psi^\dagger:=\psi_\infty keeps Corollary 6 valid as stated.

The PL condition cannot hold along a path that drifts toward infinite scale, corresponding to the confidently-wrong collapse that can occur under entropy minimization.

Thanks for the reply. I have raised the score to support this paper acceptance.

Thank you for raising your score and for your helpful feedback! We will include the promised revisions in the final version.

Reviewer_MtCnOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies test-time adaptation (TTA) for improving the utility of models trained under label differential privacy (LabelDP). Since TTA uses only the released model and unlabeled test inputs, it incurs no additional training-label privacy loss. The authors show that symmetric randomized response biases training toward a uniform-label objective, reducing prediction confidence. To recover some of the lost utility, the paper investigates two adaptation mechanisms: re-estimating batch-normalization (BN) statistics on test batches and minimizing prediction entropy by updating the BN affine parameters (TENT).

Critical strengths
  • The paper studies how existing TTA methods can improve the utility of LabelDP models without incurring additional training-label privacy loss.
  • The theoretical analysis provides useful insights into how randomized response affects the training objective and prediction confidence.
  • The experimental evaluation is extensive, covering different privacy budgets, datasets, TTA methods, and distribution shifts.
  • The paper investigates when TTA fails, particularly with small test batches, label-correlated streams, and models with poor initial accuracy.
Critical weaknesses
  • The convergence of entropy minimization does not imply an improvement in classification accuracy, which is the main objective of the paper. Although the authors discuss a gradient-alignment condition, they do not establish whether it holds in their setting.
  • The theoretical analysis focuses on symmetric randomized response, whereas the main experiments use LP-MST, for which the posterior contraction result does not directly apply. This creates a gap between the theoretical analysis and the experimental evaluation.
Questions for authors
  • Can the authors provide a stronger theoretical connection between entropy minimization and improvements in classification accuracy? In particular, is it possible to establish or empirically verify the gradient-alignment condition discussed in Remark 2?
  • Can the authors clarify to what extent the theoretical results for symmetric randomized response extend to LP-MST, which is used in the main experiments?
3 comments

Thank you for the review. We add a result that links lower entropy to lower clean risk for LabelDP models, measure the alignment condition, and extend Proposition 1 to RRWithPrior, the mechanism inside LP-MST.

Q1. Entropy minimization and accuracy

We agree that entropy convergence does not necessarily imply lower classification error. Our explanation is that LabelDP produces underconfident predictions, and entropy minimization can move these predictions toward lower clean risk. A new result makes this precise (App. F.13). Take the symmetric-RR target q=ρπ+(1−ρ)uq=\rho\pi+(1-\rho)u from Proposition 1 and its temperature family pβ∝qβp_\beta\propto q^\beta. Then

ddβR(pβ)∣β=1=−(1−ρ) Ex∑j(πj−1K)(log⁡qj+log⁡K)≤0.\frac{d}{d\beta}R(p_\beta)\Big|_{\beta=1}=-(1-\rho)\,\mathbb{E}_x\sum_j\Big(\pi_j-\tfrac{1}{K}\Big)\big(\log q_j+\log K\big)\le 0.

Each term is nonnegative because qj−1K=ρ (πj−1K)q_j-\tfrac1K=\rho\,(\pi_j-\tfrac1K). So sharpening the private target lowers clean risk while entropy falls, at a rate proportional to 1−ρ(ε)1-\rho(\varepsilon). Label randomization provides an additional benefit from TENT that is not observed to the same extent in the non-private setting. Sharpening alone keeps the predicted class, so accuracy gains also need ranking changes, which come through the shared BN parameters and statistics. We measure those directly (Table 8 and our response to Lq2q).

Q1. Gradient Alignment Condition

We evaluate the condition in Remark 2 by measuring the cosine similarity between the BN-affine gradients of clean cross-entropy and entropy, on clean CIFAR-10. Test labels are used only for this diagnostic.

Remark 2's condition is local. For a small entropy step of size η\eta on the BN affine parameters ψ\psi, a first-order expansion gives

R(ψ−η∇ψH)≈R(ψ)−η ⟨∇ψR,∇ψH⟩,R(\psi-\eta\nabla_\psi H)\approx R(\psi)-\eta\,\langle\nabla_\psi R,\nabla_\psi H\rangle,

so a small entropy step also lowers the clean cross-entropy whenever this inner product is positive, that is, when the two gradients are roughly the same direction. We measure it as a cosine, which has the same sign: cos⁡=⟨∇ψR,∇ψH⟩∥∇ψR∥ ∥∇ψH∥\cos=\frac{\langle\nabla_\psi R,\nabla_\psi H\rangle}{\|\nabla_\psi R\|\,\|\nabla_\psi H\|}, on clean CIFAR-10 test batches. Here RR is the batch cross-entropy with the true labels, used only for this measurement, and HH is the batch's mean prediction entropy. We report it at the source parameters and during TENT, next to clean error:

ModelSourceBNTENTCosine at sourceCosine during TENT
LabelDP ε=1\varepsilon=145.8941.8242.12+0.37+0.00
LabelDP ε=2\varepsilon=215.9515.3313.18+0.66+0.24
LabelDP ε=4\varepsilon=48.948.747.50+0.61+0.16
LabelDP ε=8\varepsilon=87.287.386.52+0.48+0.08
Non-private control, no mixup8.5810.449.19−0.26−0.26

The condition holds at the source for every LabelDP model and fails for the non-private model. Among the LabelDP models, alignment during TENT follows the same ordering as the entropy step's accuracy gains (from BN to TENT) does: 2.16 points at ε=2\varepsilon=2, 1.23 at ε=4\varepsilon=4, 0.86 at ε=8\varepsilon=8, and nothing at ε=1\varepsilon=1, where alignment falls to zero.

We include this experiment in our revised manuscript.

Q2. From symmetric RR to LP-MST

Thank you for asking us to clarify the relationship between symmetric RR and LP-MST. We have added a short conditional calculation for the RRWithPrior privatization step already used in our experiments, in our revised manuscript (See Appendix F.12).

At a high level, RRWithPrior restricts each label to the top-kk classes SS of a prior, then applies randomized response within that set. Given the prior, the privatized posterior is given by:

π~j(x)=ρk πj(x)+1−ρk s(x)k  (j∈S),π~j(x)=0  (j∉S),\tilde\pi_j(x)=\rho_k\,\pi_j(x)+\frac{1-\rho_k\,s(x)}{k}\ \ (j\in S),\qquad \tilde\pi_j(x)=0\ \ (j\notin S),

where s(x)=∑j∈Sπj(x)s(x)=\sum_{j\in S}\pi_j(x) and ρk=(eε−1)/(eε+k−1)\rho_k=(e^\varepsilon-1)/(e^\varepsilon+k-1).

Within the set SS, the class ranking is kept and every margin shrinks by the factor ρk\rho_k, which is at least ρ(ε)\rho(\varepsilon). Classes outside the set get zero probability. Consequently, the mechanism preserves rankings within SS, but preserves the global Bayes class only when that class is included in SS. Setting SS to the full label set recovers Proposition 1 in the submitted manuscript.

Thank you for the detailed response. I have decided to maintain my positive score.

Thank you very much for your feedback and positive assessment. We will include the revisions in the final manuscript.

Reviewer_Lq2qOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks: After training labels have been protected with differential privacy, can some model utility be recovered using only unlabeled inputs at deployment, while preserving the original training-label privacy guarantee?

The authors adopt a two-stage pipeline: they first train a label-private model using LP-MST, then apply test-time adaptation through BN statistics re-estimation or TENT. SAR, MEMO, and their variants are used to analyze the adaptation components. The theoretical analysis examines how symmetric randomized response changes the population training objective and establishes convergence properties of idealized entropy minimization. The experiments investigate privacy budgets, clean and corrupted data, source model quality, batch size, and input stream composition.The algorithmic novelty is limited, but the problem perspective has independent value. Both BN re-estimation and TENT updates come from existing methods; the original TENT paper already includes both components. Moreover, Das et al. have demonstrated that hard-label retraining can improve LabelDP models without consuming additional label-privacy budget. Thus, improving utility after private training at no additional privacy cost is not, by itself, unique to this paper.

The paper’s added perspective is to use unlabeled adaptation at deployment to recover LabelDP utility, particularly by observing benefits in some clean-data settings and systematically examining the roles of normalization, entropy updates, and deployment conditions. This differs meaningfully in its use case from approaches that retrain on source data. The contribution is primarily an empirical study with clearly defined boundaries. The theory provides a partial explanation but does not fully answer the “why” in the title.

Critical strengths
  1. The paper introduces a practically meaningful deployment perspective and establishes a distinctive clean-data finding.

The paper extends utility optimization for LabelDP to the deployment stage. The TENT gains of 0.87–3.62 percentage points on clean CIFAR-10, together with the opposite effect observed for the matched non-private model, warrant investigation. These results suggest that static inference may not fully exploit the information available after private training. This is a concrete, empirically supported observation that can motivate further research on adaptation for LabelDP models.

  1. The experimental coverage is sufficient to constitute a complete study, and failure conditions are carefully reported.

The experiments cover multiple privacy budgets, CIFAR-10/100 and their corrupted variants, CLIP priors, ImageNet64 subsets with a DINO backbone, multiple adaptation components, repeated runs, prediction flips, and stream composition. Their value lies not only in the number of experiments but also in how they address whether adaptation works, which components contribute most, and when it fails. For example, the label-correlated stream in Table 4 increases TENT’s error from the source model’s 23.86% to 66.07%, placing a clear restriction on deployment recommendations. For an empirical paper with a limited scope, the amount of work and organization of the results are sufficient to form a complete study. Covering every modality, architecture, and privacy mechanism is unnecessary for its contribution to hold.

Critical weaknesses
  1. The central explanation of “why it works” has not been fully established theoretically or empirically.

Symmetric RR preserves the population Bayes decision rule. Consequently, confidence contraction alone cannot explain the loss in accuracy, much less establish that entropy minimization can recover it. What needs explanation is how finite-sample effects, representation learning, and shared BN updates produce beneficial changes in class rankings. The existing risk bounds, BN sensitivity bound, and entropy convergence bound do not establish this connection; calibration and prediction-flip analyses also remain descriptive evidence. In particular, on clean CIFAR-10 at ε=1, BN alone improves performance by 3.76 percentage points, compared with 3.62 points for TENT, showing that the largest overall gain in this setting does not depend on entropy updates. The evidence for the role of normalization is stronger than the explanation that privacy induces high entropy and entropy minimization therefore restores accuracy. This is the main limitation on the depth of the contribution.

  1. Key controls are missing for distinguishing privacy-related effects from general training effects, and source model quality limits generalization.

Lemma 2 already shows that the expected objective under symmetric RR is equivalent to label smoothing with the corresponding strength; training also uses mixup. Without controls for label smoothing and mixup, it is difficult to distinguish effects specific to label randomization from those of softened supervision more generally, source model quality, or statistical mismatch. Equivalence of the expected objectives does not imply that the two finite-sample training procedures are identical, so empirical comparisons are still needed. Relatively weak source models introduce further uncertainty. For example, the source accuracy on CIFAR-100 at ε=4 is 26.77% in this paper, whereas Table 2 of the original LP-MST paper reports 50.15% for LP-2ST. Cross-paper numbers do not constitute a strictly matched comparison, but this gap warrants an explanation of the training differences.

The comparison with hard-label retraining in Appendix C.5 should also remain exploratory: the authors have not validated the conditions under which the original approach performs well, so these results cannot establish an overall advantage over existing LabelDP post-processing approaches.

Questions for authors
  1. Statistical robustness: Are the central values in Table 10 means over five seeds? Could you report paired source-to-TENT gains at each privacy budget and conduct the same multi-seed evaluation for the non-private control to establish whether the reversal in the direction of the effect is robust?
  2. Distinguishing mechanisms: Do you have label-smoothing controls under the same training setup, as well as TENT results that keep source BN statistics fixed and update only the BN affine parameters? These results would help identify which components drive the clean-data gains and how those gains relate to label randomization.
  3. Source model differences: Which training configurations primarily account for the gap between your results and the original LP-MST results? Do the clean-data gains and their trends across privacy budgets persist for source models that more closely reproduce the original results?
4 comments

Thank you for the careful review. The new controls help clarify where the clean-data gains come from. We show that TTA's clean-data gain on LabelDP models comes from label randomization, and it grows with the amount of randomization.

We use two different control settings. Both train on clean labels with the same pipeline and seeds as the private models.

  • The label-smoothing control smooths the labels by 1−ρ(ε)1-\rho(\varepsilon) (0.853, 0.610, 0.157 for ε=1,2,4\varepsilon=1, 2, 4) with α=4\alpha=4 mixup, which is Lemma 2's expected target for the private model.
  • The non-private control uses no smoothing (trained with mixup and no-mixup regularization).

Q1. Seeds and paired gains

The revision reports five-seed means and paired gains in Table 1. The central values in Table 15 (Formerly Table 10) have now been updated. The paired gains agree with the single run within one standard deviation (−3.62, −2.63, −1.08, −0.87), and the gain still grows monotonically with privacy.

The non-private control over 5 seeds, clean error (%):

Non-private control (ε=∞\varepsilon=\infty)SourceBN − sourceTENT − source
No mixup (as in the submission)8.58 ±\pm 0.34+1.86 ±\pm 0.35+0.61 ±\pm 0.43
α=4\alpha=4 mixup regularization7.77 ±\pm 0.24+0.20 ±\pm 0.14−0.81 ±\pm 0.23

Without mixup, BN and TENT both raise the error for the non-private control over five seeds. With α=4\alpha=4 mixup, which is close to how the private models are trained, TENT gives the control a small gain in accuracy.

W1, Q2. Where the clean-data gain comes from

Proposition 1 keeps the Bayes ranking, so the loss must come from finite-sample effects, as the reviewer suggested. BN re-estimates the BN statistics on each test batch, and TENT then adds an entropy step on the BN affine parameters.

Clean error (%) with the paired change source-to-TENT gains

ModelSourceBNTENTTENT − source
LabelDP ε=1\varepsilon=145.89 ±\pm 0.7941.82 ±\pm 1.0042.12 ±\pm 1.13−3.76 ±\pm 1.56
LabelDP ε=2\varepsilon=215.95 ±\pm 0.7415.33 ±\pm 0.4213.18 ±\pm 0.26−2.78 ±\pm 0.81
LabelDP ε=4\varepsilon=48.94 ±\pm 0.648.74 ±\pm 0.227.50 ±\pm 0.03−1.43 ±\pm 0.64
LabelDP ε=8\varepsilon=87.28 ±\pm 0.207.38 ±\pm 0.216.52 ±\pm 0.14−0.76 ±\pm 0.28
Non-private control7.77 ±\pm 0.247.96 ±\pm 0.136.96 ±\pm 0.15−0.81 ±\pm 0.23
Label-smoothing control, 0.8535.94 ±\pm 0.246.47 ±\pm 0.216.73 ±\pm 0.23+0.79 ±\pm 0.38
Label-smoothing control, 0.6106.10 ±\pm 0.166.64 ±\pm 0.156.19 ±\pm 0.15+0.08 ±\pm 0.16
Label-smoothing control, 0.1576.80 ±\pm 0.306.91 ±\pm 0.116.30 ±\pm 0.09−0.50 ±\pm 0.32

The label-smoothing control trains on Lemma 2's expected target, and we do not observe the same TTA gains. The LabelDP gain exceeds it by 4.55, 2.86 and 0.93 points at ε=1,2,4\varepsilon=1, 2, 4. At ε=8\varepsilon=8 the matched smoothing is only 0.003, so we compare with the non-private control, and the two gains are similar (0.76 and 0.81 points). What TTA recovers is the damage from training on one fixed noisy label per example.

First, under strong randomization, the stored BN statistics no longer match clean inputs. Re-normalization alone lowers error from 45.89 to 41.82 at ε=1\varepsilon=1, and it helps none of the controls. Because the statistics are shared, it moves every logit at once, and at ε=1\varepsilon=1 it fixes about twice as many predictions as it breaks (Shown in Table 8). Second, the model keeps an underconfident but usable ranking, which the entropy step sharpens: 2.16 points at ε=2\varepsilon=2 (from 15.33 to 13.18), about twice the non-private control's 1.01.

At ε=1\varepsilon=1, the source model is only 54% accurate. In this case, there is relatively little for the entropy step to improve, and most of the gain comes from re-normalization. On the theory side, a new result shows that sharpening the underconfident private target lowers clean risk, with a factor 1−ρ(ε)1-\rho(\varepsilon) that vanishes without privacy.

Q2. TENT with Source BN statistics Fixed

We performed the suggested experiment, TENT with the source BN statistics kept fixed, updating only the BN affine parameters, clean error (%):

ModelSourceTENTTENT, source statistics fixed
LabelDP ε=1\varepsilon=145.8942.1279.33 ±\pm 3.62
LabelDP ε=2\varepsilon=215.9513.1843.30 ±\pm 20.68
LabelDP ε=4\varepsilon=48.947.5010.83 ±\pm 2.81
LabelDP ε=8\varepsilon=87.286.528.30 ±\pm 0.57
Non-private control7.776.969.06 ±\pm 0.74

Without re-estimated batch statistics, the entropy step makes every model worse, is unstable across seeds at ε=2\varepsilon=2, and even reaches 82 to 88% error under severity-5 corruption. MEMO shows the same pattern (Table 2): with source statistics it stays close to the unadapted model, and it helps only with test-batch statistics. For both methods, the entropy step helps only on top of re-normalization.

W2. Controls and mixup

All privacy claims in the revision use these controls (table above). The most informative comparison is ε=8\varepsilon=8, where the private and control use the same mixup regularization: there the extra gain is near zero. It grows only as label noise grows.

LP-MST uses stronger mixup at small ε\varepsilon (table under Q3). Mixup can affect TTA in two ways, but neither explains the extra gain for label private trained models.

  1. Softer targets. Larger mixup lowers the average top target from 0.64 (α=4\alpha=4) to 0.57 (α=16\alpha=16). Our label-smoothing control at ε=1\varepsilon=1 trains on far softer targets (top target 0.23, plus α=4\alpha=4 mixup), and TTA does not improve accuracy (+0.79).
  2. Blended inputs to BN. Running BN statistics are dominated by the final training steps, which use stage 2. At ε=4\varepsilon=4, stage 2 uses α=4\alpha=4, the same as the controls, and the extra gain is still 0.93 points. At ε=8\varepsilon=8, both stages use α=4\alpha=4, and the private and control gains are close (0.76 and 0.81).

Q3. Source quality

We use the exact published repo for experiments from Ghazi et al. (2021) (Containing only CIFAR10) and the official TENT/BN code in PyTorch. For CIFAR-100, we use the listed batch size and learning rate (256 and 0.2). Their paper does not give mixup α\alpha, stage split or prior temperature for CIFAR-100. Following their guidance, we use:

ε\varepsilonMixup α (stage 1, stage 2)Stage-1 data sharePrior temperature
116, 880%0.6
28, 870%0.5
48, 460%0.5
84, 460%0.5

Part of the performance gap is present even without privacy. With α=4\alpha=4 mixup, our non-private ResNet-18 reaches 64.8% accuracy on CIFAR-100 against their 76.4%. With a stronger source model, TTA still improves accuracy at every privacy budget. Every comparison we make is between models trained with the same recipe, so the weaker baseline may change how large the gains are, but it should not change whether TTA helps. With the CLIP prior (Table 7), the ε=8\varepsilon=8 source is within 1.9 points of the non-private reference, and TENT still lowers clean error at every budget, by 1.84 to 7.40 points.

W3. Hard-label retraining

Appendix C.5 is an exploratory comparison by design, and we make no claim of advantage over Das et al. Our contribution is a different setting: improving a released LabelDP model at deployment from unlabeled target inputs, without retraining. We will emphasize this in our revised manuscript.

Thanks for the further clarification and experiment evidences. I will keep my positive score unchanged.

Thank you very much for your support and helpful feedback. We will include the discussed revisions in the final version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.