acceptodds
Under review as a conference paper at ICLR 2027

Residual Transport in Batch-Normalization Adaptation: Mechanism and Parameter-Free Correction

Abstract

Replacing BatchNorm (BN) statistics with moments estimated from an unlabeled target domain is a useful test-time adaptation baseline, yet partial replacement can help, do nothing, or harm. We isolate one mechanism behind this variability. If a BN layer keeps its source mean but adopts a target variance, the scale change transports the BN output mean by a signed, channel-specific amount . We estimate this quantity on a calibration split and test it on disjoint evaluation images. Across CIFAR-10-C and a bounded external ImageNet-C blur-family replication with pretrained ResNets, the calibration-only prediction closely tracks the held-out observation. On fully trained models, injecting the predicted into an otherwise frozen network degrades downstream performance, and exact-vector channel permutation substantially attenuates the effect, confirming that channel alignment matters. Relative to variance-only replacement, aligned residual compensation shows positive aggregate gains at both training durations and on both external ImageNet ResNet architectures, while permuted compensation yields a smaller gain. The mechanism yields a direct algorithm: Residual-Balanced BN (RB-BN) keeps the target scale and installs a mean that preserves the frozen target expected BN output, exactly realizing through BN statistics alone. In an all-layer study on CIFAR-10-C and CIFAR-100-C with two independently trained 100-epoch checkpoints per dataset, all 15 corruptions, two calibration budgets, and disjoint evaluation images, sequential RB-BN improves accuracy over frozen inference by and on CIFAR-10-C and CIFAR-100-C, respectively; sequential AdaBN improves by and . RB-BN uses no mixing weight. The evidence supports residual transport as a measurable, channel-aligned causal component of BN adaptation and provides a parameter-free correction derived from that mechanism.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
batch normalization, test-time adaptation, covariate shift, residual transport, channel alignment, causal intervention, CIFAR-10-C, corruption robustness

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_T114OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies residual transport, which is caused by replacing the variance in a BatchNorm layer while retaining its source mean. The authors verify a signed, channel-wise shift and evaluate its prediction on disjoint calibration and evaluation images. Moreover, they use interventions, sign reversal, and channel permutation to examine its effect on downstream predictions. Based on above observations, they propose Residual-Balanced BN (RB-BN). Experiments over a series backbone show the promising results.

Critical strengths
  1. The paper uses disjoint calibration/evaluation splits, signed interventions, and exact-vector channel permutations, which provide useful evidence that the identified transport has a channel-dependent effect on downstream predictions.
  2. RB-BN follows from an explicit expectation-preservation constraint and requires no gradient-based optimization or tuned mixing weight.
Critical weaknesses
  1. On CIFAR-10-C and CIFAR-100-C, sequential RB-BN only matches sequential AdaBN within the reported uncertainty.
  2. RB-BN preserves the average output of the frozen BN layer on target data. In contrast, AdaBN uses the target mean to center the features. It is unclear why preserving the original output is more beneficial for adaptation. Although the experiments support the proposed residual transport mechanism, they do not fully explain why this preservation objective should improve prediction performance.
Questions for authors
  1. The ImageNet-C calibration splits use a fixed number of images per WNID. Does constructing these splits require class labels, and do the conclusions hold under class-agnostic unlabeled sampling?
  2. What evidence links the benefit of local residual compensation to the final all-layer performance, rather than merely showing that the injected channel-wise shift affects the frozen network?
Reviewer_EuLQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how replacing only the variance in batch normalization with target variance changes the expected output when the target distribution is different from the source distribution. It derives a channel-wise expression for this shift. Based on the 'variance-only' replacement, it proposes RB-BN, which compensates for the shift by adjusting the BN centering mean. Experiments on CIFAR-C and ImageNet-C examine this mechanism and evaluate the resulting adaptation method.

Critical strengths
  1. The paper provides an analytical expression for the output mean shift and derives a closed-form correction that requires no gradient updates or target labels.
Critical weaknesses
  1. The paper is very hard to read. It introduces many non-intuitive terms without sufficiently clear explanations, e.g., “aligned intervention,” “sign control,” and “location control” in section 3, and “within-condition,” “short matrix,” “fully trained matrix,” and “PlainBN networks” in section 5. These are just a few examples throughout the paper. They make it difficult to follow the motivation and understand what each experiment is intended to show.

  2. The motivation is unclear across the problem formulation, the proposed method, and experiments:

  • It is unclear why replacing only the BN variance is an important setting to study for target-domain adaptation. Although this isolates the interaction between mean mismatch and scale changes, the paper does not sufficiently explain its practical advantage.
  • RB-BN is based on the prerequisite that only variance is replaced. However, it is unclear why this is a better adaptation strategy than simply updating both statistics. Experiments in section 5 show that RB-BN is actually worse than 'joint replacement' by a large margin.
  • What is the reason to experiment with channel permutation on the compensation vector q?
  • RB-BN is proposed to solve (9), but why should adaptation preserve the expected BN output obtained using source statistics on target data? The derivation does not explain why it is desirable for target domain performance.
  • The budgeted ablation - in what situation do we need for "only K BN layers may change"?
  1. Comparison with test-time adaptation methods such as Tent is missing.
Questions for authors

See weaknesses.

Reviewer_XQmXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies how replacing batch normalization variances while retaining source means changes the expected layer output under distribution shift. It derives a signed quantity called residual transport and evaluates its estimation using separate calibration and evaluation images. Output interventions, channel permutations, and compensation experiments examine its downstream effects. The authors then propose Residual Balanced BN, which adopts target scales while adjusting means to preserve the expected frozen output. Experiments cover CIFAR corruption benchmarks and an ImageNet blur subset. Sequential adaptation achieves results comparable to AdaBN on CIFAR, but performs consistently worse than AdaBN in the external ImageNet experiments. Simultaneous adaptation causes substantial performance degradation.

Critical strengths

The local experiments separate calibration from evaluation and include direct output interventions, sign reversal, and permutations that preserve the correction values. These controls provide useful evidence that the measured output shift has downstream effects that depend on its channel placement. This paper provides a simple analytical correction that can be implemented through BN statistics without gradient updates or a selected mixing weight. The explicit connection between an output correction and modified normalization statistics makes the proposal easy to interpret and provides a concrete basis for further investigation.

Critical weaknesses

In Sections 3.1 and 5.1, Equations 5 and 6 share the same calibration scale, source statistics, and affine parameters. Their difference therefore depends only on the difference between calibration and evaluation activation means. Stronger evidence is needed that residual transport provides useful explanatory information beyond simple mean and scale shifts, especially because Table 8 and Figure 9 show no consistent advantage over simpler selection measures.

In Section 3.3, the correction is derived by requiring the expected target output to match that of the frozen model. This constraint lacks a clear connection to classification performance. The frozen model already contains the target mean mismatch that adaptation may need to remove, so preserving its expected output can preserve a harmful shift. The uniqueness of Equation 8 only establishes a solution to the chosen constraint, not that the constraint is desirable.

In Section 5.6, sequential collection is presented as restoring the distributional premise of the method. However, Appendix D and Table 10 audit expectation preservation while upstream statistics remain frozen. This does not validate the distributions encountered by the deployed sequential algorithm, where upstream layers have already changed. Although the paper correctly states that Equation 9 is local, the connection between this identity and the successful sequential policy remains insufficiently tested.

The algorithm results provide little evidence of a practical advantage over AdaBN. Section 5.6 reports differences of only 0.16 percentage points on CIFAR 10 C and negative 0.18 points on CIFAR 100 C, with intervals including zero. Section 5.7 and Tables 6 and 7 show that RB BN trails AdaBN in every external model and corruption cell at both budgets, with average deficits of approximately 5.6 to 6.1 points. In Table 7, it also reduces ResNet 50 accuracy on defocus blur from 43.03% to 26.20%.

The comparison set in Section 4, is insufficient to establish the value of the proposed normalization rule. The interpolation comparisons use only source weights 0.7 and 0.9, while related approaches such as TTN are discussed but not evaluated. Moreover, Section 3.1 promises a mean only replacement comparison, yet its results are absent from the main local result tables. Joint replacement is consistently strong, leaving unclear how much benefit requires scale adaptation.

Section 4 restricts the local CIFAR study to two early layers, the main complete network study to severity 3, and the external study to blur corruptions. The nonresidual architecture is tested only in the short training study. In addition, Appendix B constructs ImageNet calibration sets with a fixed number of images per class, which removes an important source of variation in unlabeled target sampling.

Questions for authors

In Section 5.2, why is the paired accuracy difference between aligned and shuffled injection reported as negative 1.99 percentage points when the individual changes imply negative 1.77 points? Were different conditions or aggregation weights used?

Appendix D and Table 10 evaluate expectation preservation under frozen upstream statistics. Were corresponding errors measured under the actual upstream states produced by sequential adaptation? If so, how do these errors vary across layers, and do they help explain the external performance gap relative to AdaBN?

Section 3.1 lists mean only replacement among the evaluated methods. What are its results under the same local protocols, and how do they compare with joint replacement and residual compensation? These results would help determine whether scale adaptation provides a meaningful benefit beyond correcting the mean.

What supports preserving the expected target output as the correction objective when that output already contains a distribution mismatch? Is there an identifiable property of the evaluated conditions that explains when this constraint helps and when it causes failures such as the defocus blur results?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.