- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3(+1)
- Presentation
- 3(+1)
- Soundness
- 2
Read review
Paper summary
This paper investigates whether BatchNorm running statistics should be carried or reset across domain shifts. Paired interventions across 12 CIFAR-C receiver checkpoints show that source effect rankings transfer, but their decision boundaries do not. A threshold calibrated on other receivers improves held-edge decisions and closed-loop performance.
Critical strengths
-
The central finding is insightful. The paper distinguishes ranking transfer from decision-boundary transfer, demonstrating that high correlation does not guarantee beneficial carry/reset decisions.
-
The empirical controls are carefully designed. Held-receiver and held-edge tests support threshold transfer, while a separate closed-loop evaluation reports positive gains across all 120 streams, which also include profile edges.
Critical weaknesses
-
The generalizability remains limited. The experiments use frozen convolutional networks, known domain boundaries, and labeled calibration effects from other receivers. Moreover, leave-one-receiver-out testing retains models with the same dataset and architecture, leaving group-level transfer insufficiently tested.
-
The practical gains are modest. Closed-loop improvement is approximately 0.93 percentage points, and the evaluation includes profile transitions. Comparisons with standard batch-statistics adaptation and different BN momentum settings are needed to clarify the practical value of the proposed decision rule.
Questions for authors
-
Does the transferred threshold remain effective when an entire dataset–architecture group is held out, rather than a single receiver checkpoint?
-
How does the threshold policy compare in absolute accuracy with batch-statistics adaptation and alternative BN momentum settings?