W1. The representation-level result does not replicate in size. In the Waterbirds replication (Appendix H), BatchNorm-affine entropy changes AUROC by at most 9.4×10−4 in every seed, within the paper's own 10−3 bound for an unchanged ranking (Table 15, P4a), compared with changes of up to 0.028 on Camelyon17. The collapse results also fail to replicate. At the common learning rate, the replication predicted in advance that BatchNorm-affine entropy would lower AUROC and collapse to one predicted class in at least 5 of 6 seeds, but it lowers AUROC in 1 seed and collapses in none (P9c). All-parameter entropy was predicted to collapse in all 6 seeds and collapses in 1 (P9d). The head-only result replicates in full, so the evidence supports the overconfidence account (S2) but not the claims about what happens when the representation adapts.
W2. The theorem and the final-layer result follow from known facts. Theorem 1 is the one-coordinate case of an established result that gradient descent is not invariant to rescaling the features (Amari 1998; Martens 2020, cited). In the final-layer result, each entropy step scales the head up by 1.06 to 1.14 (Section 6.1), which raises NLL for a model that is already overconfident, and this failure of entropy-based test-time adaptation has been documented in the literature (Niu et al. 2023; Zhao et al. 2023, which are already cited). Section 3.1 calls Theorem 1 elementary, so the contribution rests on the experiments, the three checks and the overconfidence measurement (S1, S2), and W1 limits the experimental evidence to the final layer.
W3. The main text has no tables, and results are given in prose. Sections 5 and 6 report results in running text, often to six decimal places (for example, mean target risk rising from .798979 to 1.745193, and loss rising from .232994 to 5.163395), while all ten tables they refer to are in the appendices. A main-text table with one row per study, giving the update, the intervention and the result, would make clear which experiment supports which claim.
To Reviewer KDeD
Thank you for the candid review. The paper is rewritten for a first-time reader, and the answers to your three questions follow.
The problem. A classifier deployed on new data may keep learning from that data without labels. Sometimes its loss then rises. Your summary names the usual suspicion: the update makes the model rely on a feature it previously ignored, such as a spurious correlation. The other possibility is that the update only makes the model more confident in the mistakes it already made. The two call for different fixes, and the loss alone cannot tell them apart. The paper is about telling them apart.
How we address it. Three checks tell the two apart:
The evidence.
We would be glad to clarify anything that is still unclear.