- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates whether high classification accuracy can coexist with instability in the local neighborhood structure of BERT representations under synthetic input perturbations. It introduces several kNN-based metrics to characterize representation-level instability. While the general research question is potentially interesting, I have serious concerns about the validity of the experimental design, and whether the evidence supports the central claims. Several issues are fundamental rather than matters of presentation or minor experimental omissions. The work reads more like a preliminary empirical observation than a sufficiently developed methodological or scientific contribution for ICLR.
Critical strengths
As a preliminary empirical investigation or pilot study, this work has some merit.
Critical weaknesses
-
The assumption is very strong. Specifically, the central observation that stable model predictions do not necessarily imply stable internal representations is neither theoretically surprising nor, by itself, a sufficiently substantial contribution. Classification decisions depend on whether representations remain on the same side of a decision boundary, whereas nearest-neighbor relationships depend on their relative positions in the embedding space. We can't assume that they are on the same page, actually we can't expect that one does not imply the other.
-
The experiments rely on synthetically generated, template-based data and several handcrafted perturbation categories. Such a controlled setting can be useful for preliminary analysis, but it is insufficient for establishing general conclusions about transformer representation robustness. The reported trends may primarily reflect properties of the synthetic data construction rather than a general phenomenon in transformer representations.
-
This paper lacks the basic baseline, which should be carefully designed. The authors can easily attribute substantial neighborhood changes to intrinsic sensitivity of learned representations. However, it does not compare the results against basic reference conditions that would help support this conclusion.
-
I don't really understand what new insight is proposed in this manuscript. There are many benchamrks are related to this field, I can't find something new. The authors should clearly articulate the key novelty, scientific insights, and contributions of this work relative to the state of the art.
-
My greatest concern is the substantial gap between the experimental evidence and the conclusions. The experimental results are largely limited to descriptive observations of kNN neighborhood changes under synthetic perturbations. These observations alone do not establish that the measured changes reflect meaningful robustness failures.
-
The manuscript suffers from substantial formatting inconsistencies, and apparent template-related issues. The overall presentation gives the impression that the submission has not undergone adequate preparation or quality control.
Questions for authors
Pls check the weakness
General comment (to all reviewers)
We uploaded a revised PDF. The revision answers the reviewers’ requested fix: same question, same metric, with the label confound removed by using semantic labels (AG News, four topic classes; BERT-base with three seeds, DistilBERT; Sec. 4.1). The original synthetic study is in App. A.
Other changes: RS is rebuilt on one fixed clean pool, so clean RS is 0 (was 0.045–0.048; Sec. 3.2). We added random-direction noise of the same cosine displacement, neighbor margins, CKA, Procrustes, clean-twin retrieval and bootstrap intervals (Secs. 3.3, 4.2–4.6).
Headline: perturbations that keep accuracy at 97–99% replace 26–53% of an input’s ten nearest neighbors, 3.9–4.9× the random-direction noise value, while CKA stays at 0.97–0.99. Replies to each review follow.