- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes residual perturbations applied at the BERT embedding layer as an alternative to dropout-based positive pair generation for unsupervised contrastive sentence embedding learning. The authors evaluate five perturbation variants on the MTEB benchmark, with SimCSE as the primary baseline. They report that residual rolling and shuffling achieve overall performance on par with SimCSE, improving on STS and pair classification tasks but declining on retrieval and clustering tasks.
Critical strengths
All perturbations are applied at the same point in the model, and the five variants are designed as pairwise controls: with vs. without structure, with vs. without the residual addition, and preserving vs. disrupting word order. This design supports the paper's main claims with the evidence presented.
Critical weaknesses
- Lack of academic rigor The introduction frames state-of-the-art sentence embedding research using only work from 2018–2019, while relevant recent advances are omitted. [1] is the related prior work: it also constructs perturbations for unsupervised contrastive sentence embedding learning, but its perturbations are learned rather than hand-crafted. It is neither cited nor used as a baseline, making it impossible to judge whether the proposed fixed residual perturbations offer any advantage. Similarly, [2] shows that sentence embeddings can be obtained with simple prompt-based strategies, which weakens the motivation for the proposed augmentation pipeline. More broadly, the related work section reads as a raw chronological list of papers with no synthesis or positioning of the proposed method against existing work. Without engaging with these closely related lines of research, the contribution of the paper remains unclear.
[1] https://openreview.net/pdf?id=gZYbGIpFYpA [2] https://arxiv.org/pdf/2201.04337
- Insufficient experimental validation The authors cite the finding that input-level cropping strongly outperforms SimCSE as motivation for this work, yet fail to include cropping as a baseline in their experiments. The proposed method is only compared to SimCSE (2021), with no evaluation against more recent state-of-the-art approaches. Without this key comparison, the results cannot demonstrate any meaningful advance over existing work, which undermines the paper’s novelty and practical value.
Questions for authors
-
The authors cite the cropping-based augmentation method as motivation for their work, but it is not included as an experimental baseline. Why is this comparison missing?
-
Key implementation details and hyperparameters are omitted from the paper. Please provide the full details required to replicate your experiments.