- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 2
Read review
Paper summary
The paper proposes SymSelect, a framework that selects high-similarity transformed pairs for representation alignment when transformation validity is unobserved. This paper's theoretical analysis separates invalid-pair contamination from selection bias among valid pairs and provides conditions for inheriting the oracle’s local improvement. Experiments on MNS and I-RAVEN show that SymSelect recovers 77%–92% of the accuracy gain from supervised valid-pair alignment without using validity labels.
Critical strengths
-
The theoretical analysis is sound under the stated assumptions, with clear proofs that distinguish invalid-pair contamination from selection bias among valid pairs.
-
The method is simple to implement and achieves strong empirical results, recovering most of the supervised alignment gain without access to pair-validity labels.
Critical weaknesses
-
The introduction and problem formulation would benefit from clearer motivating examples and a more explicit distinction between candidate transformations and valid alignment relations, making the setting more accessible to readers unfamiliar with the area.
-
Although the paper provides quantitative gradient diagnostics and validity-matched controls, it does not establish whether Theorem 1’s sufficient condition holds in practice or assess the tightness and predictive value of Theorem 2’s finite-batch bound.
-
Although the paper includes oracle-based augmentation and several selection controls, it lacks comparisons with learned augmentation or input-dependent invariance methods, such as Augerino [1] and InstaAug [2], leaving the practical advantage of similarity-based selection over existing adaptive approaches unclear.
[1] Benton, Gregory, et al. "Learning invariances in neural networks from training data." Advances in neural information processing systems 33 (2020): 17605-17616.
[2] Miao, Ning, et al. "Learning instance-specific augmentations by capturing local invariances." arXiv preprint arXiv:2206.00051 (2022).
Questions for authors
1, How sensitive is SymSelect to the choice of similarity measure? Could the author evaluated alternative metrics for pair selection and alignment?
2, How sensitive is SymSelect to the choice of k? Could the author provide some ablation study on k?
3 comments
Thanks for the detailed explanations and experiments, my concerns are addressed. I have raised the score to 3.
Thank you!
Dear Reviewer R5SW,
We would like to respectfully offer a reminder and invite you to join the further discussion. We believe that most concerns have been reasonably addressed or substantially resolved. If you have a different view, we can give further clarification. Alternatively, we would also welcome you to raise the score or adjust the confidence.
We would also like to share important additional experimental results that may change your view of the paper. Following your suggestions, we strengthened the empirical analysis of selective alignment, including gradient diagnostics, validity-matched controls, and sensitivity to the selection rate.
We have also clarified the motivating examples, theoretical scope, and relationship to adaptive augmentation methods.
Full experimental details are provided in our responses and revision. We look forward to your reply and will be happy to answer any further questions.