- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper introduces a method to understand and isolate if contrastive learning can produce reliable cross modal scores/similarities without removing nuisance information from individual representations from encoders.
Critical strengths
The topic investigated in the paper is relevant to the community.
Critical weaknesses
The introduction is unreadable. The motivation is unclear, the framing of the problem is unclear, the related works are unclear or they are not presented in a proper frame, the list of contributions begins with a result and not a contribution. I read the introduction and related work section and still I don't have any ideas of what the paper is about and what is the motivation.
Notation should be revised. bold for vectors and matrices, otherwise are scalars or varaibles.
The problem setup section cite theorem 1 and corollary 3(i)-(ii) but they are not presented there and the authors did not write where I could find them. similarly, there are a lot of mathematical terms that are not defined or explained.
Section 3: problem setup. Section 4: main results. The method is in the main results... the authors should severely improve the readability of the paper.
The paper does not have a logical flow, there are main results and the authors explain part of the method, than there is the experiment section still with equations.
The paper might be valuable as the topic is interesting and it would be interesting as well to explore it so the idea is good, but it is very hard to understand as it is written and presented.
Questions for authors
can the authors considerably improve the readness of the paper (including motivation, organization, related works, explanations, visualizations)? I understand that it would be very hard during rebuttal period, but this is also a suggestion.
3 comments
I thank the authors for the hard work of the rebuttal.
I think that the revision improved the paper. However, the revised version is almost a new paper with respect to the one originally submitted, in which most of the sections have been drastically changed and rewritten. Also, the original paper was 80 pages, now it is 38, and the authors have not specified what they removed and the reason behind the choice.
I think that submitting a non-ready paper and then take the one-month review time to completely revise it or to completely change it after rebuttal is a bit unfair with respect to authors who have carefully submitted a ready paper.
I am sorry but I have to confirm my score.
Thank you for reading our revision and for acknowledging that it improved the paper. We apologize that our previous response did not point you to item (5) of our global response. This item lists where each main result of the submission appears in the revised paper and why we removed the others.
Theorems 1 and 2 and Proposition 1 combine statements that we proved in the submission. To address the comments of Reviewers wP3q and AvFa, we added the non-Gaussian analysis in Section 5, the effect of cosine normalization, and the tests on pretrained image–text models in Section 6.2 (items (2)–(4)).
The reduction from 80 to 38 pages comes from the appendix, which we shortened from 67 to 26 pages. The proofs of the main results took 31 pages in Appendices A–C of the submission, and we rewrote them in 15 pages. For the reasons given in item (5), we removed the weight-decay analysis (Appendix D of the submission) and the compression and ranking results (Appendix F and part of Appendix E). Since we limited the appendix to what the main text uses, we also left out the additional related work (Appendix I) and other material such as the sweep over the output dimension and the comparison with an explicit encoder-alignment regularizer in Appendix G.
Thank you for reading our paper carefully, and we are sorry that the submitted version was so hard to follow. Following your suggestion, we have rewritten the main text from the beginning, revising the motivation, the organization, the related work, and the explanations, and adding Figure 1 to illustrate the main idea in the introduction. We are grateful that you saw value in the idea despite the presentation, and we hope the revised version makes it clear.
We rewrote the introduction so that it starts from the motivation, explains what earlier analyses leave open, and states our question before the results. We kept the list of contributions as the theorems we prove, because they are our contributions.
As you suggested, we set all vectors and matrices in bold, for example x, M, and WA, and kept scalars in plain type.
In Section 3, we removed the references to results that appear later and defined the terms that the theorems use, including the signal and nuisance coordinates, the signal block, the two notions of denoising, and score–encoder separation.
We also reorganized the paper so that Section 3 defines the model, the loss, and the gradient flow before Section 4 states the theorems with the mechanism behind each. In Section 6, we kept only one equation, the definition of the metric.
If any part of the revised paper is still hard to read, we would be grateful to know where, and we would be glad to revise it.