- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks whether performance gains can be attributed to the original pairing when shuffling sample correspondences does not reduce performance. The authors design experiments to compare the original pairing with within-label partner shuffles and random subspaces. Theoretical and empirical results show that gains over random directions do not establish a pairing benefit. Pairing-specific gains are not common, particularly regarding prediction accuracy. Overall, the gradient subspace can be useful even when the original sample correspondence is not.
Critical strengths
- The paper provides extensive experiments to support its theoretical claims, which are performed across visual and multilingual dialogue tasks.
- The proposed testing strategy is convincing, which performs shuffle control on pairings to test whether performance gains are still preserved.
Critical weaknesses
- The paper writing should be improved. The introduction should describe the meaningfulness of your research topic by discussing some related works, but no references are included in the introduction section. It is hard for readers to identify the paper’s contributions.
- Limited reference papers are included. Only around 20 related papers are shown in the paper.
- Although the paper presents an interesting diagnostic finding, it offers limited solutions or implications on how to design better pairings or improve the resulting gradient subspaces.
- The AI Use Statement is not included in the paper, which should be a section to describe your AI usage in producing this paper.
Questions for authors
How well does the proposed audit extend to more complex visual or multilingual tasks, like object detection, image generation, or multilingual text generation?
4 comments
Thank you again for your review. We hope our response has addressed your concerns. As the discussion period is coming to an end, we would be glad to answer any remaining questions. If our response has resolved your concerns, we would greatly appreciate it if you could reconsider your assessment of our paper in light of it.
Thank you for the rebuttal. It was helpful but ultimately did not change my assessment. I am keeping my score.
Thank you for your response. We respect your decision, but we are still unclear about which specific concerns remain unresolved after our rebuttal and manuscript revisions.
We would appreciate some clarification on the following points:
AI Use Statement: We have added a dedicated section to the revised manuscript. Is this concern now resolved? Implications: We explicitly clarified the implications of our findings, including when pairing provides genuine benefits. What additional implications or solutions do you believe are necessary? Extensions: We explained the general requirements for extending the proposed audit to detection and generation tasks. Do you see any specific methodological limitations that prevent these extensions? Writing and Related Work: Could you identify which specific contributions remain unclear or which relevant prior works are missing? We have made a genuine effort to address each concern raised in the original review. Therefore, it would be helpful if you could identify the particular aspects of our response that you find insufficient, rather than simply reiterating the original assessment.
If there are no remaining substantive concerns beyond those already addressed, we respectfully ask you to reconsider whether maintaining the original rating is fully justified in light of the revised manuscript and our discussion.
Thank you for your time and consideration.
Thank you for finding the testing strategy convincing and the experiments extensive.
Writing. The contributions are stated in the abstract and Section 1, paragraphs 2–4. The introduction states the question first; Section 2 follows immediately with the literature on gradient subspaces, control studies and test-time adaptation.
References. The number of references is not a measure of quality. The paper cites the work it builds on in each area, including SubGD, PGrad, GaLore, LDAdam, saliency sanity checks, probe control tasks, TTT, Tent, CoTTA and EATA. If a specific relevant work is missing, we would be glad to add it.
Implications. The paper gives two: a gain over random directions should not be reported as evidence that pairing helps, and the calibration identifies when pairing does help, namely when partners share a nuisance that subtraction cancels.
Extensions. The audit needs only per-example gradients for each pair, a grouping to shuffle within, and target gradients, all of which are available for detection or generation losses.
AI Use Statement. The revised PDF includes a dedicated AI Use Statement section.
We hope these clarifications lead the reviewer to reconsider the rating.