- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates the relationship between scale and shape regularization in sliced-Gaussian latent prediction methods.
Critical strengths
This work explores an interesting theoretical property of distribution-matching regularizers and proposes a mathematically motivated mechanism for controlling representation geometry. The experiments cover both image and time-series modalities and provide useful observations about the dependence of rank and downstream performance on the regularization configuration.
Critical weaknesses
My main concern is that the proposed method is insufficiently specified and remains unconvincing. The central construction relies on quantities whose explicit definitions are not provided in the main text. And the relationship between the exact infinite-slice objective and the approximated kernel used in implementation is unclear.
Second, the theoretical curvature analysis does not establish a convincing connection to the learned representations. The observed accuracy changes are inconsistent.
Questions for authors
- The authors need to provide a complete and reproducible formulation of the proposed regularizer and clarify exactly which objective is implemented?
2, What meaningful understanding or improvement does the proposed scale–shape control enable beyond adjusting representation rank?
- How does the local curvature analysis explain the observed behavior of trained networks?
4 comments
Dear reviewer,
We have updated the manuscript for added clarity and addressed your initial concerns in the response above, and the VISReg and the K sweep results we added supports our hypothesis even better, thanks to your comments.
We have also made a top-level general comment addressing all the improvements made in an effort to make our responses easier to parse.
We would be grateful to know your thoughts on the updates and responses before the close of the review period soon.
We thank you once again for your time and contributions through this review process!
Best, Authors
Dear Reviewer,
We have made a significant effort to improve the clarity of the manuscript and also added experiments and ablations that strengthen the utility of our method for anyone using sliced regularizers.
We think our method is a useful tool for other researchers, not just in terms of potentially improving their performance on a downstream task but also as an interpretability tool that allows a rank-sweep, making it clear if their algorithm has an optimal effective rank region that existing methods like SIGReg (and now VISReg, recently added and supportive of our hypothesis) are by default blind to.
We have received one score increase so far and we would be excited to hear your thoughts and your final score on the updated manuscript before the rebuttal closes.
Thank you once again for your time and contributions to our work!
Thanks for your work on rebuttal and revision. My concern has been mostly addressed. I would keep my current rating.
Thank you for pressing on the specification; the appendix that should have carried the implemented objective was an unresolved reference in the submitted version, which we think explains much of the difficulty you report. It is now updated, and we have added experiments on what η does beyond rank and on how far the curvature analysis reaches on trained networks. We look forward to your reply!
W1 / Q1. "The central construction relies on quantities whose explicit definitions are not provided … the relationship between the exact infinite-slice objective and the approximated kernel used in implementation is unclear. Provide a complete and reproducible formulation and clarify exactly which objective is implemented."
Q2. "What meaningful understanding or improvement does the proposed scale–shape control enable beyond adjusting representation rank?"
W2 / Q3. "The theoretical curvature analysis does not establish a convincing connection to the learned representations … How does the local curvature analysis explain the observed behavior of trained networks?"
W3. "Difficult to follow despite careful reading."