- Rating
- 3(+2)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 3
Read review
Paper summary
This paper considers the problem of defining a softmax on a distribution of lines. The difficulty for doing so stems from the fact that a line is a projective feature, and its representation is hence an element of a projective rather than a linear space. Taking the averages in charts of yields discontinuities.
The authors remedy this by embedding the projective features into a space of symmetric matrices through the Veronese embedding, and then taking the average in that space. One can then extract the argmax by computing a leading eigenvector of the mean matrix. Since the Veronese embedding is well-defined on , and in particular do not depend on the chart chosen, one thus obtains a well-defined object. As the authors show, the Veronese based approach is, in contrast to previous approaches, continuous in the sence that as a distribution tends to a -peak at some point in , the readout of the Veronese softmax also approaches .
The authors use the Veronese embedding as a readout to learn lines on both symmetric and real datasets, and record an improved performance.
Critical strengths
The approach is mathematically reasonable.
The proofs are correct.
Critical weaknesses
The presentation is far from optimal. Claims are often made without enough context to understand their meaning. The line of argumentation often feels disrupted in the text, and it rather reads like a collection of arguments than a common story. Certain sentences are very hard to parse.
There is a substantial overlap with a concurrent submission with no reference to it.
Questions for authors
There are some sentences that I have a very hard time parsing the meaning of. Can the authors clarify?
-
Page 4: "The aggregation applies to arbitrary cell vectors , and needs no Hough grid. Its result is a convex combination of embedded cells, hence in general not the embedding of a single line."
-
Page 5: "Here the recovered line is the final output, so we place the objective before recovery".
-
Page 9: "Quotients by finite groups admit polynomial invariant embeddings of the same kind, and the choice of embedding fixes both the operator and the metric its loss measures in."
-
Page 19: "Only the concentration certificate is asymptotic, and it is what promotes the bound to a failure of concentration-consistency. On the fixed experimental grid we therefore assert the cancellation, not the limit".
There are further some hints at interesting connections to the literature given in the appendix that are not mentioned in the main text/expanded upon further.
- On page 19, the authors give a hint to a framework of extrinsic means on manifolds (Bhattacharya and Patrangenaru, 2003). How does their approach relate to it?
- On page 21 and 22, the authors hint at a connection Gauge equivariant frameworks and a connection between them and the Veronese softmax. Why is this not referenced in the main paper body? More importantly, can the authors make these connections more explicit? There does not seem to be any mechanisms in the Veronese embedding that resembles a Gauge-equivariant network.
4 comments
Answer to further connections and the ethics flag
Connection 1: Bhattacharya and Patrangenaru (2003) provide a framework for averaging on manifolds. The points are embedded in a Euclidean space, averaged there, and the average is projected back onto the embedded manifold. Our image lines fit their construction for real projective space. The Veronese embedding stores the distinct entries of their matrix embedding . We average these entries using the predicted heatmap probabilities. Recovering a line from this average is their projection step, computed through the leading eigenvector in Section 3.3. The recovered line is therefore the projective extrinsic mean of the heatmap distribution. In our method, the average also provides the point at which we supervise the network. Appendix B.6 uses their condition for uniqueness of the mean. The Introduction cites their work, and Section 3.3 names the extrinsic mean. In the revision, we also cite them directly in Section 3.3.
Connection 2:
We agree that this part is not properly connected to the main body of the paper, and as written, leads to confusion. Here, we want to highlight that the properties of the Hough transform's output do not only affect the readout. More specifically, the fact that the unchanged output space (single cover) is a non-orientable manifold affects the design choice of the refinement CNN. A convolution shares one kernel across all positions, which requires a consistent orientation of the space. The single cover is a Möbius strip and has no such orientation, so a standard convolution on it is not well defined. Weiler et al. (2021) resolve this with constrained kernels, whereas we convolve on the double cover, a cylinder, which is orientable. Not only can cylindrical convolutions be obtained simply by using standard convolutions with circular padding, but in Appendix C.3 we further show that they are a special case of their framework. We acknowledge that their approach adds generality, which is however not needed for our case. In Appendix C.2 we show that VSmax combines evidence on the double cover inherently in its embedding, whereas for the coordinate baselines we first have to fold back to the single cover.
We attempt to clarify this in the revised manuscript by:
- a new section title
- a clearer reference from the main paper to the appendix
- an introductory paragraph in the appendix section
- a heading for the conditions under which this interpretation holds
Ethics flag: We take this flag seriously and have asked the Area Chair to have it clarified by the Program Chairs. So that the concern does not stay unaddressed in the meantime, we assure the reviewer that the submission complies with the ICLR submission guidelines.
References
- Rabi Bhattacharya and Vic Patrangenaru. Large Sample Theory of Intrinsic and Extrinsic Sample Means on Manifolds. The Annals of Statistics, 2003.
- Maurice Weiler, Patrick Forré, Erik Verlinde, and Max Welling. Coordinate Independent Convolutional Networks: Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds. arXiv:2106.06020, 2021.
Thank you for the clarifying comments
I thank the authors for the rebuttal. The paper is much clearer to me know. In particular the new section on the connection to Gauge-equivariant networks is great for my understanding. Also the clarifying comment about the polynomial invariants makes it imaginable that there might be cases where the idea can be generalized to other cases. Therefore, I am now rather in favor of the paper being accepted, and have changed my score accordingly.
(All of this is of course assuming that the ethical flag is just due to unlucky circumstances. I trust and believe that the area and program chairs will easily handle that.)
Thank you for the updated review
We thank the reviewer for the thoughtful review and for reconsidering the score. The detailed comments helped us substantially improve the clarity of the paper. We also appreciate your suggestions for broader connections and share the interest in exploring them further in our future work.
Answer to the clarification questions
We thank the reviewer for confirming the soundness of the approach and the correctness of the proofs, and for pointing to the sentences and connections that are not explained well enough. We answer the questions in turn and describe the corresponding manuscript changes below. We hope the clarifications improve the readability of the paper and would be happy to address any further passages that remain unclear.
Clarification 1:
We thank the reviewer for this question, because it shows us that our methods section tied VSmax to the Hough grid far more closely than the method requires. What we actually want to state here is what VSmax needs as input and what it returns. As input, it just needs a finite set of candidate lines, each with associated probabilities. A Hough grid is one way to obtain them, but not a requirement. As output, it returns the probability-weighted mean of the embedded candidates. The embedded lines form a curved surface in a six-dimensional space. A weighted mean of points on it does not generally give a point on the surface. This is why we point out that VSmax's output is in general not yet a line proposal, and why we need the geometric recovery explained in Section 3.3.
In the revision, Section 3.1 introduces VSmax with a general set of candidates from the start and uses the Hough grid as one example. The input therefore needs no separate clarification, and a short transition to the subsequent sections replaces the quoted sentences. This change is also the reason for minor edits in Sections 2 and 4 and in the proofs in Appendix B. They now use the same general candidates instead of grid cells, so that the paper stays consistent. The mathematical statements themselves are unchanged.
Clarification 2:
The quoted sentence leaves the actual reason for our design implicit, which is why we gratefully make it explicit here. In order to obtain the predicted line, we have to recover it from the estimate in embedding space using the mechanism explained in Section 3.3. It involves eigendecomposition, for which gradients may get large during backpropagation. Given that in our Theorem 1(iii), we identify that a small L2 loss guarantees a small prediction error of the recovered line, we propose to avoid differentiating through the recovery.
In the revision, the end of Section 3.2 now gives this explicit reason for why we supervise in the embedding space, and the opening of Section 3 no longer suggests a different one.
Clarification 3:
This sentence is intended to give an outlook on the paradigm behind VSmax. For lines, the symmetry group is ℤ₂, and the Veronese embedding is invariant to it. Other target domains are defined up to other finite groups of symmetries. For these, invariant polynomials also provide such embeddings, in which one would average and supervise as we do for lines. The chosen embedding thus determines both the readout operator and the distance the loss measures. Which recovery and which guarantees such an embedding allows remains open.
In the revision, we split the sentence into two parts that are easier to parse.
Clarification 4:
We agree with the reviewer on the lack of clarity in this passage. We want to show that the failure of soft-argmax is not an artefact of a limit. Proposition 1 uses heatmaps with equal mass on two cells on either side of the angular border. For every such heatmap, the opposite offsets cancel, and the prediction error stays above a fixed bound. This holds without any limit. The limit adds only that the heatmaps concentrate on the target as the two cells approach the border. Evidence that concentrates while the error does not vanish is what violates concentration-consistency.
Our experiment uses a fixed grid, on which the cells cannot approach the border arbitrarily closely. It therefore shows the cancellation and the error floor, but not the limit.
In the revision, Appendix B.5 states this distinction directly, without the terms "certificate" and "promotes", and points to the experiment in Appendix D.1.