acceptodds
Under review as a conference paper at ICLR 2027

Soft-Argmax for the Projective Plane via the Veronese Embedding

Abstract

Soft-argmax is a widely used readout operator for differentiable coordinate regression, turning heatmaps into predictions by averaging candidate coordinates. While equivariant architectures account for symmetries in feature processing, the readout that turns evidence into a prediction, and the loss on that prediction, must respect the topology of the target domain as well. The parameters of undirected lines are a case in which such a symmetry breaks coordinate averaging. Each line has two sign-related representatives, so no coordinate domain admits an average that respects proximity between lines, and we prove that any readout averaging signed offsets can miss a target on which the evidence concentrates. We introduce Veronese soft-argmax, a parameter-free readout that averages sign-invariant quadratic features and recovers the line from their average. The same representation yields the training objective, whose value bounds the error of the recovered line. We prove that predictions approach the target as evidence concentrates and give sufficient conditions for a unique prediction, on real projective spaces of any dimension. Experiments on synthetic lines show that readout and objective each contribute, and our approach transfers to horizon estimation in real images. The construction applies wherever a heatmap ranges over candidates in a real projective space, and in particular to targets with antipodal symmetry.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
geometric deep learning, projective geometry, soft-argmax, line detection

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_FWRAOpenReview ↗
Rating
3(+2)
Confidence
3
Informativeness
2
Presentation
2(+1)
Soundness
3
Read review
Paper summary

This paper considers the problem of defining a softmax on a distribution of lines. The difficulty for doing so stems from the fact that a line is a projective feature, and its representation is hence an element of a projective rather than a linear space. Taking the averages in charts of RP2\mathbb{RP}^2 yields discontinuities.

The authors remedy this by embedding the projective features into a space of symmetric matrices through the Veronese embedding, and then taking the average in that space. One can then extract the argmax by computing a leading eigenvector of the mean matrix. Since the Veronese embedding is well-defined on RP2\mathbb{RP}^2, and in particular do not depend on the chart chosen, one thus obtains a well-defined object. As the authors show, the Veronese based approach is, in contrast to previous approaches, continuous in the sence that as a distribution tends to a δ\delta-peak at some point x⋆x^\star in RP2\mathbb{RP}^2, the readout of the Veronese softmax also approaches x⋆x^\star.

The authors use the Veronese embedding as a readout to learn lines on both symmetric and real datasets, and record an improved performance.

Critical strengths

The approach is mathematically reasonable.

The proofs are correct.

Critical weaknesses

The presentation is far from optimal. Claims are often made without enough context to understand their meaning. The line of argumentation often feels disrupted in the text, and it rather reads like a collection of arguments than a common story. Certain sentences are very hard to parse.

There is a substantial overlap with a concurrent submission with no reference to it.

Questions for authors

There are some sentences that I have a very hard time parsing the meaning of. Can the authors clarify?

  1. Page 4: "The aggregation applies to arbitrary cell vectors xi∈S2x_i\in S^2, and needs no Hough grid. Its result is a convex combination of embedded cells, hence in general not the embedding of a single line."

  2. Page 5: "Here the recovered line is the final output, so we place the objective before recovery".

  3. Page 9: "Quotients by finite groups admit polynomial invariant embeddings of the same kind, and the choice of embedding fixes both the operator and the metric its loss measures in."

  4. Page 19: "Only the concentration certificate d‾(Pϵ,x∗)→0\underline{d}(P_\epsilon,x^*)\to 0 is asymptotic, and it is what promotes the bound to a failure of concentration-consistency. On the fixed experimental grid we therefore assert the cancellation, not the limit".

There are further some hints at interesting connections to the literature given in the appendix that are not mentioned in the main text/expanded upon further.

  1. On page 19, the authors give a hint to a framework of extrinsic means on manifolds (Bhattacharya and Patrangenaru, 2003). How does their approach relate to it?
  2. On page 21 and 22, the authors hint at a connection Gauge equivariant frameworks and a connection between them and the Veronese softmax. Why is this not referenced in the main paper body? More importantly, can the authors make these connections more explicit? There does not seem to be any mechanisms in the Veronese embedding that resembles a Gauge-equivariant network.
4 comments
AuthorsOpenReview ↗

Answer to the clarification questions

We thank the reviewer for confirming the soundness of the approach and the correctness of the proofs, and for pointing to the sentences and connections that are not explained well enough. We answer the questions in turn and describe the corresponding manuscript changes below. We hope the clarifications improve the readability of the paper and would be happy to address any further passages that remain unclear.

Clarification 1:

We thank the reviewer for this question, because it shows us that our methods section tied VSmax to the Hough grid far more closely than the method requires. What we actually want to state here is what VSmax needs as input and what it returns. As input, it just needs a finite set of candidate lines, each with associated probabilities. A Hough grid is one way to obtain them, but not a requirement. As output, it returns the probability-weighted mean of the embedded candidates. The embedded lines form a curved surface in a six-dimensional space. A weighted mean of points on it does not generally give a point on the surface. This is why we point out that VSmax's output is in general not yet a line proposal, and why we need the geometric recovery explained in Section 3.3.

In the revision, Section 3.1 introduces VSmax with a general set of candidates from the start and uses the Hough grid as one example. The input therefore needs no separate clarification, and a short transition to the subsequent sections replaces the quoted sentences. This change is also the reason for minor edits in Sections 2 and 4 and in the proofs in Appendix B. They now use the same general candidates instead of grid cells, so that the paper stays consistent. The mathematical statements themselves are unchanged.

Clarification 2:

The quoted sentence leaves the actual reason for our design implicit, which is why we gratefully make it explicit here. In order to obtain the predicted line, we have to recover it from the estimate in embedding space using the mechanism explained in Section 3.3. It involves eigendecomposition, for which gradients may get large during backpropagation. Given that in our Theorem 1(iii), we identify that a small L2 loss guarantees a small prediction error of the recovered line, we propose to avoid differentiating through the recovery.

In the revision, the end of Section 3.2 now gives this explicit reason for why we supervise in the embedding space, and the opening of Section 3 no longer suggests a different one.

Clarification 3:

This sentence is intended to give an outlook on the paradigm behind VSmax. For lines, the symmetry group is ℤ₂, and the Veronese embedding is invariant to it. Other target domains are defined up to other finite groups of symmetries. For these, invariant polynomials also provide such embeddings, in which one would average and supervise as we do for lines. The chosen embedding thus determines both the readout operator and the distance the loss measures. Which recovery and which guarantees such an embedding allows remains open.

In the revision, we split the sentence into two parts that are easier to parse.

Clarification 4:

We agree with the reviewer on the lack of clarity in this passage. We want to show that the failure of soft-argmax is not an artefact of a limit. Proposition 1 uses heatmaps with equal mass on two cells on either side of the angular border. For every such heatmap, the opposite offsets cancel, and the prediction error stays above a fixed bound. This holds without any limit. The limit adds only that the heatmaps concentrate on the target as the two cells approach the border. Evidence that concentrates while the error does not vanish is what violates concentration-consistency.

Our experiment uses a fixed grid, on which the cells cannot approach the border arbitrarily closely. It therefore shows the cancellation and the error floor, but not the limit.

In the revision, Appendix B.5 states this distinction directly, without the terms "certificate" and "promotes", and points to the experiment in Appendix D.1.

AuthorsOpenReview ↗

Answer to further connections and the ethics flag

Connection 1: Bhattacharya and Patrangenaru (2003) provide a framework for averaging on manifolds. The points are embedded in a Euclidean space, averaged there, and the average is projected back onto the embedded manifold. Our image lines fit their construction for real projective space. The Veronese embedding stores the distinct entries of their matrix embedding xx⊤xx^\top. We average these entries using the predicted heatmap probabilities. Recovering a line from this average is their projection step, computed through the leading eigenvector in Section 3.3. The recovered line is therefore the projective extrinsic mean of the heatmap distribution. In our method, the average also provides the point at which we supervise the network. Appendix B.6 uses their condition for uniqueness of the mean. The Introduction cites their work, and Section 3.3 names the extrinsic mean. In the revision, we also cite them directly in Section 3.3.

Connection 2:

We agree that this part is not properly connected to the main body of the paper, and as written, leads to confusion. Here, we want to highlight that the properties of the Hough transform's output do not only affect the readout. More specifically, the fact that the unchanged output space (single cover) is a non-orientable manifold affects the design choice of the refinement CNN. A convolution shares one kernel across all positions, which requires a consistent orientation of the space. The single cover is a Möbius strip and has no such orientation, so a standard convolution on it is not well defined. Weiler et al. (2021) resolve this with constrained kernels, whereas we convolve on the double cover, a cylinder, which is orientable. Not only can cylindrical convolutions be obtained simply by using standard convolutions with circular padding, but in Appendix C.3 we further show that they are a special case of their framework. We acknowledge that their approach adds generality, which is however not needed for our case. In Appendix C.2 we show that VSmax combines evidence on the double cover inherently in its embedding, whereas for the coordinate baselines we first have to fold back to the single cover.

We attempt to clarify this in the revised manuscript by:

  • a new section title
  • a clearer reference from the main paper to the appendix
  • an introductory paragraph in the appendix section
  • a heading for the conditions under which this interpretation holds

Ethics flag: We take this flag seriously and have asked the Area Chair to have it clarified by the Program Chairs. So that the concern does not stay unaddressed in the meantime, we assure the reviewer that the submission complies with the ICLR submission guidelines.

References

  • Rabi Bhattacharya and Vic Patrangenaru. Large Sample Theory of Intrinsic and Extrinsic Sample Means on Manifolds. The Annals of Statistics, 2003.
  • Maurice Weiler, Patrick Forré, Erik Verlinde, and Max Welling. Coordinate Independent Convolutional Networks: Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds. arXiv:2106.06020, 2021.
Reviewer_FWRAOpenReview ↗

Thank you for the clarifying comments

I thank the authors for the rebuttal. The paper is much clearer to me know. In particular the new section on the connection to Gauge-equivariant networks is great for my understanding. Also the clarifying comment about the polynomial invariants makes it imaginable that there might be cases where the idea can be generalized to other cases. Therefore, I am now rather in favor of the paper being accepted, and have changed my score accordingly.

(All of this is of course assuming that the ethical flag is just due to unlucky circumstances. I trust and believe that the area and program chairs will easily handle that.)

AuthorsOpenReview ↗

Thank you for the updated review

We thank the reviewer for the thoughtful review and for reconsidering the score. The detailed comments helped us substantially improve the clarity of the paper. We also appreciate your suggestions for broader connections and share the interest in exploring them further in our future work.

Reviewer_YMxVOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Soft-argmax is an operation that makes argmax differentiable by taking a probability-weighted average of coordinates over a heatmap (a probability distribution over candidate coordinates).

This paper addresses line regression from a heatmap produced by a fixed Hough transform.

A line is described by an angle and its distance from the origin, and rotating the angle by 180 degrees while flipping the sign of the distance gives the same line. As a result, when the angle is restricted to the range 0 degree–180 degress, the two ends of this range are actually connected, with only the sign of the distance flipped (a Möbius strip structure). When the target lies near these ends, as for a nearly vertical line, the heatmap probability is split between the two ends. Averaging the coordinates directly, as soft-argmax does, then makes the two distance values of opposite sign cancel out, producing a wrong line.

The authors prove that no matter how the angle is handled (e.g., the sinusoidal soft-argmax, which averages the angle circularly), this problem cannot be avoided as long as the distance is averaged directly (Proposition 1).

The proposed VSmax averages, instead of coordinates, Veronese features constructed so that the two representations of the same line take identical values, and then recovers the line from this average as an eigenvector.

Since it keeps the probabilities produced by the network and the weighted averaging unchanged and only replaces the fixed values attached to each cell, it can be placed where soft-argmax is used without adding any learnable parameters.

The method is validated on synthetic images containing a single line (including evaluation under noise stronger than seen during training) and on horizon estimation in real images (HLW-visible).

Critical strengths
  1. A theoretical contribution that connects existing geometric tools to the problem of extracting lines from a Hough heatmap. Veronese features, the extrinsic mean, and the eigenvector construction used in quaternion averaging are known tools, but the paper connects them to the requirement that the operation extracting a prediction from a heatmap must respect the topology of the label space, and pinpoints where standard soft-argmax fails with a concrete counterexample (Proposition 1). Since the method is a parameter-free drop-in that only replaces soft-argmax, this is a useful perspective for the Hough-based line regression community.
Critical weaknesses
  1. The measured improvements do not come from the mechanism addressed by the theory.
  • 1-1. The failure explained by the theory (cancellation of distance values at the boundary) occurs only within a few degrees of the angular boundary; elsewhere, the existing methods are also accurate (Appendix. D.1, Fig. 7).
  • 1-2. Yet VSmax clearly outperforms the baselines only when strong noise not seen during training is added; under conditions similar to training, it shows almost no difference from the existing methods (Fig. 4, Table 3).
  • 1-3. Moreover, swapping in only the VSmax operation on the same heatmap gives a small gain; the large improvement appears only when the heatmap itself is retrained with VSmax and the new loss (Fig. 4).
  1. The practical value and generality are not demonstrated. It is hard to see the value of this technique in real-world domains.
  • 2-1. The real-image experiment (Table 1, HLW-visible horizon, U-Net + Hough pipeline) is insufficient. There is no ablation that keeps the same pipeline and replaces only the final operation with soft-argmax / sinusoidal / VSmax, so it is unclear whether the gain in Table 1 is due to VSmax.
  • 2-2. The validation is limited to the Hough space, so its extensibility is not demonstrated. The method and proofs are defined for general projective label spaces, and the abstract claims that the construction "applies wherever a heatmap ranges over candidates in a real projective space." However, all experiments are conducted on a single line in the image obtained via the Hough transform. There are no experiments in other spaces, such as vanishing points or 3D axis directions.
  • 2-3. It would be helpful if the authors could point to real tasks (in vision or other domains) where this technique can be meaningfully used.
Questions for authors
  1. W1. The failure explained by the theory occurs only near the angular boundary, whereas the actual performance gains come under strong noise not seen during training, and mainly from retraining with the new loss. How do you explain these gains with the theory in the paper?
  2. W2-1. In the HLW experiment (Table 1), do you have results that keep the same pipeline and replace only the final operation with soft-argmax / sinusoidal / VSmax?
  3. W2-2. Are there experiments showing that VSmax is effective in projective label spaces other than lines in the Hough space (e.g., vanishing points, 3D axis directions)?
  4. W2-3. What are the real tasks (in vision or other domains) where this technique can be meaningfully used?
4 comments
AuthorsOpenReview ↗

Answer to W1

We thank the reviewer for the careful summary, which captures the cancellation of Proposition 1 and the drop-in nature of VSmax exactly. The three observations in W1 describe our experiments correctly, and they helped us make explicit what connects them.

W1: We agree that the benefit of VSmax shows most clearly as the noise increases. We also agree that our contribution is twofold. VSmax improves the readout, and it makes the loss function correct. Finally, we agree that the benefit is smaller "under conditions similar to training". However, we argue that these observations do not point outside our theory. They show one mechanism at work. Coordinate averaging fails when the heatmap reaches across the angular boundary, and whether it does depends on the spread of the heatmap.

Under conditions similar to training, the network in our simple synthetic setting can produce a single, well-concentrated peak. We chose this simple setting deliberately, because it lets us control the conditions under which the readouts are compared. In real images, clutter and ambiguous visual cues can make networks less decisive in the heatmap. Such clear evidence therefore cannot be assumed. Increasing the input noise is our controlled way to study exactly this case. We now show how each of the three observations follow from this mechanism.

1-1: We agree that for sharp heatmaps the failure stays within a few degrees of the boundary, as Figure 7c shows. This is what our theory implies for sharp heatmaps, and we thank the reviewer for raising it, because it makes the prediction visible. The failure needs evidence on both sides of the boundary. A sharp heatmap has such evidence only when the target is close to the boundary. A spread heatmap has it for targets much farther away. At every spread, the error of VSmax stays bounded by Theorem 1(ii), while the coordinate readouts admit no such bound (Proposition 1). The theory therefore implies small differences in our sharp synthetic setting and larger ones as the evidence spreads.

To further show this, we extended Figure 7c to wider heatmaps and to all target angles.

Width ssStandard soft-argmaxSinusoidal soft-argmaxVSmax
0.011.05∘1.05^\circ0.75∘0.75^\circnone
0.045.35∘5.35^\circ3.85∘3.85^\circnone
0.1013.75∘13.75^\circ9.20∘9.20^\circnone
0.2029.60∘29.60^\circ18.80∘18.80^\circnone
0.3059.15∘59.15^\circ46.55∘46.55^\circnone

The range of the failure grows with the width, up to 59∘59^\circ from the boundary for standard soft-argmax. VSmax stays below the threshold for every width and target. The harder the evidence, as in real-image problems, the more the readout matters.

In the revision, we add this table to Appendix D.1 (Table 6).

1-2: We agree that under conditions similar to training, VSmax and the baselines are close. This is the sharp, concentrated case of 1-1, now for heatmaps produced by the network. Up to the training noise, the network still produces concentrated heatmaps, and the lead of VSmax is small. Beyond it, the heatmaps increasingly spread, as Table 7 shows. One step into out-of-distribution noise, the lead of VSmax grows about tenfold (Figure 4c). The gain thus follows the spread of the heatmap, as 1-1 predicts. The noise therefore is our controlled way to enforce this spread, standing in for the harder evidence of real images.

1-3: We agree that the larger gain appears after retraining, and we see this as the second part of our contribution. The frozen swap reads heatmaps learned with VSmax. Under training conditions, these heatmaps are concentrated (Table 7), so this is again the sharp case of 1-1, and the gain is small. As the noise grows, they spread, and even the frozen gain grows (Figure 4a). Retraining shows the same mechanism acting during training. Near the boundary, a coordinate readout turns even a correct heatmap into a wrong line (Proposition 1). The loss then penalises correct evidence, and the network learns worse heatmaps. With VSmax, correct evidence yields a small loss, and a small loss bounds the error (Theorem 1(iii)). Section 5.3 shows both effects. The coordinate readouts score higher on VSmax-trained heatmaps than on their own. With the same Veronese loss for every readout, VSmax still leads (Figure 5).

Together, the three points show one pattern. The coordinate readouts fall behind as the evidence spreads, in the readout itself (1-1), in the heatmaps of the network (1-2), and in what the network learns (1-3). Our new real-image experiment shows that this matters beyond our synthetic setting. On synthetic data, the frozen swap gains little under training conditions. On real images, replacing only the readout of the trained horizon network of Abbas and Zisserman (2019) improves its AUC in every seed, without retraining and without added noise. This is consistent with real images providing less clear evidence than our synthetic setting.

AuthorsOpenReview ↗

Answer to W2

W2-1 & W2-2:

We thank the reviewer for pointing to an experiment that isolates the readout in W2-1, since it asks directly whether the real-image gain is due to VSmax itself. The broader concern behind it, that the real-image evidence is not yet convincing, is shared by another reviewer, so we address it in the general comment. There we keep the trained horizon network of Abbas and Zisserman (2019) fixed and replace only its final readout operation. Their readout is a coordinate soft-argmax over the 11 most probable bins of each coordinate, so Proposition 1 applies to it. Replacing it with VSmax on the same outputs improves the AUC in every seed, by 22% on HLW-visible. This is the real-image counterpart of the synthetic swap in Figure 4a. That the frozen swap gains so much here, unlike on our sharp synthetic heatmaps, is the mechanism described throughout W1 at work.

Moreover, the same experiment shows VSmax outside Hough space, which is what the reviewer asks for in W2-2. Abbas and Zisserman (2019) represent the horizon by the normal of a plane, a unit vector in R3 up to sign, and read it from a heatmap different from a Hough grid. Formally, this is one of the label types the reviewer specifically names, an undirected 3D axis. A vanishing point is a vector of the same kind. VSmax computes the same 3x3 matrix in every case, and our guarantees hold for every heatmap over this space, whatever network produced it.

W2-3:

We thank the reviewer for asking where our contribution applies beyond horizons. One example is needle trajectory estimation in ultrasound-guided biopsy. Wijata et al. (2024) train a network to segment the needle and then obtain its line in a fixed post-processing step, the maximum of a Radon transform of the mask. This matches our pipeline, in which a network is followed by a fixed differentiable transform, except that the line itself never enters training. VSmax would make this last step differentiable as well without failing at the angular boundary, so that the line can be trained end to end. We do not claim certainty that VSmax would outperform existing methods there. We see it as a direction worth investigating. What we do claim is that wherever a network scores candidates for a projective target, VSmax applies and our guarantees hold.

References

  • Syed Ammar Abbas and Andrew Zisserman. A Geometric Approach to Obtain a Bird's Eye View From an Image. ICCV Workshops, 2019.
  • Agata M. Wijata, Bartłomiej Pyciński, and Jakub Nalepa. A Needle in a (Medical) Haystack: Detecting a Biopsy Needle in Ultrasound Images Using Vision Transformers. ICIP, 2024.
Reviewer_YMxVOpenReview ↗

Post-rebuttal Review of paper ID 49248

W1. released W2-1.partially released W2-2. partially released W2-3. partially released.

The technical contribution of this work is clear, but I still see limitations in its scalability. Although the paper does not sufficiently demonstrate its usefulness from a practical standpoint, the theoretical contribution stands, so I am leaning toward weak acceptance for this paper.

AuthorsOpenReview ↗

Response to post-rebuttal review

We thank the reviewer for the constructive discussion and the updated assessment. Separating the three observations in W1 helped us make explicit the mechanism that connects them. Moreover, the request for an ablation that replaces only the readout led directly to the new real-world example that improves the baseline on Horizon Lines in the Wild. We also appreciate the reviewer's comment on practical scope and see a broader evaluation beyond horizon estimation as a natural direction for future work.

Reviewer_1bf3OpenReview ↗
Rating
3(+1)
Confidence
2(+1)
Informativeness
3(+1)
Presentation
2
Soundness
2
Read review
Paper summary

This paper builds a differentiable heatmap readout for predicting undirected lines. Plain coordinate averaging breaks here: two nearby lines can carry opposite signed offsets across an angular boundary, so their average cancels out. Veronese soft-argmax instead averages sign-invariant quadratic features of the normalized line representation, then recovers a line from the leading eigenvector of the resulting moment matrix. Supervision is applied to the averaged representation, before recovery. The paper gives invariance and recovery guarantees and tests the method on synthetic lines and on horizon estimation from real images.

Critical strengths
  • The paper pins down a concrete geometric failure of coordinate readouts and explains it with an explicit counterexample. The case for changing the representation is immediate.
  • The readout is simple and adds no trainable parameters. Using one representation for both aggregation and supervision is coherent, and putting the loss before recovery keeps gradients off the eigendecomposition.
  • The theory connects the training loss to recovered prediction error and states when the prediction is unique.
  • The synthetic experiments separate the readout from the training objective through controlled comparisons. Tests under image corruption and shifted training support add more, and the horizon experiment shows the pipeline working on real images.
Critical weaknesses
  • Configurations for the swept baselines in the real-image comparison were picked using combined-test performance. That tuning favors the reported baseline scores, and the numbers cannot be read as an independent-test outcome. Select configurations on validation data and freeze them before touching test.
  • The real-image numbers need better evidence of stability: averages over training seeds, training variability, and paired image-level confidence intervals. The synthetic readout and supervision comparisons are convincing, but the real-image gains do not.
Questions for authors
  • Could the authors provide more details about the swept baselines in the real-image comparison?
  • Provide more evidence of stability on real-image gains.
4 comments
AuthorsOpenReview ↗

Answer to the first question

We thank the reviewer for the precise reading of the method. We agree with both concerns and provide the missing evidence.

Question 1: The swept baselines are LETR (Xu et al., 2021) and LINEA (Janampa and Pattichis, 2025). Both are transformer line detectors that do not utilize the Hough prior and predict lines directly, which is the alternative named in our introduction. They are general line detectors, and no horizon training recipe exists for them. We therefore swept their architectural hyperparameters. LETR was trained with two backbones (ResNet18 and ResNet50) and LINEA with four model sizes (N, S, M and L), each with eight query counts between 20 and 1000. This gives 16 LETR runs and 32 LINEA runs, 48 in total. Appendix D.5 lists the settings and scores of every run.

For HLW-CNN, we use the configuration that Workman et al. (2016) report as their best. To strengthen this group, we add the model of Abbas and Zisserman (2019), which is also trained with one fixed configuration. Appendix D.5.5 documents its training. Its results are in the table under Question 2.

This further leads to how the swept configurations were selected, and here we agree with the reviewer. We reported, for each transformer sweep, the run with the highest combined-test AUC. This gives both baselines their best possible combined-test score, so these scores are not independent test results. We have now followed the recommended protocol. We evaluated the same 48 trained networks on the HLW validation split and selected one configuration per family by validation AUC alone. We can confirm that validation selects the same two configurations as before. The selected configurations, and hence all reported runs, therefore do not change. In the revision, Appendix D.5 reports the validation score of every run (Tables 12 and 13) and describes the selection by validation.

AuthorsOpenReview ↗

Answer to the second question

Question 2: We thank the reviewer for acknowledging that our synthetic comparisons are convincing, and agree that the real-world comparisons need the same rigor. We achieve this by addressing the stability concerns in the order requested.

On the one hand, we trained every model of the real-world comparison on four additional seeds. So, only the seed changes for those additional runs, and provides the required variability. The table below now lists mean ±\pm standard deviation instead of single seed scores.

ModelTest IDTest OODTest combined
LETR (R50, 50 queries)0.794±0.0100.794 \pm 0.0100.764±0.0120.764 \pm 0.0120.784±0.0060.784 \pm 0.006
LINEA (N, 500 queries)0.695±0.0460.695 \pm 0.0460.726±0.0450.726 \pm 0.0450.706±0.0450.706 \pm 0.045
HLW-CNN0.797±0.0070.797 \pm 0.0070.782±0.0070.782 \pm 0.0070.792±0.0040.792 \pm 0.004
Abbas and Zisserman (2019)0.651±0.0350.651 \pm 0.0350.720±0.0200.720 \pm 0.0200.675±0.0280.675 \pm 0.028
Ours0.806±0.0080.806 \pm 0.0080.800±0.0060.800 \pm 0.0060.804±0.0070.804 \pm 0.007

With five seeds, the ranking of the submission is unchanged. Our model still has the highest mean AUC on every split, and it is above every baseline on the combined test set in each of the five seeds. The advantage therefore does not depend on a favorable seed.

On the other hand, we agree that paired image-level confidence intervals further strengthen the evaluation, and we compute them. The seeds capture how much the result varies with training, and the intervals capture how much it varies with the test images. For the submission seed, we resample the test images with replacement 10,000 times and evaluate our model and each baseline on the same resampled images. The first row gives our AUC on the submission seed, and the other rows give each AUC difference with its 95% percentile bootstrap interval.

Test IDTest OODTest combined
Ours, AUC0.8130.8130.8060.8060.8110.811
Ours − LETR+0.014+0.014 [+0.005+0.005, +0.023+0.023]+0.049+0.049 [+0.035+0.035, +0.063+0.063]+0.026+0.026 [+0.019+0.019, +0.034+0.034]
Ours − LINEA+0.118+0.118 [+0.104+0.104, +0.133+0.133]+0.057+0.057 [+0.036+0.036, +0.079+0.079]+0.097+0.097 [+0.085+0.085, +0.109+0.109]
Ours − HLW-CNN+0.017+0.017 [+0.005+0.005, +0.029+0.029]+0.020+0.020 [+0.002+0.002, +0.039+0.039]+0.018+0.018 [+0.008+0.008, +0.028+0.028]
Ours − Abbas and Zisserman (2019)+0.154+0.154 [+0.132+0.132, +0.177+0.177]+0.057+0.057 [+0.032+0.032, +0.083+0.083]+0.121+0.121 [+0.103+0.103, +0.138+0.138]

All 95% confidence intervals lie above zero, hence, the advantage does not depend on a favorable choice of test images either. In the revision of our paper, we update Table 1 (Section 5.4) to show the results over 5 seeds instead of the single seed, and we provide the bootstrapping results in Appendix D.5 (Table 11).

Together, the seeds and the bootstrapping show that the real-image gains are stable. We observe that the reviewers share the broader concern that our real-image evaluation is less convincing than the synthetic one. So beyond addressing the specific request on comparisons present in the submission, we provide further real-image evidence in a general comment including a new experiment.

References

  • Syed Ammar Abbas and Andrew Zisserman. A Geometric Approach to Obtain a Bird's Eye View From an Image. ICCV Workshops, 2019.
  • Sebastian Janampa and Marios Pattichis. LINEA: Fast and Accurate Line Detection Using Scalable Transformers. ICIP, 2025.
  • Scott Workman, Menghua Zhai, and Nathan Jacobs. Horizon Lines in the Wild. BMVC, 2016.
  • Yifan Xu, Weijian Xu, David Cheung, and Zhuowen Tu. Line Segment Detection Using Transformers Without Edges. CVPR, 2021.

Thanks for the additional clarification and response. I acknowledge your responses, and they address my concerns, especially the real-world gains. Therefore, I will raise my score and lean towards a positive recommendation.

We thank the reviewer for the careful follow-up. The requested analyses made our evaluation considerably stronger, and we are glad that the revised paper now provides a clearer basis for its assessment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General comment: real-world gains from VSmax

We thank all reviewers for their careful feedback, and observe that there is a shared concern. Namely, it is the demonstration of the real-world impact of our findings. We agree that this turns out to be the main gap in our submission. To address this gap, we ran an additional experiment with the most relevant heatmap-based baseline on Horizon Lines in the Wild, the method of Abbas and Zisserman (2019). We argue that their heatmap readout (Top-11) suffers from the same topological mismatch that our paper addresses. Simply replacing it with our readout (VSmax) at inference time improves the network's AUC by up to 22\mathbf{22}%. The gain we claim in our paper therefore depends neither on our Hough-based architecture nor on our training loss. The results are shown in the following table.

Test setReadoutTest IDTest OODTest combined
HLW-visibleTop-110.651±0.0350.651 \pm 0.0350.720±0.0200.720 \pm 0.0200.675±0.0280.675 \pm 0.028
VSmax0.824±0.0170.824 \pm 0.0170.818±0.0110.818 \pm 0.0110.822±0.0120.822 \pm 0.012
Relative gain (%)+26.8±4.1\mathbf{+26.8 \pm 4.1}+13.7±2.1\mathbf{+13.7 \pm 2.1}+21.9±3.2\mathbf{+21.9 \pm 3.2}
HLWTop-110.687±0.0200.687 \pm 0.0200.645±0.0100.645 \pm 0.0100.672±0.0120.672 \pm 0.012
VSmax0.748±0.0060.748 \pm 0.0060.671±0.0130.671 \pm 0.0130.721±0.0050.721 \pm 0.005
Relative gain (%)+8.9±2.5\mathbf{+8.9 \pm 2.5}+4.0±0.9\mathbf{+4.0 \pm 0.9}+7.2±1.8\mathbf{+7.2 \pm 1.8}

We trained five networks with different random seeds, following the training procedure of Abbas and Zisserman (2019). We acknowledge that we could not reproduce the scores reported in their paper, however, not all training details for HLW are available. We argue that this does not affect the comparison here, because both readouts get the same outputs of the same network, and we focus on the relative gain. The table reports the inference results (mean ±\pm standard deviation) with their original readout and VSmax in its place. We use HLW-visible, the test set from our evaluation that contains only images with a visible horizon, and the full HLW benchmark they report on.

Beyond these experimental results, we briefly explain from a theoretical perspective why our readout improves performance in this real-world scenario. The established horizon detection approach of Abbas and Zisserman (2019) also represents the horizon geometrically. Its coordinate representation inherits the same Z2\mathbb{Z}_2 symmetry, so their proposed top-11 readout exhibits the failure that our Proposition 1 proves. VSmax does not, because the Veronese embedding is invariant to the symmetry. Theorem 1 further guarantees accurate recovery as the network concentrates evidence around the true horizon.

In the revised manuscript we provide, all changes are marked in blue. Section 5.4 now also reports this comparison, and Appendix D.5.5 documents training details. Since this comparison also uses the full HLW test set, the Ethics statement now refers to the HLW dataset instead of only its filtered subset. We believe this addresses the reviewers' main concern. Our readout yields substantial performance gains in an established, practically relevant architecture on a real-world benchmark. This further shows that our method is of interest beyond the Hough transform community, to a broader audience working on line and geometry estimation.

Reference

  • Syed Ammar Abbas and Andrew Zisserman. A Geometric Approach to Obtain a Bird's Eye View From an Image. ICCV Workshops, 2019.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.