acceptodds
Under review as a conference paper at ICLR 2027

Relaxation-Free Robust Vanishing Point Estimation in Manhattan World

Abstract

In man-made environments, the parallel edges of a scene form three mutually orthogonal families, known as the Manhattan structure, and their projections meet at three vanishing points that fix the rotation between the camera and the scene. Recovering that rotation from a single calibrated image is difficult, because a line detector returns far more segments than the scene structure actually contains. The two halves of the task are moreover entangled, since a segment can be assigned to a direction only once the directions are known, while estimating them in turn presupposes that assignment. A single criterion, the truncated multi-selection error over the rotation group, settles assignment and estimation together. Our main contribution is MMVP, an iterative algorithm that minimizes this criterion directly, in the original problem dimension and without any relaxation (in the sense of losing tightness). Built on the majorization–minimization principle, the method assembles its surrogate in three stages, freezing the association at the current iterate, majorizing the truncation by an outlier process guided by a homotopy, and dominating the quadratic form over the rotation group by a linear function whose minimizer is available in closed form at a cost linear in the number of lines. We prove that, at every fixed level of the homotopy, the objective values it produces are non-increasing and bounded below, hence convergent, and that the iterates admit limit points, every one of them stationary. A deterministic voting-based initialization completes the pipeline, and no stage of the procedure draws a random sample. Experiments on synthetic scenes and on two real datasets, against geometric and learning-based estimators alike, show that MMVP outperforms them on association F and recall, and that no competing method reaches a higher angular accuracy at any threshold.

Primary area:
optimization
Keywords:
Manhattan world, non-convex optimization, relaxation-free optimization, rotation estimation, vanishing point estimation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_FdZfOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper estimates the camera’s orientation relative to a “Manhattan” scene, i.e. a scene built mostly from three perpendicular directions, such as buildings and rooms. The input is the line segments detected in a single calibrated image. This comes down to finding one rotation and, at the same time, deciding which lines belong to which of the three directions and which lines are outliers. The authors use the same objective as GlobustVP. Instead of GlobustVP’s convex relaxation and conic solver, they propose MMVP, an iterative solver based on majorization - minimization. Each iteration simplifies the problem in three steps, 1. It temporarily fixes which direction each line belongs to, 2. It down-weights outlier lines, moving gradually from a smooth version of the loss to the exact one, 3. It replaces the rotation step with a simpler problem that has a closed-form solution through a small SVD. As a result, every step is fast and simple, every estimate is a valid rotation, and no external optimization solver is needed. The authors also prove that the objective decreases at a fixed homotopy level. They provide a quick test that checks whether the returned rotation is globally optimal for its line grouping, and they use a deterministic initialization in place of random sampling. Experiments on synthetic data, HoliCity and YUD+ compare MMVP with nine geometric and learning-based methods. MMVP reports the best line grouping and angular accuracy scores, running in about 30 - 70 ms per image on a CPU.

Critical strengths
  • The proposed solution is a clean, solver-free way to minimize the GlobustVP objective. Each step is closed-form, costs grow linearly with the number of lines, and every iterate stays a valid rotation.
  • Failures are kept in averages at the worst score, and the YUD+ Manhattan/non-Manhattan split is informative.
  • Empirical results look strong with stable behavior as contamination rises.
Critical weaknesses
  • Theorem D.1 shows that, for a fixed value of the homotopy parameter μ\mu, the objective never increases and the iterates reach a stationary point. However, as I understand Algorithm 1 and Section 3.2, μ\mu is doubled after every outer iteration until it reaches 10e-6. Starting from μ0=c2/(2emax−c2)\mu_0 = c^2/(2e_{max} - c^2) with c = 0.03, this takes roughly 30 doublings. That matches the “thirty to forty outer iterations” reported in Appendix D.2. If this reading is correct, the algorithm takes only one step at each value of μ\mu before moving to the next. The guarantees of Theorem D.1 then apply mainly to the final phase, after μ\mu stops increasing. I appreciate that the Remark in Appendix D.1 acknowledges that monotonicity holds per level and not across levels. Still, the abstract and contribution (3) read as if the guarantee covered the full procedure.
  • MMVP is a well-engineered solver, and I see value in how its pieces fit together. However, most of the individual pieces seem to come from prior work, and the paper does not always make this clear. So the main contribution seems to be combining these known tools into one efficient solver for an existing objective. That can be a worthwhile contribution, but the current framing, especially contribution, suggests more originality than the components support.
  • The results are strong, but the main paper has no ablation study, so it is unclear which part of MMVP is responsible for the improvement. Without ablations, it's hard to tell whether the gains come mainly from the proposed solver or from the other parts, such as a good initialization followed by local polishing.
Questions for authors
  1. How many outer iterations are spent at each μ\mu level in practice? Do results change if you iterate to convergence at each level?
  2. How was c = 0.03 selected, and how sensitive are results to c on each dataset?
  3. On what data were CONSAC, PARSAC, Lin et al. and U-ARE-ME trained? Were any trained on YUD/HoliCity-like data?
2 comments

My main concerns about soundness and ablations are addressed, and I will raise my score to weak accept.

AuthorsOpenReview ↗

Sincere thanks for your follow-up and updating your score

Dear Reviewer FdZf,

Thank you very much for taking the time to reconsider, for confirming that your main concerns about soundness and ablations have been addressed, and for raising your score. We are truly grateful for your engagement and for the fairness of your assessment.

With sincere appreciation and best regards,

Authors of Submission 9457

Reviewer_WQ7COpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MMVP, a majorization-minimization algorithm for robust vanishing point estimation under the Manhattan-world assumption. The method constructs a surrogate through three stages: fixing the current line associations, majorizing the truncation using an outlier process guided by GNC, and converting the resulting quadratic optimization into a closed-form Procrustes update. The authors establish objective monotonicity and stationary limit-point properties at fixed homotopy levels. They also provide an a posteriori test for the global optimality of the rotation given the returned association. The experiments with synthetic and real data demonstrate the improvements of the method in F1, recall, and angular accuracy with geometric and learning-based methods.

Critical strengths
  1. The approach avoid to use the semidefinite lifting and conic solver to address efficiency limitation.

  2. MMVP is best or on par across synthetic, HoliCity and YUD+ benchmarks, with the largest margins under heavy contamination.

Critical weaknesses
  1. The contribution is the combination of existing techniques, including majorization-minimization, GNC, and Procrustes optimization. The guarantees are standard MM results at a fixed homotopy level, and the optimality test does not cover the association, so the value rests mainly on the empirical evidence.

  2. There is no ablation of individual stages or the deterministic initialization, and MMVP additionally uses K=6 restarts. The improvement therefore cannot be attributed to the three-stage majorization.

Questions for authors
  1. Could the authors provide ablations for the individual stages and the deterministic initialization on the real data?

  2. On instances where GlobustVP returns a frame, what objective values of (P) do the two methods attain?

  3. On HoliCity, line labels are assigned by the closest ground-truth direction within the inlier threshold. Is this threshold the same as c = 0.03 used, and how sensitive are precision, recall and F1 to it?

Reviewer_ErZ6OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper estimates a Manhattan frame from line segments in a single calibrated image. It keeps the truncated multi-selection objective of GlobustVP but replaces the SDP relaxation with a majorization-minimization solver (MMVP) that stays on SO(3). Each outer iteration fixes the line-to-direction assignment and reweights lines with a GNC-style truncated least squares outlier process under a homotopy. It then updates the rotation with Procrustes after linearly majorizing each quadratic form. The paper validates monotone descent and stationary limit points at a fixed homotopy level, gives a dual certificate for the rotation at a fixed association, and uses a deterministic voting initialization. Experiments on synthetic data, HoliCity and YUD+ compare against geometric, relaxation-based and learning-based methods, and report the best recall, F1 and angular accuracy.

Critical strengths
  • The solver is simple and practical. It is deterministic, needs no conic solver, scales linearly with the number of lines and runs in about 36 ms per image, while matching or beating GlobustVP and the other baselines on HoliCity and YUD+.

  • The evaluation is honest about failures, since failed runs are kept in the averages with the worst score, and association and angular metrics are reported together.

Critical weaknesses
  • Limited novelty. The cost function comes from GlobustVP, and the solver is built from known parts. The hard assignment is as in classical EM schemes. The weights and schedule are those of GNC-TLS (Yang et al., 2020): Eq. 7 and the initial μ are identical. The rotation step is a standard linear majorizer followed by Procrustes.

  • The theory covers less than it seems. Monotone descent and stationarity hold only at a fixed homotopy level, but μ is doubled at every iteration, so most of the trajectory is not covered (App. D.1.2 notes increases after μ updates). The certificate in Prop. D.2 only covers the rotation for a fixed association. The combinatorial part stays open, and exhaustive search finds a lower objective on 7% of the synthetic instances.

  • Global methods and related work are missing. The paper argues about global optimality but does not compare with branch-and-bound Manhattan frame estimation (Bazin et al., CVPR 2012; Joo et al., CVPR 2016). It also misses line-based quasi-global VP estimation methods (Kim et al., RA-L 2022; Li et al., TPAMI 2023).

  • The evidence for the gains is a bit limited. On HoliCity the gain over Quasi-VP is small (F1 82.6 vs. 79.8, AA@10° 80.2 vs. 78.0) at 36 times the runtime. Much of the gap to GlobustVP comes from its 415 failed crops scored at 90°. The association labels are built with the same truncation rule that MMVP optimizes.

Questions for authors
  • Q1: How does MMVP compare in accuracy and runtime with methods that search the rotation space directly? Examples are branch-and-bound (Bazin et al., CVPR 2012) and quasi-global line-based search (e.g. Kim et al., RA-L 2022). On the synthetic data, where gravity is known, all methods could be given the same vertical prior.

  • Q2: Why does GlobustVP fail on 415 of the 2,490 HoliCity crops? Was it run with its released code and settings? Please also report results with failed crops excluded for all methods.

  • Q3: Stage 2 is GNC-TLS. What does MMVP gain over plain GNC-TLS with hard assignment and a direct rotation update, e.g. a few Riemannian Gauss-Newton steps on SO(3) instead of the linear majorizer of Lemma 3.1? An ablation study on this would make the contribution a bit clearer.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.