acceptodds
Under review as a conference paper at ICLR 2027

Geometry Meets Structure: Learning Robust Monocular Visual Odometry with Scale-Preserving Constraints

Abstract

Self-supervised monocular visual odometry learns motion from appearance but lacks explicit geometric consistency at inference time. Two-view geometry provides complementary motion constraints, yet its translation is scale-ambiguous and its estimates can become unreliable under weak parallax or noisy correspondences. We investigate how learned motion predictions and explicit geometry can be selectively combined, and present GeoStruct-VO. Its Structure-Aware Feature Enhancement (SAFE) module augments shallow appearance features with spatially distributed responses for pose estimation. At inference time, Scale-Preserving Geometric Refinement (SPGR) assesses geometric reliability based on correspondence support, cheirality, epipolar consistency, and agreement with the learned prediction. Reliable estimates refine rotation and translation direction while retaining the network-predicted translation magnitude; otherwise, the learned pose is preserved. A Multi-frame Rotation Graph (MRG) further incorporates quality-filtered long-gap rotational constraints to reduce accumulated drift. Experiments on KITTI and cross-domain evaluations on vKITTI2 show that GeoStruct-VO improves translational and rotational accuracy over the evaluated learning-based methods. Component analyses further demonstrate the complementary effects of feature enhancement, selective geometric correction, and multi-frame rotation refinement. The real-time configuration processes each frame in 20.9 ms, showing that explicit geometry can improve learned odometry without online parameter adaptation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Monocular Visual Odometry, Robotics, Self-Supervised Learning, Representation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ZZ77OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GeoStruct-VO, a hybrid monocular visual odometry pipeline that combines learned motion predictions with selectively applied two-view geometry. Its depth and pose networks are trained jointly through self-supervised view reconstruction, while SAFE modifies the shallow RGB feature encoder to enhance motion-relevant structural responses. At inference time, SPGR applies rule-based reliability checks to selectively replace the network predicted rotation and translation direction while retaining the predicted translation norm. A sliding window module, MRG, estimates rotational corrections from cross-frame constraints and updates trajectory positions accordingly. Experiments on KITTI sequences 09–10 and two vKITTI2 scenes under three conditions report lower average odometry errors than the evaluated learning-based baselines. The vKITTI2 evaluation uses KITTI-trained models without fine-tuning or test-time adaptation.

Critical strengths
  • Competitive performance on the evaluated benchmarks.\textbf{Competitive performance on the evaluated benchmarks.} On KITTI sequences 09–10, the full pipeline achieves average translational and rotational errors of 2.93% and 0.46°/100 m, respectively, outperforming the learning-based baselines included in the main comparison.
  • Positive cross-domain results without parameter adaptation.\textbf{Positive cross-domain results without parameter adaptation.} KITTI-trained models are evaluated on two vKITTI2 scenes under three appearance conditions without fine-tuning or test-time parameter updates. The full pipeline obtains the lowest average ATE, translational error, and rotational error among the methods evaluated in this setting.
  • Component ablations demonstrate measurable benefits within the pipeline.\textbf{Component ablations demonstrate measurable benefits within the pipeline.} Removing SAFE increases translational error, removing SPGR increases both translational and rotational errors, and removing MRG also degrades the aggregate results. These experiments support the practical usefulness of the components within the proposed system.
  • Practical front-end inference speed.\textbf{Practical front-end inference speed.} The configuration without MRG achieves a reported front-end latency of 20.9 ms/frame while retaining competitive accuracy on the reported hardware.
Critical weaknesses

Major Weaknesses

  • Insufficient comparison with closely related methods and alternative back ends.\textbf{Insufficient comparison with closely related methods and alternative back ends.} Although the paper motivates its contribution relative to hybrid systems such as DROID-SLAM, DPVO, and DF-VO, these methods are absent from the main comparison. DF-VO appears only in a supplementary evaluation under a different protocol. Furthermore, the MRG ablation only considers removing the back end, without comparing against established alternatives such as rotation-graph optimization, pose-graph optimization (e.g., ORB-SLAM), bundle adjustment (e.g., ORB-SLAM2 and DROID-SLAM), or even local bundle adjustment. Since the benefits of geometric refinement have already been well established in prior work, the current experiments merely confirms the usefulness of adding geometric optimization, rather than demonstrating any specific advantage of MRG. Therefore, it remains unclear what MRG offers beyond conventional geometric optimization methods. I know inference time is a trade-off, but these conventional refinements are needed to build the contribution.
  • Limited evidence for SAFE’s claimed mechanism.\textbf{Limited evidence for SAFE's claimed mechanism.} The component ablation supports the usefulness of SAFE within the complete pipeline, but does not establish that its improvement specifically results from preserving motion-relevant structural cues. There are no parameter- or computation-matched controls, targeted ablations of its constituent operations, or comparisons that reduce or delay downsampling. Figure 4 shows changes in feature responses and channel participation, but these observations do not establish structural preservation or motion relevance. The current evidence therefore does not distinguish the proposed mechanism from alternative explanations such as increased model capacity or more general feature enhancement.
  • The central design rationale is insufficiently developed.\textbf{The central design rationale is insufficiently developed.} The connection between the stated problems and several design choices remains unclear. The paper motivates SAFE through boundary-related cues and the possible loss of subtle responses during early downsampling, but does not clearly establish which structural information is missing or explain how its particular filtering, aggregation, and downsampling operations address that loss. Since the filters are unconstrained and receive no explicit structural supervision, their learnability alone does not establish the intended structural interpretation. Section 3 also asserts complementarity between RGB and predicted disparity without adequately explaining the expected benefit of channel exchange. In the introduction, MRG is introduced without adequately connecting it to the preceding motivation. Although Section 3.3 later identifies residual rotational drift as its target, the rationale for choosing this particular correction formulation remains insufficiently developed. These gaps make it difficult to identify the specific hypotheses underlying the proposed components and assess whether the experiments test them.

Minor Weaknesses

  • Technical definitions are incomplete.\textbf{Technical definitions are incomplete.} The paper does not specify how the MRG confidence weight wsew_{se} is obtained, sufficiently detail the operations represented by L\mathcal{L} and R\mathcal{R}, or define the spatial CV and normalized effective-channel statistics. These omissions limit reproducibility and interpretation.
  • Figures lack sufficient labeling and explanatory context.\textbf{Figures lack sufficient labeling and explanatory context.} Figure 1(c) does not clearly distinguish the configurations and timing definitions associated with 20.9 and 31.4 ms/frame. Figure 2 mixes architectural details, training objectives, and inference stages, while omitting intermediate feature labels needed to connect it to the equations. The dense-mapping panels in Figure 3 lack sufficient explanation of their construction, visual elements, and intended takeaway. Figure 4 similarly requires a clear description of feature aggregation and normalization for meaningful visual comparison.
  • Scale-related wording and evaluation details require clarification.\textbf{Scale-related wording and evaluation details require clarification.} Preserving the network-predicted translation norm does not by itself establish recovery of metric translation magnitude, which makes the contribution statement stronger than the described setup supports. The paper should clarify this wording, specify whether trajectory alignment includes scale fitting, and describe the ATE computation and units. It should also explain how translation magnitude is assigned to the Pure Geometry baseline in Figure 5.
Questions for authors
  • What advantage does MRG provide over a conventional rotation-graph, pose-graph, or local bundle-adjustment back end attached to the same front end? Can the authors provide a controlled comparison, including the accuracy–runtime trade-off?
  • How are “motion-relevant structural cues” operationally defined? Can the authors provide evidence that SAFE preserves these cues, rather than improving performance through additional capacity or computation? A matched-capacity control and a reduced-downsampling comparison would help distinguish these explanations.
  • What specific benefit is expected from exchanging RGB and predicted-disparity features, and what evidence isolates this benefit? Can the authors provide an error analysis showing when residual rotational drift remains after SPGR and when MRG successfully reduces it?
  • Does trajectory alignment fit a global scale, and how is ATE computed? How is translation magnitude determined for the Pure Geometry baseline in Figure 5? Does “metric translation magnitude” mean that the network predicts translation in physical units before trajectory alignment? If so, what anchors this scale in the described training setup?
Reviewer_1xWyOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper deal with the problem of estimating the pose (rotation and translation) from monocular images using a neural network. It is well known that, when using monocular images, the translation is observable only up to scale. However, neural networks uses implicit prior on the metric dimensions of the environment learned during the training to provide a scale prediction. The paper proposes GeoStruct-VO, a learning–geometry framework including the following three mechanism:

  1. Structure-Aware Feature Enhancement (SAFE) : a learnable structural feature extraction and adaptive spatial–channel aggregation that strengthens motion-relevant structural representations
  2. Scale-Preserving Geometric Refinement (SPGR) : an inference-time correction of the learned pose that keeps the network-predicted translation magnitude if two-view geometry satisfies geometric reliability and refine rotation and translation direction.
  3. Multi-frame Rotation Graph (MRG) : a decoupled back end that incorporates quality-filtered cross-frame rotational constraints to further reduce accumulated rotational drift.

The proposed approach has been validated on two datasets, KITTI and vKITTI2.

Critical strengths

The authors proposes an architecture that is technically sound.

Apart from comparisons with ORB-SLAM3 the results are good compared to the considered SOTA using neural networks.

Critical weaknesses

The idea of using geometric constraints to enforce scale consistency is not new. Several approaches exists and I can recall at least one relevant that is not cited by the authors:

Mingkang Xiong, Zhenghong Zhang, Weilin Zhong, Jinsheng Ji, Jiyuan Liu and Huilin Xiong: Self-supervised Monocular Depth and Visual Odometry Learning with Scale-consistent Geometric Constraints, International Joint Conference on Artificial Intelligence 2020.

The idea of refine the predicted rotation with the geometrical motion is also not new, see the following relevant paper that is not cited by the authors:

Cimarelli, C.; Bavle, H.; Sanchez-Lopez, J.L.; Voos, H. RAUM-VO: Rotational Adjusted Unsupervised Monocular Visual Odometry. Sensors 2022.

The experimental results in real-time configuration (w/o MRG) are not impressive. OrbSLAM 3 performs better on sequence 9 and has a better rotation error in sequence 10. For these results it is not clear why one should use GeoStruct-VO instrad of OrbSLAM when rotation error matters. Moreover the authors did not report Absolute Trajectory Error (ATE) on KITTI to quantify global trajectory consistency.

Questions for authors

Why the authors did not consider the suggested relevant papers ?

Why DSO and ORB-SLAM3 have not been evaluated on vKITTI2 ?

Reviewer_4jd2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  1. Method. The paper presents GeoStruct-VO, a self-supervised monocular visual-odometry system that combines a learned pose network with explicit geometric refinement.

  2. Components. Structure-Aware Feature Enhancement (SAFE) augments shallow RGB features using depthwise convolutions, dual-branch downsampling, spatial aggregation, and channel recalibration. Scale-Preserving Geometric Refinement (SPGR) estimates rotation and translation direction from an essential matrix fitted to tracked Shi–Tomasi/Lucas–Kanade correspondences. Support, parallax, cheirality, epipolar-error, prediction-agreement, and translation-direction temporal-consistency tests determine whether the geometric estimate replaces the network rotation and translation direction while retaining the network translation magnitude. A Multi-frame Rotation Graph (MRG) additionally uses selected long-gap rotational constraints in sliding-window least-squares optimization.

  3. Validation. The method is trained with self-supervised photometric reconstruction, geometry-consistency, edge-aware smoothness, and channel-regularization losses. It is evaluated on KITTI sequences 09–10, with cross-domain tests on selected outdoor vKITTI2 scenes and conditions. It improves over the paper's learning-based baselines and achieves average KITTI errors of 2.93% translational drift and 0.46 degrees/100 m rotational drift. Ablations attribute the largest improvement to SPGR, with smaller additional gains from SAFE and MRG.

Critical strengths
  1. A practical and carefully gated hybrid pipeline. SPGR does not blindly trust two-view geometry: it checks correspondence support, parallax, cheirality, epipolar residuals, agreement with the network, and temporal consistency, and falls back to the learned pose when necessary. The aggregate vKITTI2 gate ablation supports the value of this selection under cross-domain synthetic conditions, although it does not report separate gate results for fog or other individual conditions.

  2. Strong reported accuracy with useful ablations and runtime accounting. On the reported KITTI and vKITTI2 protocols, GeoStruct-VO improves substantially over the selected learning-based baselines. The paper separately studies geometric rotation, translation direction, reliability gates, and MRG window size, and provides an unusually detailed breakdown of concurrent front-end and asynchronous back-end costs.

Critical weaknesses
  1. The technical novelty is limited and insufficiently isolated. SAFE is assembled from standard learnable convolutions, pooling, residual fusion, spatial aggregation, and channel attention. The paper explicitly states that “structure-aware” denotes an architectural bias rather than supervised edge extraction. Table 4 provides a same-backbone removal ablation, but there is no parameter/FLOP-matched residual-convolution or attention replacement showing that the gain arises from the claimed structural bias rather than added capacity or conventional attention. Likewise, combining learned motion estimates with explicit geometry and rejecting unreliable geometric updates is established in hybrid VO; the paper itself discusses DF-VO, DROID-SLAM, DPVO, CM-VO, and ORB-SfMLearner. MRG is a small-angle sliding-window rotation-graph correction based on standard least squares. The particular gating and integration may be useful, but the experiments do not isolate a substantial new principle beyond this assembly.

  2. The empirical scope is too narrow to support broad robustness and generality claims. The strictly held-out KITTI result covers only sequences 09 and 10, while vKITTI2 is synthetic; all evaluations remain within real or synthetic road-driving data, with no non-driving camera-motion domain. There is also no dedicated stress evaluation for low texture, strong rotation, motion blur, short translation baselines, or dynamic occlusion, despite reliance on calibrated two-view geometry, fixed thresholds, sufficient parallax, and stable sparse tracking. TUM RGB-D or vision-only EuRoC evaluations would test these regimes more appropriately than relocalization datasets such as 7-Scenes or ScanNet. DPVO and a loop-closure-disabled DROID-VO configuration are the most relevant modern learned VO references, although their synthetic supervision and optimization protocols should be reported separately. Full DROID-SLAM, MASt3R-SLAM, and VGGT-based SLAM would provide broader context but are not fair drop-in baselines because they use longer-window/global optimization, loop closure, or different calibration and scale assumptions. Thus, the strongest empirical criticism is the narrow domain coverage and absence of a matched modern hybrid comparison, not simply omission of every full SLAM system.

Questions for authors
  1. What exact alignment and scale correction are applied before computing KITTI and vKITTI2 trelt_{\mathrm{rel}}, rrelr_{\mathrm{rel}}, and ATE? Given the scale gauge freedom of monocular self-supervised view synthesis, in what precise sense is the network-predicted translation magnitude “metric,” and how is its scale fixed without pose, depth, stereo, or inertial supervision?

  2. How does GeoStruct-VO compare with DPVO or a DROID-VO configuration under a matched monocular, odometry-only protocol and the same scale alignment? These supervised systems should be labeled separately. If full optimization-based SLAM systems such as MASt3R-SLAM or VGGT-based variants are outside scope, please state that boundary and avoid implying a comparison with the broader learning–geometry literature.

  3. Does the method remain effective on indoor or handheld sequences with frequent rotation, short translation baselines, low texture, dynamic occlusion, and motion blur? Results on an appropriate VO benchmark such as TUM RGB-D or vision-only EuRoC, together with SPGR acceptance rates and failure-case analysis, would materially affect my assessment of the claimed robustness and generality.

  4. For SAFE, can the authors provide parameter/FLOP-matched controls using ordinary residual convolutions or standard spatial/channel attention, and quantitative evidence that the learned responses correspond to motion-relevant structure? The current response visualizations and distribution statistics do not isolate structural modeling from increased capacity.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.