- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes Structural Radial Aggregation for Yaw (S-RAY), a framework for cross-view yaw estimation under camera position uncertainty. Building on the radial-invariant column-matching and yaw-voting framework of LAYS, S-RAY introduces two complementary components. First, a structural score aligns ground-view structural direction distributions with aerial structure-axis representations to identify promising yaw candidates at each possible camera position. Second, Fourier-based continuous voting replaces discrete hard-bin accumulation with a learned band-limited angular kernel. By exploiting the angular structure of the voting formulation, the method computes scores efficiently through FFT-based correlations without explicitly constructing position–match pairs. Experiments on KITTI, MGL, and VIGOR report substantial improvements in fine-grained yaw estimation. The paper also presents ablation studies, computational comparisons, and downstream localization results using the estimated yaw for initialization.
Critical strengths
- An elegant and computationally meaningful formulation of continuous yaw voting.
The Fourier-based voting formulation is a notable technical contribution. By representing the angular voting kernel using a finite cosine series and exploiting the angle-difference structure of the induced yaw, the method transforms expensive position–match voting into a small number of FFT-based correlations.
For the selected band-limited kernel, this factorization provides an exact computation of the corresponding continuous voting score rather than an additional numerical approximation of the direct sum. The resulting formulation is mathematically well motivated and offers a practical way to scale yaw scoring across candidate positions and orientations.
The integration of structural orientation information with radial matching also provides a reasonable approach to resolving ambiguous yaw hypotheses that may remain difficult to distinguish using individual correspondence votes alone.
- Substantial empirical improvements supported by useful ablations and downstream evaluation.
The reported results demonstrate substantial improvements in fine-grained yaw estimation across multiple benchmarks. In particular, R@1° increases from 51.07% to 87.65% on KITTI Same Area, from 48.44% to 72.91% on KITTI Cross Area, and from 34.81% to 55.22% on MGL relative to the reported LAYS baseline.
The component ablations provide useful evidence that structural guidance and continuous voting contribute to the overall performance. The paper also reports computational advantages for the voting module and demonstrates that the estimated yaw can improve downstream Loc² localization when used for initialization.
The inclusion of failure-case analysis, particularly for large angular ambiguities, further helps characterize the remaining limitations of the approach.
Critical weaknesses
- The contributions of structural alignment, continuous voting, and training differences are not sufficiently isolated.
Although the overall performance improvements are substantial, the method builds directly on the column matching, radial geometry, and voting framework established by LAYS. Therefore, carefully isolating the contributions of the newly introduced components is important for assessing the methodological novelty.
The ablation results in Table 4 raise a specific attribution concern. The hard-bin variant, which retains structural scoring while replacing continuous voting with hard-bin accumulation, already achieves 78.69% R@1° on KITTI Same Area, compared with 51.07% for the reported LAYS baseline. The full method further increases this value to 87.65%.
Thus, a substantial portion of the overall improvement is already present before replacing hard-bin voting with the proposed continuous formulation. However, this difference cannot be attributed exclusively to structural alignment because other training or implementation changes may also contribute.
In particular, the structural score incorporates an existing aerial-only pose plausibility component, while the new alignment term and the additional training objective have not been sufficiently separated in the reported comparisons.
A matched evaluation of Fourier voting with the original LAYS plausibility score, the complete structural score, and the original LAYS model under equivalent training settings would help determine which design elements are responsible for the gains. A separate analysis of the additional training loss would further strengthen the attribution.
This issue is consequential because the individual contributions of structural alignment and Fourier voting constitute the main methodological claims.
- The comparison protocol and efficiency claims require more precise qualification.
Some compared methods appear to operate under different yaw assumptions or prediction conventions. In particular, methods that estimate orientation modulo 180° are not directly comparable to methods evaluated over the complete ±180° range without an appropriate treatment of the angular ambiguity.
The paper should more clearly distinguish methods designed for full-range yaw estimation from those relying on narrower orientation assumptions. Furthermore, while S-RAY demonstrates strong improvements in fine-grained recall, its reported mean yaw error on KITTI Cross Area is not uniformly better than that of the strongest competing method. Presenting these trade-offs more explicitly would provide a more balanced interpretation of the results.
The computational comparison also requires clarification. The reported 23× acceleration corresponds to the finest-level voting module, where runtime decreases from approximately 19.16 ms to 0.83 ms. However, the complete inference time decreases from approximately 103 ms to 84 ms, corresponding to an end-to-end speedup of about 1.23×.
The module-level improvement remains valuable, but emphasizing the 23× figure without consistently distinguishing it from full-system latency may overstate the practical efficiency advantage.
Finally, some implementation details of the structural score, including the orientation representation, gating mechanism, and auxiliary supervision, are deferred to the appendix. A clearer main-text explanation would improve reproducibility and make the structural component easier to assess.
Questions for authors
Q1. Attribution of performance improvements: Can the authors clarify the results of using Fourier voting with the original LAYS pose plausibility score but without the proposed structural alignment term? Are matched results available for LAYS under the same backbone and training configuration, including the additional loss described in Section 3.5? These comparisons would help establish the independent contributions of structural alignment and continuous voting.
Q2. Structural alignment mechanism: Could the authors clarify how the structural alignment term learns correspondence between ground-view structural directions and aerial orientation axes? In particular, how should the statement that training learns the relationship between angular axes be interpreted when Eq. (4) already specifies a fixed angular correspondence? A concise explanation of the learned quantities and the role of the structural gate would help clarify this mechanism.
Q3. Angular resolution and Fourier representation: Is the final yaw estimate restricted to a 1° angular grid, or is sub-bin refinement performed? How sensitive are R@1° and the reported sub-degree median errors to the output grid resolution? Additionally, can the authors clarify the learned angular kernel's shape and whether the selected Fourier order introduces side lobes or other effects that influence the voting distribution?
Q4. Baseline comparability and efficiency: How are baselines with different yaw parameterizations or orientation priors handled in the evaluation? Can the authors clarify the difference between module-level and end-to-end acceleration and describe the implementation and hardware configuration used for hard-bin voting? These details would help assess the fairness and practical significance of the reported improvements.
4 comments
Response 2/3
Q2. Our wording in Sec. 3.3 ("training learns how their angle axes correspond") was inaccurate: the correspondence is fixed, and what is learned are the two quantities it compares.
- What Eq. (4) fixes. If the camera has yaw , a structure that the ground image shows at direction relative to the camera runs along the absolute direction in the aerial image. Eq. (4), , applies this geometric rule; nothing about it is learned.
- What training learns. The two inputs of the rule: the ground direction distribution , i.e. in which directions the ground image shows structure (a 1D CNN over ground columns, softmax over direction bins), and the aerial axis scores , i.e. along which absolute directions the aerial image has structure at each position (a small CNN on aerial features). is high at yaws where the two line up.
- Role of the gate. is one learned scalar per pyramid level that weights within . It starts at zero, so enters the score only as far as it lowers the training loss.
- How it is trained. Two losses reach , and both use only the ground-truth pose: (i) the pose losses on the total score, like every other term; (ii) the structural alignment loss on alone, a circular cross-entropy over yaw between at the ground-truth position and a von Mises target () at the ground-truth yaw, weight 0.5. cannot separate from , so (ii) concentrates it on that pair, and decides the direction.
The revised Sec. 3.3 states the fixed correspondence and the learned and and defines the structural alignment loss; Sec. 3.5 gives the full objective, and Appendix B.3 describes the gate (this also addresses the last point of W2).
Q3, output grid. S-RAY evaluates scores on a coarse-to-fine grid of 4° (90 bins), 2° (180 bins), and 1° (360 bins) across the three pyramid levels, and outputs the argmax bin center at the finest 1° level directly, without post-hoc interpolation (Appendix C). Sub-degree median errors (e.g., 0.49° on KITTI) occur naturally because the ground-truth yaw is continuous: when a prediction selects the nearest 1° bin center, its error against the continuous ground truth lies in with an expected median near 0.25°. To test sensitivity to the readout grid, we re-decoded the final score of trained models, without retraining, on coarser grids (4° and 2°), and with a 0.5° sub-bin readout that linearly interpolates the 1° score, which places the estimate between bin centers toward the stronger neighbouring bin. R@1° in % (median yaw error):
| Readout | 4° | 2° | 1° (paper) | 0.5° sub-bin |
|---|---|---|---|---|
| VIGOR Same | 35.3 (1.41°) | 50.1 (1.00°) | 55.1 (0.88°) | 55.6 (0.87°) |
| VIGOR Cross | 29.7 (1.67°) | 39.0 (1.38°) | 40.3 (1.31°) | 42.0 (1.26°) |
| KITTI Same | 47.3 (1.06°) | 72.8 (0.59°) | 87.8 (0.48°) | 90.9 (0.47°) |
Grids coarser than 1° cost many points, while the sub-bin readout adds 0.5 points on VIGOR Same Area, 1.7 on Cross Area and 3.1 on KITTI, so the 1° grid limits the reported recall only slightly. The paper's numbers keep the 1° readout; this table is in Appendix E.
Q3, kernel shape and side lobes. The supplementary material of the submission already described the kernel shape; it is now Appendix B.1 of the revised PDF, which includes the appendix. With the initial kernel falls to 43% of its peak at ±15°, crosses zero near ±24°, and its ripples beyond stay within 16% of the peak. The learned kernels keep this width, 26°, 27° and 28° at half maximum on KITTI, MGL and VIGOR (figure in Appendix B.1), against 120° for . We now also report their side lobes, computed from the learned coefficients: the first zero crossing is at 21°, 22° and 23°, the largest positive side lobe is 14%, 11% and 0.2% of the peak, and the deepest negative lobe is −60%, −39% and −25%, located at 180° on KITTI and MGL and at 35° on VIGOR. A negative lobe is a penalty, so on KITTI and MGL the learned kernel penalizes the opposite heading, most strongly on KITTI.
Response 3/3
Q4 / W2, yaw conventions of the baselines; mean error; scorer vs. end-to-end speed
Yaw conventions. CCVPE, SliceMatch, Loc², VIRD, OGH-Net and LAYS report full-circle (±180°) yaw. ARC-Loc is the exception. Its line solver defines yaw modulo 180° and outputs a value in [−90°, 90°). Our full-circle metrics evaluate that output, so we agree its yaw columns are not directly comparable with the other rows.
Scored modulo 180° with ARC-Loc's own evaluator, ARC-Loc reaches 22.61% within 1° and 66.50% within 5° on KITTI Same Area (median 2.83°), and 11.59% and 41.29% on Cross Area (median 7.97°). Under the same axial metric, S-RAY reaches 79.42% and 89.92% on Cross Area (median 0.53°), against 72.91% and 82.37% full-circle. The ranking therefore does not depend on the yaw convention.
The revised Appendix D.1 states the yaw convention of every baseline and gives these modulo-180° results.
Mean error. On KITTI Cross Area, S-RAY's mean yaw error (23.00°) is slightly higher than VIRD's (22.03°), while its median is lower (0.59° vs. 1.87°) and its R@1° higher (72.91 vs. 30.39). The mean is dominated by two groups of large errors: 9.92% of queries near 90° and 7.56% near 180°. Sec. 4.2 states this comparison and Appendix A.1 analyzes the two groups.
Scorer vs. end-to-end speed. We agree that the 23× figure covers only the finest-level scorer. We measured both methods end to end on an NVIDIA RTX 6000 Ada: S-RAY is 2.50× faster with each method's own backbone and 4.23× faster with the matched VGG16 U-Net backbone. The submission's end-to-end times (about 103 vs. 84 ms) included implementation overhead outside the scorer, in inference code that both methods share. We removed it for both methods and re-measured them on one GPU (RTX 6000 Ada): S-RAY now takes 40.3 ms and LAYS 100.8 ms (2.50×). LAYS changes little because its time is dominated by hard-bin scoring (81.8 ms after feature extraction, against 6.0 ms for S-RAY).
| Configuration | S-RAY | LAYS | Speedup |
|---|---|---|---|
| Finest-level scorer (Table 5) | 0.834 ms | 19.162 ms | 22.97× |
| End to end (default backbones) | 40.3 ms | 100.8 ms | 2.50× |
| Post-backbone scoring | 6.0 ms | 81.8 ms | 13.6× |
| End to end (matched VGG16 U-Net) | 23.7 ms | 100.3 ms | 4.23× |
Protocol and hardware. All comparisons run with batch size 1, gradients disabled, and inputs resident in GPU memory. The finest-level scorer row reproduces Table 5 on an NVIDIA RTX PRO 6000 Blackwell over 102×102 candidate positions and 360 yaw bins, reporting the median of 5 rounds of 21 timed runs after 5 warm-up runs. The three end-to-end rows time one MGL query on an otherwise idle NVIDIA RTX 6000 Ada.
Hard-bin implementation. The hard-bin scorer is the official LAYS implementation, which we obtained from its authors. It computes a bin index for every position–match pair and scatters the votes. Both methods run on the same GPU with identical inputs, excluding data loading.
Where the time goes. After feature extraction, scoring across the three-level pyramid takes 6.0 ms for S-RAY and 81.8 ms for LAYS (13.6×), so the speedup comes from the Fourier scorer. Feature extraction takes 34.6 of S-RAY's 40.3 ms with DINOv2-B against 17.6 ms for LAYS's VGG16 U-Net, which reduces the end-to-end ratio to 2.50×. With the shared VGG16 U-Net it takes 20.1 ms for S-RAY and 19.7 ms for LAYS, and the ratio is 4.23×. Both S-RAY configurations are also more accurate than LAYS (backbone table under Q1).
In the revised paper, the abstract, introduction and conclusion state the end-to-end speedup alongside the scorer speedup and recall gain, Sec. 4.3 reports the timings above, and Appendix D.3 describes each configuration and the timing protocol.
Thank you for your rebuttal. After careful consideration of the manuscript quality, I have decided to retain my original score.
Response 1/3
We thank you for the thorough review and for recognizing the exactness and efficiency of our continuous Fourier voting formulation. Paper changes are marked in red.
Q1 / W1, attribution of the gains
Fourier voting with the LAYS plausibility score vs. our structural score S. "F only" in Table 4 drops the second scoring term altogether and does not compare S (Eq. 4) directly against LAYS's aerial-only pose plausibility score. Retraining with Fourier voting and LAYS's plausibility score reaches 25.04 yaw recall within 1° (R@1°, %; median 2.21°) on VIGOR Same Area, against 55.01 (0.88°) with S. The full 2×2 with hard bins, at fixed backbone, objective and support (R@1° / median, Same Area):
Separating the method from the backbone. We retrained S-RAY with LAYS's trained VGG16 U-Net in place of frozen DINOv2-B (±180° prior, R@1° % / median yaw error °):
With the same VGG16 U-Net backbone, S-RAY is ahead of LAYS on KITTI and MGL and of VGG hard-bin voting on VIGOR, by 29.32, 14.08, 11.76, and 9.60 points in the four rows. On KITTI Same Area, the step from published LAYS (51.07) to S-RAY (87.65) reflects both the method and the backbone: in our setup, DINOv2-B hard bins reach 78.69 with S and 79.46 with the plausibility score (LAYS: 51.07), and DINOv2-B adds 7.26 points to S-RAY (80.39 to 87.65), which leaves hard-bin voting nearly 9 points behind.
The additional loss of Sec. 3.5 (retrained, VIGOR Same Area).
The revised paper reports the 2×2, the backbone results and both loss ablations in Sec. 4.3, with full tables in Appendix E.