- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Proposes DLS, a dense local attention operator for spherical images. Combines geometry-aware positional encoding with row-column KV aggregation to avoid irregular sparse sampling. Achieves approximately translation-consistent features with improved accuracy and inference speed across multiple spherical perception tasks.
Critical strengths
-
Novel and well-motivated formulation Geometry-aware dense attention avoids irregular sparse sampling while maintaining approximate translation consistency
-
Strong accuracy and efficiency. Improves segmentation mIoU by 2.9 points with 4.9× speedup. Consistent improvements across multiple tasks (Tables 1–2).
Critical weaknesses
-
DLS requires substantially more GPU memory despite lower latency (Table 2). May limit practical deployment at high resolutions.
-
Translation consistency relies on a first-order approximation, with increasing errors near the poles and for larger receptive fields (Sec. 3.3). Robustness under these conditions needs further validation.
Questions for authors
- Can the memory overhead be reduced without sacrificing inference speed?
- How does performance change near the poles or with larger receptive fields?
- How much of the speedup comes from the dense formulation versus the optimized Triton implementation?
Thank you for the practical questions about memory, polar behavior, and implementation efficiency. We agree that practical efficiency should account for peak memory as well as latency, and your suggestions helped us improve the implementation and clarify its behavior.
Reducing memory overhead
Our original inference implementation prioritized latency. We now provide two optimized variants: latency-optimized (L) and latency- and peak-memory-optimized (LM). LM shares identical read-only buffers, reschedules independent operations, and processes attention heads in groups to reduce peak memory usage.
For the trained 256×512 segmentation model, buffer sharing and rescheduling alone reduce peak allocated memory to approximately 0.93 GB with latency essentially unchanged. Grouping attention heads further reduces memory to 0.472 GB, about 55% below the latency-optimized variant's 1.049 GB. Latency changes from 13.11 to 13.20 ms, an increase of only 0.09 ms (0.7%). The L and LM variants produce bit-identical outputs on all 40 validation panoramas for this trained model, preserving the evaluated accuracy.
Across the four tested resolutions, the final LM variant reduces peak memory by approximately 41-56%, with measured latency changes between -1.0% and +3.3%. Tables 2 and F now report both variants, and Appendices C and E describe the implementation and measurements. These results directly address reducing memory overhead with little latency cost and without changing the model.
Performance at high latitudes
Stanford2D3D has limited valid coverage near the poles, so we additionally evaluated DLS-SegFormer, S2-SegFormer, and SphereUFormer on the full-latitude Structured3D dataset. Figure C in Appendix D reports segmentation accuracy in 15∘ latitude bands over 1,687 test images. Section 4.4 now summarizes these findings.
DLS exceeds S2-SegFormer in every plotted band, while SphereUFormer performs better in several high-latitude bands. The overall mIoU values are 53.7, 53.2, and 51.7 for DLS, S2, and SphereUFormer, respectively. The continuous S2 attention formulation respects spherical geometry, but this property alone does not guarantee higher segmentation accuracy at high latitudes. We hypothesize that latitude-dependent responses can provide useful cues to latitude-specific scene statistics. Their usefulness may depend on the dataset and task, and the present comparison does not isolate this mechanism.
Larger angular supports
The default angular-support schedules closely match those of the baseline, as reported in Section 4.1. In preliminary segmentation experiments at 256×512, doubling the angular supports of all layers reduced mIoU by 0.89 percentage points for DLS and 0.84 percentage points for S2. We therefore retain the original, closely matched angular-support schedules for the main experiments. We report this sensitivity check and its numerical results in Section 4.4 and Appendix D.
The similar decreases indicate comparable sensitivity to this support enlargement. They do not isolate curvature error from dataset- and task-dependent effects of enlarging the receptive field.
Contribution of implementation optimization to the speedup
Appendix E reports both PyTorch-reference and optimized timings to examine the computational advantage under both implementation settings. Table F gives the following default-model SegFormer latencies at 256×512:
Relative to their respective reference implementations, optimization accelerates S2 by approximately 31.6× and DLS by 14.5× (L) or 14.4× (LM). Both algorithms benefit substantially from the implementation improvements, although the relative speedup is larger for S2.
For the spherical Transformer, we additionally applied pointwise/GELU and resampling fusion using Triton to the already optimized S2 implementation. At 256×512, latency decreases from 11.2 ms to 10.9 ms (Table G). This modest additional gain shows that the original CUDA implementation already captures most of the measured speedup over its PyTorch reference.