- Rating
- 1
- Confidence
- 4(+1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper addresses monocular 3D reconstruction for central cameras, aiming to handle pinhole, fisheye, catadioptric and panoramic images without intrinsics or a camera-type label. The authors argue that a single fixed spherical-harmonic basis, as used in UniK3D, is a poor fit for all cameras, and split the ray field into three layers. A 16-knot monotone radial profile maps image radius to incidence angle, a degree-7 spherical-harmonic residual is added on top, and a convolutional decoder supplies a per-pixel free-form residual. Both residuals are applied as rotations on the sphere. A DINOv2 and DPT distance branch predicts per-pixel radial distance, which is multiplied with the ray to obtain the point cloud. The experiments start with an oracle study that fits each representation directly to ground-truth rays. The model is then trained on 2D3DS and 360Loc and compared with UniK3D, DA3, GeoCalib and AnyCalib on seen cameras, on held-out catadioptric and GoPro cameras, and on the PanoCity and CO3Dv2 datasets, followed by an ablation of the ray representation.
Critical strengths
The layered ray representation is a reasonable idea. The oracle study in Table 1 shows that a radial model cannot express equirectangular projection, that spherical harmonics lack precision on fisheye, and that the composition fits all three geometries to within 0.25 degrees. This analysis is clear and is the most valuable part of the paper. On strongly distorted catadioptric and panoramic images, the numerical advantage over existing baselines is also large.
Critical weaknesses
My main concern is that the comparisons are not fair enough to support the central claims. OmniRay is trained on synthetic lenses rendered from panoramas, and the random-lens augmentation already covers annular image domains. The held-out cameras are kept unseen only by a rejection rule that is not described in the main text. Among the baselines, only UniK3D is fine-tuned on the same data; DA3 is not, and GeoCalib and AnyCalib only perform calibration. The gap on seen cameras, 0.14 degrees against 6.34 degrees, is essentially an in-distribution versus out-of-distribution comparison. All baselines exceed 75 degrees of error on the catadioptric camera, which tells me they have simply never seen such inputs, so this result says little about the representation itself. Recent camera-aware methods such as DAC, UniDAC, MoGe-2, Wid3R and CAM3R are also missing. The claim of a new state of the art by a large margin is therefore not yet supported.
Second, the method is not better on ordinary images, and the ablations are thin. On CO3Dv2 pinhole images, OmniRay obtains 2.97 degrees ray error and 0.038 Chamfer distance, clearly worse than DA3 at 2.04 degrees and 0.028. This contradicts the statement that flexibility for wide-angle cameras does not come at the expense of pinhole imagery. Training uses only two panorama datasets, so scene coverage is limited. The ablation in Table 3 is run on a small held-out subset with coarsely rounded numbers and no variance, and adding the free-form residual makes GoPro worse, from 5.00 to 5.50 degrees. In Table 1 the composite is also worse than the radial base alone on fisheye and catadioptric cameras, 0.075 against 0.002 and 0.174 against 0.023. The paper does not explain when the composition helps and when it hurts.
Questions for authors
- What exactly is the rejection rule that keeps the catadioptric and GoPro cameras out of the random-lens augmentation, and how close are the nearest accepted training lenses to these test cameras?
- What happens if UniK3D is trained with the same random-lens augmentation? This experiment is essential to tell whether the gain comes from the representation or from the training data.
- Why is the composite worse than the radial base on fisheye and catadioptric in Table 1, and why does the free-form residual hurt GoPro in Table 3? Does the spherical-harmonic residual interfere with a radial base that should already suffice?
- Could you add depth metrics such as AbsRel and δ1 to the main table, and compare with DAC, UniDAC and MoGe-2 on public benchmarks such as ScanNet++ fisheye and KITTI-360?
Thank you for the additional experiments and explanations. The comparisons using the same augmentation and the ablations on the full test set have eased my concerns about fairness and whether the individual components help. However, after reading the revised results, I still have reservations about the significance of the contribution and lean toward rejection.The paper combines a radial model, spherical harmonics, and a per-pixel residual into a layered ray prediction module. This design has some novelty, and the additional experiments support its effectiveness. My concern is not that using existing components makes a method unoriginal. Rather, I am not yet convinced that the improved representation brings a sufficiently meaningful gain in 3D reconstruction. The ablations show that the components complement each other, but they do not fully establish how much this matters for practical reconstruction.
The largest gains still come from synthetic-camera tests, particularly the catadioptric camera. The new real-data experiments provide some support, but the reconstruction evidence remains limited. ScanNet++ evaluates ray error only. After both methods are fine-tuned on ADT, CD decreases from UniK3D’s 0.0569 to 0.0552, a relatively small improvement. The CO3Dv2 pinhole results also show no clear reconstruction advantage. I acknowledge that PanoCity provides an outdoor street-scene evaluation. Still, the evidence for better reconstruction under real imaging conditions is less convincing than the evidence for better ray estimation on synthetic cameras. My remaining concern is therefore about the scope and significance of the demonstrated gains, rather than the absence of a particular dataset or baseline.
I also have one question about the rejection rule. For GoPro, the rule ensures that a training lens differs from the test camera by more than 15° at at least one radius. It does not imply that the mean ray-angle difference exceeds 15°. The catadioptric rule instead constrains the inner and rim angles. Please clarify what distance measure “at least 15° away” refers to, and whether you measured the nearest distance between the actual training samples and each test configuration. I agree with calling these tests “held-out camera configurations,” but the description of their geometric separation should be precise.