acceptodds
Under review as a conference paper at ICLR 2027

PanoDPG: Decoupled Pose and Geometry Estimation for High-Resolution Panoramic 3D Reconstruction

Abstract

Feed-forward multi-view reconstruction has achieved remarkable progress in jointly recovering camera poses and dense geometry, yet scaling it to high-resolution panoramas remains prohibitively expensive. At the core of this inefficiency is a task-agnostic design that applies the same high-resolution representation and global interaction to both pose estimation and geometry reconstruction, despite their distinct computational and representational requirements. Pose estimation relies primarily on global context and is less sensitive to spatial resolution, whereas dense geometry reconstruction requires fine-grained visual details and is strongly affected by panoramic projection distortion. Based on this observation, we propose PanoDPG, a dual-branch framework that decouples pose estimation from dense geometry reconstruction in both resolution and projection representation. PanoDPG performs global pose reasoning on low-resolution equirectangular projections (ERP), while reconstructing detailed geometry from high-resolution cubemaps with reduced spherical distortion. To preserve the geometric dependencies between the two branches, we introduce a register-token-based interaction mechanism that efficiently transfers global information, aligns the scales of estimated poses and reconstructed geometry, and promotes multi-view consistency without full global attention over high-resolution features. Experiments on multi-datasets show that PanoDPG achieves reconstruction accuracy while reducing computational cost, demonstrating an effective and scalable solution for high-resolution panoramic reconstruction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
panoramic image, 3D reconstruction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_BzE6OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PanoDPG, a task-decoupled dual-branch framework for high-resolution panoramic 3D reconstruction. The key motivation is that pose estimation and geometry reconstruction have different requirements: pose estimation benefits more from global context and is robust to resolution, while geometry reconstruction requires high-resolution local details and lower projection distortion. PanoDPG therefore uses low-resolution ERP features for global pose estimation and high-resolution cubemap features for detailed geometry reconstruction, coupling the two branches via register-token communication. Experiments show competitive pose and point-cloud performance with better efficiency and scalability than recent feed-forward panoramic reconstruction methods.

Critical strengths

The paper has a very good motivation. It points out a real and important asymmetry: pose estimation does not need high-resolution input, while geometry reconstruction does. I agree with this problem formulation. The idea of decoupling pose and geometry into different resolution/representation branches is well aligned with this observation.

Critical weaknesses

The 2072 comparison is not fair. The paper says PanoVGGT was only retrained at 1036×518, so there is no 2072 training result for PanoVGGT. Listing its 2072 collapse in the table is therefore not meaningful. I think the 2072 setting has no comparability.

The framework design in this paper is not novel, and the use of cubemap operations is also fairly common. Some experimental results still need further explanation.

Questions for authors
  1. Since PanoVGGT was not trained at 2072×1036, is the 2072 comparison in Table 1 fair? Explain where this data comes from and why it collapses.
  2. At 1036×518, can you provide the ablation with only the cubemap branch + intra-register attention, with both pose and point heads from that branch? This would isolate the contribution of the ERP branch and task decoupling.
  3. Table 4 claims 2K efficiency, but the text says baselines were trained at 1036. Could you provide efficiency comparison at 1036×518, matching PanoVGGT's official 1024 setting?
  4. In Table 3, dual-branch without RBI drops pose AUC from 0.903 to 0.819, and RBI only recovers it to 0.903. Does this mean decoupling itself harms pose estimation and RBI merely compensates? Explain this phenomenon.

I initially leaned toward borderline accept, but that option isn't available, so I'll wait and see the authors' response.

1 comment

W1/Q1: Unfair comparison

Thank you for pointing this out. PanoVGGT* was fine-tuned at 1036×518 and performs poorly at the unseen 2072×1036 resolution. We have removed this unfair comparison.

Our previous 1K PanoDPG used 1036×518 source panoramas but retained six 686×686 cubemap faces, matching the 2K configuration’s processing resolution. It did not constitute a fully matched 1K configuration. We have therefore trained a matched 1K configuration with 1036×518 ERP inputs and six 350×350 cubemap faces for comparison at the baselines’ fine-tuning resolution. The updated results are reported in Tables 1 and 2 of the revised manuscript.

W2: Framework design novelty

Our contribution is a task-decoupled architecture tailored to panoramic reconstruction, with two key designs:

(1) Joint decoupling of resolution and representation. We combine a moderate-resolution ERP branch for global pose estimation with a high-resolution cubemap branch for detailed geometry recovery. This allocation follows our quantitative analysis of their different resolution requirements (Fig. 2; revised Suppl. Tabs. 8–11).

(2) Hierarchical aggregation with unified communication. Dense attention connects the six faces within each cubemap group, while compact register tokens support both ERP–cubemap feature exchange and communication across panoramas. This connects task-specific representations and propagates global information without dense attention across all faces and frames.

Q2: Cubemap branch + register token attention

Thank you for this suggestion.

[Experimental setup] We have added a cubemap-only ablation, retaining global attention within each six-face group and register-token attention across groups, with both pose and point heads attached to the cubemap branch. The variant is trained using the same protocol as PanoDPG.

FE and TE denote a frozen encoder and a trainable encoder, respectively.

[Experimental results]

ConfigurationAcc. ↓Comp. ↓AUC@5 ↑Rmean ↓Tmean ↓
Single-branch ERP (FE)0.9470.9690.9030.3981.037
Single-branch ERP (TE)0.1230.0960.9010.3671.057
Single-branch Cubemap + RBI (FE)0.2720.1790.14412.1720.46
Dual-branch (FE)0.1260.0890.8190.6551.612
Dual-branch + RBI (FE)0.1050.0760.9030.3801.035

[Experimental conclusion] As shown in the table, removing the ERP branch degrades both pose and point-cloud accuracy. This demonstrates the contribution of the ERP branch even when cross-group register communication is retained, supporting our design of combining ERP-based pose estimation with cubemap-based geometry recovery.

Q3: Efficiency evaluation

We have added runtime and GPU memory comparisons at 1036×518 (Tab. 15 in the suppl. and Fig. 6 in revision). All methods use ERP inputs at this resolution; PanoDPG additionally uses six 350×350 cubemap faces per panorama. Each entry reports inference time (s) / peak GPU memory (GiB).

PanoDPG generally ranks second in runtime for shorter sequences and becomes the fastest from 50 frames onward. At 200 frames, it takes 88.85 s, reducing runtime by 33.3% compared with PanoVGGT (133.26 s) as below.

Input framesPanoVGGTWid3RArgusPanoDPG (Ours)
30.27 / 4.200.51 / 6.740.38 / 6.720.43 / 5.38
304.66 / 8.608.12 / 34.496.73 / 12.535.04 / 11.64
5010.59 / 11.86Runtime error†15.29 / 14.859.92 / 16.29
10035.78 / 20.00Runtime error†51.96 / 25.1627.65 / 27.90
200133.26 / 36.29OOM193.56 / 45.7988.85 / 51.11

† Wid3R encounters a 32-bit indexing limit in its reflection-padding operator, distinct from OOM.

Q4: dual-branch without RBI

Thank you for highlighting this result. Without RBI, the branches exchange no features, but their predictions remain coupled through supervision: ERP translations and cubemap points share a scale alignment to ground truth and jointly contribute to the global point-cloud loss. They must therefore produce geometrically consistent predictions without direct feature coordination, which can explain the pose degradation.

RBI enables this coordination through feature exchange. We agree that decoupling without RBI reduces pose accuracy in this configuration. However, the complete model restores the single-branch pose AUC while substantially improving geometry, supporting the combined design of task decoupling and inter-branch communication.

Reviewer_Fb2POpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes PanoDPG, a dual-branch feed-forward model for multi-view panoramic 3D reconstruction. The goal is to get globally consistent poses and high-resolution, fine-grained geometry at an acceptable cost. A pose branch runs on downsampled panoramas (1036×518, equirectangular) with global attention across frames to estimate camera poses. A geometry branch runs on high-resolution cubemap faces (6×686×686 from a 2K panorama) with attention only inside each frame's cubemap group, and predicts local point maps. The two branches exchange information through register tokens instead of dense attention. On Holo360D, Matterport3D, Stanford2D3D and PanoCity, PanoDPG reports pose and point-cloud accuracy that is competitive with or better than PanoVGGT, Argus and Wid3R, while it can run with more frames and running faster at 2K input.

Critical strengths
  1. Well-motivated, practical design. Splitting the task into a low-resolution global pose branch and a high-resolution local geometry branch is a sensible way to keep global consistency while handling high-resolution panoramas at a reasonable cost. It lets the model train with more input frames and run much faster than global-attention models at 2K.
  2. Competitive results with fine-tuned baselines. PanoDPG is best on Holo360D for both pose and point clouds and competitive on the other three datasets, and the baselines are also fine-tuned on the same training splits.
Critical weaknesses
  1. Limited novelty: the core design and motivation were already proposed in DAGE [1] (Ngo et al., CVPR 2026, which is not cited). DAGE uses the same dual-stream idea for perspective images up to 2K: a low-resolution stream with alternating frame/global attention that estimates camera poses and global context, and a high-resolution per-frame stream for fine geometry. It has the same motivation (poses do not need fine-grained features, while geometry does) and the same efficiency argument (global attention only at low resolution). PanoDPG mainly differs in the task (panoramas instead of perspective images, using ERP for pose and cubemaps for geometry) and in how the two streams are synchronized (register tokens instead of a cross-attention adapter). There is no comparison with a DAGE-style synchronization, so the benefit of the proposed register-token design is not shown. The observation that ERP suits pose and perspective views suit geometry was also already reported by Holo360D (Ou et al., 2026).

  2. The main claim (high-resolution, fine-grained, temporally consistent geometry) is not evaluated.

    • There is no depth evaluation at all (no AbsRel/δ, which PanoVGGT reports), no boundary-sharpness metric, and no temporal-consistency evaluation on panoramic videos, even though Holo360D provides continuous trajectories. Even with 2072×1036 input, all metrics are computed after projecting the predictions onto a 1036×518 grid (l379–381; the resolutions in Tab. 1–2 are input resolutions), which cannot reward high-resolution detail, and only ≤5 frames are used. Qualitative results show only fused point clouds, without depth maps or analysis of seams between cube faces.
    • The paper's own numbers do not show a clear benefit of high-resolution input for geometry. It is expected that 2072×1036 input does not improve pose over 1036×518 (Tab. 1), since the pose branch always uses the downsampled 1036×518 panorama. But point-cloud reconstruction is exactly where the high-resolution cubemap branch should help, and the gains in Tab. 2 are small and mixed: Holo360D improves (Acc/Comp 0.117/0.097 → 0.105/0.076), Stanford2D3D changes little (0.035/0.033 → 0.034/0.028), and accuracy gets worse on Matterport3D (0.031 → 0.032) and PanoCity (0.112 → 0.143), with only completeness improving.
    • The key observation in Fig. 2 (geometry improves with resolution, pose peaks at 1036×518) has no stated model, dataset, number of frames or protocol, and never tests 2072×1036. If the model was trained at 1036×518 and tested at other widths, the pose peak may simply reflect the training resolution.
    • The ablation baseline is not credible: in Tab. 3, the single-branch ERP variant gets Acc/Comp 0.947/0.969 on Holo360D, far worse than every single-branch ERP baseline on the same data (e.g., PanoVGGT* 0.115/0.084, Tab. 2). Against PanoVGGT*, the gain of PanoDPG is small (0.105/0.076 at 2K), and at 1036×518 input PanoDPG (0.117/0.097) is slightly worse.
  3. The evaluation is not reliable or comparable with the baselines.

    • Near-saturated metric: pose is reported only as AUC@5, which is ≥0.94 for most methods on Matterport3D, Stanford2D3D and PanoCity, instead of the AUC@30 used in all baseline papers.
    • In-domain only: all four evaluation datasets are also training datasets. Unlike the baseline papers (PanoVGGT on Pano3D-GibsonV2, Argus on Matterport3D/Stanford2D3D, Wid3R on Stanford2D3D), there is no zero-shot evaluation.
    • Baseline settings: baselines are fine-tuned at 1036×518 (Argus was designed for 560×196, Wid3R for 518×336) and fine-tuning sometimes makes them worse than their public checkpoints. I agree training baselines at 1036×518 for cost reasons, but then the evaluation should compare each method at its intended operating point. Instead, the "PanoVGGT* (2072×1036)" row, which is out of distribution and collapses, is used to claim that PanoDPG is "less sensitive to input resolution" (line 387–392).
    • Efficiency comparison: runtime and memory (Fig. 5, Tab. 4) are only reported at 2K. They should be compared at both resolutions (1036×518 and 2072×1036).
    • Unclear setup: the main text and appendix disagree on key details: the encoder (frozen π³ in Sec. 3.2 vs. VGGT in App. A.1), the training data (Holo360D vs. CO3D), the input to the cubemap branch (2K vs. 1036×518 ERP), and the block layout. The training resolution, optimizer, learning rate and iterations are not stated.

[1] Tuan Duc Ngo, Jiahui Huang, Seoung Wug Oh, Kevin Blackburn-Matzen, Evangelos Kalogerakis, Chuang Gan, Joon-Young Lee. DAGE: Dual-Stream Architecture for Efficient and Fine-Grained Geometry Estimation. CVPR, 2026.

Questions for authors

Overall: The paper proposes a well-motivated dual-stream design for globally consistent, high-resolution panoramic reconstruction at an acceptable cost. However, the same design and motivation already appear in DAGE (CVPR 2026), which is not cited, and this lowers the contribution. The experiments also have many problems: the main high-resolution, fine-detail, temporally consistent depth claim is not evaluated, the pose metric is near-saturated, all evaluation sets are in-domain, and the comparison with baselines is not fully fair. Thus, I lean toward rejecting this paper.

Question

  1. Please discuss DAGE [1] (Ngo et al., CVPR 2026) and compare against a DAGE-style synchronization to show the benefit of register tokens.
  2. Please evaluate the main claim directly: depth at native 2K resolution against baselines (run at their native resolution, with outputs upsampled), a boundary-sharpness metric, and temporal consistency over long panoramic videos (e.g., Holo360D trajectories), with depthmap visualizations.
  3. Please report AUC@30 and longer sequences (e.g., 10+ frames).
  4. Please report zero-shot results on held-out panoramic data (e.g., Pano3D-GibsonV2 as in PanoVGGT, or Realsee3D as in Argus).
  5. For Fig. 2: which model and dataset are used, and is the model retrained at each resolution or trained once at 1036×518? Please include 2072×1036.
  6. For Tab. 3: how was the single-branch ERP variant trained and evaluated, and why does its point cloud fail (0.947/0.969)?
  7. Please state the training resolution and fix the main-text/appendix inconsistencies (encoder, training data, cubemap input, block layout), report the baseline fine-tuning recipe, and compare runtime/memory with baselines at both 1036×518 and 2072×1036.
8 comments

W1/Q1: Limited novelty

We thank the reviewer for pointing us to DAGE. We agree that DAGE is an important baseline for our work. Although both DAGE and PanoDPG use dual branches to address the different resolution needs of pose and geometry estimation, PanoDPG is motivated by a distinct finding on panoramic resolution sensitivity, which guides its task-specific resolution allocation. Below, we clarify the differences in key insight, architecture, and interaction mechanisms.

1. Empirical analysis and key insight

DAGE reports limited pose accuracy gains from increasing the LR-stream resolution at a substantial efficiency cost (Tab. 7 in the DAGE). Accordingly, DAGE adopts a relatively low resolution (252 pixels) for its pose branch to maintain high reconstruction efficiency.

However, we find that in panoramic settings, overly low resolutions impair pose accuracy, while overly high resolutions do not provide sustained benefits.

[Experimental setup] We train a single PanoVGGT model and a single PanoDPG model using variable-resolution inputs ranging from 504×252 to 1400×700 and from 504×252 to 2072×1036, respectively, both under PanoDPG’s training data with fixed 5-inframe input. For each model, we evaluate the checkpoint across all training resolutions on Holo360D and Stanford2D3D with fixed 5-inframe input. Full results are provided in Tables 8–11 of the revised supplementary material. Below, we present selected PanoDPG results on Holo360D and Stanford2D3D.

[Experimental results]

ERP ResolutionHolo360DStanford2D3D
AUC@5 ↑Rmean (°) ↓Tmean (°) ↓AUC@5 ↑Rmean (°) ↓Tmean (°) ↓
504×2520.73231.39732.21310.77635.44772.9349
1036×5180.87650.52091.19870.92523.30150.8181
1120×5600.87690.51101.19650.94050.66680.8739
1400×7000.85620.91451.44490.90922.60931.6076
2072×10360.71901.02942.20010.80994.63822.0767

[Experimental Analysis] Across the evaluated models and datasets, we observe a consistent overall trend: pose accuracy initially improves as resolution increases, but declines at higher resolutions. For PanoDPG on Holo360D, AUC@5 rises from 0.7323 at 504×252 to 0.8765 at 1036×518, then drops to 0.8562 at 1400×700 and 0.7190 at 2072×1036.

[Experimental conclusion] To our knowledge, we provide the first systematic, quantitative analysis of this non-monotonic pose response within the training resolution range of feed-forward panoramic reconstruction models.

Validated across models and datasets, this finding goes beyond DAGE’s reported limited gains from increasing pose-stream resolution and provides an empirical basis for task-specific resolution allocation in panoramic reconstruction. It motivates our choice of 1036×518 ERP inputs for pose estimation, while allocating higher-resolution cubemap inputs to geometry reconstruction.

2. Architectural design

① PanoDPG jointly decouples resolution and input representation according to task requirements, whereas DAGE decouples tasks only by resolution.

② Hierarchical attention for cubemaps. DAGE uses standard within-frame self-attention in its high-resolution branch. PanoDPG combines global attention among the six faces within each cubemap group with compact register-token attention across groups, promoting geometric consistency within and across panoramas with limited computational overhead.

We acknowledge Holo360D’s prior observations; our contribution is a unified, task-decoupled network with compact and effective communication.

3. Interaction mechanisms

PanoDPG and DAGE have different communication requirements.

DAGE’s per-frame adapter addresses only the resolution gap within the same perspective projection.

PanoDPG additionally requires cross-representation exchange between ERP and cubemaps and global communication across cubemap groups.

Our register tokens aggregate spatial features into compact global summaries that are not tied to fixed patch locations. This helps bridge the representational gap between ERP and cubemap inputs, facilitating information exchange and coordinated pose and geometry prediction.

[Experimental setup] Following the reviewers’ suggestions, we compare PanoDPG with our register token based interaction with retrained variants using FlashVGGT-style attention (Reviewer Qyui) and a DAGE-style adapter (Reviewer Fb2P) in the table below.

Parenthesized changes are relative to the DAGE-style adapter. AUC changes are reported in percentage points (pp), while error changes are relative percentages.

[Experimental results]

DatasetsInteraction designR_mean(°)↓T_mean(°)↓AUC@5 ↑AUC@30 ↑Acc. ↓Comp. ↓
MP3DFlashVGGT-style attention0.6300.8640.9080.9840.0360.024
MP3DDAGE-style adapter0.4980.5100.9640.9940.0360.025
MP3DRegister tokens (Ours)0.465(↓6.6%)0.481(↓5.7%)0.969(↑0.5 pp)0.995(↑0.1 pp)0.032(↓11.1%)0.021(↓16.0%)
Holo360DFlashVGGT-style attention0.9531.8540.8150.9550.1080.071
Holo360DDAGE-style adapter0.9011.2920.8930.9750.1060.072
Holo360DRegister tokens (Ours)0.380(↓57.8%)1.035(↓19.9%)0.903(↑1.0 pp)0.980(↑0.5 pp)0.105(↓0.9%)0.076(↑5.6%)

[Experimental conclusion] Our register-token design achieves better pose estimation and point-cloud accuracy on both datasets, due to the register tokens’ effective global information aggregation, which enables compact communication across representations and cubemap groups.

W2-1/Q2: depth evaluation

Thank you for the suggestion.

[Experimental setup] We have added depth accuracy, boundary sharpness, and cross-view depth consistency evaluations across all datasets (Holo360D, Matterport3D, Stanford2D3D, Panocity, and Realsee3D) on a 2072×1036 ERP. We evaluate depth accuracy using AbsRel and δ₁, depth-boundary sharpness using Boundary F1 and dbe_chamfer, and cross-view consistency using consistency error.

[Experimental results] For brevity, we show Stanford2D3D results here; full results are provided in Table 12 of the revised supplementary material.

MethodAbsRel ↓δ₁ ↑Boundary F1 ↑dbe_chamfer ↓Consistency error ↓
Argus*0.040797.22%0.045615.6333.37%
Wid3R0.151685.96%0.018732.8336.44%
Wid3R*0.035297.64%0.041416.9953.41%
PanoVGGT0.036397.30%0.045015.8812.96%
PanoVGGT*0.035297.62%0.048016.4202.88%
PanoDPG (1K)0.034997.63%0.052013.9323.95%
PanoDPG (2K)0.033797.66%0.062812.9973.37%

* denotes fine-tuned baselines. Bold indicates the best result.

Depth-map comparisons in Fig. 7 of the revised supplementary material show finer details and sharper boundaries.

[Experimental conclusion] PanoDPG (2K) achieves the best depth accuracy and boundary quality, while improving cross-view consistency over its 1K counterpart.

We also quantify cubemap overlap consistency on Matterport3D. Mean relative depth and 3D discrepancies are both below 0.7%, indicating strong geometric consistency between adjacent faces.

MetricRelative depth discrepancyRelative 3D discrepancy
Mean error ↓0.586%0.662%
Proportion with error <1% ↑88.96%88.41%
Proportion with error <5% ↑99.24%99.24%

W2-2: Benefit of high-resolution input

We apologize for the unclear description. The original 1K PanoDPG used 1036×518 source panoramas but retained six 686×686 cubemap faces, matching the geometry branch resolution of the 2K configuration. We have removed this unfair comparison.

[Experimental setup] Instead, we train and evaluate a matched 1K PanoDPG configuration (six 350×350 faces from 1036×518 panoramas) alongside the 2K configuration (six 686×686 faces from 2072×1036 panoramas).

[Experimental results]

For brevity, we show Stanford2D3D results here; full results are provided in Table 12 of the revised supplementary material.

MethodAbsRel ↓δ₁ ↑Boundary F1 ↑dbe_chamfer ↓Consistency error ↓
PanoDPG (1K)0.034997.63%0.052013.9323.95%
PanoDPG (2K)0.033797.66%0.062812.9973.37%

[Experimental conclusion] Across all five datasets, the 2K configuration improves depth accuracy, boundary sharpness, and cross-view consistency.

We acknowledge the mixed gains in global point-cloud metrics, which also depend on pose and alignment. The added depth and boundary evaluations more directly assess the geometric accuracy and detail recovered by the high-resolution branch.

W2-3/Q5: For Fig. 2

The original Fig. 2 uses the PanoVGGT trained under PanoDPG’s training data at seven resolutions ranging from 504×252 to 1400×700, with fixed five-frame inputs for both training and evaluation. The checkpoint is evaluated on Holo360D at all seven resolutions. Thus, the observed peak cannot be attributed to training exclusively at 1036×518.

We further extend the analysis across models, datasets, and resolutions. We train a single *PanoDPG model with eleven input resolutions for the ERP branch, ranging from 504×252 to 2072×1036. Evaluating the checkpoint on Holo360D and Stanford2D3D reproduces the non-monotonic pose trend (Tables 8–11 of the revised supplementary material). These results extend the observation beyond a single model or dataset and cover the requested 2072×1036 setting.

W2-4/Q6: The ablation baseline

① ERP baseline. Like PanoDPG, the single-branch ERP baseline uses a frozen encoder, as encoder fine-tuning had limited impact on pose accuracy in our experiments. We have clarified this setting and added an unfrozen ERP baseline for comparison. As shown below, unfreezing substantially improves point-cloud accuracy, while pose performance remains comparable.

ConfigurationAcc. ↓Comp. ↓AUC@5 ↑R_mean ↓T_mean ↓
Single-branch ERP (frozen encoder)0.9470.9690.9030.3981.037
Single-branch ERP (trainable encoder)0.1230.0960.9010.3671.057

② Gains over PanoVGGT*. PanoDPG achieves better depth accuracy and sharper depth boundaries than PanoVGGT*. Boundary F1 increases from 0.0480 to 0.0628 on Stanford2D3D, showing benefits in geometric detail beyond the global point-cloud metrics. For brevity, we show Stanford2D3D results here; full results are provided in Table 12 of the revised supplementary material.

MethodAbsRel ↓δ₁ ↑Boundary F1 ↑dbe_chamfer ↓
PanoVGGT0.036397.30%0.045015.881
PanoVGGT*0.035297.62%0.048016.420
PanoDPG (1K)0.034997.63%0.052013.932
PanoDPG (2K)0.033797.66%0.062812.997

③Pole-region depth. We additionally evaluate depth in 10° regions around both ERP poles, where spherical distortion is severe. PanoDPG achieves lower depth error than all evaluated fine-tuned baselines on Holo360D, PanoCity, and Realsee3D. Full results are provided in Table 13 of the revised supplementary material. Matterport3D and Stanford2D3D are excluded from this evaluation because their ERP images lack valid polar-region data.

Depth evaluation in the polar regions on Holo360D:

MethodAbsRel ↓RMSE ↓δ₁ ↑
Argus*0.02830.246398.21%
Wid3R*0.03110.223598.57%
PanoVGGT*0.02010.201598.89%
PanoDPG (1K)0.01470.172499.00%
PanoDPG (2K)0.01330.145699.15%
Reviewer_QyuiOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

PanoDPG addresses an existing problem: estimating camera poses and reconstructing dense 3D geometry from multiple pano images, which is an important problem in robotics or VR.

High-res pano images help both pose (limited gains) and geometry, but are computationally expensive. Infact, it hurts pose accuracy if you keep on increasing the resolution. Following this observation, PanoDPG proposes to use low-res ERP for pose and high-res cube views for geometry. The two branches communicate via register tokens (compact learned representations for each cubemap group) instead of full cross-attention between their high-resolution cubemap and low-resolution ERP patch tokens. This helps to reduce memory and computation.

The paper reports competitive or state-of-the-art results with reduced computational and memory costs.

Critical strengths
  1. This is an important problem, particularly for panoramic images, which are high-resolution. Interestingly, even when computational cost is not a concern, directly using higher-resolution images can actually degrade pose estimation accuracy. While this observation has also been discussed in Halo360D, demonstrating how to effectively exploit high-resolution information without hurting pose accuracy and less computation is a novel and practically valuable contribution.

  2. The formulation based on dual branch makes sense and the results support the overall architecture design including register for information exchange between the two branches.

[1.] Holo360D: A Large-Scale Real-World Dataset with Continuous Trajectories for Advancing Panoramic 3D Reconstruction and Beyond (https://arxiv.org/abs/2604.22482)

Critical weaknesses
  1. The empirical finding that the pose accuracy decreases as image resolution increases is not explained properly. Moreover, this finding is already supported by existing works (Halo360D), the paper primarily presents empirical results without investigating/explaining the underlying cause. Providing further analysis or insights into why this occurs, such as whether the degradation is more pronounced in specific scenarios (e.g., highly textured regions, repetitive patterns, or other challenging visual conditions), would help better understand it and strengthen the contribution.

  2. Alternatives to register tokens are not discussed. It would be useful to compare other efficient methods for cross-branch attention, such as used in FlashVGGT [1] or Splatter-360 [2] etc (although not the same tasks), to better justify the choice of register tokens.

[1.] FlashVGGT: Efficient and Scalable Visual Geometry Transformers with Compressed Descriptor Attention (https://arxiv.org/abs/2512.01540)

[2.] Splatter-360: Generalizable 360◦ Gaussian Splatting for Wide-baseline Panoramic Images (https://arxiv.org/pdf/2412.06250)

Questions for authors

LIne 407: PanoVGGT∗ (2072×1036) ==> Why does PanoVGGT perform so poorly at this resolution? Could this be due to issues with the training setup or implementation?

Since some of the reported results are very close, it would be helpful to assess their statistical significance by running multiple times (maybe 3 times ?)?

Line 047: panoramicreconstruction  ==> typo

2 comments

W1: Discussion on pose-resolution sensitivity

We further investigate this phenomenon through four experiments:

1. Across models and datasets

[Experimental setup] We initially evaluated retrained variable-resolution PanoVGGT at varying resolutions only on Holo360D. To assess generalization across methods and datasets, we additionally train a single PanoDPG model at eleven ERP resolutions, from 504×252 to 2072×1036, and evaluate the checkpoint on Holo360D and Stanford2D3D.

[Experimental results]

Full results are provided in Tables 8–11 of the revised supplementary material. Below, we present selected PanoDPG results on Holo360D, with the cubemap face resolution fixed at 686×686.

ERP ResolutionAUC@5 ↑Rmean (°) ↓Tmean (°) ↓Mean angular error (°) ↓
504×2520.73231.39732.213139.436
1036×5180.87650.52091.198737.802
1120×5600.87690.51101.196537.988
1400×7000.85620.91451.444938.495
2072×10360.71901.02942.200140.383

[Experimental conclusion] The non-monotonic pose trend persists across models and datasets, supporting its generality in panoramic pose estimation.

2. Scene dependence

[Experimental setup] We compare PanoVGGT’s pose degradation trends as resolution increases from 1036×518 to 1400×700 across scenes with high texture, low texture, repetitive patterns, and low illumination.

[Experimental results]

ConditionAUC@5 ↑Change (pp)
High texture0.9416 → 0.9434+0.18
Low texture0.9191 → 0.9173−0.18
Repetitive patterns0.9402 → 0.9369−0.33
Low illumination0.9093 → 0.9014−0.79

[Experimental conclusion] Increasing resolution improves pose accuracy in highly textured scenes but degrades it under low texture, repetitive patterns, and low illumination. This suggests that pose degradation is more pronounced when reliable visual cues are limited, particularly in low-light scenes.

3. Image detail

[Experimental setup] To assess the effect of image detail on pose estimation, We use the same retrained PanoDPG checkpoint on Holo360D. We compare native 2072×1036 images with images upsampled from 1036×518 to 2072×1036. Upsampling increases resolution but does not restore missing image details. This comparison keeps the input size fixed to examine the contribution of additional fine-scale detail. Results are reported as upsampled / native-resolution inputs.

[Experimental results]

AUC@5 (%) ↑AUC@30 (%) ↑R_mean (°) ↓T_mean (°) ↓
PanoDPG76.61 / 78.0294.07 / 94.811.360 / 1.1771.974 / 1.759

[Experimental conclusion] Native high-resolution inputs achieve better pose accuracy than upsampled inputs, suggesting that additional fine-scale detail can benefit pose estimation and is unlikely to be the main cause of the observed degradation at higher resolutions.

4. Global attention correspondences

[Experimental setup] Pose estimation may be closely related to cross-view correspondences captured by global attention. We therefore evaluate cross-view correspondence quality using ** global attention** of PanoDPG on Holo360D across ERP resolutions from 504×252 to 2072×1036. Using the same checkpoint and query locations, we report the mean correspondence angular error alongside pose AUC@5.

[Experimental results]

ERP ResolutionAUC@5 ↑Mean correspondence angular error (°) ↓
504×2520.732339.436
1036×5180.876537.802
1120×5600.876937.988
1400×7000.856238.495
2072×10360.719040.383

[Experimental conclusion]

The correspondence accuracy follows the same non-monotonic trend as pose accuracy, suggesting that pose degradation is associated with less reliable cross-view correspondence.

[Final conclusion]

The trend persists across the tested models and datasets (Experiment 1). Experiments 2–3 highlight the importance of sufficient, distinctive visual cues, while Experiment 4 associates pose accuracy with the quality of cross-view correspondences captured by global attention. Together, these findings motivate the following explanation.

We infer that reliable cross-view matching requires enough clear and distinctive visual cues. At low resolutions, insufficient image detail limits reliable matching. Increasing resolution initially provides clearer cues, improving correspondences and pose accuracy. At excessively high resolutions, however, fixed-size patches cover smaller angular regions and may capture insufficient distinctive geometric or semantic information, making cross-view matching less reliable and reducing pose accuracy.

W2:Alternatives to register tokens

FlashVGGT uses full-resolution queries for global attention over bilinearly downsampled patch descriptors.

Splatter-360 applies cross-view attention within each branch and fuses cubemap features reprojected onto the ERP with spherical geometry.

DAGE [1] injects low-resolution features into same-frame high-resolution tokens through cross-attention, followed by high-resolution self-attention.

Our register tokens aggregate spatial features into compact global summaries that are not tied to fixed patch locations. This helps bridge the representational gap between ERP and cubemap inputs, facilitating information exchange and coordinated pose and geometry prediction.

[Experimental setup] Following the reviewers’ suggestions, we compare PanoDPG with our register token based interaction with retrained variants using FlashVGGT-style attention (Reviewer Qyui) and a DAGE-style adapter (Reviewer Fb2P) in the table below.

Parenthesized changes are relative to the DAGE-style adapter. AUC changes are reported in percentage points (pp), while error changes are relative percentages.

[Experimental results]

DatasetsInteraction designR_mean(°)↓T_mean(°)↓AUC@5 ↑AUC@30 ↑Acc. ↓Comp. ↓
MP3DFlashVGGT-style attention0.6300.8640.9080.9840.0360.024
MP3DDAGE-style adapter0.4980.5100.9640.9940.0360.025
MP3DRegister tokens (Ours)0.465(↓6.6%)0.481(↓5.7%)0.969(↑0.5 pp)0.995(↑0.1 pp)0.032(↓11.1%)0.021(↓16.0%)
Holo360DFlashVGGT-style attention0.9531.8540.8150.9550.1080.071
Holo360DDAGE-style adapter0.9011.2920.8930.9750.1060.072
Holo360DRegister tokens (Ours)0.380(↓57.8%)1.035(↓19.9%)0.903(↑1.0 pp)0.980(↑0.5 pp)0.105(↓0.9%)0.076(↑5.6%)

[Experimental conclusion] Our register-token design achieves better pose estimation and point-cloud accuracy on both datasets, due to the register tokens’ effective global information aggregation, which enables compact communication across representations and cubemap groups.

Q1: PanoVGGT∗ (2072×1036)

PanoVGGT* performs poorly at 2072×1036 because it was trained at 1036×518 and generalizes poorly to this unseen resolution. We have removed this unfair comparison.

Our previous 1K PanoDPG used 1036×518 source panoramas but retained six 686×686 cubemap faces, matching the 2K configuration’s processing resolution. It did not constitute a fully matched 1K configuration. We have therefore trained a matched 1K configuration with 1036×518 ERP inputs and six 350×350 cubemap faces for comparison at the baselines’ fine-tuning resolution. The updated results are reported in **Tables 1 and 2 of the revised manuscript.

Q2: Close results

We report the mean and standard deviation over three independent retraining runs of PanoDPG to quantify training variability (Tab. 18 in the revised supplementary material). While some gains over existing methods remain modest, additional evaluations demonstrate PanoDPG’s practical benefits:

  • ① Depth accuracy and depth-boundary quality (Tab. 12 in the revised supplementary material).
  • ② Depth quality in the two ERP polar regions (Tab. 13 in the revised supplementary material).
  • ③ Performance on 10-frame Holo360D sequences (Tab. 5 in the revised main paper).
  • ④ Depth-map visualizations showing sharper boundaries and clearer structural details (Fig. 7 in the revised supplementary material).

Q3: Typo

Thank you for pointing this out. We have corrected the typo in the revised manuscript.

[1] Tuan Duc Ngo, Jiahui Huang, Seoung Wug Oh, Kevin Blackburn-Matzen, Evangelos Kalogerakis, Chuang Gan, Joon-Young Lee. DAGE: Dual-Stream Architecture for Efficient and Fine-Grained Geometry Estimation. CVPR, 2026.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Follow-up Before the Discussion Closes

Thanks again for your insightful comments and valuable time devoted to our paper. As the author-reviewer discussion period is coming to an end, please let us know if there's any further information we can provide to facilitate the discussion process. We are happy to answer any questions or concerns you may have during the author-reviewer discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.