- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes PanoDPG, a task-decoupled dual-branch framework for high-resolution panoramic 3D reconstruction. The key motivation is that pose estimation and geometry reconstruction have different requirements: pose estimation benefits more from global context and is robust to resolution, while geometry reconstruction requires high-resolution local details and lower projection distortion. PanoDPG therefore uses low-resolution ERP features for global pose estimation and high-resolution cubemap features for detailed geometry reconstruction, coupling the two branches via register-token communication. Experiments show competitive pose and point-cloud performance with better efficiency and scalability than recent feed-forward panoramic reconstruction methods.
Critical strengths
The paper has a very good motivation. It points out a real and important asymmetry: pose estimation does not need high-resolution input, while geometry reconstruction does. I agree with this problem formulation. The idea of decoupling pose and geometry into different resolution/representation branches is well aligned with this observation.
Critical weaknesses
The 2072 comparison is not fair. The paper says PanoVGGT was only retrained at 1036×518, so there is no 2072 training result for PanoVGGT. Listing its 2072 collapse in the table is therefore not meaningful. I think the 2072 setting has no comparability.
The framework design in this paper is not novel, and the use of cubemap operations is also fairly common. Some experimental results still need further explanation.
Questions for authors
- Since PanoVGGT was not trained at 2072×1036, is the 2072 comparison in Table 1 fair? Explain where this data comes from and why it collapses.
- At 1036×518, can you provide the ablation with only the cubemap branch + intra-register attention, with both pose and point heads from that branch? This would isolate the contribution of the ERP branch and task decoupling.
- Table 4 claims 2K efficiency, but the text says baselines were trained at 1036. Could you provide efficiency comparison at 1036×518, matching PanoVGGT's official 1024 setting?
- In Table 3, dual-branch without RBI drops pose AUC from 0.903 to 0.819, and RBI only recovers it to 0.903. Does this mean decoupling itself harms pose estimation and RBI merely compensates? Explain this phenomenon.
I initially leaned toward borderline accept, but that option isn't available, so I'll wait and see the authors' response.
W1/Q1: Unfair comparison
Thank you for pointing this out. PanoVGGT* was fine-tuned at 1036×518 and performs poorly at the unseen 2072×1036 resolution. We have removed this unfair comparison.
Our previous 1K PanoDPG used 1036×518 source panoramas but retained six 686×686 cubemap faces, matching the 2K configuration’s processing resolution. It did not constitute a fully matched 1K configuration. We have therefore trained a matched 1K configuration with 1036×518 ERP inputs and six 350×350 cubemap faces for comparison at the baselines’ fine-tuning resolution. The updated results are reported in Tables 1 and 2 of the revised manuscript.
W2: Framework design novelty
Our contribution is a task-decoupled architecture tailored to panoramic reconstruction, with two key designs:
(1) Joint decoupling of resolution and representation. We combine a moderate-resolution ERP branch for global pose estimation with a high-resolution cubemap branch for detailed geometry recovery. This allocation follows our quantitative analysis of their different resolution requirements (Fig. 2; revised Suppl. Tabs. 8–11).
(2) Hierarchical aggregation with unified communication. Dense attention connects the six faces within each cubemap group, while compact register tokens support both ERP–cubemap feature exchange and communication across panoramas. This connects task-specific representations and propagates global information without dense attention across all faces and frames.
Q2: Cubemap branch + register token attention
Thank you for this suggestion.
[Experimental setup] We have added a cubemap-only ablation, retaining global attention within each six-face group and register-token attention across groups, with both pose and point heads attached to the cubemap branch. The variant is trained using the same protocol as PanoDPG.
FE and TE denote a frozen encoder and a trainable encoder, respectively.
[Experimental results]
[Experimental conclusion] As shown in the table, removing the ERP branch degrades both pose and point-cloud accuracy. This demonstrates the contribution of the ERP branch even when cross-group register communication is retained, supporting our design of combining ERP-based pose estimation with cubemap-based geometry recovery.
Q3: Efficiency evaluation
We have added runtime and GPU memory comparisons at 1036×518 (Tab. 15 in the suppl. and Fig. 6 in revision). All methods use ERP inputs at this resolution; PanoDPG additionally uses six 350×350 cubemap faces per panorama. Each entry reports inference time (s) / peak GPU memory (GiB).
PanoDPG generally ranks second in runtime for shorter sequences and becomes the fastest from 50 frames onward. At 200 frames, it takes 88.85 s, reducing runtime by 33.3% compared with PanoVGGT (133.26 s) as below.
† Wid3R encounters a 32-bit indexing limit in its reflection-padding operator, distinct from OOM.
Q4: dual-branch without RBI
Thank you for highlighting this result. Without RBI, the branches exchange no features, but their predictions remain coupled through supervision: ERP translations and cubemap points share a scale alignment to ground truth and jointly contribute to the global point-cloud loss. They must therefore produce geometrically consistent predictions without direct feature coordination, which can explain the pose degradation.
RBI enables this coordination through feature exchange. We agree that decoupling without RBI reduces pose accuracy in this configuration. However, the complete model restores the single-branch pose AUC while substantially improving geometry, supporting the combined design of task decoupling and inter-branch communication.