acceptodds
Withdrawn from ICLR 2027

Hybrid-VGGT: Scaling Visual Geometry Grounded Transformers to High Resolution via Multi-Scale Detail Injection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Feed-forward foundation models have significantly accelerated uncalibrated 3D reconstruction by inferring geometry in a single pass. However, their reliance on global self-attention introduces a severe quadratic computational bottleneck at high resolutions, forcing a compromise between inference efficiency and the preservation of fine-grained geometric details. To address this accuracy-efficiency trade-off, we introduce Hybrid-VGGT, an asymmetric dual-branch extension of VGGT that allocates global multi-view reasoning and high-resolution detail extraction to complementary streams. A low-resolution branch efficiently anchors global scene context and camera prediction, while a parallel Hybrid Detail Extractor (HDE) captures per-view, fine-grained structural cues directly from high-resolution inputs. By integrating multi-layer local convolutions with scalable linear attention, the HDE extracts structural cues across multiple scales. These multi-scale features are then progressively injected into the global low-resolution stream via a customized hierarchical fusion head, effectively refining the coarse geometric representation with high-frequency boundary cues. Comprehensive experiments demonstrate that Hybrid-VGGT achieves state-of-the-art performance on multiple benchmarks for camera pose recovery, dense depth, and point map estimation, yielding high-fidelity 3D geometry and the best overall mean rank in our comparisons. On 100-frame 1036p clips, Hybrid-VGGT runs at 38.56 FPS, compared with 3.12 FPS for VGGT and 5.19 FPS for Pi3.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Feed-forward Reconstruction, High-resolution Reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_289zOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Feed-forward 3D foundation models (VGGT, Pi3) are bottlenecked by the quadratic cost of global self-attention, capping inputs at ~518p and blurring fine geometric detail. This paper proposes Hybrid-VGGT, an asymmetric dual-branch architecture: a low-resolution branch (initialized from pretrained VGGT, max side 518px) performs global multi-view reasoning and camera pose estimation, while a parallel Hybrid Detail Extractor (HDE) processes 1036px inputs with a cascade of hybrid blocks (depth-wise conv → linear attention à la Shen et al. 2021 → depth-wise conv), producing multi-scale detail features at O(N) cost. A customized multi-scale DPT head (RefineNet-style top-down path) progressively injects resolution-matched HDE features additively into the global pyramid at every stage, yielding native 1036p depth and point maps. Trained end-to-end on 16 datasets (64×A800, 160 epochs), the method reports the best mean rank on camera pose (1.62), multi-view depth (1.6), and point map estimation (1.67) versus Fast3R, CUT3R, FLARE, VGGT, Pi3, and DAGE, while running at 38.56 FPS on 1036p 100-frame clips (vs. 3.12 for VGGT) on a single RTX 5090.

Critical strengths

The problem targets real-application bottlenecks and is well-formulated.

The architectual design and illustration is clear and self-consistent.

The ablation is informative, supporting the claim that detail extraction for geometry needs dedicated local inductive biases rather than off-the-shelf semantic features.

Critical weaknesses

The difference and improvements over DAGE is not evident. Comparisons to DAGE only exists in Table 3.

For VGGT and Pi3, why evaluate them by "518p input then upsample to 1036p"? As Tab. 1 points out, it is operable for VGGT and Pi3 on 1036p, although inefficient. Using this fairer setting will demonstrate its actual improvement.

The method is instantiated exclusively on VGGT, yet newer open backbones (VGGT-Ω\Omega, DA3, etc.) have since demonstrated stronger geometry and pose accuracy at comparable resolutions. It also bears on real-world impact as practitioners choosing a feed-forward geometry model now start from the newer ones, so a method pinned to VGGT risks limited adoption upon release. Anyway, it is a "nice-to-have" suggestion. Also, the results of VGGT-Ω\Omega, DA3 are not included in comparison. It is unknown whether the newer backbones would surpass Hybrid's result in high resolution.

It is also suggested to include performance of your proposed module initialised with these backbones.

Questions for authors

Major Issues

Regarding efficiency, although it outperforms VGGT, how does it compare with LiteVGGT/FastVGGT/StreamVGGT/InfiniteVGGT (just pick a few) and DAGE under such high-resolution setting, and how much improvement in performance it obtained among others?

Where does 1036p depth ground truth come from for each dataset (especially 7-Scenes, whose sensor depth is 640×480)? Is GT also upsampled, and if so, how do you interpret boundary-sharpness wins against GT that contains no real high-frequency detail? I do suggest following the evaluation of DAGE (See its Tab. 1), to not strictly setting 1036p.

Figure 3 shows better details of proposed method. But I would suggest increasing point size or setting input resolution to 1036p for VGGT & Pi3, otherwise it isn't a fair comparison.

Minor Issues

The definition of "518p"/"1036p" in the paper is "resized to a maximum dimension of 518/1036 pixels," i.e., the long side. Yet in real-world, "p, progressive scan" mostly refers to short side, (e.g.: 1080p is 1920$\times$ 1080 under 16:9 aspect ratio). Please follow traditions.

I fixed the review in last minute as I found out CARVE isn't yet open-sourced, making you unable to compare. Please point it out once encountering such issue.

Citations of CARVE and DAGE are not accepted version. Please update them with CVPR 2026 published version.

Although I can not recommend acceptance with current version, I will certainly raise my score toward acceptance if the questions and weaknesses are resolved.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 289z (Part 1/2)

Thanks for your careful and valuable comments. These comments have been of great value to my approach. We will explain your concerns point by point.

Weakness 1. Difference and gains over DAGE.

Our contribution is the geometry-oriented detail pathway and its integration with global reasoning and dense refinement. HDE combines local spatial processing with per-view context; pre-Aggregator injection allows these cues to inform camera prediction, while multi-scale DPT fusion supports dense reconstruction. This targets how efficiently high-resolution information is used within the LR/HR framework.

The original Hypersim-test ablations quantify the extractor and fusion choices: replacing DINO with HDE lowers XYZ error 0.2490→0.2262 (9.2%); removing the Feature Injector raises it to 0.2387. Separately, Tables 1–3 (Tables 2–4 in manuscript) now compare DAGE-LR518 at the same 1036 external input. Ours improves all 8 pose metrics, all 5 depth AbsRel scores and all 3 DTU metrics, with DTU completion error falling 1.811→1.562 (13.7%). These additions establish benefits across tasks beyond the original depth-only comparison.

Weakness 2. VGGT/Pi3 with actual 1036 inputs.

Tables 1–3 (Tables 2–4 in manuscript) now include VGGT/Pi3 with actual 1036 inputs, including DTU. At matched input resolution, Ours improves all 8 pose metrics and all 5 depth AbsRel scores over both. The advantage therefore persists when the baselines receive the same number of input pixels. Each Ours-518/1036 pair uses the same checkpoint and protocol; only external input resolution changes.

Table 1. Camera pose (Table 2 in manuscript).

MethodCO3Dv2 AUC30↑RE10K AUC30↑TUM ATE↓RPE-t↓RPE-r↓ScanNet++ ATE↓RPE-t↓RPE-r↓Rank↓
Fast3R73.4361.680.0900.1011.4250.1550.1233.49110.0
CUT3R75.8281.470.0470.0150.4510.0940.0220.6297.9
FLARE73.9980.010.0260.0130.4750.0640.0230.9718.1
VGGT-51888.5977.620.0120.0100.3110.0350.0150.3824.6
VGGT-103688.0775.880.0180.0120.3650.0370.0310.9587.3
Pi3-51888.4185.900.0140.0090.3120.0310.0130.3472.5
Pi3-103688.6084.630.0200.0140.3820.0360.0180.3795.6
DAGE88.1784.080.0140.0100.3110.0350.0170.3774.4
Ours-51889.9584.260.0120.0100.3100.0340.0140.3793.1
Ours-103690.2684.940.0110.0090.3080.0330.0140.3721.6

Table 2. Depth (Table 3 in manuscript).

MethodSintel AbsRel↓Sintel δ1↑ETH3D AbsRel↓ETH3D δ1↑7-Scenes AbsRel↓7-Scenes δ1↑BlendedMVS AbsRel↓BlendedMVS δ1↑ScanNet++ AbsRel↓ScanNet++ δ1↑Rank↓
DUSt3R0.4560.5090.2410.6200.4120.9140.1470.8630.1450.8138.2
MASt3R0.3800.5330.1520.8040.4420.9210.1660.8780.0430.9746.7
Fast3R0.5470.4290.2510.5560.5060.7330.1710.8260.1830.71810.4
CUT3R0.6300.4590.1540.7600.4140.9590.1500.8860.0310.9656.7
FLARE0.4960.4180.2320.6300.4800.8520.2120.8200.1460.8009.8
VGGT-5180.2260.6800.0360.9860.3790.9460.0200.9920.0610.9623.6
VGGT-10360.2850.5880.0960.9110.6810.9580.0620.9850.0960.9246.4
Pi3-5180.2310.6740.0450.9860.4170.9660.0470.9730.0470.9743.8
Pi3-10360.2870.6190.0610.9800.5620.9570.0560.9690.0670.9705.8
DAGE-10360.2140.6950.0340.9790.4230.9710.0300.9880.0260.9732.9
Ours-10360.2080.6850.0330.9890.3680.9400.0210.9930.0250.9751.8

Table 3. Point maps (Table 4 in manuscript).

MethodDTU ACC↓DTU Comp↓DTU NC↑7-Scenes ACC↓7-Scenes Comp↓7-Scenes NC↑ETH3D ACC↓ETH3D Comp↓ETH3D NC↑Rank↓
Fast3R3.3402.9290.6710.0950.1440.6730.8320.9780.6679.0
CUT3R4.7423.4000.6790.0930.1020.7040.6170.7470.7547.7
FLARE2.5413.1740.6840.0850.1450.6960.4640.6640.7447.1
VGGT-5181.3381.8960.6760.0440.0560.7330.2800.3050.8534.1
VGGT-10361.7682.1240.6730.1310.2010.6440.4160.5510.6788.4
Pi3-5181.1981.8490.6780.0470.0750.7420.1940.2100.8833.0
Pi3-10361.2532.1170.6750.1000.1690.6790.3270.3410.8177.0
DAGE1.1681.8110.6800.0450.0580.7250.2080.2320.8662.7
Ours-5181.1761.9250.6810.0460.0610.7180.2540.2580.8533.7
Ours-10361.1171.5620.6960.0410.0540.7290.2710.2730.8502.3

Weakness 3. Comparison with newer geometry backbones.

The added DA3-Large-1.1 experiment shows that the current VGGT-based model remains competitive with a newer released geometry model on the Bonn test. At 1036×784, Ours achieves AbsRel/δ1 of 0.0405/0.9878 versus 0.0768/0.8937. On the separate CO3Dv2 resource benchmark at 1036×588, throughput is 38.56 versus 10.91 FPS (3.53×). VGGT-Ω evaluation awaits checkpoint access approval from hugging face; we will add it once access is granted.

AuthorsOpenReview ↗

Response to Reviewer 289z (Part 2/2)

Weakness 4. HDE transfer to newer backbones.

We agree that transfer to newer backbones is a valuable extension. It requires backbone-specific integration and retraining, which exceeds our available training resources, and we leave it to future work. The completed DA3 comparison addresses the immediate practical question: whether the current VGGT-based model remains competitive with a newer released model at high resolution. Its accuracy and throughput advantage on the added benchmarks supports that use case; cross-backbone transfer remains to be evaluated.

Major Question 1. Efficient VGGT variants and DAGE at high resolution.

At the same LR518 global stream and 1036 external input, Ours improves all 8 pose metrics, all 5 depth AbsRel scores and all 3 DTU metrics over DAGE. On the added Bonn test, AbsRel/δ1 is 0.0405/0.9878 versus 0.0411/0.9865. Complete inference on 100-frame CO3Dv2 clips reaches 38.56 versus 20.08 FPS (1.92×), with memory 20.11 versus 20.27 GB.

DAGE-LR252 uses a smaller global stream to reduce memory; its Bonn AbsRel is 0.0568 versus 0.0411 for LR518. The matched LR518 comparison shows our accuracy–speed advantage without reducing global-stream resolution. Against FastVGGT at 1036 input, Ours has lower Bonn AbsRel (0.0405 vs 0.0731) and 4.73× CO3Dv2 throughput.

Accuracy: Bonn, inputs 518×392/1036×784, GT 1036×784, one scale/clip; Bonn is absent from our training mixture. Resources: CO3Dv2, 100 frames, 518×294/1036×588, RTX 5090, complete inference and the same timing scope.

Table 1. Bonn accuracy / CO3Dv2 resources.

Method/long side (px)AbsRel↓δ1↑FPS↑Mem↓
VGGT/5180.04950.972344.6917.51
VGGT/10360.07340.95113.1228.98
Pi3/5180.04210.981556.1117.42
Pi3/10360.04560.98005.1923.35
FastVGGT/5180.04890.973746.5817.86
FastVGGT/10360.07310.95108.1519.31
DA3-Large-1.1/5180.06870.926455.7415.47
DA3-Large-1.1/10360.07680.893710.9120.51
DAGE (LR252)/5180.05590.977150.7311.98
DAGE (LR252)/10360.05680.977226.2219.21
DAGE (LR518)/10360.04110.986520.0820.27
Ours/5180.04180.979840.0919.32
Ours/10360.04050.987838.5620.11

Major Question 2. GT sources and the resolution of 7-Scenes depth.

The common evaluation grid provides a consistent reference for comparing all methods: identical GT depths, valid-depth masks and per-sequence scale alignment. GT comes from supplied rendered depth for Sintel/BlendedMVS, scans for ETH3D, sensors for 7-Scenes and registered geometry for ScanNet++. Resampling changes the scoring grid, not the information in GT. In particular, 7-Scenes depth originates at 640×480, so the current scores measure agreement with sensor-supported depth; finer boundary appearance in Figure 3 is qualitative.

We will additionally report AbsRel/δ1 at native GT resolution, keeping GT unchanged and resizing predictions to its grid (640×480 for 7-Scenes). This will test whether the accuracy comparison is preserved without GT resampling. The DTU scan-based gains already provide separate quantitative evidence of reconstruction quality.

Major Question 3. Figure 3: matched inputs and point size.

We will revise Figure 3 with actual 1036-input VGGT/Pi3 and Ours, identical viewpoints and point size, and explicit input labels. This update is pending. The completed matched-input depth and point-map experiments establish the quantitative comparison; the revised figure will align the visual comparison with that setting.

Minor Question 1. Explicit input dimensions.

We use “input long side 518/1036 pixels” and will replace the “p” suffix throughout the manuscript. Method suffixes in multi-dataset tables denote long-side pixels. Single-dataset shapes are explicit: Bonn 518×392/1036×784; CO3Dv2 resources 518×294/1036×588.

Minor Question 2. CARVE implementation availability.

CARVE inference code and checkpoints were unavailable when these comparisons were prepared. We will state this explicitly; the added experiments include the runnable DAGE, FastVGGT and DA3 baselines.

Minor Question 3. Update the reference versions.

Thank you. We will update DAGE's citation to its official CVPR 2026 proceedings version and check the remaining reference metadata.

As the discussion deadline approaches, we would appreciate your feedback on the expanded DAGE comparisons and actual 1036-input VGGT/Pi3 evaluations across pose, depth and point maps. At matched 1036 input, our method improves all eight pose metrics and all five depth scores over these baselines. We also added FastVGGT and DA3 comparisons and clarified the GT resolution and evaluation protocol. Could you let us know whether these additions address your main concerns, or which remaining issue would benefit from clarification? Thank you for your time and consideration.

Reviewer_HWHJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper keeps VGGT's multi-view aggregation at low resolution while processing each image through a high-resolution detail encoder that combines depth wise convolution and linear attention. Those detail features are then injected into a multi-scale DPT to predict high-fidelity depth and point maps, while retaining camera-pose prediction.

Critical strengths
  1. The reported high-resolution throughput is compelling: 38.56FPS at 1036p, compared with 3.12 for VGGT and 5.19 for Pi3.
  2. The evaluation covers pose, depth, and point maps across several datasets, with competitive results overall.
Critical weaknesses
  1. The novelty is narrow. DAGE already separates low-resolution multi-view reasoning from high-resolution detail extraction, and CARVE uses low- and high-resolution features in a VGGT-based model. What measurable gains comes from this paper's specific design?

  2. The high-resolution encoder is not convincingly justified. Table 5 includes a DINO extractor variant, but no matched comparison with standard alternatives such as ConvNext and Swin Transformer. It also does not isolate the convolution and linear-attention components. Why is the proposed encoder preferable at comparable resolution, model size, and compute?

  3. The quality-speed comparison mixed operating points. The main depth table compares native 1036p output with mostly 518p predictions upsampled for evaluation, while the headline speed comparison runs baselines at 1036p. At 518p, VGGT and Pi3 are faster. Matched quality–runtime–memory results, including DAGE, are needed.

Questions for authors

see the weakness

3 comments
AuthorsOpenReview ↗

Response to Reviewer HWHJ (Part 1/2)

Thanks for your careful and valuable comments. These comments have been of great value to my approach. We will explain your concerns point by point.

Weakness 1. Specific contribution beyond DAGE/CARVE.

Our contribution is a geometry-oriented detail pathway that informs both multi-view reasoning and dense reconstruction while global attention stays on a low-resolution grid. Within the LR/HR framework shared with DAGE/CARVE, HDE combines local spatial processing and per-view context, then injects these cues before the Aggregator and throughout the multi-scale DPT hierarchy. This connects efficient detail extraction to both camera and dense outputs.

The design is supported at two levels. In encoder ablations, HDE lowers Hypersim-test XYZ error 0.2502→0.2262 (9.6%) versus Swin-S, with 2.89× encoder throughput (Table 1); removing only linear attention raises XYZ error to 0.2481. In the matched LR518/1036 system comparison, Ours improves all 8 pose metrics, all 5 depth AbsRel scores and all 3 DTU metrics over DAGE, while complete inference is 1.92× faster on 100-frame CO3Dv2 clips. Thus the specific design delivers measurable geometry and efficiency gains within the shared framework.

Weakness 2. Budget-matched ConvNeXt/Swin and component ablations.

The added Swin-S replacement and w/o LA ablations support the combination of local processing and context. HDE improves all three Hypersim Test metrics over Swin-S using 1.61M versus 48.84M encoder parameters. Removing only linear attention increases XYZ error 0.2262→0.2481; retaining it reduces error by 8.8% relative to the convolution-only variant.

Table 1 measures each detail encoder alone: HDE has lower GFLOPs and 2.89×/1.78× throughput versus Swin-S/ConvNeXt-S. These results establish an efficient alternative to the standard Swin-S replacement. Strict equal-budget retraining and attention-only training are not included; ConvNeXt accuracy remains unmeasured. We will distinguish these standard-backbone and component ablations explicitly.

Table 1. Encoder comparison. Accuracy: Hypersim Test. Encoder-only FPS/GFLOPs: 1036×588, RTX 5090.

VariantPose↑AbsRel↓XYZ↓Encoder FPS (repro.)↑Encoder GFLOPs↓
Swin-S0.96130.02280.2502150.15228.64
ConvNeXt-S---244.31205.11
HDE (ours)0.96820.02150.2262433.7113.60
AuthorsOpenReview ↗

Response to Reviewer HWHJ (Part 2/2)

Weakness 3. Matched quality, runtime and memory, including DAGE.

The matched operating points show a joint gain in high-resolution geometry and inference efficiency. Tables 1–3 (Tables 2–4 in manuscript) now include actual VGGT/Pi3-1036 runs, DAGE-LR518 and both Ours resolutions where requested. At 1036 input, Ours has lower AbsRel than VGGT/Pi3/DAGE on every depth dataset and ranks first across all three tasks. Each Ours resolution pair uses the same checkpoint and protocol.

On 100-frame CO3Dv2 clips at 1036×588, complete inference reaches 38.56 FPS versus 3.12/5.19/20.08 for VGGT/Pi3/DAGE-LR518, with memory 20.11 versus 28.98/23.35/20.27 GB. On the separate Bonn test, Ours achieves the best AbsRel/δ1 among the reported operating points. The comparison therefore measures accuracy and resources at both input resolutions, rather than combining low-resolution baseline accuracy with high-resolution baseline timing.

All evaluation scenes/sequences are held out; † marks training on a separate split of the source dataset. Rank is mean metric midrank, rounded to one decimal. Method suffixes denote input long side in pixels.

Table 1. Camera pose (Table 2 in manuscript).

MethodCO3Dv2† AUC30↑RE10K AUC30↑TUM ATE↓RPE-t↓RPE-r↓ScanNet++† ATE↓RPE-t↓RPE-r↓Rank↓
Fast3R73.4361.680.0900.1011.4250.1550.1233.49110.0
CUT3R75.8281.470.0470.0150.4510.0940.0220.6297.9
FLARE73.9980.010.0260.0130.4750.0640.0230.9718.1
VGGT-51888.5977.620.0120.0100.3110.0350.0150.3824.6
VGGT-103688.0775.880.0180.0120.3650.0370.0310.9587.3
Pi3-51888.4185.900.0140.0090.3120.0310.0130.3472.5
Pi3-103688.6084.630.0200.0140.3820.0360.0180.3795.6
DAGE88.1784.080.0140.0100.3110.0350.0170.3774.4
Ours-51889.9584.260.0120.0100.3100.0340.0140.3793.1
Ours-103690.2684.940.0110.0090.3080.0330.0140.3721.6

Table 2. Depth (Table 3 in manuscript).

MethodSintel AbsRel↓Sintel δ1↑ETH3D AbsRel↓ETH3D δ1↑7-Scenes AbsRel↓7-Scenes δ1↑BlendedMVS† AbsRel↓BlendedMVS† δ1↑ScanNet++† AbsRel↓ScanNet++† δ1↑Rank↓
DUSt3R0.4560.5090.2410.6200.4120.9140.1470.8630.1450.8138.2
MASt3R0.3800.5330.1520.8040.4420.9210.1660.8780.0430.9746.7
Fast3R0.5470.4290.2510.5560.5060.7330.1710.8260.1830.71810.4
CUT3R0.6300.4590.1540.7600.4140.9590.1500.8860.0310.9656.7
FLARE0.4960.4180.2320.6300.4800.8520.2120.8200.1460.8009.8
VGGT-5180.2260.6800.0360.9860.3790.9460.0200.9920.0610.9623.6
VGGT-10360.2850.5880.0960.9110.6810.9580.0620.9850.0960.9246.4
Pi3-5180.2310.6740.0450.9860.4170.9660.0470.9730.0470.9743.8
Pi3-10360.2870.6190.0610.9800.5620.9570.0560.9690.0670.9705.8
DAGE-10360.2140.6950.0340.9790.4230.9710.0300.9880.0260.9732.9
Ours-10360.2080.6850.0330.9890.3680.9400.0210.9930.0250.9751.8

Table 3. Point maps (Table 4 in manuscript).

MethodDTU ACC↓DTU Comp↓DTU NC↑7-Scenes ACC↓7-Scenes Comp↓7-Scenes NC↑ETH3D ACC↓ETH3D Comp↓ETH3D NC↑Rank↓
Fast3R3.3402.9290.6710.0950.1440.6730.8320.9780.6679.0
CUT3R4.7423.4000.6790.0930.1020.7040.6170.7470.7547.7
FLARE2.5413.1740.6840.0850.1450.6960.4640.6640.7447.1
VGGT-5181.3381.8960.6760.0440.0560.7330.2800.3050.8534.1
VGGT-10361.7682.1240.6730.1310.2010.6440.4160.5510.6788.4
Pi3-5181.1981.8490.6780.0470.0750.7420.1940.2100.8833.0
Pi3-10361.2532.1170.6750.1000.1690.6790.3270.3410.8177.0
DAGE1.1681.8110.6800.0450.0580.7250.2080.2320.8662.7
Ours-5181.1761.9250.6810.0460.0610.7180.2540.2580.8533.7
Ours-10361.1171.5620.6960.0410.0540.7290.2710.2730.8502.3

Accuracy: Bonn, , inputs 518×392/1036×784, GT 1036×784; Bonn is absent from training. Resources: CO3Dv2, 100 frames, 518×294/1036×588, RTX 5090, complete inference and the same timing scope.

Table 4. Bonn accuracy / CO3Dv2 resources.

Method/long side (px)AbsRel↓δ1↑FPS↑Mem↓
VGGT/5180.04950.972344.6917.51
VGGT/10360.07340.95113.1228.98
Pi3/5180.04210.981556.1117.42
Pi3/10360.04560.98005.1923.35
DAGE (LR252)/5180.05590.977150.7311.98
DAGE (LR252)/10360.05680.977226.2219.21
DAGE (LR518)/10360.04110.986520.0820.27
Ours/5180.04180.979840.0919.32
Ours/10360.04050.987838.5620.11

As the discussion deadline approaches, we would appreciate your feedback on the added Swin-S replacement and linear-attention ablation, together with the matched-resolution DAGE comparison. These results support the contribution of the detail pathway through improvements in geometry and efficiency, including 1.92× complete-inference throughput over DAGE at the same LR518/1036 setting. Could you let us know whether these additions address your main concerns about the design’s contribution and practical value? We would be happy to clarify any remaining points. Thank you for your time and consideration.

Reviewer_AfznOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Hybrid-VGGT keeps global multi-view reasoning on a 518p token grid while a parallel Hybrid Detail Extractor — depthwise convolutions plus linear attention, strictly per-view — processes the full 1036p image; the resulting multi-scale features are injected into the low-resolution stream through a DPT-style head. The motivation is the quadratic cost of global attention at high resolution. Validated on pose, multi-view depth, point maps, throughput, and an internal ablation on Hypersim (Table 5). Headline claims are the best mean rank in each table (1.62, 1.6, 1.67) and 38.56 FPS at 1036p against 3.12 and 5.19.

Critical strengths

Table 3 contains a resolution-matched head-to-head and the method wins it. DAGE is the only baseline that also outputs natively at 1036p, and the caption says so. Across five depth datasets the method beats DAGE on AbsRel in all five: 0.208 vs 0.214, 0.033 vs 0.034, 0.368 vs 0.423, 0.021 vs 0.030, 0.025 vs 0.026. This is the one comparison where input resolution is held constant and the advantage is therefore not guaranteed by construction. It is also the best-constructed table: 7 of 9 rank entries reproduce exactly, and the 1.6 versus DAGE's 2.8 is robust to every tie-breaking convention. The "first place in 7 out of 10 metrics" count is exactly right, including its three losses.

The held-out portion of the evaluation is genuinely competitive. Of the four evaluation datasets that appear in the training list, three are unflagged (see weakness 1), but the truly held-out results are not weak: DTU is a clean sweep of ACC/Comp/NC (1.117/1.562/0.696 against Pi3's 1.198/1.849/0.678), and 7-Scenes is won on ACC and Comp. The 1.67 mean rank in Table 4 reproduces exactly, and the prose states the losses correctly. So the method is not merely memorising its training mixture, and an AC should not read the in-domain problem as evidence of overfitting — it is an evidence-labelling problem, not a performance one.

Critical weaknesses

The "zero-shot" framing is not supported by the paper's own training list, and the ablation runs in-domain. The training corpus includes BlendedMVS (Train Set), Hypersim, ScanNet++ and CO3Dv2 (Train Set). Appendix B is titled a "rigorous zero-shot evaluation protocol." Yet only ScanNet++ carries a "(Seen)" flag; BlendedMVS appears unflagged while being a training set, and CO3Dv2 — the paper's largest pose win, +1.85 AUC30 — is a training set. The Hypersim ablation is likewise on a training dataset with no split qualifier, in contrast to the careful "(Train Set)" spellings used elsewhere. This matters for how Table 3 is weighted: the best-in-table BlendedMVS δ of 0.993 is an in-domain number competing against zero-shot baselines, and the 9.2% XYZ reduction credited to the HDE is measured on data the model has seen. At minimum every table needs a seen/unseen column.

The verdict is carried by rank aggregation over gaps at or below noise, and the rank arithmetic does not reproduce. In Table 2 the method beats Pi3 on 3 of 8 metrics, loses 4, and ties one; it is worse than Pi3 on all three ScanNet++ metrics — the dataset the paper itself calls in-domain. In Table 4 it loses all three ETH3D metrics decisively (ACC 0.271 vs 0.194), so the 1.67 is partly a disaggregation artefact. In Table 3, four of the seven winning cells are 0.001 margins. Not one number carries a seed, standard deviation, or confidence interval. Worse, the Rank columns are unreliable: Table 2's VGGT entry prints 2.63 but its per-metric ranks sum to 23, giving 2.875; recomputing with shared ranks gives Ours 1.5625, Pi3 1.8125 — a 0.25 margin, not the stated 0.13. Table 4 prints Fast3R 5.67 (true 5.556), CUT3R 4.78 (4.667) and FLARE 4.11 (4.333). Compounding this, the pose evidence contradicts the method section: Section 3.2 routes camera regression from Fg, i.e. the 518p stream, while Appendix B.1 states "we evaluate from our 1036p outputs" — there is no described mechanism by which 1036p imagery reaches the camera head. The only pose ablation is one Hypersim column with a total spread of 0.0094.

Questions for authors
  1. Report accuracy at matched input resolution. Table 1 proves VGGT and Pi3 were run at 1036p. Add those runs to Tables 2-4, or explain what makes it infeasible. Decisive: this is the only experiment that separates "the HDE is better" from "more pixels are better," and the compute demonstrably exists.
  2. It is able to add DAGE to Tables 2 and 4, and report a 518p version of your own model so the pose path can be decomposed. Decisive: DAGE is the one resolution-matched competitor and appears only in depth; the "best overall mean rank" claim currently rests on a field that excludes it from two of three tasks.
  3. Label every table column as seen or unseen, and report variance. Report at least three seeds with standard deviations for the cells that decide the rankings, and recompute the Rank columns of Tables 2 and 4. Decisive: without it, "state of the art" and "not distinguishable from DAGE/Pi3" are the same claim.
  4. Reconcile Section 3.2 with Appendix B.1 on the camera pathway. If the pose head reads only the 518p stream, say so and remove the high-resolution pose justification; if it does not, describe the mechanism. Decisive: the abstract's pose claim depends on which statement is true.
  5. Fix Eq. 8. Lines 771-774 write AbsRel=1N∑i=1N∣di−d^i∣d^i\mathrm{AbsRel} = \frac{1}{N}\sum_{i=1}^{N}\frac{\lvert d_i - \hat{d}_i\rvert}{\hat{d}_i}, with did_i the ground truth and d^i\hat{d}_i the prediction, so the normalizer is the prediction rather than the ground truth, contrary to the protocol the paper says it follows. State the exact formula used and confirm per-sequence scale alignment. Decisive: the choice of normalizer changes the numbers materially, and if the implementation differs from the paper this is a reporting error.
3 comments
AuthorsOpenReview ↗

Response to Reviewer Afzn (Part 1/2)

Thanks for your careful and valuable comments. These comments have been of great value to my approach. We will explain your concerns point by point.

Weakness 1. Training/test separation and zero-shot scope.

All evaluation scenes/sequences are excluded from training. We use official sequence-level splits for CO3Dv2 and scene-level splits for BlendedMVS, ScanNet++ and Hypersim; only train partitions enter training. Section 4.4 already specifies Hypersim Test: the 9.2% gain in manuscript Table 5 is measured on disjoint test scenes. Thus the ablation tests generalization to unseen scenes. We will label each dataset's split and training exposure in the main text and tables, use “held-out evaluation” throughout, and reserve dataset-level “zero-shot” for source datasets absent from training.

Weakness 2. Evidence for the gains and reliability of Rank.

The gains are supported by matched-input comparisons and improvements in absolute errors. At 1036 input, Ours improves all 8 pose metrics and all 5 depth AbsRel scores over VGGT/Pi3. Against DAGE-LR518, DTU completion error falls 1.811→1.562 (13.7%), alongside improvements in all three DTU metrics. These results establish benefits beyond the smallest depth margins.

Ours ranks first in pose/depth/point maps (1.6/1.8/2.3), leading 7/10 depth and 5/9 point-map metrics. Equal dataset weighting preserves first place. We corrected the rank arithmetic using tied midranks; original accuracy values are unchanged. The three-run variability and paired-resolution camera results are reported in Questions 3–4.

All evaluation scenes/sequences are held out; † marks training on a separate split of the source dataset. Rank is mean metric midrank, rounded to one decimal. Method suffixes denote input long side in pixels.

Table 1. Camera pose (Table 2 in manuscript).

MethodCO3Dv2† AUC30↑RE10K AUC30↑TUM ATE↓RPE-t↓RPE-r↓ScanNet++† ATE↓RPE-t↓RPE-r↓Rank↓
Fast3R73.4361.680.0900.1011.4250.1550.1233.49110.0
CUT3R75.8281.470.0470.0150.4510.0940.0220.6297.9
FLARE73.9980.010.0260.0130.4750.0640.0230.9718.1
VGGT-51888.5977.620.0120.0100.3110.0350.0150.3824.6
VGGT-103688.0775.880.0180.0120.3650.0370.0310.9587.3
Pi3-51888.4185.900.0140.0090.3120.0310.0130.3472.5
Pi3-103688.6084.630.0200.0140.3820.0360.0180.3795.6
DAGE88.1784.080.0140.0100.3110.0350.0170.3774.4
Ours-51889.9584.260.0120.0100.3100.0340.0140.3793.1
Ours-103690.2684.940.0110.0090.3080.0330.0140.3721.6

Table 2. Depth (Table 3 in manuscript).

MethodSintel AbsRel↓Sintel δ1↑ETH3D AbsRel↓ETH3D δ1↑7-Scenes AbsRel↓7-Scenes δ1↑BlendedMVS† AbsRel↓BlendedMVS† δ1↑ScanNet++† AbsRel↓ScanNet++† δ1↑Rank↓
DUSt3R0.4560.5090.2410.6200.4120.9140.1470.8630.1450.8138.2
MASt3R0.3800.5330.1520.8040.4420.9210.1660.8780.0430.9746.7
Fast3R0.5470.4290.2510.5560.5060.7330.1710.8260.1830.71810.4
CUT3R0.6300.4590.1540.7600.4140.9590.1500.8860.0310.9656.7
FLARE0.4960.4180.2320.6300.4800.8520.2120.8200.1460.8009.8
VGGT-5180.2260.6800.0360.9860.3790.9460.0200.9920.0610.9623.6
VGGT-10360.2850.5880.0960.9110.6810.9580.0620.9850.0960.9246.4
Pi3-5180.2310.6740.0450.9860.4170.9660.0470.9730.0470.9743.8
Pi3-10360.2870.6190.0610.9800.5620.9570.0560.9690.0670.9705.8
DAGE-10360.2140.6950.0340.9790.4230.9710.0300.9880.0260.9732.9
Ours-10360.2080.6850.0330.9890.3680.9400.0210.9930.0250.9751.8

Table 3. Point maps (Table 4 in manuscript).

MethodDTU ACC↓DTU Comp↓DTU NC↑7-Scenes ACC↓7-Scenes Comp↓7-Scenes NC↑ETH3D ACC↓ETH3D Comp↓ETH3D NC↑Rank↓
Fast3R3.3402.9290.6710.0950.1440.6730.8320.9780.6679.0
CUT3R4.7423.4000.6790.0930.1020.7040.6170.7470.7547.7
FLARE2.5413.1740.6840.0850.1450.6960.4640.6640.7447.1
VGGT-5181.3381.8960.6760.0440.0560.7330.2800.3050.8534.1
VGGT-10361.7682.1240.6730.1310.2010.6440.4160.5510.6788.4
Pi3-5181.1981.8490.6780.0470.0750.7420.1940.2100.8833.0
Pi3-10361.2532.1170.6750.1000.1690.6790.3270.3410.8177.0
DAGE1.1681.8110.6800.0450.0580.7250.2080.2320.8662.7
Ours-5181.1761.9250.6810.0460.0610.7180.2540.2580.8533.7
Ours-10361.1171.5620.6960.0410.0540.7290.2710.2730.8502.3
AuthorsOpenReview ↗

Response to Reviewer Afzn (Part 2/2)

Question 1. VGGT/Pi3 at matched 1036 input.

The added VGGT/Pi3-1036 runs cover pose, depth and point maps, including DTU. At the same 1036 input, Ours improves all 8 pose metrics and all 5 depth AbsRel scores over both models. Ours-518 also improves 6/8 pose metrics over VGGT-518 and ties two. The advantage therefore persists at equal input resolution, establishing an improvement beyond simply supplying more pixels.

Question 2. DAGE comparisons and Ours-518.

DAGE-LR518 with 1036 input and Ours-518 are now included in pose and point maps. Ours-1036 improves all 8 pose metrics and all 3 DTU metrics over DAGE. Each Ours resolution pair uses the same checkpoint and protocol, enabling a direct test of input resolution within our model.

We extend the evaluation to Bonn, absent from our training mixture: 55 ten-frame clips from 5 scenes. At 1036×784, Ours achieves AbsRel/δ1 of 0.0405/0.9878 versus 0.0411/0.9865 for DAGE-LR518. Separately, complete inference on 100-frame CO3Dv2 clips at 1036×588 reaches 38.56 versus 20.08 FPS (1.92×). Together, these comparisons demonstrate dataset-unseen accuracy and practical high-resolution efficiency.

Table 1. Bonn accuracy / CO3Dv2 resources.

Accuracy: Bonn, inputs 518×392/1036×784, GT 1036×784, one scale/clip, same Ours checkpoint. Resources: CO3Dv2, 100 frames, 518×294/1036×588, RTX 5090, full inference; memory in GB.

Method/long side (px)AbsRel↓δ1↑FPS↑Mem↓
VGGT/5180.04950.972344.6917.51
VGGT/10360.07340.95113.1228.98
Pi3/5180.04210.981556.1117.42
Pi3/10360.04560.98005.1923.35
DAGE (LR252)/5180.05590.977150.7311.98
DAGE (LR252)/10360.05680.977226.2219.21
DAGE (LR518)/10360.04110.986520.0820.27
Ours/5180.04180.979840.0919.32
Ours/10360.04050.987838.5620.11

Question 3. Training exposure, three-run variation and ranking.

Across three independent training runs, Ours-1036 retains first-place mean rank in pose, depth and point maps in every run against the reported baselines. The advantage is also supported by absolute errors: DTU completion is 1.562 ± 0.0036 versus 1.811 for DAGE, a 13.7% reduction that is large relative to the observed training variability. CO3Dv2/RE10K AUC30 sample SDs are both 0.0265 percentage points; depth AbsRel sample SDs are 0.0010/0.0020/0.0020 on ETH3D/BlendedMVS/ScanNet++. We will provide complete per-metric mean ± sample SD and all three runs in the supplement.

All evaluation scenes/sequences are held out from training; † denotes use of a separate training partition of the source dataset. We have recomputed the expanded Rank columns using tied midranks and equal metric weights. Together, the repeated first-place rankings and the larger absolute-error improvements support the reliability of the overall comparison beyond individual small margins.

Question 4. Camera pathway in Section 3.2 and Appendix B.1.

High-resolution information does reach the Camera Head: HDE → Feature Injector → Aggregator → fused features FgF_g → Camera Head, as shown in Figure 1. The “518” description in Section 3.2 refers to the global token grid, while “1036” in Appendix B.1 refers to the external image input. Global attention uses the low-resolution grid after HDE injection. We will replace “Operating exclusively on Ilow” and state this full path in both sections.

The paired rows use the same checkpoint and protocol, changing only external input resolution. From 518 to 1036, AUC30 rises 0.31/0.68 percentage points on CO3Dv2/RE10K; five trajectory metrics improve and one ties. This directly tests the contribution of higher-resolution input to camera prediction.

Question 5. Eq. 8 and per-sequence scale alignment.

We will correct the denominator notation in Eq. 8: AbsRel=N−1∑i∣sd^i−di∣/di\mathrm{AbsRel}=N^{-1}\sum_i|s\hat d_i-d_i|/d_i, where di>0d_i>0 is GT depth and d^i\hat d_i is predicted depth. Evaluation follows CUT3R: one scale is fitted per sequence and shared across all frames, without a shift. The implementation already uses the GT denominator, so this correction leaves the reported accuracy unchanged.

As the discussion deadline approaches, we would appreciate your feedback on the added matched-1036-input comparisons, three-run results, and clarifications of training/test separation and depth evaluation. Across the three training runs, our method retains the best mean rank in all three tasks against the reported baselines. Could you let us know whether this evidence addresses your concerns about the fairness and reliability of the comparisons, or whether any specific issue needs further clarification? Thank you for your time and consideration.

Comments on the paper

1 comment

Dear Reviewers, the authors have posted their comments in response to the reviews. Please make an effort to discuss your review and the authors' response. - regards AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.