acceptodds
Under review as a conference paper at ICLR 2027

CAGE-GS: Camera-Aware Geometry Enhancement for Generalizable Sparse-View Human Gaussian Splatting

Abstract

Sparse-view observations provide limited geometric evidence for generalizable human reconstruction, making reliable cross-view correspondence and view-consistent Gaussian prediction difficult. Existing methods either depend on parametric human templates or rely on depth estimates without sufficient cross-view evidence, leading to incomplete or spatially inconsistent geometry and unreliable cross-view aggregation. We present CAGE-GS, a generalizable framework that progressively transforms coarse geometry into reliable cross-view evidence for Gaussian reconstruction. Specifically, a Camera-Aware Warp Refiner establishes camera-grounded correspondence from coarse inverse depth and uses cross-view semantic consistency to guide iterative geometry refinement. The refined geometry then guides a Projection-Aware Cross-View Attention to construct spatially consistent cross-view candidates and selectively aggregate complementary multi-view features for Gaussian prediction. Together, these components turn coarse geometry into reliable camera-grounded evidence for geometry-consistent Gaussian prediction from sparse calibrated views, without relying on parametric human templates or ground-truth depth. Across three benchmarks under the four-view setting, CAGE-GS improves PSNR and SSIM by up to 3.34 dB and 0.0164, respectively, while reducing LPIPS by up to 22.8%, with consistent advantages across varying input-view configurations.

Primary area:
generative models
Keywords:
3D Gaussian Splatting, Novel View Synthesis, Sparse-View Human Reconstruction, Camera-Aware Geometry, Cross-View Correspondence

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_PEUxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents CAGE-GS, a feed-forward framework for human 3D Gaussian reconstruction from sparse calibrated views. Its main contributions are Camera-Aware Warp Refinement (CAWR), which iteratively improves depth estimation, and Projection-Aware Attention (PAA), which enhances geometrically consistent cross-view feature aggregation. Experiments demonstrate improved reconstruction quality and generalization to unseen human subjects and different datasets without per-subject optimization.

Critical strengths
  1. Improved Reconstruction Quality: CAGE-GS effectively reduces 3DGS artifacts around self-occluded body regions and produces sharper facial, hand, and clothing details. These improvements are particularly valuable for sparse-view human reconstruction, where geometric ambiguity often causes visible distortions.

  2. Generalizable Sparse-View Reconstruction: The framework enables feed-forward reconstruction of unseen human subjects without requiring parametric body templates or ground-truth depth. The reported cross-dataset results further demonstrate promising generalization across different human datasets.

Critical weaknesses
  1. Dependence on Accurate Camera Calibration: Both CAWR and PAA rely on calibrated camera poses, but the paper does not evaluate robustness to pose errors or synchronization inaccuracies, raising concerns about practical applicability.
Questions for authors
  1. Choice of VGGT Backbone:
  • Since CAGE-GS uses known camera poses rather than VGGT's predicted poses, what motivates the choice of VGGT over alternative geometry foundation models such as Pi³ or Depth Anything 3?

  • Could CAWR and PAA operate with DINOv2 features and Depth Anything estimates, similar to geometry-guided reconstruction frameworks such as GeoLRM (NeurIPS 2024)? How much of the improvement depends on VGGT rather than the proposed modules?

  • If VGGT's predicted camera poses differ significantly from the provided calibrated poses, how is geometric consistency between the predicted depth and known camera geometry ensured?

  1. Computational Efficiency: CAGE-GS requires 467.11 ms per reconstruction under the four-view setting. What are the individual runtime costs of VGGT, CAWR, PAA, and Gaussian prediction? Which component constitutes the primary computational bottleneck, and how does inference time scale with the number of input views?

  2. The cross-dataset evaluation in Appendix Table 6 provides important evidence supporting the paper's generalization claim. I suggest moving these results to the main paper, clearly distinguishing cross-subject reconstruction from cross-dataset generalization.

1 comment
AuthorsOpenReview ↗

Response to Reviewer PEUx

We thank the reviewer for recognizing our reconstruction quality and cross-dataset generalization. We provide additional experiments and clarifications on backbone choice, camera calibration, runtime, and generalization, with corresponding updates incorporated into the revised manuscript and appendix.

Re: Geometry Backbone Choice and Generality (Q1).

We select VGGT for its depth initialization and geometry-aware multi-view features, rather than its camera prediction capability, as calibrated cameras are available in our setting. Importantly, CAWR and PAA refine geometry and aggregate cross-view features through camera-grounded projections, without relying on VGGT-specific representations. To examine whether our gains depend on VGGT, we replace its depth and features with DA3-LARGE-1.1 and compare a frozen-backbone Gaussian baseline against our full pipeline. Table 1 shows substantial PSNR improvements with both VGGT (+8.32 dB) and DA3 (+7.35 dB), demonstrating consistent architectural benefits across different geometry backbones.

Table 1. Backbone and architecture comparison.

Exp.Frozen backboneOur componentsPSNR ↑SSIM ↑LPIPS ↓
AVGGTNone22.890.92590.1062
BVGGTFull31.210.97430.0334
CDA3-LARGE-1.1None22.440.92630.1008
DDA3-LARGE-1.1Full29.790.96830.0398

Re: Camera Calibration and Depth Consistency (W1/Q1).

Following prior human reconstruction methods (e.g., GHG, RoGSplat, and GHGS-MVSC), we use calibrated cameras for fair comparison. To assess calibration dependence, we replace them with VGGT-predicted reference cameras and jointly train the camera head. As shown in Table 2, our method achieves 27.24 dB PSNR with predicted cameras, demonstrating its feasibility under estimated camera geometry. Regarding depth–camera consistency, CAWR iteratively updates cross-view correspondences under the adopted camera parameters and refines depth accordingly, promoting consistency between the estimated geometry and camera projections.

Table 2. Comparison of calibrated and VGGT-predicted reference cameras.

Reference camerasPSNR ↑SSIM ↑LPIPS ↓
Calibrated31.210.97430.0334
VGGT-predicted27.240.94620.0785

Re: Runtime Breakdown and View-Count Scaling (Q2).

We profile inference with 2–5 input views on an RTX 4090 at 512×512 (FP16, batch size 1), using 100 samples, 20 warm-up runs, and three measurement passes. The repeated four-view latency is 466.92 ms, consistent with the reported 467.11 ms.

Table 3 reports the component-wise runtime and view-count scaling. At four views, PAA accounts for 207.85 ms (44.5% of total latency), followed by VGGT at 185.84 ms (39.8%). Across 2–5 views, the largest runtime contributor shifts from VGGT to PAA, reflecting the increasing cost of cross-view feature aggregation. End-to-end latency increases from 198.88 to 655.00 ms, while peak memory grows from 5.00 to 6.97 GiB. Component timings are measured using CUDA Events, whereas end-to-end latency is independently measured with GPU synchronization.

Table 3. Inference efficiency across input-view counts. PAA includes four feature-level calls.

Measurement2 views3 views4 views5 views
VGGT (ms)105.53146.71185.84242.34
CAWR (ms)21.2833.1843.6054.16
PAA (ms)50.37116.10207.85325.20
Gaussian prediction (ms)14.5119.2923.5828.49
Rendering (ms)2.342.462.542.63
End-to-end (ms)198.88322.33466.92655.00
FPS5.033.102.141.53
Peak memory (GiB)5.005.666.316.97

Re: Cross-Subject and Cross-Dataset Generalization (Q3).

We thank the reviewer for this suggestion. In the revised manuscript, we have moved Appendix Table 6 to the main paper and explicitly distinguished cross-subject generalization (unseen identities within the training dataset) from cross-dataset generalization (THuman2.0-trained models evaluated on 2K2K/HuMMan without fine-tuning).

Reviewer_1KZoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

As I understand it, this paper proposes a generalizable human GS framework using sparse calibrated images. It first extracts coarse depth and multi-view features from a frozen VGGT. The Camera-aware warp refiner (CAWR) iteratively refines the depth using camera-based reprojection and cross-view semantic consistency. Then, projection-aware cross-view attention (PAA) uses the refined geometry to aggregate spatially consistent features from other views. Finally, these features are decoded into 3D Gaussians for NVS.

Critical strengths

S1. Strong quantitative and qualitative results. The paper achieves strong results across multiple human reconstruction datasets and different numbers of input views. In particular, I appreciate the cross-domain experiments where the model trained on THuman2.0 is directly evaluated on other datasets without fine-tuning. The improvements are quite meaningful, especially in sparse-view settings.

2. Reasonable framework without human templates or ground-truth depth. I think the overall framework is intuitive and easy to follow. Using cross-view consistency to refine geometry and then using the refined geometry for feature aggregation makes sense. The method also avoids relying on fitted human templates or ground-truth depth, which is practically useful

Critical weaknesses

W1. Limited methodological novelty. My main concern is the limited technical novelty of the proposed framework. Camera-based reprojection, iterative depth refinement using cross-view consistency, and geometry-guided attention have been widely explored in previous multi-view reconstruction methods. While combining these components is reasonable, I find it difficult to identify a technically distinctive contribution beyond their integration.

W2. Also, most of the proposed techniques seem fairly general and not particularly specific to human Gaussian reconstruction. It is unclear why the method is designed and evaluated specifically for humans, rather than being presented as a more general geometry-guided reconstruction framework. I think the paper could better clarify what challenges are uniquely addressed in human reconstruction.

W3. The method heavily relies on the strong pretrained VGGT backbone, but the paper does not sufficiently isolate the benefits of the proposed modules from this prior. In particular, removing CAWR causes a very large performance drop, but this does not necessarily demonstrate that the proposed iterative refinement is better than simpler existing geometry refinement approaches.

Questions for authors

Q1. Since VGGT can also predict camera parameters, why does the proposed framework assume calibrated camera poses? How sensitive is the method to camera pose errors, and could VGGT-predicted cameras be used instead?

Q2. What makes the proposed geometry refinement and cross-view attention particularly suitable for human Gaussian reconstruction? Could the same framework be directly applied to general object or scene reconstruction?

Q3. The improvement from PAA seems relatively small (0.28 dB PSNR). Could the authors explain why this module provides only a limited gain, and compare it against simpler geometry-guided feature aggregation methods?

Q4. Could the authors provide a baseline using the frozen VGGT depth and features with a simpler Gaussian prediction head? This would help separate the contribution of the proposed architecture from the pretrained backbone.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 1KZo

We thank the reviewer for recognizing our reconstruction quality, cross-domain generalization, and reduced supervision requirements. We address the concerns below, with additional experiments and clarifications included in the revised manuscript or appendix.

Re: Technical Contribution and Controlled Comparisons (W1/W3/Q3)

(1) Methodological Distinctions.
Our contribution lies in a geometry–correspondence feedback mechanism for Gaussian reconstruction, rather than individual reprojection or depth refinement operations. CAWR iteratively updates camera-grounded cross-view correspondences and refines depth using their feature evidence. The refined geometry then guides PAA's projection-aware feature aggregation and Gaussian placement.

(2) Controlled Comparisons.
Table 1 compares simpler CAWR/PAA replacements with other components fixed. Full CAWR outperforms the single-step residual CNN replacement by 2.08 dB PSNR, supporting iterative cross-view refinement. Full PAA exceeds simple geometry-guided aggregation by 1.46 dB, highlighting the benefit of consistency-aware feature selection beyond geometric alignment.

Table 1. Simpler CAWR/PAA replacements using VGGT; other modules are retained.

ModuleSettingPSNR ↑SSIM ↑LPIPS ↓
CAWRNo CAWR (original Table 4)22.870.92710.1026
Single-step residual CNN replacement29.130.96250.0502
Full CAWR (original Table 4)31.210.97430.0334
PAANo PAA (original Table 4)30.930.97280.0340
Simple geometry-guided aggregation29.750.96790.0353
Full PAA (original Table 4)31.210.97430.0334

(3) PAA Contribution Analysis.
Building on CAWR-refined geometry, PAA further improves cross-view feature aggregation, increasing PSNR from 30.93 to 31.21 dB while improving SSIM and LPIPS. In contrast, simple geometry-guided aggregation reduces PSNR to 29.75 dB, below the no-PAA baseline (Table 1). This indicates that accurate geometric projection alone does not guarantee effective feature fusion, highlighting the benefit of PAA's consistency-aware feature selection.

(4) Fine-Grained CAWR Analysis.
Table 2 further isolates CAWR's three mechanisms. Cross-view warping improves PSNR by 5.49,dB, iterative updates add 1.90,dB, and semantic weighting contributes another 0.95,dB, demonstrating their complementary benefits beyond basic geometric correction.

Table 2. Fine-grained ablation of CAWR.

Depth refinerWarpIterativeSemanticPSNR ↑SSIM ↑LPIPS ↓
No CAWR–––22.870.92710.1026
Single-step Warp✓––28.360.95800.0541
Iterative Warp✓✓–30.260.97040.0368
Full CAWR✓✓✓31.210.97430.0334

Re: Human Reconstruction Motivation and Scope (W2/Q2)

Sparse-view human reconstruction faces challenges from articulation, self-occlusion, and complex clothing geometry, causing ambiguous correspondences and incomplete observations. These challenges motivate CAWR's camera-grounded geometry refinement and PAA's projection-aware feature aggregation, making human reconstruction a challenging testbed with established benchmarks. While our method requires no human-specific priors, extending its evaluation to broader scenes remains a natural direction for future work.

Re: Backbone vs. Architecture (W3/Q4)

To distinguish backbone effects from architectural gains, we evaluate VGGT and DA3-LARGE-1.1 with a simple Gaussian head and our full pipeline (Table~3). First, replacing VGGT with DA3 under the simple baseline reduces PSNR by 0.45,dB, indicating that backbone substitution alone cannot explain our gains. Second, our architecture improves PSNR by 8.32 and 7.35,dB with VGGT and DA3, respectively, demonstrating substantial gains across backbones. These results support the effectiveness of our architecture beyond backbone-specific geometry priors.

Table 3. Backbone and Architecture Comparison.

Exp.Frozen backboneOur componentsPSNR ↑SSIM ↑LPIPS ↓
AVGGTNone22.890.92590.1062
BVGGTFull31.210.97430.0334
CDA3-LARGE-1.1None22.440.92630.1008
DDA3-LARGE-1.1Full29.790.96830.0398

Re: Calibrated vs. Predicted Cameras (Q1)

Following prior human reconstruction methods (e.g., GHG, RoGSplat, and GHGS-MVSC), we use calibrated cameras for fair comparison. We further evaluate VGGT-predicted cameras (Table 4), demonstrating that our method remains effective without calibrated reference cameras.

Table 4. Comparison of calibrated and VGGT-predicted reference cameras.

Reference-camera sourcePSNR ↑SSIM ↑LPIPS ↓
Calibrated cameras31.210.97430.0334
VGGT-predicted cameras27.240.94620.0785

I appreciate the authors' detailed response. My primary concerns are mostly addressed. However, I have a few more questions.

As the author noted, I think the fine details such as clothing, hair, facial expression and other details require more Gaussians than relatively textureless regions.

Given that the method uses pixel-aligned Gaussians, how could each Gaussian is reconstructed to represent details more carefully? Is it possible to allocate more Gausians to regions with fine details, or does it primarily improve the placement and parameters of the existing gaussians?

Another remaining question is, if the approach can be generally applied to other scenes, why is the evaluation limited to human avatars? What aspects of CAGE-GS account for its better performance than the template-based baselines? Could CAGE-GS also improve reconstruction when used with an SMPL or SMPL-X template?

AuthorsOpenReview ↗

Response to the Follow-up Questions of Reviewer 1KZo (1/2)

We thank the reviewer for the constructive follow-up questions and for acknowledging that our previous responses have addressed the main concerns. We provide further clarification on Gaussian allocation, cross-category generalization, and template compatibility below.

Re: Gaussian Allocation and Detail Reconstruction

(1) Gaussian Refinement and Comparative Evidence. CAGE-GS improves fine details by refining the geometry and attributes of existing Gaussians rather than adaptively allocating more Gaussians, retaining one per foreground reference pixel. Importantly, pixel alignment does not fix Gaussian positions in 3D space. CAWR refines depth for unprojection, while the Gaussian prediction head estimates position offsets, rotations, anisotropic scales, and opacities. PAA further integrates consistency-aware cross-view features and local RGB information to improve Gaussian prediction in geometrically and visually complex regions.

Table 1 provides further comparative evidence. ROI-LPIPS and ROI-PSNR evaluate texture-rich regions automatically selected from GT images using Sobel gradients, while surface distance measures the mean distance from Gaussian centers to the GT clothed surface. Compared with the evaluated baselines, CAGE-GS achieves the lowest ROI-LPIPS (0.1500) and surface distance (9.77mm), alongside the highest ROI-PSNR (24.06dB). These results demonstrate superior fine-detail preservation and more accurate alignment of Gaussian centers with the GT human surface, without increasing the Gaussian count.

Table 1. Gaussian allocation, geometric accuracy, and fine-detail reconstruction on THuman2.0 under the four-view setting.

MethodROI-LPIPS ↓ROI-PSNR ↑Surface Dist. (mm) ↓
EVA-Gaussian (adapted)0.212121.7512.77
RoGSplat0.197322.4111.45
GHGS-MVSC0.172023.0510.23
CAGE-GS (Ours)0.150024.069.77

(2) Ablation on Fine-Detail Reconstruction. Table 2 further evaluates the contributions of individual components to fine-detail reconstruction. Removing CAWR substantially reduces ROI-PSNR from 24.06 to 14.80 dB, highlighting the importance of accurate depth refinement for preserving fine details. The ablations of PAA and LE further demonstrate the complementary benefits of consistency-aware cross-view aggregation and local appearance information for fine-detail reconstruction.

Table 2. Component ablations for local detail fidelity and Gaussian geometry.

VariantROI-LPIPS ↓ROI-PSNR ↑Surface Dist. (mm) ↓
w/o CAWR0.619314.8044.41
w/o PAA0.155223.6610.32
w/o LE0.152623.559.60
Full0.150024.069.77
Reviewer_Q1qrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CAGE-GS reconstructs human 3D Gaussians from sparse calibrated images without target-task depth or parametric-body supervision. It combines a frozen VGGT prior, recurrent camera-guided inverse-depth refinement, and projection-aware semantic/geometric attention, trained with two-stage rendering supervision. Evaluation covers three datasets, two to five input views, component ablations, cross-domain transfer, and efficiency. The strongest evidence concerns sparse-input rendering quality; improved geometry itself receives only qualitative validation.

Critical strengths
  • Meaningful sparse-view gains: On THuman2.0, CAGE-GS improves three-view PSNR over GHGS-MVSC by 1.93 dB (Table 1). Appendix Table 6 also shows cross-domain gains, including 3.53 dB over RoGSplat on 2K2K, supporting usefulness beyond dataset-specific fitting.
  • Practical supervision advantage: The geometry-to-aggregation pipeline avoids target-task depth and SMPL supervision while handling unseen identities. This reduces dependence on template fitting for clothing and non-body structures, an important limitation of several compared methods.
Critical weaknesses
  • The methodological advance is not sufficiently distinguished from prior work: GHGS-MVSC already combines VGGT, depth-based 3D lifting, and semantic cross-view attention. Recurrent depth estimation and rendering-supervised geometry learning have precedents in IterMVS, GPS-Gaussian+, and DepthSplat. The particular integration may be valuable, but matched simpler alternatives and strong pretrained-depth replacements, such as Depth Anything 3, are needed to establish the benefit beyond these ingredients.
  • Geometry attribution and baseline comparability remain unresolved. The 8.34 dB CAWR ablation does not isolate scale adaptation from correspondence/recurrence; no quantitative geometry metrics or matched GT-depth reference are provided. Table 1's EVA scores exactly match EVA-Gaussian's original 1024×1024 stereo 90°/72° results, while CAGE-GS describes 512×512 four/five-view evaluation. Camera-pool versus consumed-view conventions and score provenance require reconciliation. Relevant GPS-Gaussian+, GBC-Splat and Snap-Snap comparisons are also missing; the architecture-only artifact cannot verify the core execution.
Questions for authors
  • Mechanism validation: What exactly replaces CAWR in Table 4, including inverse-depth scaling and training? With the same backbone/decoder and matched training, compare scale-aligned VGGT depth, a simple residual refiner, and CAWR, alongside GT reference-view depth as a diagnostic control. Report depth/surface errors and rendering metrics to separate geometric correction from rendering-specific adaptation.
  • Clarification needed for reproduction: The artifact specifies iterations and numerical attention weights, but omits core execution. How is normalized inverse depth restored to calibrated scale, and what are the exact correspondence/confidence, attention-score, and validity equations? Can you supply executable CAWR/PAA and the training/evaluation configuration underlying the tables?
  • Comparison validity: EVA's Table 3 reports 26.31/27.54 dB at 1024×1024 with stereo angles 90°/72°, matching Table 1's four/five-view entries. Were these quoted or recomputed? Please reconcile available cameras versus jointly consumed views, resolution, splits, and metric regions, and provide matched baseline comparisons or justify exclusions.
5 comments
AuthorsOpenReview ↗

Response to Reviewer Q1qr (1/2)

We thank the reviewer for recognizing our sparse-view gains and reduced supervision. We address the concerns through methodological clarifications, comparisons, and geometric evaluations. All additional experiments and clarifications are included in the revised manuscript or appendix.

Re: Methodological Contributions and Controlled Comparisons (W1)

(1) Methodological Distinctions.

Our key contribution is an iterative geometry–correspondence feedback mechanism that refines depth to guide projection-aware feature aggregation and Gaussian placement. Unlike conventional iterative depth estimation (IterMVS) or rendering-supervised geometry learning (GPS-Gaussian+ and DepthSplat), CAWR explicitly updates camera-grounded cross-view correspondences using semantic consistency. Unlike GHGS-MVSC's depth-lifted 3D neighborhood aggregation, PAA leverages refined geometry for projection-consistent feature aggregation and Gaussian placement.

(2) Controlled Comparisons.

Table 1 evaluates simpler alternatives under identical settings. CAWR improves PSNR by 2.08 dB over single-step Residual CNN depth correction, while PAA outperforms simple geometry-guided aggregation by 1.46 dB, supporting the benefits of iterative correspondence refinement and projection-aware aggregation.

Table 1. Comparison with simpler CAWR/PAA alternatives under identical settings.

ModuleSettingPSNR ↑SSIM ↑LPIPS ↓
CAWRNo CAWR (original Table 4)22.870.92710.1026
Single-step residual CNN replacement29.130.96250.0502
Full CAWR (original Table 4)31.210.97430.0334
PAANo PAA (original Table 4)30.930.97280.0340
Simple geometry-guided aggregation29.750.96790.0353
Full PAA (original Table 4)31.210.97430.0334

(3) Pretrained Backbone Comparisons.

To isolate architectural gains from backbone effects, we compare our full pipeline against a simple Gaussian baseline (trainable DPT decoder and Gaussian MLPs) using frozen VGGT or DA3-LARGE-1.1.

As shown in Table 2, (i) Architectural contribution: Our pipeline improves PSNR by 8.32 dB with VGGT (A→B), whereas replacing VGGT with DA3 alone reduces PSNR by 0.45 dB (A→C), indicating that the substantial gains stem from our proposed components rather than backbone replacement. (ii) Cross-backbone effectiveness: Our pipeline also improves DA3 by 7.35 dB (C→D), demonstrating consistent architectural benefits across both backbones.

Table 2. Comparison of pretrained backbones and proposed components.

Exp.Frozen backboneOur componentsPSNR ↑SSIM ↑LPIPS ↓
AVGGTNone22.890.92590.1062
BVGGTFull31.210.97430.0334
CDA3-LARGE-1.1None22.440.92630.1008
DDA3-LARGE-1.1Full29.790.96830.0398

Re: Geometry Attribution and Fine-Grained Controls (W2/Q1)

(1) Fine-Grained Mechanism Controls.

Table 3 progressively introduces cross-view warping, iterative refinement, and semantic weighting with the downstream architecture fixed. These mechanisms yield successive PSNR gains of 5.49, 1.90, and 0.95 dB, respectively, while reducing depth MAE from 0.1537 to 0.0206, demonstrating the progressive benefits of each refinement mechanism.

Table 3. Fine-grained CAWR ablation. MAE and RMSE denote depth errors.

Depth refinerWarpIter.SemanticPSNR ↑SSIM ↑LPIPS ↓MAE ↓RMSE ↓
No CAWR------22.870.92710.10260.15370.1647
Single-step Warp✓----28.360.95800.05410.04140.0533
Iterative Warp✓✓--30.260.97040.03680.02310.0318
Full CAWR✓✓✓31.210.97430.03340.02060.0300

(2) Geometry and Rendering Accuracy.

Table 4 compares geometric accuracy and rendering quality across VGGT, Residual CNN, and CAWR under identical settings, alongside a GT-depth reference. CAWR improves PSNR by 2.08 dB over Residual CNN and reduces depth MAE from 0.0351 to 0.0206, demonstrating gains in both geometric accuracy and rendering quality. GT reference depth achieves 34.44 dB PSNR (+3.23 dB over CAWR), indicating further room for geometric improvement. Depth errors are evaluated without test-GT scale fitting. The GT-depth experiment uses reference-view GT depth with a retrained downstream network.

Table 4. Geometry accuracy and GT-depth reference comparison.

Geometry providerPSNR ↑SSIM ↑LPIPS ↓MAE ↓RMSE ↓
VGGT (normalized)22.870.92710.10260.15370.1647
Residual CNN29.130.96250.05020.03510.0451
Full CAWR31.210.97430.03340.02060.0300
GT reference depth34.440.98430.0169----
AuthorsOpenReview ↗

Response to Reviewer Q1qr (2/2)

Re: Additional Baseline Comparisons (W2/Q3)

GPS-Gaussian+, GBC-Splat, and Snap-Snap adopt different input protocols (rectified stereo pairs, four stereo pairs, and front/back or five-view inputs, respectively), preventing direct reproduction under our setting. We therefore excluded them from the original main table. For completeness, Table 1 compares published THuman2.0 results under the respective input protocols. Notably, CAGE-GS with only two input views achieves higher PSNR, SSIM, and lower LPIPS than all reported baselines, demonstrating its effectiveness under sparse-view settings.

Table 1. THuman2.0 comparison under respective input protocols.

MethodNative input settingPSNR ↑SSIM ↑LPIPS ↓
GPS-Gaussian+2 selected / 8-camera pool24.720.89400.1290
GBC-Splat (Subdiv.)8 images / 4 stereo pairs25.860.89700.1188
Snap-Snap2 images, front/back22.440.88780.1324
Snap-Snap (multi-view)5-view extension24.730.91590.1067
CAGE-GS2 jointly consumed27.570.95400.0560
CAGE-GS4 jointly consumed31.210.97430.0334
CAGE-GS5 jointly consumed31.610.97560.0320

Re: EVA Protocol Alignment (W2/Q3)

We updated Table 1 of the revised manuscript with reproduced EVA-Gaussian results under the same THuman2.0 split, 512×512512\times512 resolution, and evaluation protocol as CAGE-GS, using four/five input views. CAGE-GS achieves 3.62/3.50 dB higher PSNR than EVA-Gaussian for four/five views, respectively, even without GT depth supervision.

Re: Ablation Configurations and Training Protocol (W2/Q1)

w/o CAWR directly uses normalized VGGT depth; w/o PAA feeds original VGGT tokens to DPT; w/o LE removes local RGB encoding and fusion; w/o Joint Training freezes the pretrained CAWR.

All variants follow the four-view, 150K-step Stage 2 protocol. The full model, w/o PAA, and w/o LE include 0.01La0.01\mathcal{L}_a alongside RGB/SSIM/mask losses, whereas w/o CAWR and w/o Joint Training omit it. La\mathcal{L}_a uses a frozen initial refiner rather than GT depth.

Re: Scale, Execution, and Reproducibility (W2/Q1/Q2)

We clarify the depth scaling, CAWR/PAA implementation, and training/evaluation protocols below. The complete code will be publicly released upon acceptance.

(1) Inverse-Depth Scaling.

VGGT-predicted inverse depth is normalized using foreground median statistics to match a predefined reference inverse-depth scale and clipped to [10−4,10][10^{-4},10]. Subsequently, CAWR iteratively refines the inverse-depth estimates under known camera intrinsics and extrinsics to improve cross-view geometric consistency.

(2) CAWR Correspondence and Confidence.

For at most two nearest source cameras, correspondence weights are

wsm=VsmM~s→qm(0.2+0.8csm),γqm=1∣Sq∣∑s∈Sqwsm.w_s^m=V_s^m\widetilde M_{s\rightarrow q}^m(0.2+0.8c_s^m),\quad \gamma_q^m=\frac{1}{|\mathcal S_q|}\sum_{s\in\mathcal S_q}w_s^m.

Here, VsmV_s^m enforces positive depth and in-bounds projection, M~s→qm\widetilde M_{s\rightarrow q}^m is the warped foreground mask, and csmc_s^m is cosine similarity clipped to [0,1][0,1]. Features and masks use bilinear and nearest-neighbor sampling, respectively. CAWR performs three half-resolution updates with 32-dimensional features and 64-dimensional hidden states.

(3) PAA Scores and Validity.

The attention terms in Eq. (9) are:

  • Semantic similarity: Asemj=max⁡(0,cos⁡(Sq,Sj))A_{\mathrm{sem}}^j=\max(0,\cos(S_q,S_j)).
  • Projection offset: Aoffj=∣δj∣2A_{\mathrm{off}}^j=\\|\delta_j\\|_2.
  • Depth discrepancy:

Adepj=∣ρjsampled−ρjprojected∣∣ρjsampled∣+∣ρjprojected∣+10−6.A_{\mathrm{dep}}^j=\frac{|\rho_j^{\mathrm{sampled}}-\rho_j^{\mathrm{projected}}|}{|\rho_j^{\mathrm{sampled}}|+|\rho_j^{\mathrm{projected}}|+10^{-6}}.

Their weights are (λsem,λdep,λoff)=(0.5,1,0.15)(\lambda_{\mathrm{sem}},\lambda_{\mathrm{dep}},\lambda_{\mathrm{off}})=(0.5,1,0.15). PAA samples the projection center and four axial neighbors. Valid candidates require positive projected depth, in-bounds coordinates, sampled inverse depth >10−6>10^{-6}, and foreground mask >0.2>0.2; invalid logits are set to −104-10^4 before softmax.

(4) Training and Evaluation.

We use 426 training and 100 held-out THuman2.0 identities, batch size 1, seed 1314, and AdamW (weight decay 10−510^{-5}). Learning rates are 10−410^{-4} for the backend and 10−510^{-5} for pretrained refiners during joint training. Loss weights are α=0.8\alpha=0.8, λr=1\lambda_r=1, λc=0.05\lambda_c=0.05, λp=0.03\lambda_p=0.03, λa=0.01\lambda_a=0.01, and λm=0.05\lambda_m=0.05; colored Chamfer uses predicted cross-view points.

Evaluation uses 100 held-out identities and 16 target views per identity (1,600 images). Fixed reference indices are 0,4,8,12\\{0,4,8,12\\} (four views) and 0,4,7,10,13\\{0,4,7,10,13\\} (five views), jointly processed. PSNR, SSIM, and VGG-LPIPS are averaged per image over full 512×512512\times512 black-background images, with VGGT padding cropped before evaluation.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer Q1qr, Thank you for your time and thoughtful review of our paper. As the discussion period is approaching its deadline, we would like to kindly follow up on our rebuttal and ensure that we have adequately addressed your concerns.

We would greatly appreciate any further feedback you may have. If any questions or concerns remain, we would be happy to provide additional clarification before the discussion period ends. Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.