acceptodds
Under review as a conference paper at ICLR 2027

AESplat: Advancing Pose-Free Feed-Forward 3D Gaussian Splatting via Decoupled Appearance Modeling

Abstract

Pose-free feed-forward 3D Gaussian Splatting (3DGS) has demonstrated remarkable potential for generalized novel view synthesis. However, existing methods typically predict Gaussian appearance attributes represented by spherical harmonics (SH) in the same manner, overlooking the fundamental distinction between view-independent and view-dependent appearance, which results in suboptimal rendering quality. In this paper, we present AESplat, a novel and general framework for pose-free feed-forward 3DGS that introduces an effective decoupled appearance modeling strategy based on an analysis of SH, enabling higher-quality rendering. Specifically, AESplat directly derives the zeroth-order SH coefficient, which represents the base view-independent appearance component, from the input images without training. The higher-order SH coefficients are subsequently predicted by a shallow multilayer perceptron equipped with two efficient 3D-aware inductive biases to model view-dependent appearance variations. Extensive experiments across multiple datasets demonstrate that our method significantly outperforms state-of-the-art approaches, achieving a dB improvement in PSNR over the pose-free method NAS3R and a dB improvement over the pose-required method DepthSplat on the RealEstate10K dataset.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
novel view synthesis, 3D Gaussian Splatting, pose-free feed-forward reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_wAFWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AESplat improves pose-free feed-forward 3D Gaussian Splatting through a decoupled appearance prediction strategy. Built on NAS3R, it fixes each pixel-aligned Gaussian’s zeroth-order SH coefficient using the corresponding input RGB value, and predicts higher-order coefficients with a shallow MLP. The MLP receives geometric relationships to the context cameras, warped cross-view colors, and intermediate geometry features. Experiments show consistent improvements over NAS3R on RealEstate10K and ACID, gains in three cross-dataset evaluations, and successful integration into two additional baselines, with a small increase in reconstruction time.

Critical strengths
  1. Consistent and meaningful empirical improvements. AESplat improves NAS3R by 0.803 dB on RealEstate10K and 0.847 dB on ACID, with corresponding improvements in SSIM and LPIPS. The gains also hold across three zero-shot evaluations and when the method is applied to NAS3R-m and SPFSplatV2 (Tables 1–3). This provides useful evidence that the approach benefits more than one dataset or implementation.

  2. A simple design with a favorable quality–efficiency trade-off. The method retains a standard Gaussian representation that can be reused across target viewpoints. Table 7 reports only 5 ms of additional reconstruction time and essentially unchanged rendering throughput. The combination of simplicity, consistent gains, and limited overhead makes the approach practically useful.

Critical weaknesses
  1. The novelty and benefit of the specific SH decomposition are not fully established. RGB-anchored color prediction has precedents in Splatt3R, and the direction and log-distance encoding follows ViewSplat. The main distinctive choice is therefore the fixed RGB-derived DC term combined with context-conditioned higher-order SH prediction. Table 4 shows that the components help, but does not compare against the same appearance head with the same GVSRE/WCM inputs while predicting all SH coefficients or allowing a learned correction to the DC term. Moreover, these ablations use 50K steps, whereas the main models use 400K. The results do not yet separate the benefit of this decomposition from additional conditioning or easier optimization. As acknowledged in Section 5.4, source RGB also only approximates view-independent appearance.

  2. The advantage over the closest appearance-modeling comparison is limited. In Table 6, the appearance-only NAS3R+ViewSplat adaptation achieves 26.653 dB on RealEstate10K, compared with 26.691 dB for AESplat. This small difference does not establish a substantial quality advantage over an existing appearance refinement strategy. AESplat’s reusable representation is appealing, but the claimed consistency advantage is not directly measured, and the efficiency comparison does not include NAS3R+ViewSplat. This limits the evidence for a broader advance over closely related approaches.

Questions for authors
  1. Controlled validation: With the same MLP, GVSRE/WCM inputs, and optimization settings, how does the method compare with jointly predicting all SH coefficients or learning a residual correction to the RGB-derived DC term? A key comparison at the full training budget would help establish whether the specific decomposition remains beneficial after convergence.

  2. Closest comparison: How exactly is ViewSplat adapted in Table 6, including which modules are trained and which attribute updates are retained? Given the small RealEstate10K difference, can you substantiate the claimed consistency advantage or quantify the total runtime advantage when rendering multiple target views?

  3. Protocol clarification: Please specify the initialization and pretrained weights used for AESplat and its NAS3R baseline, and confirm that their training settings and use of camera intrinsics are matched. Please also clarify how test-time target camera parameters are obtained and at what output resolution the metrics are computed for the compared methods.

2 comments
AuthorsOpenReview ↗

Response to Reviewer wAFW

We thank reviewer wAFW for the detailed review and helpful suggestions. For clarity, we abbreviate Weakness, Question, and Answer as W, Q, and A, respectively.

W1 & Q1: Novelty and Advantages of the Decoupled SH Modeling Strategy

A1.1: Differences from and Advantages over Splatt3R.

We acknowledge that Splatt3R has already adopted I2DC and uses a Gaussian Head to predict residual corrections to the zeroth-order SH coefficients. However, the ablation studies in the original Splatt3R paper show that additionally predicting higher-order SH coefficients with the Gaussian Head leads to degraded rendering quality. This observation further supports our motivation that a unified prediction strategy is not optimal.

In contrast, the key innovation of AESplat lies in decoupling view-independent and view-dependent appearance modeling and adopting different modeling strategies for these two appearance components. This allows the low- and high-order SH components to play complementary roles, thereby improving the overall appearance modeling capability.

A1.2: Additional Controlled Ablation Studies

We conduct comprehensive ablation studies on the ACID dataset using the full training schedule of 400K iterations, with results reported in Table A. Despite the differences in training datasets and schedules compared to the initial ablation studies conducted on RealEstate10K with 50K iterations, the performance trends of individual components remain consistent. This further demonstrates the stability of our ablation findings.

Furthermore, to investigate the effectiveness of decoupled modeling of low- and high-order SH coefficients, we design two alternative strategies under the same MLP architecture, GVSRE/WCM inputs, and optimization settings. Both variants are trained for the full 400K iterations, with results reported in Table B:

  1. Joint Prediction of All SH Coefficients: We remove the I2DC module from AESplat and increase the output dimension of the MLP by three, enabling it to jointly predict the low-order SH coefficients. This variant leads to degraded performance, indicating that treating low- and high-order SH coefficients in the same manner is suboptimal.

  2. RGB Residual-based DC Correction: For each Gaussian, we extract the RGB values of the center pixel and its surrounding 3×33\times3 neighborhood from its source image. These nine RGB values, together with the Gaussian's three-axis scales, are fed into an independent MLP to predict a three-channel color residual. The residual is then applied in RGB space, and the corrected color is converted into the zeroth-order SH coefficient. This strategy yields a modest performance improvement, suggesting that separately modeling the correction of the zeroth-order SH coefficient and the prediction of higher-order coefficients is more effective.

Overall, these results further validate the rationale and effectiveness of AESplat's decoupled modeling strategy for low- and high-order SH coefficients.

Table A. Ablation studies on ACID with the full 400K training iterations.

MethodPSNR ↑SSIM ↑LPIPS ↓
Full27.6790.8250.151
w/o GVSRE27.3630.8220.153
w/o WCM27.4920.8230.152
I2DC only26.7160.8090.163
Baseline (NAS3R)26.8320.8130.160
Baseline (NAS3R) + I2DC26.9510.8150.159

Table B. Comparison of different SH modeling strategies.

MethodPSNR ↑SSIM ↑LPIPS ↓
AESplat (Ours)27.6790.8250.151
Full SH Prediction27.0620.8170.157
Add DC Residual27.7310.8260.150

W1 & Q2: Comparison with the Existing Appearance Modeling Method ViewSplat

A1: Adapting ViewSplat to NAS3R. ViewSplat is a recently proposed refinement method built upon pretrained pose-free feed-forward 3DGS models. Specifically, it introduces a shallow View MLP to predict view-dependent residuals conditioned on the target camera pose, which are used to refine the initial Gaussian representation produced by the pretrained 3DGS model. This enables adaptive adjustment of all Gaussian attributes, including 3D centers, scales, rotations, opacities, and SH coefficients, for each target viewpoint. In addition, a DPT head is attached after the geometry transformer to predict the parameters of the View MLP. During training, only the View MLP and the DPT head are trainable, while all other components of the pretrained model remain frozen.

To ensure a fair comparison, we integrate ViewSplat with NAS3R and restrict its attribute refinement to SH coefficients only, retaining its view-dependent residual prediction and refinement of SH coefficients while excluding updates to Gaussian positions, opacities, rotations, and scales.

To further demonstrate the advantages of our method, we provide additional comparisons between NAS3R+ViewSplat and AESplat in terms of cross-dataset generalization and computational efficiency. All methods are trained on RealEstate10K and evaluated in a zero-shot setting on ACID, DL3DV, and ScanNet++. As shown in Table C, AESplat consistently outperforms NAS3R+ViewSplat across all metrics on ACID, DL3DV and ScanNet++. These results demonstrate the stronger overall cross-dataset generalization capability of our decoupled appearance modeling strategy.

Furthermore, as shown in Table D, using the same NVIDIA RTX 4090 GPU, AESplat achieves an inference time of 0.074 s, slightly lower than the 0.076 s of NAS3R+ViewSplat. More notably, AESplat achieves a rendering speed of 333 FPS, approximately 2.33 times that of NAS3R+ViewSplat (143 FPS). These results demonstrate that AESplat achieves substantially higher rendering efficiency while maintaining a comparable inference time.

Table C. Cross-dataset generalization comparison between NAS3R+ViewSplat and AESplat. All methods are trained on RealEstate10K and evaluated in a zero-shot setting on ACID, DL3DV, and ScanNet++.

MethodACID PSNR ↑SSIM ↑LPIPS ↓DL3DV PSNR ↑SSIM ↑LPIPS ↓ScanNet++ PSNR ↑SSIM ↑LPIPS ↓
NAS3R+ViewSplat27.1880.8150.15920.2400.6390.26721.7050.8040.206
AESplat (Ours)27.3750.8190.15620.4300.6480.25921.8920.8120.199

Table D. Comparison of computational efficiency between NAS3R+ViewSplat and AESplat on RealEstate10K using an NVIDIA RTX 4090 GPU.

MethodInference Time (s) ↓Rendering Speed (FPS) ↑
NAS3R+ViewSplat0.076143
AESplat (Ours)0.074333

Q3: Clarification of the Experimental Protocol

A3: Training Settings. During training, both AESplat and the NAS3R baseline are initialized with pretrained VGGT weights. Neither method takes camera intrinsics as input; instead, the camera intrinsics are predicted by the model. All other training settings are kept identical to ensure a fair comparison.

Target Camera Parameter Estimation at Test Time. During testing, the context views and target view are jointly fed into the model, which predicts the camera parameters of the target view. These predicted parameters are then used to render the target viewpoint.

Output Resolution. The output resolutions used by different methods are as follows:

  • 224 × 224: YoNoSplat, SPFSplatV2-L, NAS3R, AESplat.
  • 256 × 256: pixelSplat, MVSplat, DepthSplat, Splatt3R, NoPoSplat, SelfSplat, PF3Splat, SPFSplat, SPFSplatV2, SPFSplatV2+Ours, NAS3R-m, NAS3R-m+Ours.
Reviewer_vCH3OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work investigates the problem of decoupled appearance modeling in pose-free feed-forward 3D Gaussian Splatting reconstruction. Specifically, it separates the conventional spherical harmonics (SH) representation into zeroth-order SH and higher-order SH coefficients, which represent view-independent and view-dependent appearance components, respectively. The experiments partially validate the effectiveness of the proposed modules.

Critical strengths
  1. The paper is clearly written.

  2. Considering zeroth-order SH and higher-order SH coefficients separately can indeed improve 3DGS appearance prediction to some extent.

Critical weaknesses
  1. Training Robustness: The prediction of higher-order SH coefficients for view-dependent appearance relies on accurate geometry, which raises concerns about whether training robustness can be guaranteed when following the NAS3R training paradigm. Have the authors introduced any additional pretrained models? If so, are the experimental comparisons fair?

  2. Concerns About the Zeroth-Order SH Design: The zeroth-order SH coefficients depend only on the current reference image, which is not necessarily an advantage. When there are large viewpoint differences, fixing the zeroth-order SH coefficients in this way may instead increase the difficulty of appearance prediction.

  3. Occlusion Issues: The proposed VDAM involves extensive cross-view transformations, raising concerns about their accuracy in the presence of occlusions.

  4. Limited Experimental Evidence: Although the paper claims that its main advantage lies in separately modeling view-independent and view-dependent appearance, the qualitative analysis of relevant surfaces, such as mirrors, tabletops, and glass, is limited. This raises concerns about the effectiveness of the proposed method.

  5. Limited Insight: The core contribution of this work is to introduce decoupled SH modeling into the pose-free feed-forward 3D Gaussian Splatting task, but this seems more like a combination of existing methods. The authors should more clearly explain the importance of decoupled SH modeling in feed-forward 3D Gaussian Splatting.

Questions for authors

Does the model rely on additional pretrained models during training? If not, how is training stability ensured, given that inaccurate geometry may affect camera projection?

3 comments
AuthorsOpenReview ↗

Response to Reviewer vCH3

We sincerely thank reviewer vCH3 for the careful review and valuable feedback. For clarity, we abbreviate Weakness, Question, and Answer as W, Q, and A, respectively.

W1 & Q1: Use of Pretrained Weights

A1: NAS3R supports two training settings: (1) training without initializing from pretrained 3D foundation model weights, and (2) initializing the model with pretrained 3D foundation model weights before training. In all our experiments, we follow the second setting. For a fair comparison, the NAS3R baselines are also initialized with the same pretrained backbone weights as our method. As shown in Table A, AESplat consistently outperforms NAS3R under both VGGT and MASt3R initializations. Specifically, with VGGT initialization, AESplat improves PSNR from 25.888 to 26.691, SSIM from 0.861 to 0.872, and reduces LPIPS from 0.136 to 0.128. Similarly, with MASt3R initialization, AESplat improves PSNR from 25.814 to 26.294, SSIM from 0.856 to 0.860, and reduces LPIPS from 0.149 to 0.141. These results demonstrate that the performance gains of AESplat are not merely attributed to pretrained foundation model weights, but also to the effectiveness of our proposed approach.

Table A: Comparison with NAS3R using pretrained foundation model weights.

MethodBackboneResolutionPSNR ↑SSIM ↑LPIPS ↓
NAS3RVGGT224×22425.8880.8610.136
AESplat (Ours)VGGT224×22426.6910.8720.128
NAS3R-mMASt3R256×25625.8140.8560.149
AESplat-m (Ours)MASt3R256×25626.2940.8600.141

W2: Concerns About the Zeroth-Order SH Design

A2: We clarify that fixing the zeroth-order SH coefficients does not prevent the model from adapting Gaussian appearance based on different viewpoints. According to Eq. (2), the final appearance of each Gaussian is jointly determined by its zeroth-order and higher-order SH coefficients. In our method, the zeroth-order SH coefficients are derived from the current reference image, providing a reliable initialization of the base appearance. Meanwhile, the View-dependent Appearance Prediction module predicts higher-order SH coefficients based on information across different context viewpoints to model view-dependent appearance variations. Therefore, when there are large viewpoint differences, the model can compensate for appearance variations that cannot be represented by the fixed zeroth-order SH coefficients through the predicted higher-order SH coefficients. Furthermore, our model is supervised solely by image reconstruction losses, which constrain the consistency between rendered results and target images, encouraging the model to learn appropriate view-dependent appearance corrections rather than simply relying on a fixed appearance representation.

W3: Occlusion Issues

A3: In VDAM, GVSRE encodes the spatial relationships between each Gaussian center and the camera origins of all context views, without involving visibility or occlusion reasoning. The current validity mask in WCM only checks whether a projected point is in front of the camera and within the image boundaries. Consequently, WCM may introduce incorrect color information in the presence of occlusions.

To mitigate this issue, we first initialize the model with pretrained VGGT weights, enabling it to acquire reasonable geometric reconstruction and camera pose estimation capabilities from the beginning of training, thereby providing a better geometric foundation for cross-view projection. In addition, we progressively increase the sampling interval between context views during the first 200K training iterations, allowing the model to gradually adapt to cross-view correspondences under larger viewpoint variations. More importantly, the colors provided by WCM do not directly determine the final rendered appearance. Instead, they are jointly processed with the current-view colors, image features, and geometric features by the appearance network. Through image reconstruction supervision, the model learns to exploit reliable cross-view color information while reducing its dependence on inaccurate color cues.

W4: Additional Experimental Evidence

A4: We conduct additional experiments on challenging specular datasets, RefReal and GlossyReal, to evaluate the robustness of our method under complex reflective conditions. As shown in Table B, AESplat consistently outperforms NAS3R on both datasets across all three metrics, demonstrating its stronger generalization ability to challenging reflective appearances. Moreover, as illustrated in Fig. 8 of the appendix, our method produces high-quality renderings with more faithful appearance details in challenging specular regions.

Table B. Quantitative comparison on challenging specular datasets.

MethodRefReal PSNR ↑SSIM ↑LPIPS ↓GlossyReal PSNR ↑SSIM ↑LPIPS ↓
NAS3R22.9030.6680.21720.5060.5960.234
AESplat (Ours)23.8390.6930.20220.7390.6200.218

W5: Clarification of the Key Insight

A5: Our core innovation lies in transforming Gaussian appearance modeling from a unified prediction paradigm into a decoupled modeling paradigm. This design is motivated by our analysis of the Gaussian appearance formulation in Eq. (2): the zeroth-order SH component represents the view-independent base appearance, whereas the higher-order SH components capture view-dependent appearance variations. Although these two components play distinct roles in appearance representation, existing feed-forward 3DGS methods typically predict all SH coefficients in a unified manner. In contrast, AESplat explicitly exploits this distinction and adopts different modeling strategies for the two types of appearance components. Although the conversion from RGB to DC coefficients follows the original 3DGS formulation, our contribution lies in incorporating this operation into a decoupled feed-forward appearance modeling framework and designing a dedicated strategy for view-dependent appearance modeling. Specifically, we explicitly introduce the 3D spatial relationships between each Gaussian and all input views, together with cross-view color information, to facilitate the prediction of view-dependent appearance. Extensive experiments demonstrate that our decoupled appearance modeling strategy substantially improves the novel view synthesis performance of the baseline, validating the effectiveness of our core design.

Reviewer_vCH3OpenReview ↗

I thank the authors for their response.

I have carefully reviewed the authors' response as well as the comments from the other reviewers. The authors' response has partially addressed my concerns. However, my concerns regarding Training Robustness, Limited Experimental Evidence, and Limited Insight remain unresolved. Therefore, I have decided to maintain my original score.

Reviewer_3vbxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AESplat proposes a decoupled appearance modeling strategy for pose-free feed-forward 3D Gaussian Splatting. It directly derives the zeroth-order SH coefficients from input pixel colors, while predicting higher-order coefficients with an MLP guided by GVSRE and WCM. The results show consistent improvements over NAS3R on RealEstate10K and ACID, with additional zero-shot generalization experiments. The idea is useful, but the novelty of the individual components and the strength of the experimental evidence still need further clarification.

Critical strengths
  1. Good rendering results. AESplat improves PSNR by 0.80 dB on RealEstate10K and 0.85 dB on ACID over NAS3R, with consistent improvements in SSIM and LPIPS.

  2. Some generalization evidence. The zero-shot experiments on ACID, DL3DV, and ScanNet++ are useful and show that the method can work beyond the training distribution.

  3. Reasonable efficiency. The reported inference time increases only from 0.056 s to 0.061 s, while rendering speed remains almost unchanged.

Critical weaknesses
  1. Limited novelty. Directly deriving the DC coefficient from RGB follows the standard 3DGS color initialization. GVSRE and WCM also look like relatively straightforward combinations of spatial encoding and cross-view color warping. The overall framework is useful, but the methodological advance over existing appearance modeling approaches is not fully convincing.

  2. Ablation studies are not strong enough. The ablations are conducted for 50K iterations, while the main model is trained for 400K iterations. It is unclear whether the contribution of each component remains the same after full training. More controlled comparisons would help.

  3. Fairness of comparisons. The paper uses publicly released baseline weights without retraining all methods at a unified resolution. This is understandable, but it makes the performance gains harder to attribute solely to the proposed method. The improvements also lack variance estimates or results from multiple independent runs.

  4. WCM may introduce incorrect color cues. The validity mask checks whether a projected point is in front of the camera and inside the image, but this does not fully handle occlusion. How the model deals with incorrect cross-view color correspondences remains unclear, especially in complex scenes.

  5. Some claims are too broad. AESplat performs well in cross-dataset evaluation, but it does not achieve the best PSNR on ScanNet++: NoPoSplat reports 22.136 dB, compared with 21.892 dB for AESplat. The general SOTA claim should be stated more carefully.

Questions for authors
  1. What is the key novelty beyond existing techniques? In particular, how much of the gain comes from the DC coefficient initialization itself, and how much comes from GVSRE and WCM?

  2. Can you provide stronger controlled ablations? It would be helpful to compare against simpler camera-direction embeddings and to report component-wise results after the full 400K training iterations, preferably with multiple runs.

  3. How robust is WCM to occlusion and inaccurate geometry? Since its validity mask does not appear to enforce visibility consistency, can the authors explain how incorrect warped colors are handled and provide results under challenging specular or exposure-varying conditions?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 3vbx

We sincerely thank Reviewer 3vbx for the thorough and careful review of our paper and for providing constructive feedback. For clarity, we abbreviate Weakness, Question, and Answer as W, Q, and A, respectively.

W1 & Q1: Clarification of our novelty and the contributions of individual components.

A1.1: Core Innovation. Our core innovation lies in shifting Gaussian appearance modeling from a unified prediction paradigm to a disentangled modeling paradigm. Our motivation stems from the analysis of the Gaussian appearance representation in Eq. 2: the zeroth-order SH term represents the base view-independent appearance component, while higher-order SH terms capture view-dependent appearance variations. Although these two components serve different roles in appearance representation, existing feed-forward 3DGS methods typically adopt a unified approach to predict all SH coefficients. In contrast, AESplat explicitly exploits this distinction and adopts different modeling strategies for the two types of appearance components. Although the conversion from RGB to DC coefficients follows the original 3DGS formulation, our contribution lies in incorporating this conversion into a disentangled feed-forward appearance modeling framework and designing a dedicated modeling strategy for view-dependent appearance by explicitly introducing the 3D spatial relationships between each Gaussian and all input views, as well as its cross-view color information.

A1.2: Contributions of Individual Components. We conduct comprehensive ablation studies on the ACID dataset with the full 400K training iterations, as shown in Tab. A. Introducing I2DC alone improves the baseline PSNR by 0.119 dB. Furthermore, removing GVSRE and WCM individually from the full model leads to PSNR drops of 0.316 dB and 0.187 dB, respectively.

Table A: Ablation Studies on ACID with Full 400K Training Iterations.

MethodPSNR ↑SSIM ↑LPIPS ↓
Full27.6790.8250.151
w/o GVSRE27.3630.8220.153
w/o WCM27.4920.8230.152
I2DC only26.7160.8090.163
Baseline (NAS3R)26.8320.8130.160
Baseline (NAS3R) + I2DC26.9510.8150.159

W2 & Q2: More Comprehensive Ablation Studies and Comparison with a Simpler Camera-Direction Embedding.

A2.1: More Comprehensive Ablation Studies. We conduct comprehensive ablation studies on the ACID dataset with the full 400K training iterations, as shown in Tab. A. Compared with the initial ablation studies conducted on RealEstate10K with 50K training iterations, the performance trends of individual components remain consistent despite the differences in training datasets and durations, further validating the consistent contributions of each component.

A2.2: Comparison with a Simpler Camera-Direction Embedding. We replace GVSRE with the simpler Plücker Ray Embedding for comparison, as shown in Tab. B. GVSRE achieves better performance across all three metrics, primarily because it explicitly models the 3D spatial relationships between each Gaussian and all input views, aggregating multi-view geometric information to provide more comprehensive spatial constraints for view-dependent appearance prediction. Furthermore, the model maintains competitive performance when GVSRE is replaced with Plücker Ray Embedding, demonstrating the compatibility of our disentangled modeling paradigm, which does not rely on a specific spatial-relation encoding strategy.

Table B: Comparison of Different Camera-Geometry Embedding Strategies on ACID.

MethodPSNR ↑SSIM ↑LPIPS ↓
Ours (GVSRE)27.6790.8250.151
Ours (Plücker Ray Embedding)27.6040.8240.152

W3: Fairness of Comparisons and Multiple Independent Runs.

A3.1: Fairness of Comparisons. We follow the experimental settings of the NAS3R baseline, adopting the corresponding input resolutions and backbone configurations, including 224×224 and 256×256. To further validate the performance of our method, we conduct comparisons using the same backbone and input resolution, as shown in Tab. C. With the VGGT backbone at a resolution of 224×224, AESplat outperforms NAS3R by 0.803 dB in PSNR. With the Mast3R backbone at a resolution of 256×256, AESplat-m improves PSNR by 0.480 dB over NAS3R-m, while also achieving better SSIM and LPIPS. These results further demonstrate the performance advantages of AESplat under comparable experimental settings.

Table C: Comparison with NAS3R under Matched Backbone and Input Resolution Settings.

MethodBackboneResolutionPSNR ↑SSIM ↑LPIPS ↓
NAS3RVGGT224×22425.8880.8610.136
AESplat (Ours)VGGT224×22426.6910.8720.128
NAS3R-mMast3R256×25625.8140.8560.149
AESplat-m (Ours)Mast3R256×25626.2940.8600.141

A3.2: Multiple Independent Runs. To evaluate the stability of our method, we conduct two additional independent runs on the ACID dataset, resulting in three runs in total, as shown in Tab. D. The PSNR values across the three runs are 27.679, 27.688, and 27.676 dB, respectively, exhibiting minimal variation. SSIM and LPIPS also remain highly consistent across runs, demonstrating the good reproducibility and stability of AESplat.

Table D: Results of Three Independent Runs on ACID.

RunPSNR ↑SSIM ↑LPIPS ↓
127.6790.8250.151
227.6880.8240.152
327.6760.8250.151

W4 & Q4: WCM Robustness

A4: We thank the reviewers for raising this important concern. We agree that the current validity mask in WCM only checks whether a projected point is in front of the camera and within the image boundaries, without explicitly enforcing visibility consistency. Consequently, WCM may introduce incorrect color information in the presence of occlusions or inaccurate geometry.

To mitigate this issue, we employ pretrained VGGT weights to initialize the model, providing a reasonable geometric foundation for cross-view projection from the beginning of training. In addition, the frame interval between context views is progressively increased during the first 200K training iterations, allowing the model to gradually adapt to cross-view correspondences under larger viewpoint variations. More importantly, WCM-provided colors do not directly determine the final rendered appearance. Instead, they are jointly processed with the current-view colors, image features, and geometric features by the appearance network. Supervision from the image reconstruction losses enables the model to exploit reliable cross-view color cues while reducing its dependence on inaccurate ones. Nevertheless, these mechanisms do not guarantee the complete elimination of incorrect correspondences.

Furthermore, we conduct additional experiments on challenging specular datasets, RefReal and GlossyReal, to evaluate the robustness of our method under complex reflective conditions. As shown in Table E, AESplat consistently outperforms NAS3R on both datasets across all three metrics, demonstrating its stronger generalization ability to challenging reflective appearances. Moreover, as illustrated in Fig. 8 of the appendix, removing WCM leads to degraded rendering quality in these challenging scenes, further highlighting the contribution of WCM to modeling complex view-dependent appearances.

Table E. Quantitative comparison on challenging specular datasets.

MethodRefReal PSNR ↑SSIM ↑LPIPS ↓GlossyReal PSNR ↑SSIM ↑LPIPS ↓
NAS3R22.9030.6680.21720.5060.5960.234
AESplat (Ours)23.8390.6930.20220.7390.6200.218

W5: Broad Generalization Claim

A5: We have revised the original statement:

As shown in Tab. 2 and Fig. 5, our method achieves the best overall performance across all three datasets, outperforming existing methods on nearly all metrics.

to provide a more precise description:

As shown in Tab. 2 and Fig. 5, our method achieves the best performance across all evaluated metrics on ACID and DL3DV. On ScanNet++, our method also demonstrates strong performance, with its PSNR only slightly lower than that of NoPoSplat.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.