- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
AESplat improves pose-free feed-forward 3D Gaussian Splatting through a decoupled appearance prediction strategy. Built on NAS3R, it fixes each pixel-aligned Gaussian’s zeroth-order SH coefficient using the corresponding input RGB value, and predicts higher-order coefficients with a shallow MLP. The MLP receives geometric relationships to the context cameras, warped cross-view colors, and intermediate geometry features. Experiments show consistent improvements over NAS3R on RealEstate10K and ACID, gains in three cross-dataset evaluations, and successful integration into two additional baselines, with a small increase in reconstruction time.
Critical strengths
-
Consistent and meaningful empirical improvements. AESplat improves NAS3R by 0.803 dB on RealEstate10K and 0.847 dB on ACID, with corresponding improvements in SSIM and LPIPS. The gains also hold across three zero-shot evaluations and when the method is applied to NAS3R-m and SPFSplatV2 (Tables 1–3). This provides useful evidence that the approach benefits more than one dataset or implementation.
-
A simple design with a favorable quality–efficiency trade-off. The method retains a standard Gaussian representation that can be reused across target viewpoints. Table 7 reports only 5 ms of additional reconstruction time and essentially unchanged rendering throughput. The combination of simplicity, consistent gains, and limited overhead makes the approach practically useful.
Critical weaknesses
-
The novelty and benefit of the specific SH decomposition are not fully established. RGB-anchored color prediction has precedents in Splatt3R, and the direction and log-distance encoding follows ViewSplat. The main distinctive choice is therefore the fixed RGB-derived DC term combined with context-conditioned higher-order SH prediction. Table 4 shows that the components help, but does not compare against the same appearance head with the same GVSRE/WCM inputs while predicting all SH coefficients or allowing a learned correction to the DC term. Moreover, these ablations use 50K steps, whereas the main models use 400K. The results do not yet separate the benefit of this decomposition from additional conditioning or easier optimization. As acknowledged in Section 5.4, source RGB also only approximates view-independent appearance.
-
The advantage over the closest appearance-modeling comparison is limited. In Table 6, the appearance-only NAS3R+ViewSplat adaptation achieves 26.653 dB on RealEstate10K, compared with 26.691 dB for AESplat. This small difference does not establish a substantial quality advantage over an existing appearance refinement strategy. AESplat’s reusable representation is appealing, but the claimed consistency advantage is not directly measured, and the efficiency comparison does not include NAS3R+ViewSplat. This limits the evidence for a broader advance over closely related approaches.
Questions for authors
-
Controlled validation: With the same MLP, GVSRE/WCM inputs, and optimization settings, how does the method compare with jointly predicting all SH coefficients or learning a residual correction to the RGB-derived DC term? A key comparison at the full training budget would help establish whether the specific decomposition remains beneficial after convergence.
-
Closest comparison: How exactly is ViewSplat adapted in Table 6, including which modules are trained and which attribute updates are retained? Given the small RealEstate10K difference, can you substantiate the claimed consistency advantage or quantify the total runtime advantage when rendering multiple target views?
-
Protocol clarification: Please specify the initialization and pretrained weights used for AESplat and its NAS3R baseline, and confirm that their training settings and use of camera intrinsics are matched. Please also clarify how test-time target camera parameters are obtained and at what output resolution the metrics are computed for the compared methods.
2 comments
W1 & Q2: Comparison with the Existing Appearance Modeling Method ViewSplat
A1: Adapting ViewSplat to NAS3R. ViewSplat is a recently proposed refinement method built upon pretrained pose-free feed-forward 3DGS models. Specifically, it introduces a shallow View MLP to predict view-dependent residuals conditioned on the target camera pose, which are used to refine the initial Gaussian representation produced by the pretrained 3DGS model. This enables adaptive adjustment of all Gaussian attributes, including 3D centers, scales, rotations, opacities, and SH coefficients, for each target viewpoint. In addition, a DPT head is attached after the geometry transformer to predict the parameters of the View MLP. During training, only the View MLP and the DPT head are trainable, while all other components of the pretrained model remain frozen.
To ensure a fair comparison, we integrate ViewSplat with NAS3R and restrict its attribute refinement to SH coefficients only, retaining its view-dependent residual prediction and refinement of SH coefficients while excluding updates to Gaussian positions, opacities, rotations, and scales.
To further demonstrate the advantages of our method, we provide additional comparisons between NAS3R+ViewSplat and AESplat in terms of cross-dataset generalization and computational efficiency. All methods are trained on RealEstate10K and evaluated in a zero-shot setting on ACID, DL3DV, and ScanNet++. As shown in Table C, AESplat consistently outperforms NAS3R+ViewSplat across all metrics on ACID, DL3DV and ScanNet++. These results demonstrate the stronger overall cross-dataset generalization capability of our decoupled appearance modeling strategy.
Furthermore, as shown in Table D, using the same NVIDIA RTX 4090 GPU, AESplat achieves an inference time of 0.074 s, slightly lower than the 0.076 s of NAS3R+ViewSplat. More notably, AESplat achieves a rendering speed of 333 FPS, approximately 2.33 times that of NAS3R+ViewSplat (143 FPS). These results demonstrate that AESplat achieves substantially higher rendering efficiency while maintaining a comparable inference time.
Table C. Cross-dataset generalization comparison between NAS3R+ViewSplat and AESplat. All methods are trained on RealEstate10K and evaluated in a zero-shot setting on ACID, DL3DV, and ScanNet++.
| Method | ACID PSNR ↑ | SSIM ↑ | LPIPS ↓ | DL3DV PSNR ↑ | SSIM ↑ | LPIPS ↓ | ScanNet++ PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|---|---|---|---|---|---|---|---|---|---|
| NAS3R+ViewSplat | 27.188 | 0.815 | 0.159 | 20.240 | 0.639 | 0.267 | 21.705 | 0.804 | 0.206 |
| AESplat (Ours) | 27.375 | 0.819 | 0.156 | 20.430 | 0.648 | 0.259 | 21.892 | 0.812 | 0.199 |
Table D. Comparison of computational efficiency between NAS3R+ViewSplat and AESplat on RealEstate10K using an NVIDIA RTX 4090 GPU.
| Method | Inference Time (s) ↓ | Rendering Speed (FPS) ↑ |
|---|---|---|
| NAS3R+ViewSplat | 0.076 | 143 |
| AESplat (Ours) | 0.074 | 333 |
Q3: Clarification of the Experimental Protocol
A3: Training Settings. During training, both AESplat and the NAS3R baseline are initialized with pretrained VGGT weights. Neither method takes camera intrinsics as input; instead, the camera intrinsics are predicted by the model. All other training settings are kept identical to ensure a fair comparison.
Target Camera Parameter Estimation at Test Time. During testing, the context views and target view are jointly fed into the model, which predicts the camera parameters of the target view. These predicted parameters are then used to render the target viewpoint.
Output Resolution. The output resolutions used by different methods are as follows:
- 224 × 224: YoNoSplat, SPFSplatV2-L, NAS3R, AESplat.
- 256 × 256: pixelSplat, MVSplat, DepthSplat, Splatt3R, NoPoSplat, SelfSplat, PF3Splat, SPFSplat, SPFSplatV2, SPFSplatV2+Ours, NAS3R-m, NAS3R-m+Ours.
Response to Reviewer wAFW
We thank reviewer wAFW for the detailed review and helpful suggestions. For clarity, we abbreviate Weakness, Question, and Answer as W, Q, and A, respectively.
W1 & Q1: Novelty and Advantages of the Decoupled SH Modeling Strategy
A1.1: Differences from and Advantages over Splatt3R.
We acknowledge that Splatt3R has already adopted I2DC and uses a Gaussian Head to predict residual corrections to the zeroth-order SH coefficients. However, the ablation studies in the original Splatt3R paper show that additionally predicting higher-order SH coefficients with the Gaussian Head leads to degraded rendering quality. This observation further supports our motivation that a unified prediction strategy is not optimal.
In contrast, the key innovation of AESplat lies in decoupling view-independent and view-dependent appearance modeling and adopting different modeling strategies for these two appearance components. This allows the low- and high-order SH components to play complementary roles, thereby improving the overall appearance modeling capability.
A1.2: Additional Controlled Ablation Studies
We conduct comprehensive ablation studies on the ACID dataset using the full training schedule of 400K iterations, with results reported in Table A. Despite the differences in training datasets and schedules compared to the initial ablation studies conducted on RealEstate10K with 50K iterations, the performance trends of individual components remain consistent. This further demonstrates the stability of our ablation findings.
Furthermore, to investigate the effectiveness of decoupled modeling of low- and high-order SH coefficients, we design two alternative strategies under the same MLP architecture, GVSRE/WCM inputs, and optimization settings. Both variants are trained for the full 400K iterations, with results reported in Table B:
Joint Prediction of All SH Coefficients: We remove the I2DC module from AESplat and increase the output dimension of the MLP by three, enabling it to jointly predict the low-order SH coefficients. This variant leads to degraded performance, indicating that treating low- and high-order SH coefficients in the same manner is suboptimal.
RGB Residual-based DC Correction: For each Gaussian, we extract the RGB values of the center pixel and its surrounding 3×3 neighborhood from its source image. These nine RGB values, together with the Gaussian's three-axis scales, are fed into an independent MLP to predict a three-channel color residual. The residual is then applied in RGB space, and the corrected color is converted into the zeroth-order SH coefficient. This strategy yields a modest performance improvement, suggesting that separately modeling the correction of the zeroth-order SH coefficient and the prediction of higher-order coefficients is more effective.
Overall, these results further validate the rationale and effectiveness of AESplat's decoupled modeling strategy for low- and high-order SH coefficients.
Table A. Ablation studies on ACID with the full 400K training iterations.
Table B. Comparison of different SH modeling strategies.