acceptodds
Under review as a conference paper at ICLR 2027

ATSplat: Compact Feed-forward 3D Gaussian Splatting with Adaptive Token Expansion

10 views

Abstract

3D Gaussian Splatting (3DGS) achieves high-quality novel-view synthesis by optimizing freely placed primitives in 3D and adaptively densifying them in under-reconstructed regions. However, this scene-adaptive capacity allocation is largely lost in existing feed-forward 3DGS methods, where primitive counts and placement depend on input image grids rather than scene complexity. We present ATSplat, a feed-forward 3DGS framework that restores scene-adaptive capacity allocation through Adaptive 3D Tokens. ATSplat first lifts coarse patch-level depth and camera cues into sparse 3D anchor tokens, forming a compact scene scaffold. The tokens are progressively and selectively expanded and refined during decoding, guided by an Adaptive Token Expansion module that predicts per-token expansion scores at each stage. The expansion scores are supervised by the rendering error reduction associated with token expansion. Each refined token is then decoded into local Gaussians with learnable 3D offsets, decoupling primitive placement from input image grids. Experiments on the RealEstate10K and DL3DV datasets highlight that ATSplat achieves competitive rendering quality while using up to fewer Gaussians than dense pixel-aligned 3DGS methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Feed-forward novel view synthesis, 3D Gaussian Splatting, 3D reconstruction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_WRnwOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ATSplat, a feed-forward 3D Gaussian Splatting framework that restores scene-adaptive capacity allocation, overcoming the limitations of existing pixel-aligned methods whose primitive counts and placements depend on input image grids rather than scene complexity. By initializing sparse 3D anchor tokens and progressively expanding them in under-reconstructed regions via an Adaptive Token Expansion (ATE) module supervised by rendering error reduction, ATSplat decouples primitive placement from input pixel grids. ATSplat achieves competitive rendering quality while using up to 61x fewer Gaussians than dense pixel-aligned methods on the RealEstate10K and DL3DV benchmarks.

Critical strengths
  1. Scene-Adaptive Capacity Allocation in Feed-Forward 3DGS: The paper successfully transfers the "sparse-to-adaptive" principle of optimization-based 3DGS into a feed-forward paradigm by dynamically allocating Gaussians based on rendering error reduction rather than rigid pixel grids.

  2. Compression and Performance Trade-off: ATSplat achieves equivalent or superior performance on RealEstate10K and DL3DV datasets with a dramatically smaller number of Gaussian primitives (up to 61x reduction).

Critical weaknesses
  1. Marginal Performance Gain of ATE over Simple Heuristics (Table 6): As shown in Table 6, using simple heuristic selections like Random (28.70 PSNR) or Farthest Point Sampling (28.65 PSNR) already achieves surprisingly strong performance—outperforming many existing feed-forward baselines—while trailing only slightly behind the proposed error reduction supervision (29.23 PSNR). This raises concerns about whether the complex, learned ATE module provides a proportional benefit given its added training and inference overhead.

  2. Limited Scalability to Dense Multi-View Inputs: The evaluation is restricted to relatively sparse view settings (up to 16 views). In dense multi-view scenarios (e.g., 32+ views), overlapping captures generate massive redundancy. Without an explicit token pruning mechanism to remove redundant tokens during decoding, progressive token expansion combined with dense inputs leads to severe memory and computational bottlenecks, aggravated by the quadratic complexity of global multi-view attention.

Questions for authors
  1. Justification of ATE vs. Simple Heuristics: Given that random or FPS token selection already yields competitive PSNR, could the authors elaborate on the exact justification for the learned ATE module? Does the performance gap widen significantly under extremely tight Gaussian budgets (e.g., < 10K Gaussians)?

  2. Handling Redundancy in Dense Multi-View Settings: How does ATSplat handle massive information redundancy in dense multi-view captures (e.g., 32+ views)? Since initial tokens scale linearly with input views and the framework relies solely on progressive expansion without token pruning, does scaling to dense views cause an unmanageable token explosion and memory bottleneck? Furthermore, how does ATSplat's scalability and efficiency compare against dense feed-forward baselines optimized for multi-view inputs, including Zpressor (Wang et al., NeurIPS 2026) and MVP (Kang et al., CVPR 2026)?

  3. GPU Memory Footprint and Computational Complexity: While inference runtime and Gaussian counts are reported, peak GPU memory (VRAM) consumption is omitted. Could the authors provide VRAM footprint analyses under high-resolution and multi-view setups to evaluate real-world deployment feasibility?

  4. Generalization of Threshold τ\tau: Can the inference threshold τ\tau (e.g., 0.1 vs. 0.01) be applied consistently across different datasets or new domains (indoor vs. outdoor), or does it need to be manually re-tuned based on scene complexity or scale?

8 comments
AuthorsOpenReview ↗

Response to Reviewer WRnw (1/3)

We thank the reviewer for the thoughtful review. We are encouraged that the reviewer finds ATSplat successfully transfers the sparse-to-adaptive principle to feed-forward 3DGS while achieving equivalent or superior performance with far fewer Gaussians. We first clarify the benefit of learned token selection over simple heuristics, then address questions on scalability, efficiency, and generalization.

ATE vs. Simple Heuristics

Random (28.70 PSNR) or Farthest Point Sampling (28.65 PSNR) already achieves surprisingly strong performance [...] This raises concerns about whether the complex, learned ATE module provides a proportional benefit given its added training and inference overhead. [...] Does the performance gap widen significantly under extremely tight Gaussian budgets (e.g., < 10K Gaussians)?

Apart from the selection strategy, the Random and FPS variants share the rest of ATSplat's key components, including progressive expansion and refinement and the flexible 3D anchor design. As requested, we evaluate the selection strategies at a tighter budget of about 7K Gaussians on RealEstate10K with 2 input views at 256×256256\times256, which corresponds to τ=0.2\tau=0.2 for our model.

BudgetSelectionPSNRSSIMLPIPSΔ\DeltaPSNR vs. no expansion
4KNo expansion27.340.8670.160n/a
~7KRandom27.950.8820.138+0.62
~7KFPS27.930.8810.140+0.59
~7KOurs (τ=0.2\tau=0.2)28.620.8910.128+1.29
13-15K (Tab. 6)Random28.700.8970.112+1.36
13-15K (Tab. 6)FPS28.650.8950.120+1.31
13-15K (Tab. 6)Ours (τ=0.1\tau=0.1)29.230.9040.111+1.89

Starting from 4K Gaussians without expansion, heuristic selection strategies improve PSNR by about 0.6 dB, while our learned selection roughly doubles the gain to 1.29 dB. While the heuristic selections also achieve competitive results thanks to these shared components, our learned expansion scores consistently outperform the heuristic variants, with the gap becoming larger under the tighter budget (about 0.7 dB higher PSNR). At about 7K Gaussians, our learned selection reaches 28.62 dB PSNR, close to the 28.70 and 28.65 dB achieved by Random and FPS with 13-15K Gaussians, showing more effective use of a limited Gaussian budget. We also note that our learned selection offers dynamic Gaussian budgets based on scene complexity, whereas heuristic selections can only produce a fixed number of Gaussians regardless of the complexity of each scene.

Overhead. We would like to emphasize that ATSplat predicts the expansion score using a lightweight MLP at each decoder block. The three MLPs have fewer than 1.8M parameters in total, and their inference time stays below 5 ms even at 540×960540\times960 resolution with 16 input views, which is negligible compared to the total reconstruction time of 0.243 s. During training, score supervision accounts for about 9% of the training step time.

We have included the tighter-budget results in Appendix D.5 and the overhead analysis in Appendix E.2.

AuthorsOpenReview ↗

Response to Reviewer WRnw (2/3)

Scalability and Efficiency

[...] progressive token expansion combined with dense inputs leads to severe memory and computational bottlenecks, aggravated by the quadratic complexity of global multi-view attention. [...] does scaling to dense views cause an unmanageable token explosion and memory bottleneck? Furthermore, how does ATSplat's scalability and efficiency compare against dense feed-forward baselines optimized for multi-view inputs, including Zpressor and MVP?

ATSplat starts from sparse anchor tokens initialized from coarse patches, which correspond to 32×3232\times32 patches in our high-resolution setups, and the encoder performs global multi-view attention on these coarse tokens. While the token count increases as selected tokens are expanded across stages, progressive expansion reduces decoder computation since the full set of tokens is processed only at the last block. We report the average number of tokens at each stage on DL3DV at 540×960540\times960 using our 16-view model with 16 and 32 input views, along with the number of tokens for the 8×88\times8 and 16×1616\times16 patches commonly used in the baselines.

ViewsMethodInitialStage 1Stage 2Stage 3
168×88\times8 patches130.6K130.6K130.6K130.6K
16×1616\times16 patches32.6K32.6K32.6K32.6K
Ours (τ=0.1\tau=0.1)8.2K10.5K13.5K17.1K
Ours (τ=0.01\tau=0.01)8.2K14.1K25.2K44.8K
328×88\times8 patches261.1K261.1K261.1K261.1K
16×1616\times16 patches65.3K65.3K65.3K65.3K
Ours (τ=0.1\tau=0.1)16.3K21.2K27.5K35.4K
Ours (τ=0.01\tau=0.01)16.3K28.0K49.9K89.1K

As shown in the table, ATSplat maintains a moderate token count even with dense input views, thanks to its progressive expansion formulation starting from sparse anchor tokens. At τ=0.1\tau=0.1, it processes fewer tokens at every stage than a model using dense 16×1616\times16 patch tokens throughout the decoder.

While MVP focuses on efficient encoding of dense multi-view inputs and reconstructs faster than ATSplat, ATSplat focuses on scene-adaptive Gaussian allocation, using up to 61×61\times fewer Gaussians (Tab. 4). We also highlight that ATSplat is not limited to a specific multi-view encoder design and can incorporate efficient designs such as the hierarchical attention of MVP.

The revised manuscript discusses scalability in Sec. 5 and reports per-stage token counts in Appendix E.1.

Redundancy in Dense Views

In dense multi-view scenarios (e.g., 32+ views), overlapping captures generate massive redundancy. [...] How does ATSplat handle massive information redundancy in dense multi-view captures (e.g., 32+ views)?

We agree that redundancy becomes more apparent with dense multi-view inputs. While ATSplat does not explicitly prune redundant tokens during decoding, as noted in our limitations, such redundancy can be alleviated by a simple voxel-based subsampling strategy without retraining, since our initial anchor tokens are defined in 3D space. Specifically, after the sparse anchor initialization, we keep one anchor token in each voxel of size 0.05 and discard the rest, passing the remaining tokens to the decoder. We demonstrate this capability on DL3DV at 256×448256\times448 under the ZPressor setup with 12, 24, and 36 input views, and compare with ZPressor.

ViewsMethodPSNRSSIMLPIPS#GS
12ZPressor24.280.8190.146688K
Ours29.980.9150.094133K
Ours + voxel subsampling29.430.9050.104115K
24ZPressor24.240.8180.147688K
Ours30.480.9230.089256K
Ours + voxel subsampling29.910.9130.098198K
36ZPressor23.870.8130.150688K
Ours29.890.9150.101369K
Ours + voxel subsampling29.590.9080.106261K

With more input views, voxel subsampling removes more anchor tokens (14.6%, 24.9%, and 32.0%), which reflects the growing redundancy, while PSNR stays within 0.6 dB of the model without subsampling. At 36 views, it reduces the number of anchor tokens from 16,128 to 10,962 (-32.0%) and the number of Gaussians from 369K to 261K (-29.3%) while showing the smallest PSNR drop of 0.30 dB. We also note that ZPressor compresses the inputs into a fixed budget of 6 anchor views regardless of the number of input views or scene complexity, resulting in a higher compression ratio with more input views. In contrast, ATSplat adaptively allocates Gaussians by selectively expanding tokens in under-reconstructed regions, offering dynamic Gaussian budgets based on scene complexity. The dense-view redundancy analysis is now included in Appendix E.3.

AuthorsOpenReview ↗

Response to Reviewer WRnw (3/3)

GPU Memory Footprint

While inference runtime and Gaussian counts are reported, peak GPU memory (VRAM) consumption is omitted. Could the authors provide VRAM footprint analyses under high-resolution and multi-view setups to evaluate real-world deployment feasibility?

To evaluate real-world deployment feasibility, we report peak GPU memory (GB) averaged over the test scenes on DL3DV at 540×960540\times960 using our 16-view model with 16, 32, and 64 input views, and compare with the baselines.

Method16 views32 views64 views
iLRM3.916.7412.40
MVP4.868.3315.27
ReSplat-init16.2931.6162.25
Ours (τ=0.1\tau=0.1)4.126.8812.61
Ours (τ=0.01\tau=0.01)4.287.3820.06

As shown in the table, ATSplat (τ=0.1\tau=0.1) maintains moderate GPU memory usage up to 64 input views, with peak memory comparable to or lower than that of dense pixel-aligned baselines. This memory efficiency originates from our sparse-to-adaptive formulation, which begins with a coarse scene scaffold and progressively expands and refines tokens in under-reconstructed regions. Peak GPU memory results are now reported in Appendix E.1.

Generalization of Threshold τ\tau

Can the inference threshold τ\tau (e.g., 0.1 vs. 0.01) be applied consistently across different datasets or new domains (indoor vs. outdoor), or does it need to be manually re-tuned based on scene complexity or scale?

The expansion score is supervised by the reduction in 2D rendering error from token expansion, making τ\tau a threshold on the expected improvement from expansion rather than a dataset- or scene-specific hyperparameter. As shown in the experiments throughout the manuscript, τ=0.1\tau=0.1 generally works well without re-tuning across different datasets, numbers of input views, and resolutions. The threshold controls the trade-off between rendering quality and the number of Gaussians, and a lower threshold such as τ=0.01\tau=0.01 yields higher rendering quality at the cost of a larger Gaussian budget. The discussion of threshold generalization now appears in Appendix B.

Reviewer_1gvmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes ATSplat, a feed-forward 3DGS method. Unlike prior methods that predict dense pixel-aligned Gaussians, ATSplat lifts coarse patch-level depth into sparse 3D anchor tokens. It then progressively expands the tokens in under-reconstructed regions during decoding, guided by a learned expansion score supervised by rendering error reduction. Each refined token is decoded into local Gaussians with 3D offsets. This yields a compact Gaussian set that is decoupled from the input pixel grid, with competitive or better rendering quality than recent baselines.

Critical strengths
  1. Novel and well-motivated design. The method combines the strengths of feed-forward and optimization-based 3DGS. Adaptive token expansion mirrors the densification step in per-scene optimization, but runs in a single forward pass. As a result, the model predicts a compact set of Gaussians instead of dense pixel-aligned outputs.

  2. Strong efficiency–quality trade-off. Experiments on RealEstate10K and DL3DV show competitive or better quality than recent feed-forward methods while using far fewer Gaussians. For example, on DL3DV (Table 2) ATSplat at τ=0.1 achieves the best results on all three metrics with a much smaller Gaussian count. In several settings (e.g., Table 3) it also gives a clear inference speedup. The appendix adds useful results on viewpoint overlap, extrapolation and cross-dataset generalization.

Critical weaknesses
  1. Insufficient discussion of token expansion novelty. The related work cites Generative Densification, F4Splat and TokenGS, but does not clearly state how progressive token expansion differs from them. Is the expansion mechanism itself new, or is the contribution the combination with anchor tokens and the error-reduction supervision? The STE ablation (Table 6) only follows the design of Nam et al. rather than comparing with that method directly. I suggest a clearer positioning and, if possible, a direct comparison with F4Splat or Generative Densification.

  2. Limited evaluation scope. The main experiments use two datasets (RealEstate10K and DL3DV), and the appendix adds ACID for cross-dataset testing. All of these are video-derived scenes. Evaluation on more diverse real-world data (e.g., ScanNet++, Tanks and Temples) would better support the generality claim.

Questions for authors

NA

2 comments
AuthorsOpenReview ↗

Response to Reviewer 1gvm

We appreciate the reviewer's constructive feedback and recognition of our design as novel and well-motivated with a strong efficiency-quality trade-off. Below, we clarify how ATSplat differs from prior methods, provide a direct comparison, and address the concern about the evaluation scope.

Discussion on Token Expansion Novelty

The related work cites Generative Densification, F4Splat and TokenGS, but does not clearly state how progressive token expansion differs from them. Is the expansion mechanism itself new, or is the contribution the combination with anchor tokens and the error-reduction supervision? [...] I suggest a clearer positioning and, if possible, a direct comparison with F4Splat or Generative Densification.

Our contribution is to bring the sparse-to-adaptive formulation of per-scene 3DGS optimization into feed-forward 3DGS through progressive token expansion during decoding. The anchor tokens and error reduction supervision form the key components enabling this progressive expansion, with the anchor tokens providing a scene-conditioned sparse initialization and error reduction supervision identifying the tokens to expand at each stage.

In contrast to our progressive expansion and refinement, the prior methods noted by the reviewer operate either on densely predicted Gaussians or on a fixed set of learnable tokens.

  • Generative Densification densifies individual Gaussians on top of predicted dense pixel-aligned Gaussians in a post-hoc manner and refines them through self-attention among Gaussians. On the other hand, ATSplat expands compact anchor tokens and refines them with fine image features at each decoding stage. The fully learnable selection strategy of Generative Densification is also evaluated in our ablation (Tab. 6), as noted by the reviewer.
  • F4Splat first predicts dense multi-scale Gaussian maps and selects among them at the output. Within our framework, single-stage expansion with the same maximum expansion factor achieves 28.48 and 28.43 dB, compared to 29.23 dB with progressive expansion (Tab. 7).
  • TokenGS uses a fixed number of learnable tokens that are not conditioned on the scene. Replacing our anchor tokens with learnable initial tokens lowers PSNR from 29.23 to 28.02 dB (Tab. 5), demonstrating that our scene-conditioned anchors provide a more effective initialization. ATSplat also outperforms TokenGS with far fewer Gaussians (Tab. 1).

Direct comparison. As suggested, we additionally compare with Generative Densification and F4Splat on RealEstate10K with 2 input views at 256×256256\times256. For Generative Densification, we report the official results under the same evaluation protocol. Since F4Splat takes unposed images as input, we evaluate it with its test-time pose optimization on the same test split, following its evaluation protocol. We also include TokenGS from Tab. 1.

MethodPSNRSSIMLPIPS#GS
Generative Densification27.080.8790.120347K
F4Splat (unposed)26.250.8570.13726K
F4Splat (unposed)26.790.8730.119131K
TokenGS (Tab. 1)28.410.9030.135262K
Ours (τ=0.1\tau=0.1)29.230.9040.11115K
Ours (τ=0.01\tau=0.01)29.410.9070.10528K

ATSplat (τ=0.1\tau=0.1) outperforms all three methods on all metrics while using fewer Gaussians. The revised manuscript covers these distinctions in Sec. 5 and the direct comparison in Appendix C.6.

Evaluation on Diverse Real-World Data

All of these are video-derived scenes. Evaluation on more diverse real-world data (e.g., ScanNet++, Tanks and Temples) would better support the generality claim.

As suggested, we perform zero-shot evaluation on Tanks and Temples using models trained on DL3DV without fine-tuning at 540×960540\times960 with 16 and 32 input views, following the LongLRM protocol. For ATSplat, we use the 16-view model from Tab. 4 for both view counts.

ViewsMethodPSNRSSIMLPIPS#GS
16LongLRM16.770.5600.4418356K
iLRM17.790.6100.4102089K
MVP18.330.6480.3508356K
Ours (τ=0.1\tau=0.1)18.820.6430.39797K
Ours (τ=0.01\tau=0.01)19.310.6870.327328K
32LongLRM18.590.6260.36716712K
iLRM18.570.6310.3844178K
MVP19.540.7070.27716712K
Ours (τ=0.1\tau=0.1)19.710.6820.349205K
Ours (τ=0.01\tau=0.01)20.160.7220.287676K

ATSplat (τ=0.1\tau=0.1) outperforms iLRM and LongLRM on all metrics and achieves higher PSNR than MVP with over 80×80\times fewer Gaussians. With τ=0.01\tau=0.01, ATSplat achieves the highest PSNR and SSIM at both view counts. These results highlight that ATSplat generalizes well across diverse datasets and real-world scenes. The zero-shot evaluation is now included in Appendix C.5 of the revision.

Dear Reviewer 1gvm,

Once again, we appreciate your constructive comments on our work.

We would welcome your feedback on whether our responses have resolved your concerns. To summarize, our responses provide:

  • An expanded discussion in Sec. 5 of the revision distinguishing our progressive token expansion from Generative Densification, F4Splat, and TokenGS — [W1]
  • A direct comparison with Generative Densification and F4Splat on RealEstate10K, demonstrating that ATSplat (τ=0.1\tau=0.1) performs better across all metrics with fewer Gaussians — [W1]
  • Zero-shot results on Tanks and Temples using 16 and 32 input views, showing that ATSplat (τ=0.1\tau=0.1) achieves higher PSNR than MVP despite using over 80×80\times fewer Gaussians — [W2]

If you have any remaining questions, we would be happy to address them during the discussion period.

Best,

Authors

Reviewer_1QxfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

ATSplat is a feed-forward 3DGS method that tries to bring back the sparse-to-adaptive capacity allocation of per-scene 3DGS. It works in four steps:

  1. Coarse patch-level depths are unprojected into sparse 3D anchor tokens.
  2. An image-to-3D decoder refines the tokens by cross-attending to fine image features.
  3. At each decoder block, an Adaptive Token Expansion (ATE) module predicts a per-token score and splits high-score tokens into M children. The score is supervised by the rendering-error reduction of expanded tokens relative to parent-averaged tokens.
  4. Each final token is decoded into K local Gaussians with 3D offsets.

On RealEstate10K, the method matches ReSplat-init with about 4.7× fewer Gaussians. On DL3DV (256×448, 512×960, 540×960) it reports clear PSNR/SSIM gains with far fewer Gaussians, up to 61× fewer than MVP. Ablations cover the anchor design, the selection strategy, and progressive versus single-stage expansion.

Critical strengths
  1. Good quality–compactness trade-off, strongest on DL3DV. At 6 views, ATSplat (τ=0.2, 32K Gaussians) beats ReSplat-init (43K) by about 1.7 dB PSNR. It also beats pixel-aligned methods that use more than 20× as many Gaussians. This holds across view counts and resolutions, and the extrapolation and cross-dataset results point the same way. It supports the claim that adaptive allocation beats uniform downsampling.
  2. Simple, well-motivated expansion signal. Supervising the score with expected error reduction, rather than raw error, separates "needs more capacity" from "lacks input information." Table 6 and Fig. 6(a) show a consistent gain over rendering-error supervision, especially at small budgets.
Critical weaknesses
  1. The source of the gains is unclear. The model combines several strong components: a DINOv2 backbone, PRoPE, a global multi-view transformer, coarse-to-fine features, and (per Appx. A.2) rendering intermediate Gaussians back to the input views and fusing them into the features. That last step is a ReSplat-like feedback loop, and it is not ablated. The ATE-specific ablations on RealEstate10K show modest gaps, e.g. random selection is only 0.5 dB behind (28.70 vs 29.23). The large DL3DV margins may therefore come partly from the architecture rather than from adaptive expansion. Training data and compute also differ from the baselines.
  2. The efficiency claims are only partly supported.
    • Compactness is measured by Gaussian count alone. Rendering FPS, memory, and storage are not reported, even though they are the motivation for compactness.
    • Inference time is not consistently better: 0.405 s vs 0.091 s for MVP in Table 4, and 0.028 s vs 0.025 s for iLRM in Table 1.
    • At the compact setting (τ=0.1), LPIPS is often worse than the baselines (Tables 1, 3, 4). The "competitive quality with 61× fewer" headline therefore hides a perceptual-quality cost.
    • Novelty over F4Splat (score-based densification), Generative Densification, and AnchorSplat/Scaffold-GS-style anchors is incremental and should be discussed more directly.

Questions For Authors

Questions for authors
  1. Can you ablate the intermediate render-and-fuse feedback in Appx. A.2, and report a no-ATE variant at matched Gaussian count on DL3DV? (Affects how much of the gain I credit to ATE.)
  2. Training uses a fixed 0.5 selection ratio, while inference uses a threshold τ. Score supervision is also applied only to pixels covered by expanded tokens. How well calibrated are scores for unselected tokens, and how much does the Gaussian count vary across scenes at a fixed τ?
  3. Please report rendering FPS and peak memory against ReSplat-init and MVP at the high-resolution settings.
  4. Why is ReSplat compared only at iteration 0? How does ATSplat compare to full ReSplat, or to ATSplat combined with ReSplat's refinement?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 1Qxf (1/3)

We thank the reviewer for the detailed review. We are glad that the reviewer recognizes the quality-compactness trade-off of ATSplat and the motivation for supervising expansion with rendering error reduction. Below, we clarify the sources of the gains, compare with full ReSplat and its refinement, and then address efficiency and the remaining concerns.

Source of the Gains

The source of the gains is unclear. The model combines several strong components [...] That last step is a ReSplat-like feedback loop, and it is not ablated. The ATE-specific ablations on RealEstate10K show modest gaps [...] The large DL3DV margins may therefore come partly from the architecture rather than from adaptive expansion. Training data and compute also differ from the baselines. [...] Can you ablate the intermediate render-and-fuse feedback in Appx. A.2, and report a no-ATE variant at matched Gaussian count on DL3DV?

The gains of ATSplat originate from its sparse-to-adaptive formulation, in which the 3D anchor tokens, progressive expansion and refinement, and learned selection each improve rendering quality (Tabs. 5-7). As requested, we further ablate the render-and-fuse step, and compare selection strategies at matched Gaussian counts on DL3DV.

Render-and-fuse step. We evaluate a variant without the render-and-fuse step on RealEstate10K with 2 input views at τ=0.1\tau=0.1, following the setup of our ablations.

VariantPSNRSSIMLPIPS
w/o render-and-fuse28.870.9000.114
Full29.230.9040.111

The render-and-fuse step further improves PSNR from 28.87 to 29.23 dB on top of the gains from the main components (0.53 to 1.89 dB), namely our flexible 3D anchor design (Tab. 5), learned selection (Tab. 6), and progressive expansion (Tab. 7).

Selection at matched Gaussian counts on DL3DV. We replace our learned selection with the alternatives in Tab. 6 while retaining the rest of ATSplat, including the 3D anchor tokens and progressive expansion and refinement. All variants are matched to the Gaussian count of our model at τ=0.1\tau=0.1 on DL3DV at 256×448256\times448 with 2, 4, and 6 input views. We also report a variant without token expansion.

Selection2 views4 views6 views
PSNRSSIMLPIPSPSNRSSIMLPIPSPSNRSSIMLPIPS
No expansion21.640.6840.32825.140.8000.21826.570.8390.181
Random22.350.7180.27226.390.8420.15828.060.8800.124
FPS22.330.7170.27526.240.8370.16427.840.8740.131
STE22.310.7160.27226.390.8420.15728.090.8810.123
Rendering error22.400.7200.27626.610.8490.15728.370.8880.123
Ours (τ=0.1\tau=0.1)22.540.7250.26826.810.8500.15128.530.8880.119

Consistent with the ablation results on RealEstate10K (Tabs. 6, 7), our learned selection achieves the highest PSNR and the lowest LPIPS among the selection strategies, and removing token expansion substantially reduces rendering quality across all input view counts.

Training data, compute, and backbone. ATSplat follows the backbone and training protocols of recent baselines: a pretrained DINOv2 backbone that also underlies the Depth Anything V2 backbone used by DepthSplat and ReSplat, the same training datasets and splits, and training compute (Appendix A.1) similar to or lower than that of most baselines.

We have included these ablations in Appendix D.4 and D.5 of the revised manuscript.

Comparison with ReSplat Refinement

Why is ReSplat compared only at iteration 0? How does ATSplat compare to full ReSplat, or to ATSplat combined with ReSplat's refinement?

We compare with ReSplat at iteration 0 since its recurrent refinement is a post-hoc stage that can be applied to other feed-forward methods. As requested, we additionally train the refinement module of ReSplat on top of the compact Gaussians of ATSplat, and evaluate it on RealEstate10K.

MethodPSNRSSIMLPIPS#GS
ReSplat-init29.400.9070.104131K
ReSplat29.750.9120.100131K
Ours (τ=0.1\tau=0.1)29.230.9040.11115K
Ours (τ=0.1\tau=0.1) + refinement29.530.9080.10615K
Ours (τ=0.01\tau=0.01)29.410.9070.10528K
Ours (τ=0.01\tau=0.01) + refinement29.690.9100.10028K

As shown in the results, the refinement module of ReSplat can also be applied to our compact Gaussians, improving rendering quality at both thresholds. With refinement, ATSplat at τ=0.01\tau=0.01 achieves rendering quality close to full ReSplat with 4.7×4.7\times fewer Gaussians. The ReSplat refinement comparison is now reported in Appendix C.7.

AuthorsOpenReview ↗

Response to Reviewer 1Qxf (2/3)

Efficiency

The efficiency claims are only partly supported. Compactness is measured by Gaussian count alone. Rendering FPS, memory, and storage are not reported, even though they are the motivation for compactness. Inference time is not consistently better: 0.405 s vs 0.091 s for MVP in Table 4, and 0.028 s vs 0.025 s for iLRM in Table 1. [...] Please report rendering FPS and peak memory against ReSplat-init and MVP at the high-resolution settings.

Rendering speed, storage, and memory. As requested, we report rendering FPS, storage, and peak GPU memory on DL3DV at 540×960540\times960 with 16 input views, following the setting of Tab. 4, and compare with ReSplat-init and MVP.

Method#GSStorage (MB)Memory (GB)FPS
ReSplat-init518K128.316.29352.8
MVP8356K1036.14.86184.1
Ours (τ=0.1\tau=0.1)137K20.84.12774.8
Ours (τ=0.01\tau=0.01)358K54.54.28585.6

At both thresholds, the smaller number of Gaussians in ATSplat translates into lower storage and faster rendering than both ReSplat-init and MVP. ATSplat also uses lower peak GPU memory than both methods.

Inference time. While MVP focuses on efficient encoding of dense multi-view inputs and reconstructs faster than ATSplat, ATSplat focuses on scene-adaptive Gaussian allocation, using up to 61×61\times fewer Gaussians (Tab. 4). We also highlight that ATSplat is not limited to a specific multi-view encoder design and can incorporate efficient designs such as the hierarchical attention of MVP. ATSplat (τ=0.1\tau=0.1) exhibits faster inference than iLRM under the high-resolution setups (Tabs. 3, 4), and is substantially faster than ReSplat-init across all settings.

Rendering speed, storage, and memory results are included in Appendix E.1 and E.2.

LPIPS at the Compact Setting

At the compact setting (τ=0.1), LPIPS is often worse than the baselines (Tables 1, 3, 4). The "competitive quality with 61× fewer" headline therefore hides a perceptual-quality cost.

We agree that "competitive rendering quality" could hide the perceptual-quality cost at τ=0.1\tau=0.1, where LPIPS is higher than some baselines (Tabs. 1, 3, 4), and have replaced it with "comparable PSNR and SSIM" in the revised manuscript. At τ=0.01\tau=0.01, ATSplat achieves the lowest LPIPS in Tabs. 3, 4 and outperforms MVP across all three quality metrics while still using 23×23\times fewer Gaussians.

AuthorsOpenReview ↗

Response to Reviewer 1Qxf (3/3)

Novelty of Progressive Token Expansion

Novelty over F4Splat (score-based densification), Generative Densification, and AnchorSplat/Scaffold-GS-style anchors is incremental and should be discussed more directly.

Our main contribution is to bring the sparse-to-adaptive formulation of per-scene 3DGS optimization into feed-forward 3DGS through progressive token expansion during decoding.

  • F4Splat first predicts dense multi-scale Gaussian maps and selects among them at the output, while ATSplat starts from sparse tokens and decides each expansion during decoding, with expansion scores supervised by rendering error reduction.
  • Generative Densification densifies individual Gaussians on top of predicted dense pixel-aligned Gaussians in a post-hoc manner, whereas ATSplat expands compact anchor tokens and refines them with fine image features at each decoding stage.

While our anchor and local 3D Gaussian design is inspired by Scaffold-GS-style methods, the core of our framework lies in the feed-forward sparse-to-adaptive reconstruction enabled by this anchor design. Scaffold-GS grows anchors using gradients accumulated during per-scene optimization and AnchorSplat predicts Gaussians from a fixed set of anchors. On the other hand, ATSplat progressively expands anchor tokens with learned expansion scores in a single forward pass.

The comparison with these methods is now discussed in Sec. 5 of the revision.

Training with a Fixed Selection Ratio

Training uses a fixed 0.5 selection ratio, while inference uses a threshold τ. Score supervision is also applied only to pixels covered by expanded tokens. How well calibrated are scores for unselected tokens, and how much does the Gaussian count vary across scenes at a fixed τ?

The expansion scores are predicted by a lightweight MLP shared across all tokens, including tokens that remain unselected at inference. During training, the fixed selection ratio exposes the score head to tokens with both large and negligible rendering error reduction, allowing it to learn low scores for tokens that gain little from expansion. We observe that this fixed-ratio training transfers well to threshold-based selection at inference. As shown in Fig. 5, high scores are localized around fine structures and textured regions, and the selected tokens at stage 3 concentrate in these regions.

At τ=0.1\tau=0.1 on RealEstate10K, the number of Gaussians per scene ranges from 4.2K to 31.8K, with an average of 15K. The discussion of fixed-ratio training now appears in Appendix B.

Dear Reviewer 1Qxf,

Thank you once again for your detailed review and constructive suggestions.

We would be grateful to hear whether our responses have addressed your concerns. In summary, we have provided:

  • An ablation showing that the render-and-fuse step provides an additional PSNR improvement beyond the main components — [W1/Q1]
  • An evaluation of selection strategies at matched Gaussian counts on DL3DV, along with a variant without token expansion, showing that our learned selection consistently outperforms the other selection strategies — [W1/Q1]
  • Results from integrating ATSplat with the refinement module of ReSplat, producing rendering quality close to full ReSplat while requiring 4.7×4.7\times fewer Gaussians at τ=0.01\tau=0.01 — [Q4]
  • Rendering FPS, storage, and GPU memory measurements on DL3DV at 540×960540\times960, showing that ATSplat requires less storage and GPU memory while rendering faster than ReSplat-init and MVP — [W2/Q3]
  • An adjustment to the claim regarding the compact setting, changing "competitive rendering quality" to "comparable PSNR and SSIM" — [W2]
  • Additional discussion in Sec. 5 of the revision, clarifying the distinctions between ATSplat and F4Splat, Generative Densification, Scaffold-GS, and AnchorSplat — [W3]
  • Further explanation of fixed-ratio training and the variation in Gaussian counts across scenes at τ=0.1\tau=0.1, ranging from 4.2K to 31.8K on RealEstate10K — [Q2]

Please let us know if any questions remain, and we would be glad to address them during the discussion period.

Best,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.