acceptodds
Under review as a conference paper at ICLR 2027

Look Where You Splat: Splat Guided Feed-Forward 3D Gaussian Prediction

Abstract

Feed-forward 3D Gaussian Splatting has recently moved from predicting a Gaussian per pixel to predicting Gaussians from a fixed set of learnable tokens, so the number of Gaussians no longer scales with the input images. These token-based feed-forward models reconstruct compact scenes with as few as 2K Gaussians, yet we find that their tokens do not always attend to where their own Gaussians project. We observe that for 31.2% of the tokens, the cross-attention weights do not align with their splat regions, where their predicted Gaussians project in the input images. A misaligned token aggregates image patch features from outside its splat region, leaves part of the region unattended, or both. We hypothesize that a token predicts its Gaussians more accurately when it attends to image patches within its own splat region. To exploit this insight, we propose Gaussian Splat Attention Bias (GSAB), which aligns each token's attention weights to its own splat region in the input images. Because GSAB only adds an attention bias and an intermediate decoding of existing tokens, it applies to any token-based model, which we verify on two that differ in pose assumption and scale: a pose-free model predicting 2K Gaussians and a posed model predicting 262K. GSAB improves PSNR by up to 0.80 dB on the posed model and by 0.71 dB on the pose-free model.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Feed-Forward Models, 3D Gaussian Splatting, Novel View Synthesis, 3D Reconstruction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_TzQpOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper investigates token-based feedforward 3D Gaussians splatting and build on the observation that, in such methods, attention between tokens and image regions does not seem to correlate well (in terms of being spatially aligned). This is then demonstrated on a toy example. The authors propose Gaussian Splat Attention Bias (GSAB) that decodes intermediate 3D Gaussians at an earlier transformer layer, projects them onto the input views using ground-truth or predicted camera poses, and uses the resulting Gaussian weights (modulated by a learned scale and spatial spread) as an additive attention bias in the final cross-attention block. GSAB is evaluated on two baselines: the pose-free C3G (2K Gaussians on RealEstate10K) and the posed TokenGS (262K Gaussians on DL3DV), showing modest improvements over their reproduced baselines.

Critical strengths
  • The core idea that a token responsible for predicting a specific 3D Gaussian should aggregate visual features from the 2D image regions where that Gaussian actually projects is geometrically intuitive and connects nicely to spatially modulated query mechanisms in 2D detection. However, as I will explain in the weaknesses section I am not sure whether this really is a real problem in practice.
Critical weaknesses
  • The paper claims to introduce a general mechanism that solves why token-based feed-forward methods lag behind pixel-aligned ones in terms of accuracy. However, the empirical validation is simply an incremental modification of C3G. While the authors also apply GSAB to TokenGS (Table 2), their own reproduction (TokenGS*) significantly underperforms the official TokenGS numbers. Adding GSAB (TokenGS* + GSAB) merely brings the performance back to the originally reported numbers. Moreover, a substantial performance gap (~1.4-2.7 dB PSNR on RealEstate10K) remains between token-based C3G+GSAB and compact/pixel-aligned competitors like F4Splat or SPFSplatV2. Thus, GSAB comes across as an incremental and minor patch for C3G rather than a general paradigm shift for token-based 3DGS.
  • The central assumption that misaligned cross-attention is a primary bottleneck in feed-forward reconstruction is treated more as a hunch than something rigorously proven. The toy experiment (Fig. 2) is too simplistic. It freezes 3D geometry and image encoder, re-initializes the transformer, and trains only color and opacity on 6 scenes for 1K iterations. Showing that local patch attention helps estimate local color when 3D geometry is fixed is trivial and does not prove that global/unconstrained cross-attention is detrimental when jointly learning 3D geometry and appearance from scratch. Let me note that multi-layer transformers already have high redundancy and often use broader context in cross-attention to resolve multi-view geometry. The fact that forcing alignment via GSAB decreases the misalignment ratio from 31.2% to 22.1% (i.e., large portion of the misaligned tokens remain "misaligned") while leading only to minor PSNR gains across the board, suggests that the network has redundancy and that strict attention-to-splat alignment is not really a bottleneck.

All in all, the paper at this point is an incremental patch to C3G rather than a mechanism to improve all token-based feedforward reconstructors. Moreover, the insight it build upon is not really analysed and proven.

Questions for authors
  • In Table 2, TokenGS* + GSAB only matches the published TokenGS baseline. Can the authors clarify why the reproduction gap is so large?
  • In the toy experiment (Fig. 2), geometry is frozen and only color and opacity are learned. What happens in a full training setting if attention is hard-constrained to the splat region vs. soft-biased via GSAB? Could the remaining 22.1% 'misaligned' tokens in C3G+GSAB actually be performing important non-local context aggregation (e.g., looking at adjacent textures or epipolar regions to resolve depth/pose ambiguity)?
  • How does the method work with any other token-based feedforward Gaussian methods?
5 comments
AuthorsOpenReview ↗

Response to Reviewer TzQp (1/3)

We thank TzQp for the careful assessment. We address the concerns below.

W1. Clarification on the comparison with pixel-aligned methods.

GSAB does not aim to resolve the difference between pixel-aligned methods and token-based ones. GSAB methodologically addresses misaligned attention in token-based models and improves them while keeping their compact set of Gaussians.

  • Our claim. We acknowledge that our sentence in Sec. 4.2, "GSAB narrows the gap between these methods and C3G without adding Gaussians," could be read as a claim that GSAB resolves the difference between token-based and pixel-aligned methods, which was not our intent. To clarify this, we have revised it to "GSAB improves C3G while keeping its compact set of 2K Gaussians."
  • Empirical validation. GSAB is evaluated not only on C3G but also on TokenGS. W1 / Q3 shows that it improves both models and discusses a third model, and W1 / Q1 explains the TokenGS reproduction gap.
  • Remaining gap to pixel-aligned methods. The pixel-aligned methods in Table 1, such as F4Splat and SPFSplatV2, are included as reference points rather than as direct competitors, as they use 13 to 64 times more Gaussians than C3G+GSAB. To show this visually, we have added Fig. 8 to Appendix C.1 of the revised paper, which plots PSNR against the number of Gaussians.

W1 / Q3. Generalization.

GSAB improves two token-based models that differ substantially, each over a baseline trained in the same setting:

  • C3G: pose-free, 2K Gaussians with one Gaussian per token (Table 1).
  • TokenGS: posed, 262K Gaussians with 64 Gaussians per token (Table 2). The published results are trained on a dataset that is not public, so a reproduction trained on the same data is the only baseline that allows a fair comparison, and GSAB improves over it in every seed. We explain the reproduction gap and report the results of each seed in W1 / Q1 below.

We therefore ask TzQp to consider the TokenGS result, compared on the same data, as a valid second case, and the two cases together as support that the improvement by GSAB generalizes beyond C3G.

We agree that showing the improvement on a third token-based model, such as GlobalSplat, would further strengthen this claim. Training GlobalSplat with and without GSAB takes several weeks on our resources, so we cannot guarantee results within the discussion period. We plan to include the results in the camera-ready version.

W1 / Q1. TokenGS reproduction gap.

The gap between the published TokenGS results and our reproduction stems from training, not evaluation. Our evaluation code matches the official evaluation: it reproduces the published numbers when evaluating the released TokenGS checkpoint (4 views: reproduced 22.98 vs. published 23.02 dB).

The training gap is likely due to two factors:

  • Non-public training data. The published results use an internally processed DL3DV dataset that is not publicly available (issue #9 of the official TokenGS repository).
  • Seed variance. Training varies by up to 0.57 dB across seeds (4 views).

We therefore compare TokenGS* and TokenGS*+GSAB trained in the same setting, reporting PSNR with 4 input views for each of the three seeds:

Methodseed 42seed 44seed 55Min / MaxMean ± std
TokenGS*22.7022.1322.2422.13 / 22.7022.36 ± 0.30
TokenGS*+GSAB23.1323.1022.8422.84 / 23.1323.02 ± 0.16

GSAB improves PSNR in every seed, and even the lowest seed of TokenGS*+GSAB (22.84) exceeds the highest seed of TokenGS* (22.70).

To let the reproductions be checked directly, we have uploaded our code as the supplementary material. It includes:

  • Training and evaluation code for both our reproductions of the baselines and GSAB.
  • Dataset preparation. For DL3DV, a script that holds out the test scenes from training and undistorts the frames following the public script suggested by the TokenGS authors in issue #9, as their internal dataset preprocessing is not available. For RealEstate10K, instructions to obtain the preprocessed version used for C3G.
AuthorsOpenReview ↗

Response to Reviewer TzQp (2/3)

W2. Does alignment matter beyond the toy experiment?

The toy experiment is a controlled test that isolates the effect of alignment: fixing the geometry keeps the splat regions identical across settings, so any difference in accuracy comes from the attention alone. Accordingly, we limit its conclusion to this toy experiment setting (L260), and we have revised the paragraph title (L238) to "In a controlled setting, better alignment improves reconstruction."

This controlled setting is close to the final block of C3G, where GSAB is applied. In this block, the splat regions of the intermediate and final Gaussians mostly overlap (Fig. 6), so the geometry is already largely determined by the earlier block, as in the toy experiment.

Other observations also link alignment to reconstruction quality in the full model:

  • Across the scenes of the released C3G, attention precision (Eq. (6)) correlates with PSNR, with a Spearman correlation coefficient of 0.61 (Sec. 3.2).
  • The intermediate reconstruction loss alone does not improve C3G reconstruction quality (Table 3), so the gain more likely comes from GSAB.

We also group the 1,000 test scenes into those whose misaligned ratio decreased and those whose misaligned ratio increased when comparing C3G*+GSAB to C3G* and to the released C3G, and report the share of scenes and the mean PSNR change of each group:

C3G*+GSAB vs.Misalignment decreased Scenes (%) / Δ\DeltaPSNR (dB)Misalignment increased Scenes (%) / Δ\DeltaPSNR (dB)Spearman ρ\rho (Δ\DeltaMisaligned, Δ\DeltaPSNR)
C3G*85.7 / +1.5214.3 / +0.60−0.53
C3G (released)88.5 / +0.7811.5 / +0.22−0.36

PSNR increases in both groups, so factors other than alignment also contribute to the gain of GSAB. However, two results show that the increase is larger where alignment improves:

  • The mean PSNR increase is 2.5 to 3.5 times larger in scenes whose misaligned ratio decreased.
  • The Spearman correlation coefficient ρ\rho between the per-scene changes in the misaligned ratio and in PSNR is negative for both baselines, so the larger the decrease in the misaligned ratio, the larger the PSNR increase tends to be.
AuthorsOpenReview ↗

Response to Reviewer TzQp (3/3)

Q2a. Soft bias (GSAB) vs. hard mask.

As suggested, we have run this comparison in full training. Our first results show that encouraging alignment with the soft bias of GSAB improves over C3G, while strictly enforcing it with a hard mask does not.

We compare the two in the final block of C3G, keeping everything else identical, including the intermediate decoding and the intermediate reconstruction loss:

  • Soft bias (GSAB): an attention bias added to the attention logits encourages each Gaussian token to attend more strongly to the splat region of its projected intermediate Gaussian, but the token can still attend to the patches outside it.
  • Hard mask: each Gaussian token attends only to the patches within the 3σ ellipse of its projected intermediate Gaussian, as the patches outside it are masked out.

Because of the training time, we first compare at 25K iterations on the 1,000 test scenes:

MethodPSNR
C3G19.50
C3G+GSAB19.91
C3G+hard mask19.50

At this early stage, GSAB increases PSNR by 0.41 dB over C3G, while the hard mask gives the same PSNR as C3G. This may be because the intermediate Gaussians are not yet well placed, so the hard mask can confine each token to a misplaced region, while the soft bias of GSAB still lets the token attend to the patches outside it.

We expect the 100K results within the discussion period and will post them in a follow-up comment.

W2 / Q2b. Could the remaining 22.1% be useful non-local context aggregation?

Some of them may be. Most of the remaining misaligned tokens (15.1% of the 22.1%) cover their own splat region and extend beyond it. This is also because GSAB reduces the other types of misalignment, where attention is placed elsewhere or covers too little.

As defined in Sec. 3.2, a token is misaligned when the precision or the recall of its attention region with respect to its splat region (Eq. (6)) falls below τ, and we split misaligned tokens into three types:

  • Too wide: the recall is at least τ but the precision is below τ, so the attention region covers the splat region and extends beyond it.
  • Too narrow: the precision is at least τ but the recall is below τ, so the attention region covers only part of the splat region.
  • Wrong place: both are below τ, so the attention region barely overlaps the splat region.

Compared to the released C3G on the 1,000 test scenes, GSAB reduces the two types in which attention misses part or all of the splat region:

  • Wrong place almost disappears (9.4% → 1.5%), so attention is directed to the right location. This reduction is linked to part of the gain of GSAB: scenes with a larger reduction of wrong place tend to show a larger PSNR increase, with a Spearman correlation coefficient of −0.34 between the per-scene change in its share and the change in PSNR.
  • Too narrow is halved (10.9% → 5.6%), so attention covers more of the splat region.

The remaining too wide tokens (15.1%) may perform such non-local context aggregation, and we agree that context from adjacent textures and epipolar regions can be important to resolve geometric ambiguities.

However, we expect such context to be needed less in the final block, where GSAB is applied. The splat regions of the final Gaussians differ little from those of the intermediate Gaussians (Fig. 6), which suggests that most of the geometric ambiguity is already resolved before this block.


We deeply appreciate the careful review and the in-depth questions of TzQp, which have helped us present the claims of our paper and the evidence for them more clearly. We would be happy to answer any further questions.

Reviewer_nUGNOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies misalignment between Gaussian-token attention and the projected regions of the predicted Gaussians. GSAB uses intermediate Gaussian predictions to construct a spatial attention bias with learned strength and spread. Experiments on C3G and TokenGS show improved novel-view synthesis on RealEstate10K and DL3DV without increasing the output Gaussian count.

Critical strengths
  1. The paper connects an interpretable failure mode to a targeted intervention. The controlled toy experiment supports the alignment hypothesis, while GSAB reduces the measured C3G misalignment rate from 31.2% to 22.1%, providing useful evidence for the method’s motivation.

  2. Improvements occur across models with different pose assumptions and substantially different Gaussian budgets. TokenGS gains of 0.46–0.80 dB over matched reproductions are supported by three-seed results, strengthening the evidence beyond a single architecture.

Critical weaknesses
  1. The specific benefit of precise splat geometry remains insufficiently isolated. Table 3 compares GSAB with no attention bias, but does not include a simpler spatial prior under matched supervision. Consequently, it remains unclear how much the projected Gaussian shape contributes beyond generic spatial guidance.

  2. The magnitude of the final C3G improvement is not fully isolated from camera and rendering choices. Appendix C.1 provides a useful matched-field-of-view comparison at 200k iterations, showing a 0.37 dB gain, but the corresponding control is absent at the full 450k training budget.

Questions for authors
  1. Under identical intermediate supervision and camera settings, how does GSAB compare with an isotropic spatial bias centered at the same projected Gaussian location?
  2. At 450k iterations, what improvement remains when C3G and C3G + GSAB use the same camera-estimation procedure and rendering field of view?
2 comments
AuthorsOpenReview ↗

Response to Reviewer nUGN

We thank nUGN for the constructive review. We answer both questions below.

W1 / Q1. Comparison with an isotropic attention bias.

We are running the requested comparison and will post the results in a follow-up comment.

We train C3G with an isotropic attention bias in the same setting as C3G+GSAB, except for the shape of the attention bias:

  • GSAB uses the anisotropic shape of the projected intermediate Gaussian.
  • The isotropic bias uses a circle centered at the projected intermediate Gaussian.

We will compare C3G, C3G+GSAB, and C3G with the isotropic bias at 100K iterations on the 1,000 test scenes, and we expect the results within the discussion period.

W2 / Q2. Comparison under the same camera condition with fully trained models (450K training iterations).

At 450K iterations, GSAB still improves when all models use the same GT camera parameters from the dataset and rendering field of view, by 0.45 dB over the released C3G and by 1.12 dB over C3G*.

We compare PSNR on the 1,000 test scenes of RealEstate10K for the models in Table 1, with the addition of C3G*+GSAB when using the same camera setting as the baselines:

MethodCameraPSNR
C3G (released)GT22.53
C3G*GT21.86
C3G*+GSABGT22.98
C3G*+GSABPredicted23.24

When evaluating C3G*+GSAB with the GT camera, we use it for both the GSAB bias and rendering, because GSAB relies on the projected intermediate Gaussians being aligned with the camera used for rendering, and mixing cameras from different sources breaks this alignment (Appendix C.3, Table 7).


We sincerely thank nUGN for the constructive questions, which have led us to examine the source of the gain of GSAB more carefully. We will share the remaining results in a follow-up comment.

AuthorsOpenReview ↗

Follow-up Response to W1 / Q1

W1 / Q1. Comparison with an isotropic attention bias at 100K iterations.

The isotropic bias improves the PSNR of C3G as much as GSAB does. When we further compare the PSNR at image edges, GSAB gives a higher PSNR than the isotropic bias.

We set up the comparison as follows:

  • Isotropic bias: a circle centered at the projected intermediate Gaussian, as described in W1 / Q1 of our earlier response. All other settings are the same as in GSAB.
  • Edge PSNR: the PSNR over the edge pixels, which we detect in each test image with the Canny edge detector (thresholds 100 and 200). We report this because edges are likely where the attention most needs to follow the shape and orientation of each projected intermediate Gaussian.

We compare PSNR and Edge PSNR at 100K training iterations on the 1,000 test scenes:

MethodPSNREdge PSNR
C3G20.6417.95
C3G+GSAB21.6118.70
C3G+isotropic bias21.6118.55

On the edge pixels, the mean per-scene PSNR gain of GSAB over the isotropic bias is 0.155 dB, about 10 times its standard error of 0.016 dB. On the remaining pixels, the two biases give the same PSNR of 22.05 dB.

This suggests that the gain over C3G comes from guiding the attention of each token to the location of its own intermediate Gaussian. The anisotropic shape gives GSAB an advantage at edges, which may matter more in scenes with many thin structures and sharp boundaries.


We thank you for raising this question, which has helped us state more precisely which part of GSAB brings the gain. We would be happy to provide any further details.

Reviewer_Vh5LOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Asks whether a Gaussian token attends to the image region its own predicted Gaussian covers. Attention region and splat region are each the smallest set of patch tokens carrying 90% of the respective mass; a token is misaligned when precision or recall against the splat region falls below τ = 0.4. In released C3G on RealEstate10K, 31.2% of tokens are misaligned. GSAB decodes Gaussians at block L/2, projects them into the views, and adds their Gaussian weight to the final block's cross-attention logits, scaled by a learned strength and spread — needed because logit standard deviation grows to 232 after 50K iterations while the weight is bounded by 1. An intermediate reconstruction loss keeps the two sets close. Applied to C3G (2K Gaussians, pose-free) and TokenGS (262K, posed).

Critical strengths
  • The observation is worth making, and it is made falsifiably. One threshold, two regions, one number: 31.2% misaligned tokens in a released model.
  • The intervention is minimal and transfers. One extra decode by the shared head and one bias added to existing logits, no extra forward pass — working on a pose-free model with 2K Gaussians and a posed model with 262K under a different token granularity.
  • Seed variance is reported where it matters: three seeds for TokenGS, with gains exceeding deviations.
Critical weaknesses
  • Two baselines are used, each flattering the result. C3G* reproduces at 21.86 dB against the released 22.53, a 0.67 dB gap never explained; TokenGS* gives 19.00/22.36/23.07 against published 19.35/23.02/23.69. Headlines quote the released checkpoint for C3G (+0.71 dB) and the weaker reproduction for TokenGS (+0.46/0.66/0.80); against published TokenGS the gains are +0.11 dB at 2 views, zero at 4 (23.02 versus 23.02) and +0.18 at 6.
  • Two experiments disagree on the effect size. Table 3 ablates at 200K iterations: 21.64 → 22.33, or +0.69. Headline numbers come from 450K: 21.86 → 23.24, or +1.38. Either GSAB doubles its advantage after 200K or the settings differ unreportedly. C3G numbers are single-seed, despite seed sensitivity being flagged one section earlier.
  • The toy experiment cannot support its conclusion. Its "learned attention" re-initializes tokens, transformer blocks and head and trains 1K iterations, finishing at 17.73 dB, 0.14 dB above uniform attention. That is converged fixed attention versus unconverged learned attention; the 1.23 dB margin measures optimisation budget, not the value of explicit alignment.
  • The causal chain is asserted between two weakly connected points. Misalignment falls only to 22.1% while PSNR rises 1.38 dB, and nothing links improved tokens to reduced error in those tokens. τ = 0.4 is never varied, and neither latency nor the L/2 choice, unexamined for TokenGS's twelve blocks, is reported.
Questions for authors
  1. What causes the 0.67 dB reproduction gap on C3G, and does GSAB help under the original recipe?
  2. Report GSAB versus baseline at matched budgets and with seeds, for C3G.
  3. Target cameras are optimised 100 iterations before metrics — how much of the gain survives without that fitting?
4 comments
AuthorsOpenReview ↗

Response to Reviewer Vh5L (1/2)

We thank Vh5L for the thorough review. We address each point below.

W1 / Q1. Choice of baselines and the C3G reproduction gap.

We understand the concern, and we would like to clarify that the baselines were not chosen to favor GSAB:

  • For C3G, we state the gain of GSAB against the released checkpoint, which performs better than our reproduction C3G*.
  • For TokenGS, the published results are trained on an internally processed version of DL3DV that is not public (issue #9 of the official TokenGS repository), so our reproduction TokenGS* is the baseline trained on the same data as TokenGS*+GSAB.

Regarding the 0.67 dB reproduction gap of C3G, we could not identify the exact cause, and we believe the most likely cause is training instability.

  • The official code is unstable and collapses during training (issue #15 of the official C3G repository), so both C3G* and C3G*+GSAB are trained with gradient gating added to the official code (Appendix A.2).
  • Because training is unstable, a reproduction can vary considerably across runs, but unfortunately our compute resources do not allow multiple seeds in the official training setting of C3G (effective batch size 64, 450K iterations).

Regarding the TokenGS reproduction gap, we believe the most likely cause is the difference in training data, and we therefore compare GSAB with TokenGS* trained on the same data.

  • Our evaluation code reproduces the published numbers with the released TokenGS checkpoint (4 views: 22.98 vs. 23.02 dB), so the gap comes from training rather than evaluation.
  • We followed the preprocessing code that the authors pointed to and the settings of the official code and the paper as closely as possible.
  • Our only changes to the official code are two stability safeguards: steps with a NaN or infinite gradient are skipped, and the Gaussian scale is capped to avoid overflow.

In addition, we report the PSNR of TokenGS* and TokenGS*+GSAB for each of the 3 seeds with 4 input views:

Methodseed 42seed 44seed 55Mean ± std
TokenGS*22.7022.1322.2422.36 ± 0.30
TokenGS*+GSAB23.1323.1022.8423.02 ± 0.16

Even the lowest PSNR of TokenGS*+GSAB exceeds the highest PSNR of TokenGS* among the 3 seeds.

To let the reproductions be checked directly, we have uploaded our code as the supplementary material. It includes:

  • Training and evaluation code for our reproductions of C3G and TokenGS and for GSAB on both.
  • Dataset preparation. For DL3DV, a script that follows the public script suggested by the TokenGS authors in issue #9. For RealEstate10K, instructions to obtain the preprocessed version used for C3G.

W2 / Q2. Effect size at 200K vs. 450K, and seeds for C3G.

We thank the reviewer for pointing this out. The two settings differ, and while we reported the training iterations of Table 3, we omitted its batch size. Table 3 uses a reduced setting so that the ablations could run in parallel:

SettingGPUsEffective batch sizeIterationsSamples seen
Table 18 H10064450K28.8M
Table 38 V10032200K6.4M

We see two reasons why the gain of GSAB over our reproduction is larger in Table 1 (+1.38 dB) than in Table 3 (+0.69 dB):

  • Table 1 is trained on about 4.5 times more samples than Table 3.
  • As discussed in W1 / Q1, C3G training is unstable, so single runs can vary.

We have stated the setting of Table 3 in Sec. 4.2 of the revised paper.

For additional seed results, we are training more seeds of C3G and C3G+GSAB with a matched budget in the setting of Table 3 (200K iterations, effective batch size 32), and we will report the mean and standard deviation in a follow-up comment.

W3. Is the learned attention in the toy experiment under-trained?

To check whether the PSNR gap in Fig. 2 comes from under-trained learned attention, we trained all toy settings 10 times longer (10K iterations). The learned attention converges at around 2K iterations, while attention fixed to the splat region still gives a 0.65 dB higher PSNR at 10K. The PSNR gap is therefore not due to the learned attention being under-trained.

The splat region, learned, and uniform settings are the same as in Sec. 3.2 and Fig. 2. We evaluated every 1K iterations on the same six scenes, and attention fixed to the splat region gives a higher PSNR than learned and uniform attention at every checkpoint.

The table shows the mean PSNR over the six scenes at selected checkpoints:

IterationsSplat region (k = 1)LearnedUniform
1K18.8917.7817.68
2K18.9918.1818.19
10K18.7918.1418.12

The 1K values differ slightly from Fig. 2 (within 0.1 dB) because all settings were trained again for this extended experiment.

AuthorsOpenReview ↗

Response to Reviewer Vh5L (2/2)

W4a. Relationship between improved alignment and reduced error.

The PSNR increase from GSAB is larger in scenes whose misaligned ratio decreased, for both baselines (the released C3G checkpoint and our reproduction C3G*).

For each scene, we compute the misaligned ratio, which is the percentage of its Gaussian tokens that are misaligned. We then measure how the misaligned ratio and the PSNR change from each baseline to C3G*+GSAB.

We group the scenes by whether the misaligned ratio decreased or increased, and compare the mean PSNR change of the two groups:

C3G*+GSAB vs.Misalignment decreased Scenes (%) / Δ\DeltaPSNR (dB)Misalignment increased Scenes (%) / Δ\DeltaPSNR (dB)Spearman ρ\rho (Δ\DeltaMisaligned, Δ\DeltaPSNR)
C3G*85.7 / +1.5214.3 / +0.60−0.53
C3G (released)88.5 / +0.7811.5 / +0.22−0.36

Spearman ρ\rho is the rank correlation coefficient between the per-scene change in the misaligned ratio and the change in PSNR, computed over all 1,000 test scenes. The negative ρ\rho for both baselines (−0.53 and −0.36) also shows that a larger improvement in alignment comes with a larger reduction in reconstruction error.

W4b. Varying the misalignment threshold τ\tau.

GSAB reduces misalignment consistently across thresholds.

We compare the misaligned ratio (%) of the released C3G and C3G*+GSAB at τ=0.3\tau = 0.3, 0.40.4, and 0.50.5:

Methodτ=0.3\tau = 0.3τ=0.4\tau = 0.4τ=0.5\tau = 0.5
C3G (released)19.231.245.4
C3G*+GSAB11.922.135.6

We use τ=0.4\tau = 0.4 in the paper to count misaligned tokens conservatively. We have added the misaligned ratio at different thresholds to Table 9 in Appendix C.5 of the revised paper.

W4c. Computational cost.

GSAB adds 3.9% to the inference time of C3G with 2 input views and 15.2% to that of TokenGS with 4 input views, while peak memory increases by at most 0.14 GB and the number of parameters by less than 0.5%.

We measure the cost of Gaussian prediction on a single RTX A5000:

MethodInference time (ms)Peak memory (GB)#Params
C3G (released)130.84.331.013B
C3G*+GSAB135.84.401.017B
TokenGS*119.01.51222.0M
TokenGS*+GSAB137.11.65223.1M

The increase is larger for TokenGS likely because its bias is computed from more intermediate Gaussians, 64 per token compared with one per token in C3G.

We have added this cost measurement to Table 5 in Appendix A.4 of the revised paper.

W4d. Transformer block choice for intermediate Gaussian decoding.

GSAB is robust to the choice of the block that decodes the intermediate Gaussians, and block L/2 performs well for both models.

For TokenGS, we compare blocks 3, 6 (L/2), and 11 of its 12 blocks with 4 input views, after training for 50K iterations out of the 150K iterations of stage 1 because of time constraints:

No biasBlock 3Block 6Block 11
PSNR18.8220.2820.3720.08

Every decoding block improves over no bias by more than 1.2 dB, while the PSNR differs by less than 0.3 dB across the blocks. We have added this comparison to Table 4 in Sec. 4.2 of the revised paper.

C3G has only two decoder blocks, so block 1 (L/2) is its only intermediate position. A possible alternative is to build the bias from the Gaussians of the final block, which requires running the final block twice.

We compare the two choices with models trained for 200K iterations with an effective batch size of 32. To exclude any effect of the camera heads trained separately for each model, both use the GT camera parameters instead:

No biasBlock 1Block 2 (two passes)
PSNR21.6422.2222.09

Block 1 gives a slightly higher PSNR than block 2, which makes block 1 a reasonable choice for decoding the intermediate Gaussians of GSAB in C3G.

Q3. Gain without test-time pose optimization.

The gain of GSAB remains even without test-time pose optimization.

We compare the released C3G, C3G*, and C3G*+GSAB on the 1,000 test scenes of RealEstate10K, repeating the evaluation of Table 1 with test-time pose optimization turned off and everything else unchanged:

Test-time pose optimizationC3G (released)C3G*C3G*+GSAB
On (Table 1)22.5321.8623.24
Off20.3920.0421.89

Without test-time pose optimization, C3G*+GSAB still gives the highest PSNR, above both the released C3G and our reproduction C3G*.

We have added this evaluation to Table 8 in Appendix C.4 of the revised paper, which also reports SSIM and LPIPS.


We are grateful to Vh5L for the detailed comments, which have led us to report our experimental settings and analyses more completely. We would be happy to discuss any remaining concerns.

Thanks for your reply, I will raise my score.

AuthorsOpenReview ↗

Follow-up Response to W2 / Q2

Thank you for your reply and for raising your score. Below we report the additional seeds for C3G that we mentioned in our earlier response.

W2 / Q2. Additional seeds for C3G.

GSAB improves the PSNR of C3G in each of 3 seeds, by 0.72 dB on average.

We repeat the training of Table 3 (200K iterations with an effective batch size of 32) with two additional seeds, and evaluate on the 1,000 test scenes as in Table 3:

MethodSeed 111123 (Table 3)Seed 4Seed 5Mean ± std
C3G21.6421.6221.6721.64 ± 0.02
C3G+GSAB22.3322.3722.3922.36 ± 0.03

Seed 111123 is the default seed of the official C3G code.

The gain from GSAB is consistent across seeds, ranging from 0.69 to 0.75 dB, and the lowest PSNR of C3G+GSAB exceeds the highest PSNR of C3G. Therefore GSAB improves C3G consistently in the setting of Table 3.


We are grateful for your thorough review and detailed questions, which have helped us state the experimental settings more completely and strengthen the analyses in the paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their constructive reviews and the discussion. In their reviews, the reviewers recognized the following strengths:

  • An interpretable failure mode, misaligned attention, measured on a released model (Vh5L, nUGN)
  • A lightweight method that adds one intermediate decoding and one attention bias, without an extra forward pass (Vh5L)
  • Gains across models with different pose assumptions and Gaussian budgets, exceeding the variation across 3 seeds of TokenGS (Vh5L, nUGN)
  • Clear presentation (Vh5L, nUGN, TzQp)

The reviewers' questions and comments led to additional analyses, results, and clarifications. Below we highlight a few of our answers:

  • Generalization (TzQp W1 / Q3)
    • Concern: GSAB should be validated on another token-based model to support the claim of generalizability.
    • Our Response: GSAB improves the PSNR of GlobalSplat by 1.00 to 1.07 dB at 12, 24, and 36 input views. GSAB now improves three token-based models, each over a baseline trained in the same setting.
  • Contribution of alignment to the gain (TzQp W2 / Q2a)
    • Concern: Part of the gain may come from the extra modulation of the final block rather than from improved alignment.
    • Our Response: Two results suggest that alignment itself also contributes to the gain:
      • Across test scenes: the PSNR gain of GSAB is 2.5 to 3.5 times larger where it reduces the misaligned ratio.
      • Hard mask (new experiment): we replace the GSAB bias with a hard mask that only restricts attention to the splat. Although the hard mask has no learned strength or spread, it improves C3G by 0.45 dB at 100K training iterations.
  • Reproduction and seeds (Vh5L W1 / Q1 and W2 / Q2, TzQp W1 / Q1)
    • Concern: Our reproductions of TokenGS and C3G fall below the published results, and the C3G results used a single seed.
    • Our Response:
      • Reproduction gap: for TokenGS, the gap likely arises because the published model was trained on an internally processed version of DL3DV, which is not public. For C3G, the most likely cause is unstable training of the official code.
      • Seeds: for both TokenGS and C3G, GSAB improves each of 3 seeds. For C3G, we trained two additional seeds in the setting of Table 3, and the mean PSNR gain over the 3 seeds is 0.72 dB.
      • Code: we have added our code for reproducing both baselines and for GSAB to the supplementary material.
  • Gain comparison under the same camera (nUGN W2 / Q2)
    • Concern: The PSNR gain on C3G after 450K training iterations was not isolated from the choice of camera parameters.
    • Our Response: When all models use the same GT camera parameters, C3G*+GSAB still exceeds the released C3G by 0.45 dB and our reproduction C3G* by 1.12 dB.

We have revised the paper as follows, with all changes marked in red:

  • Scope of our claim, in the abstract and Sec. 4.2 (TzQp)
  • Baseline of each reported gain, in Sec. 1 (Vh5L)
  • Conclusion of the toy experiment, in Sec. 3.2 (TzQp, Vh5L)
  • Training settings, in Sec. 4.2 (Table 3) and Appendices A.2 and C.3 (Vh5L)
  • Evaluation protocol, in Sec. 4.1 (Vh5L)
  • Added analyses (TzQp, Vh5L): the choice of the decoding block (Table 4), the computational cost (Table 5), PSNR against the number of Gaussians (Fig. 8), the evaluation without test-time pose optimization (Table 8), and the misaligned ratio at other thresholds (Table 9)

We have also added to the supplementary material:

  • Training and evaluation code for our reproductions of C3G and TokenGS baselines, and for GSAB on both.
  • Data preparation: a script for DL3DV and instructions for obtaining the preprocessed RealEstate10K used by C3G.

We thank the reviewers and the AC, and hope this summary is helpful for the remaining discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.