acceptodds
Under review as a conference paper at ICLR 2027

RemixGS: Scalable Feed-Forward Gaussian Splatting with View-Register Mixing and Interleaved Gaussian Queries

Abstract

Feed-forward Gaussian Splatting models predict 3D Gaussians from multi-view inputs in a single network pass, replacing iterative per-scene optimisation. The prevalent methods nevertheless scale poorly in viewpoint span, input views, and representation fidelity. Their joint attention involves every patch token of every view, although cross-view alignment is chiefly geometric; their decoders entangle geometry and appearance, allowing geometric errors to be baked into appearance. This paper presents RemixGS, a novel feed-forward Gaussian Splatting method that efficiently reconstructs high-fidelity scenes from large-scale multi-view inputs. Specifically, RemixGS pairs a scalable alternating multi-view encoder with a geometry-texture interleaved Gaussian decoder. The encoder alternates between a local block confined to a single view and a global view-register mixer spanning all views, routing all cross-view interaction through a few per-view registers instead of dense attention across all patch tokens. The decoder carries a fixed set of query tokens through interleaved geometry and texture blocks, where texture reads geometry but not vice versa, preventing geometric errors from being baked into appearance. RemixGS is first trained at a base input and representation scale and then extended to larger-scale inputs and finer representations. Experiments show that our RemixGS consistently surpasses state-of-the-art methods in reconstruction quality as the input scale increases, with substantially lower training and inference cost. Code will be publicly released to facilitate reproducible research.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Feed-forward, Gaussian Splatting,3D Reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9fGfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents RemixGS, a feed-forward Gaussian Splatting model designed to scale to larger multi-view inputs. Its encoder alternates per-view attention with a view-register mixer that routes cross-view interactions through a small set of learned registers, avoiding dense attention across all image patches. Its decoder uses interleaved geometry and texture blocks with one-way geometry-to-texture guidance, separately predicting geometric attributes and color. Experiments demonstrate competitive rendering quality and improved computational efficiency.

Critical strengths
  1. Clear Insight for the Scalable Cross-View Interaction.

The view-register mixer provides a clear and effective mechanism for reducing cross-view attention costs. The efficiency analysis and measured runtime results support its scalability as the number of input views increases.

  1. Comprehensive Experimental Results.

The experimental results demonstrate strong reconstruction quality and computational efficiency across a range of input settings.

Critical weaknesses

1. Insufficient Component Analysis.

The contributions of the two main components are not sufficiently disentangled. The register-count ablation does not establish the effect of replacing dense attention, while the decoder ablation changes both the processing structure and attribute separation. A controlled comparison combining each encoder with each decoder would better establish their individual quality–efficiency benefits.

2. Limited Baseline Comparisons.

The experiments are inadequate to support the broad performance claims. The main evaluation includes only TokenGS and QuerySplat; additional critical baselines, such as VolSplat [1], FreeSplat [2], GaussianGraphNetwork [3] and ZPressor [4], should be considered under matched settings. The supplementary zero-shot evaluation does not establish a clear advantage over the strongest compared methods, and differences in training data and evaluation scenes further limit its conclusiveness.

References

[1] Weijie Wang et al. “VolSplat: Rethinking Feed-Forward 3D Gaussian Splatting with Voxel-Aligned Prediction.” ECCV, 2026.

[2] Yunsong Wang et al. “FreeSplat: Generalizable 3D Gaussian Splatting Towards Free-View Synthesis of Indoor Scenes.” NeurIPS, 2024.

[3] Shengjun Zhang et al. “Gaussian Graph Network: Learning Efficient and Generalizable Gaussian Representations from Multi-view Images.” NeurIPS, 2024.

[4] Weijie Wang et al. “ZPressor: Bottleneck-Aware Compression for Scalable Feed-Forward 3DGS.” NeurIPS, 2025.

Questions for authors
  1. Clarify the Individual Component Contributions.

Could the authors provide a controlled ablation to isolate their individual contributions of the encoder and decoder?

  1. Broader Baseline Evaluation.

Could the authors include comparisons with additional relevant baselines under matched settings to substantiate the claimed advantages?

  1. Evidence for Reduced Geometry–Appearance Compensation.

What direct evidence supports the claim that the decoder prevents appearance from absorbing geometric errors?

6 comments
AuthorsOpenReview ↗

Response to Reviewer 9fGf (Part 1 of 2)

We thank the reviewer for the careful reading and for recognising the clear insight of the view-register mixer and the comprehensive experiments. Below we respond to each weakness (W) and question (Q) in turn.


W1 / Q1 — controlled encoder × decoder ablation

We add this comparison in the revision: the dense encoder of TokenGS and our view-register encoder are each combined with the single-path decoder of TokenGS and our interleaved decoder, and all four models are trained from scratch for 150k steps under one recipe (Table 4 and Section 4.3).

PSNR↑40frames180framesFulltrajectory
EncoderDecoderV=824V=1664V=3296
Dense (TokenGS)TokenGS22.2923.5518.8519.4718.2118.63
OursTokenGS21.5122.4618.4919.0717.8118.23
Dense (TokenGS)Interleaved (ours)23.5125.6219.5720.9719.1819.89
OursInterleaved (ours)23.2325.0720.0120.8519.4019.91

PSNR in dB at 256×256 on DL3DV-10K.

Decoder. Our interleaved decoder raises PSNR by 1.52 to 2.61 dB with our encoder and by 0.72 to 2.07 dB with the dense one.

Encoder. With our interleaved decoder, the dense encoder of TokenGS is only 0.05 dB higher than our encoder in average PSNR over the six settings, but at a much higher cost: its cost grows quadratically with the view count, whereas ours grows close to linearly, because views exchange information only through a few registers (Section 3.1).


W2 / Q2 — additional baselines and the zero-shot evaluation

In the revision, we add ZPressor [1] on our DL3DV settings (Appendix D), and VolSplat [2] and GGN [3], which release no DL3DV weights, on the RealEstate10K zero-shot protocol (Appendix C). FreeSplat [4] releases ScanNet weights only, so we do not evaluate it. Our RemixGS has higher PSNR than all three in every setting, and its lead grows with the view count: over ZPressor from 2.68 to 3.04 dB (32 to 96 views, full trajectory) and over VolSplat from 1.27 to 5.01 dB (12 to 36 views). From 8 to 96 views, it takes 0.047–0.218 s and 1.4–3.6 GB, against 0.190–0.249 s and 3.7–4.3 GB for ZPressor. ZPressor is better in LPIPS (0.361 vs. 0.440 at 96 views), as is VolSplat at 12 and 24 views (0.111 and 0.145 vs. 0.170 and 0.160); both are trained with an LPIPS loss.

Zero-shot evaluation. Our RemixGS is trained on DL3DV only (due to limited GPU resources), whereas the other methods in Table 5 are trained on RealEstate10K, so this evaluation measures transfer rather than ranking. Even so, our RemixGS is on par with ZPressor-6 [1] at 24 and 36 views (0.11 dB lower and 0.01 dB higher in PSNR); only the state-of-the-art GlobalSplat-32K [5] is clearly ahead. From 12 to 36 views, our PSNR rises from 27.91 to 28.51 dB, while that of GlobalSplat-32K falls from 29.54 to 29.39 dB and that of LVSM [6] from 28.65 to 26.38 dB. Our test set holds 5,345 of the 5,601 scenes, as the rest are no longer available; VolSplat and GGN are evaluated on these same scenes.

Method12 views24 views36 views
PSNR↑ / SSIM↑ / LPIPS↓PSNR↑ / SSIM↑ / LPIPS↓PSNR↑ / SSIM↑ / LPIPS↓
GGN19.99 / 0.714 / 0.27018.40 / 0.676 / 0.29617.68 / 0.659 / 0.307
VolSplat26.64 / 0.896 / 0.11124.65 / 0.852 / 0.14523.50 / 0.817 / 0.168
RemixGS (ours)27.91 / 0.885 / 0.17028.40 / 0.895 / 0.16028.51 / 0.897 / 0.157
RemixGS† (ours, fine-tuned)28.43 / 0.893 / 0.16229.03 / 0.904 / 0.14929.19 / 0.907 / 0.146

Fine-tuning on RealEstate10K. To reduce the difference in training data, we fine-tune our RemixGS on the RealEstate10K training split for 10k steps with the same losses (no perceptual loss) and evaluate it under the same protocol. Its PSNR rises by 0.52–0.68 dB, to 28.43, 29.03 and 29.19 dB at 12, 24 and 36 views: 0.52 and 0.69 dB above ZPressor-6 at 24 and 36 views, and within 0.45 and 0.20 dB of GlobalSplat-32K, with higher SSIM (0.904 and 0.907 vs. 0.901 and 0.899). Both remain better in LPIPS.


References

[1] Weijie Wang et al. "ZPressor: Bottleneck-Aware Compression for Scalable Feed-Forward 3DGS." NeurIPS, 2025.

[2] Weijie Wang et al. "VolSplat: Rethinking Feed-Forward 3D Gaussian Splatting with Voxel-Aligned Prediction." ECCV, 2026.

[3] Shengjun Zhang et al. "Gaussian Graph Network: Learning Efficient and Generalizable Gaussian Representations from Multi-view Images." NeurIPS, 2024.

[4] Yunsong Wang et al. "FreeSplat: Generalizable 3D Gaussian Splatting Towards Free-View Synthesis of Indoor Scenes." NeurIPS, 2024.

[5] Roni Itkin et al. "GlobalSplat: Efficient Feed-Forward 3D Gaussian Splatting via Global Scene Tokens." ECCV, 2026.

[6] Haian Jin et al. "LVSM: A Large View Synthesis Model with Minimal 3D Inductive Bias." ICLR, 2025.

AuthorsOpenReview ↗

Response to Reviewer 9fGf (Part 2 of 2)

Q3 — direct evidence for reduced geometry–appearance compensation

In the revision, we measure geometry directly against a pseudo ground truth from Depth Anything 3 [1] (Appendix F). With our encoder, our interleaved decoder raises PSNR by 1.52 and 1.78 dB, and at the same time lowers depth AbsRel from 0.555 to 0.497 and from 0.343 to 0.329. With the dense encoder, AbsRel rises instead, as its features show almost no cross-view consistency (0.001–0.050 vs. 0.243–0.408 for ours), so the geometry stream, which does not read appearance, has no consistent cue. The interleaved decoder is itself less sensitive to feature noise than the single-stream decoder with both encoders, and least with ours (2.15 vs. 4.37 at V=64).

V=16V=64
EncoderDecoderAbsRel↓δ1↑Cons.↑Sens.↓AbsRel↓δ1↑Cons.↑Sens.↓
Dense (TokenGS)TokenGS0.5740.5740.0014.960.4300.6430.0273.08
OursTokenGS0.5550.5790.3537.030.3430.6800.4084.37
Dense (TokenGS)Interleaved (ours)0.7130.5370.0324.020.4440.6550.0502.75
OursInterleaved (ours)0.4970.6320.2433.120.3290.7000.2762.15

Cons. is the cross-view consistency of the features the decoder reads, and Sens. the depth sensitivity to 1% feature noise.

These results show that, in our RemixGS, the interleaved decoder reduces the compensation of geometric errors by appearance.


References

[1] Haotong Lin et al. "Depth Anything 3: Recovering the Visual Space from Any Views." ICLR, 2026.

AuthorsOpenReview ↗

Following Up on Our Responses

Dear Reviewer 9fGf,

We are grateful for your constructive suggestions, which have helped us improve the paper considerably. Our rebuttal responds to all of your comments point by point. Among other additions, we ran an encoder × decoder ablation with a dense-attention control and isolated the effect of the branch split (W1). We also compared with VolSplat, FreeSplat, GGN, and ZPressor under matched settings, clarified the RE10K zero-shot results (W2), and provided direct evidence for the decoder mechanism (Q3).

We would be thankful for a brief note on whether these additions address your concerns before the discussion ends in less than one day. Please feel free to let us know if further clarification would help.

Best regards,

The Authors of Submission 16065

Reviewer_VMehOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RemixGS for efficient feed-forward 3D Gaussian reconstruction from many input images. It exchanges information across views through a small number of register tokens and predicts Gaussian attributes using an interleaved geometry and texture decoder. The Gaussian count is independent of the number of input views. A subsequent adaptation stage increases the query count and input resolution. Experiments evaluate reconstruction on DL3DV across different view counts and viewpoint spans, with additional zero-shot evaluation on RealEstate10K.

Critical strengths
  • Reducing cross-view attention costs is a reasonable approach to handling more input images. The paper reports both runtime and memory measurements. Training takes 63.6 hours compared to 97.3 hours for TokenGS on the same hardware, demonstrating a useful improvement in efficiency.
  • The reconstruction gains over TokenGS are substantial. With the same number of Gaussians, Table 1 shows PSNR improvements of 1.93–2.21 dB and better SSIM and LPIPS across all settings. Evaluation covers 8 to 96 input views and different viewpoint spans. The zero-shot results on RealEstate10K also support generalization.
Critical weaknesses
  • The explanation of the decoder’s benefits in Sec. 3.2 is not sufficiently supported. Passing information only from geometry to texture and using fewer texture blocks does not guarantee that appearance cannot compensate for geometric errors. Table 4 compares a single-stream decoder with the proposed architecture, making it difficult to separate the effects of branch separation and geometry guidance from those of the additional texture blocks.
  • The camera inputs used in the main comparisons are unclear. Appendix A describes using either dataset poses or externally predicted poses, but does not specify which are used in Table 1. If RemixGS and QuerySplat use different pose sources, the results may reflect both reconstruction quality and pose accuracy. This distinction should be clarified.
  • The comparison with existing efficient feed-forward methods could be broader. For example, Off The Grid [1] also reduces redundant Gaussians through adaptive primitive placement, but is neither discussed nor compared. Given the paper’s emphasis on efficiency, a comparison of rendering quality, Gaussian count, runtime, and memory would help assess the benefits beyond query-based methods. Such a comparison should account for differences in camera inputs and supported view counts.

[1] Moreau et al. “Off The Grid: Detection of Primitives for Feed-Forward 3D Gaussian Splatting.” CVPR 2026.

Questions for authors
  • Which poses are provided to RemixGS and TokenGS in Table 1? How are the target cameras aligned with QuerySplat’s predicted coordinate system? Please also clarify whether the runtime measurements in Figure 4 include the pose estimation required by each method.

I will reconsider the score after the author rebuttal.

3 comments
AuthorsOpenReview ↗

Response to Reviewer VMeh (Part 1 of 2)

We thank the reviewer for the careful reading and for recognising the efficiency gains, the substantial improvement over TokenGS and the zero-shot generalisation. Below we respond to each weakness (W) and question (Q) in turn.


W1 — branch separation and guidance vs. additional texture blocks

Block count. Our RemixGS decoder has 12 blocks in total (8 geometry + 4 texture), the same as the 12-layer single-path decoder of TokenGS. In the revision, we add an encoder–decoder ablation (Table 4 and "Encoder–Decoder Ablation" in Section 4.3): the dense encoder of TokenGS and our view-register encoder are each combined with the two decoders, and all four models are trained from scratch under one recipe. The two decoders are thus compared at an equal block count, and with our encoder our interleaved decoder raises PSNR by 1.52 to 2.61 dB. Depth alone does not close this gap: 8 layers is the best depth we found for the single path in the original Table 4, and 6 and 10 layers both trained worse (training PSNR −0.24 and −0.17 dB at the same step).

PSNR↑40frames180framesFulltrajectory
EncoderDecoderV=824V=1664V=3296
Dense (TokenGS)TokenGS22.2923.5518.8519.4718.2118.63
OursTokenGS21.5122.4618.4919.0717.8118.23
Dense (TokenGS)Interleaved (ours)23.5125.6219.5720.9719.1819.89
OursInterleaved (ours)23.2325.0720.0120.8519.4019.91

PSNR in dB at 256×256 on DL3DV-10K.

Geometry measurement. With our encoder, we compare the rendered depth of the two decoders against a pseudo ground truth from Depth Anything 3 [1] (Appendix F). Our interleaved decoder raises PSNR by 1.52 and 1.78 dB, lowers depth AbsRel from 0.555 to 0.497 and from 0.343 to 0.329, and raises δ1 from 0.579 to 0.632 and from 0.680 to 0.700. With the dense encoder, AbsRel rises instead, as its features show almost no cross-view consistency (0.001–0.050 vs. 0.243–0.408 for ours), so the geometry stream, which does not read appearance, has no consistent cue. The interleaved decoder is itself less sensitive to feature noise than the single-stream decoder with both encoders, and least with ours (2.15 vs. 4.37 at V=64).

V=16V=64
EncoderDecoderAbsRel↓δ1↑Cons.↑Sens.↓AbsRel↓δ1↑Cons.↑Sens.↓
Dense (TokenGS)TokenGS0.5740.5740.0014.960.4300.6430.0273.08
OursTokenGS0.5550.5790.3537.030.3430.6800.4084.37
Dense (TokenGS)Interleaved (ours)0.7130.5370.0324.020.4440.6550.0502.75
OursInterleaved (ours)0.4970.6320.2433.120.3290.7000.2762.15

Cons. is the cross-view consistency of the features the decoder reads, and Sens. the depth sensitivity to 1% feature noise.

These results show that, in our RemixGS, the interleaved decoder mitigates the entanglement of geometry and appearance.


References

[1] Haotong Lin et al. "Depth Anything 3: Recovering the Visual Space from Any Views." ICLR, 2026.

AuthorsOpenReview ↗

Response to Reviewer VMeh (Part 2 of 2)

W2 / Q1 — pose sources and target-camera alignment

Pose sources. In Table 1, our RemixGS and TokenGS take the dataset poses for the input and target views. QuerySplat [1] takes no poses; it builds on VGGT-Ω, which already learns to predict camera poses.

Target-camera alignment. To compute the metrics on the same target views as the other methods, each target camera of QuerySplat is mapped from the dataset pose into its predicted frame and refined for 30 steps against the target image, with the Gaussians fixed. For wide viewpoint spans, this alignment scores QuerySplat higher than its official protocol. Over all nine settings, the mean PSNR of QuerySplat under the two alignments differs by only 0.07 dB. The revision states both in Appendix A.

Predicted poses. To remove the difference in pose source, we take the RemixGS$^{512}$ of Table 1, which is trained with dataset poses only, and feed it input poses predicted by VGGT-Ω at test time, without any retraining. We evaluate it under the protocol of QuerySplat$^{512}$, with the same target-camera mapping and 30-step refinement. Even so, it still leads QuerySplat in PSNR in 6 of 9 settings and in SSIM in 8 (mean 21.50 dB / 0.670 vs. 21.41 dB / 0.660). QuerySplat keeps its LPIPS lead, which we attribute to its perceptual loss. Relative to Table 1, PSNR changes by −0.66 to −0.92 dB on the 40-frame span, −0.16 to −0.22 dB on the 180-frame span and at most −0.04 dB on the full trajectory.

PSNR↑40frames180framesFulltrajectory
V=81624V=163264V=326496Mean
QuerySplat$^{512}$23.3223.4323.5520.8020.7920.6920.0120.0920.0021.41
RemixGS$^{512}$ (dataset poses, Table 1)23.8124.6125.0220.5321.0321.2019.8420.2320.3421.85
RemixGS$^{512}$ (VGGT-Ω poses)23.1523.7524.1020.3720.8220.9919.8320.2320.3021.50

W3 — comparison with other efficient feed-forward methods

In the revision, we compare with two efficient feed-forward methods, Off The Grid (OTG) [2] and ZPressor [3], in Appendix D (Tables 6 and 7), using their released weights. OTG takes 518×294 input without camera poses, and ZPressor takes the dataset poses, as our RemixGS does. Our RemixGS has the highest PSNR in every setting, leading ZPressor by 0.03 to 3.04 dB and OTG by 0.53 to 3.57 dB. Both methods are better in LPIPS: they are trained with an LPIPS loss, and they mostly place more Gaussians at pixel locations of the input views, which gives sharper renderings. This also offers useful directions for improving our RemixGS in future work. The efficiency from 8 to 96 input views on one A800 GPU is as follows:

MethodGaussiansLatency (s)Peak memory (GB)
OTG174K–2.09M0.28–5.926.6–26.3
ZPressor688K0.190–0.2493.7–4.3
RemixGS (ours)262K0.047–0.2181.4–3.6

Q1 — does Figure 4 include pose estimation?

Figure 4 includes no separate pose estimation. VGGT-Ω is the backbone of QuerySplat, so its cost is already part of QuerySplat's latency; our RemixGS and TokenGS take the dataset poses. To show what pose estimation would add, we time VGGT-Ω alone at 448×256 on the same A800 (latency in s):

V8162432486496
VGGT-Ω alone0.1060.2020.3090.4250.6901.0131.786
RemixGS (ours)0.0470.0630.0780.0940.1250.1560.218
QuerySplat0.3130.4350.5680.7111.0291.4042.282

Even if our RemixGS first estimated its poses with VGGT-Ω, the total of 0.153 to 2.004 s would remain below the 0.313 to 2.282 s of QuerySplat.


References

[1] Yinglong Li et al. "QuerySplat: Decoupling Geometry and Appearance Representations in 3DGS Prediction." arXiv:2608.01186, 2026.

[2] Arthur Moreau et al. "Off The Grid: Detection of Primitives for Feed-Forward 3D Gaussian Splatting." CVPR, 2026.

[3] Weijie Wang et al. "ZPressor: Bottleneck-Aware Compression for Scalable Feed-Forward 3DGS." NeurIPS, 2025.

AuthorsOpenReview ↗

Gentle Reminder on the Discussion

Dear Reviewer VMeh,

We hope this message finds you well. We truly appreciate your careful reading of our paper, and especially your note that you would reconsider the score after the rebuttal.

We have carefully addressed every point you raised. In particular, we have added direct evidence for the decoder mechanism together with an isolated test of the branch split (W1). We have also clarified the pose source, the target alignment for QuerySplat, and the timing in Fig. 4 (W2), and included a comparison with Off The Grid (W3).

Since the discussion closes in less than one day, any feedback from you would mean a great deal to us. We remain available to answer any further questions.

Best regards,

The Authors of Submission 16065

Reviewer_xebNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a novel feed-forward Gaussian Splatting method that efficiently reconstructs high-fidelity scenes from large-scale multi-view inputs. RemixGS pairs a scalable alternating multi-view encoder with a geometry-texture interleaved Gaussian decoder. The encoder alternates between a local block confined to a single view and a global view-register mixer spanning all views, routing all cross-view interaction through a few per-view registers. The decoder carries a fixed set of query tokens through interleaved geometry and texture blocks, where texture reads geometry but not vice versa, preventing geometric errors from being baked into appearance.

Critical strengths
  1. The method figure and ablation figure are very clear and easy to understand.
  2. The writing is fairly clear and easy to follow.
Critical weaknesses

The core innovation of the author is that the proposed "geometry-texture interleaved Gaussian decoder" is mainly used to solve the problem of “Gaussians’ attributes are usually treated as a whole, leaving geometry and appearance entangled", and "a geometry that departs from the truth may leave a residual that appearance absorbs, lowering the loss while the geometric error remains". However, there is no in-depth analysis of this phenomenon, nor is it proven whether it is really a performance bottleneck. Relying solely on Tab. 4 for simple ablation is not enough to support the starting point and conclusion of the paper. 2. The PSNR and SSIM effects of RemixGS compared to QuerySplat are not significantly improved, and the LPIPS is slightly lower. Furthermore, comparisons under sparse-view settings (V=2, 4) are missing. In addition, in Fig. 2 and 3, the visualization of the supplementary materials is not very good, and no significant improvement has been seen compared to QuerySplat.

Questions for authors

1.How does RemixGS perform under sparse-view conditions? 2.Why was “RemixGS is first trained at a base input and representation scale and then extended to larger-scale inputs and finer representations.” chosen—was it for resource considerations or for effectiveness considerations?

4 comments
AuthorsOpenReview ↗

Response to Reviewer xebN (Part 1 of 3)

We thank the reviewer for the careful reading and for finding the method figure, the ablation figure and the writing clear. Below we respond to each weakness (W) and question (Q) in turn.


W1 — the entanglement phenomenon is not analysed beyond Table 4

We agree that Table 4 alone is not enough and add two analyses. First, a controlled encoder × decoder 2×2, all four models trained from scratch under one recipe (revised Table 4, Section 4.3): the interleaved decoder raises PSNR by 1.52–2.61 dB with our encoder and by 0.72–2.07 dB with the dense one (PSNR in dB at 256×256 on DL3DV-10K):

PSNR↑40frames180framesFulltrajectory
EncoderDecoderV=824V=1664V=3296
Dense (TokenGS)TokenGS22.2923.5518.8519.4718.2118.63
OursTokenGS21.5122.4618.4919.0717.8118.23
Dense (TokenGS)Interleaved (ours)23.5125.6219.5720.9719.1819.89
OursInterleaved (ours)23.2325.0720.0120.8519.4019.91

Second, a direct geometry measurement against a pseudo ground truth from Depth Anything 3 [1] (Appendix F): with our encoder, the interleaved decoder lowers depth AbsRel from 0.555 to 0.497 and from 0.343 to 0.329 while raising PSNR. With the dense encoder, AbsRel rises instead, as its features show almost no cross-view consistency (0.001–0.050 vs. 0.243–0.408 for ours), so the geometry stream, which does not read appearance, has no consistent cue. The interleaved decoder is itself less sensitive to feature noise than the single-stream decoder with both encoders, and least with ours (2.15 vs. 4.37 at V=64).

V=16V=64
EncoderDecoderAbsRel↓δ1↑Cons.↑Sens.↓AbsRel↓δ1↑Cons.↑Sens.↓
Dense (TokenGS)TokenGS0.5740.5740.0014.960.4300.6430.0273.08
OursTokenGS0.5550.5790.3537.030.3430.6800.4084.37
Dense (TokenGS)Interleaved (ours)0.7130.5370.0324.020.4440.6550.0502.75
OursInterleaved (ours)0.4970.6320.2433.120.3290.7000.2762.15

Cons. is the cross-view consistency of the features the decoder reads, and Sens. the depth sensitivity to 1% feature noise.

These results show that, in our RemixGS, the interleaved decoder mitigates the entanglement of geometry and appearance.


References

[1] Haotong Lin et al. "Depth Anything 3: Recovering the Visual Space from Any Views." ICLR, 2026.

AuthorsOpenReview ↗

Response to Reviewer xebN (Part 2 of 3)

W2 — the gain over QuerySplat is not significant; LPIPS is worse

QuerySplat [1], released on arXiv in August 2026, is concurrent work. It builds on the pretrained VGGT-Ω [2] and adds 1,118M parameters of its own, four times the 279M of our RemixGS. Its base stage alone uses twice our training steps and twice our batch size, about 15× our training compute in total, and is followed by three fine-tuning stages. Our RemixGS is trained from scratch on DL3DV-10K and, with half the Gaussians, leads in PSNR in 7 of 9 settings and in SSIM in all 9 (Table 1).

QuerySplat is trained with a perceptual loss, which our RemixGS does not use. To test whether this explains its LPIPS lead, we fine-tune the RemixGS$^{512}$ of Table 1 for 5k steps with the LPIPS loss that QuerySplat uses, at weight 0.05, and evaluate it in the same 9 settings. LPIPS drops by 0.042 to 0.047 in every setting, and the fine-tuned model now matches or beats QuerySplat in 4 of 9 settings and is within 0.018 in the rest. Averaged over the 9 settings, its LPIPS equals that of QuerySplat (0.387), while PSNR and SSIM are lower than in Table 1 by only 0.14 dB and 0.005, and it still leads QuerySplat in PSNR in 7 settings and in SSIM in 8 of 9.

LPIPS↓40frames180framesFulltrajectory
V=81624V=163264V=326496Mean
QuerySplat$^{512}$0.3110.3070.3080.4110.4110.4160.4400.4400.4400.387
RemixGS$^{512}$ (ours, Table 1)0.3530.3330.3260.4720.4620.4590.5000.4930.4890.432
RemixGS$^{512}$ + LPIPS (ours)0.3110.2900.2820.4290.4160.4120.4560.4460.4430.387

As an academic lab with limited GPUs, we cannot currently train our RemixGS at the scale of QuerySplat for a matched comparison. Our fine-tuning takes only 5k steps, whereas QuerySplat trains with the LPIPS loss for 300k steps; we expect our RemixGS to improve further under a comparable training setup.


W2 — the improvement over QuerySplat is not visible in Figures 2 and 3

In the revision, we improve the visualization of all qualitative figures (Figures 2 and 3 and Appendix E). We add boxes and zoomed insets on the regions where the methods differ, and embed the renderings losslessly, so that zooming into the PDF shows the details.

With the insets, the improvement of our RemixGS over QuerySplat is clearly visible. In Scene 1 of Figure 3, QuerySplat dissolves the entrance steps into streaks and blurs the door frames, while our RemixGS keeps the step edges and the door frames distinct. In Scene 2, QuerySplat blurs the shelf goods until neighboring boxes merge, while our RemixGS keeps the shelf edges and the box outlines.


References

[1] Yinglong Li et al. "QuerySplat: Decoupling Geometry and Appearance Representations in 3DGS Prediction." arXiv:2608.01186, 2026.

[2] Jianyuan Wang et al. "VGGT-Ω." CVPR, 2026.

AuthorsOpenReview ↗

Response to Reviewer xebN (Part 3 of 3)

W2 / Q1 — how does our RemixGS perform under sparse views?

We evaluate our RemixGS at V=2/4/6 on the DL3DV benchmark of DepthSplat [1].

MethodV=2V=4V=6
PSNR↑ / SSIM↑ / LPIPS↓PSNR↑ / SSIM↑ / LPIPS↓PSNR↑ / SSIM↑ / LPIPS↓
TokenGS16.28 / 0.456 / 0.53520.03 / 0.619 / 0.43321.85 / 0.684 / 0.390
RemixGS (ours)17.32 / 0.503 / 0.50221.78 / 0.689 / 0.37023.49 / 0.747 / 0.326
QuerySplat$^{512}$20.63 / 0.654 / 0.38922.88 / 0.718 / 0.31923.41 / 0.730 / 0.306
RemixGS$^{512}$ (ours)17.00 / 0.535 / 0.52621.59 / 0.692 / 0.40123.44 / 0.744 / 0.359
RemixGS$^{512}$ + input augment (ours)17.51 / 0.558 / 0.50122.13 / 0.717 / 0.37623.86 / 0.761 / 0.343

Rows marked 512^{512} are evaluated at 512×512, the others at 448×256.

Our RemixGS leads TokenGS [2] at every view count, by 1.0 to 1.8 dB. QuerySplat leads at V=2 and V=4, and the gap closes as views are added: at V=6, our RemixGS is ahead in PSNR and SSIM. As discussed under W2 in Part 2, we attribute the LPIPS lead of QuerySplat largely to its perceptual loss. QuerySplat builds on the pretrained VGGT-Ω, a strong geometry prior when views are few. We acknowledge that the extremely sparse-view setting is not the focus of our design. Our RemixGS exchanges global information across views through per-view registers, which benefits from more views, and the training does not specifically target extremely sparse-view inputs.

To test this, we apply input-view augmentation at test time, without retraining: the same input is augmented to 24 views, each input view kept and complemented by random crops, resized back to 512×512 with the intrinsics adjusted accordingly. The crops add no new pixels or viewpoints. The augmentation raises PSNR by 0.42 to 0.54 dB and SSIM by 0.017 to 0.025, and lowers LPIPS by 0.016 to 0.025, at every view count (last row). Duplicating each view instead leaves the output unchanged within numerical precision, so the gain comes from views that differ. We attribute it to the richer cross-view exchange they allow: each crop carries its own 16 registers, and the augmented input matches the view count our RemixGS is trained with. The same augmentation hurts QuerySplat instead, lowering its PSNR by more than 4 dB, and by up to 10 dB in some scenes. This likely stems from QuerySplat relying on VGGT-Ω to predict the cameras and geometry, as multiple crops of the same view confuse VGGT-Ω and lead it to large prediction errors.

The strength of our RemixGS remains many-view input: with 8 to 96 views, it leads QuerySplat in PSNR in 7 of 9 settings and in SSIM in all 9 (Table 1). These sparse-view results show that our RemixGS begins to surpass QuerySplat in PSNR and SSIM from V=6, and with input-view augmentation the lead widens to 0.45 dB in PSNR and 0.031 in SSIM. In future work, we will build on our RemixGS to improve its performance in extremely sparse-view settings such as V=2 and V=4.


Q2 — why a base stage followed by adaptation?

The staged schedule is a common training strategy that we adopt for compute efficiency; the model itself has no such constraint (Section 3.3). The base stage trains for 150K steps at 256×256 with 1,024 query tokens; the adaptation stage then grows them to 4,096 and raises the resolution to 448×256 or 512×512 in short fine-tuning. Our RemixGS can be trained directly at 512×512 or with more query tokens, at a higher cost. We reason that learning the basic representation of a scene is the hardest part of training and a coarser representation suffices for it; resolution and token count are then raised on top of it. This coarse-to-fine schedule is common in image generation, where cascaded models first generate at low resolution and then super-resolve [3], and in feed-forward 3DGS, e.g., TokenGS [2] and QuerySplat [4].


References

[1] Haofei Xu et al. "DepthSplat: Connecting Gaussian Splatting and Depth." CVPR, 2025.

[2] Jiawei Ren et al. "TokenGS: Decoupling 3D Gaussian Prediction from Pixels with Learnable Tokens." CVPR, 2026.

[3] Jonathan Ho et al. "Cascaded Diffusion Models for High Fidelity Image Generation." JMLR, 2022.

[4] Yinglong Li et al. "QuerySplat: Decoupling Geometry and Appearance Representations in 3DGS Prediction." arXiv:2608.01186, 2026.

AuthorsOpenReview ↗

Kind Follow-up on Our Rebuttal

Dear Reviewer xebN,

Thank you again for your thoughtful review. In our rebuttal submitted on November 13, we have responded to each of your comments and questions, and we have since continued to strengthen it with new results. We now provide direct geometric evidence for the decoder mechanism (W1), demonstrate the performance of our RemixGS under extremely sparse-view inputs (W2/Q1), and clarify the LPIPS gap, the qualitative figures, and the two-stage training (Q2).

With less than one day left in the discussion period, we would greatly appreciate hearing whether these responses resolve your concerns. If anything remains unclear, we would be glad to address it.

Best regards,

The Authors of Submission 16065

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank all reviewers for their careful reading and constructive comments. The reviewers found our view-register mixer a clear and effective way to reduce cross-view attention cost, supported by measured runtime and memory (9fGf, VMeh); noted the substantial gains over TokenGS at the same number of Gaussians, the broad evaluation from 8 to 96 views, and the zero-shot generalisation to RealEstate10K (VMeh); and found the figures and writing clear (xebN).

We have responded to every weakness and question of each reviewer and revised the paper accordingly, with changes marked in blue. The main concerns and our evidence are summarised below.

  • Contributions of the encoder and decoder (xebN, VMeh, 9fGf). A new 2×2 ablation crosses the dense encoder of TokenGS and our view-register encoder with the single-stream decoder of TokenGS and our interleaved decoder, all trained from scratch under one recipe (Table 4). Our interleaved decoder raises PSNR by 1.52–2.61 dB with our encoder and by 0.72–2.07 dB with the dense one. With our interleaved decoder, our encoder is within 0.05 dB of the dense one in average PSNR, while its cost grows close to linearly with the input view count.
  • Geometry–appearance compensation (xebN, VMeh, 9fGf). Against a pseudo ground truth from Depth Anything 3 [1] (Appendix F), our interleaved decoder lowers depth AbsRel while raising PSNR, and it is less sensitive to feature noise than the single-stream decoder with both encoders.
  • Pose sources (VMeh). Table 1 uses the dataset poses for our RemixGS and TokenGS (Appendix A). Given poses predicted by VGGT-Ω at test time, without any retraining, our RemixGS still leads QuerySplat [2] in PSNR in 6 of 9 settings and in SSIM in 8 of 9.
  • Comparison with QuerySplat (xebN). Trained from scratch with about 1/15 of its training compute, our RemixGS leads in PSNR in 7 of 9 settings and in SSIM in all 9. With only 5k steps of LPIPS fine-tuning, it matches the average LPIPS of QuerySplat (0.387).
  • Additional baselines (VMeh, 9fGf). We add ZPressor [3] and Off The Grid [4] on DL3DV (Appendix D), and VolSplat [5] and GGN [6] on RealEstate10K (Appendix C). Our RemixGS has higher PSNR than all four in every setting, with lower latency and memory than ZPressor and Off The Grid. Fine-tuned on RealEstate10K for only 10k steps, its PSNR rises by 0.52–0.68 dB, exceeding ZPressor-6 by 0.52 and 0.69 dB at 24 and 36 views and coming within 0.45 and 0.20 dB of GlobalSplat-32K.
  • Sparse views and qualitative figures (xebN). We report results with 2 to 6 input views, and add zoomed insets to all qualitative figures (Figures 2 and 3, Appendix E).

We thank the reviewers for these suggestions, which have strengthened the analysis and experiments of our RemixGS.

References

[1] Haotong Lin et al. "Depth Anything 3: Recovering the Visual Space from Any Views." ICLR, 2026.

[2] Yinglong Li et al. "QuerySplat: Decoupling Geometry and Appearance Representations in 3DGS Prediction." arXiv:2608.01186, 2026.

[3] Weijie Wang et al. "ZPressor: Bottleneck-Aware Compression for Scalable Feed-Forward 3DGS." NeurIPS, 2025.

[4] Arthur Moreau et al. "Off The Grid: Detection of Primitives for Feed-Forward 3D Gaussian Splatting." CVPR, 2026.

[5] Weijie Wang et al. "VolSplat: Rethinking Feed-Forward 3D Gaussian Splatting with Voxel-Aligned Prediction." ECCV, 2026.

[6] Shengjun Zhang et al. "Gaussian Graph Network: Learning Efficient and Generalizable Gaussian Representations from Multi-view Images." NeurIPS, 2024.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.