acceptodds
Under review as a conference paper at ICLR 2027

PixGR: Pixel-Space Generative Decoding for Fast Feed-Forward 3D Reconstruction

Abstract

We present PixGR, an efficient framework that integrates lightweight pixel-space generative modeling directly into feed-forward 3D Gaussian reconstruction. Rather than conditioning generation on an explicit and imperfect reconstruction, PixGR encodes the context observations into a 3D-aware scene latent and generatively reconstructs a global 3D Gaussian representation that jointly supports context and target views. We formulate this process with pixel-space flow matching, enabling generative supervision of 3D reconstruction through differentiable rendering without relying on compressed VAE latents. To retain feed-forward efficiency, PixGR encodes the input scene only once and performs iterative inference with a lightweight DiT decoder. Experiments demonstrate improved reconstruction over regression-based approaches in challenging sparse-view settings, particularly with widely spaced observations and view extrapolation, while maintaining sub-second inference and remaining competitive with substantially heavier pretrained generative models. Code will be publicly available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Reconstruction, Feed-Forward, Pixel-Space Flow Matching

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_YbGcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The proposed PixGR uses pixel-space flow matching for sparse-view 3D reconstruction. The scene encoder extracts and caches input features in a single pass, while a lightweight generative decoder combines a noisy target image and camera parameters to iteratively predict a shared 3D Gaussian representation, which is then rendered to produce a denoised image. The proposed method outperforms in reconstruction quality on RealEstate10K, DL3DV, and ACID. However, there are clear precedents for this type of integrated generative reconstruction, and the current experiments have not sufficiently prove the value of the incremental design proposed in this paper.

Critical strengths
  1. The performance improvement is clear. The PSNR of the RE10K extrapolation reaches 26.03 dB, which is higher than the 23.98 dB achieved by the SOTA generative baseline in Table 2. The deterministic decoder of the same scale in Table 3 achieves 24.34 dB, which supports the effectiveness of generative decoding in this architecture.

  2. It shows practical efficiency and outputs reusable explicit scenes. Scene encoding is computed only once. The DL3DV experiment constructs a shared 3DGS using three target keyframes and then renders all 42 evaluation viewpoints.

Critical weaknesses
  1. The distinctions from existing methods need to be further clarified. DiffusionGS [1] has integrated Gaussian prediction and rendering into the denoising process and demonstrated results in real-world scenes. PixWorld [2] also employs pixel-space flow matching and differentiable rendering.

  2. The ablation studies seems insufficient. Table 3 supports the effectiveness of generative decoding, but lacks pixel-space/latent-space comparisons, making it difficult to confirm the specific benefits of avoiding VAE. Furthermore, the number of parameters does not account for the computational cost of iterations.

[1] Cai, Yuanhao, et al. "Baking gaussian splatting into diffusion denoiser for fast and scalable single-stage image-to-3d generation and reconstruction." ICCV, 2025. [2] Gao, Sensen, et al. "PixWorld: Unifying 3D Scene Generation and Reconstruction in Pixel Space." arXiv preprint arXiv:2607.05373 (2026).

Questions for authors
  1. Compared to DiffusionGS and PixWorld, what specific new techniques have been introduced? Please specify whether PixWorld is a prior or concurrent work relative to this paper.

  2. Can the authors provide comparisons with more closely related generative methods, as well as evidence isolating the benefits of the pixel-space design and iterative computation? DiffusionGS was compared only in a single-view setting, and PixWorld has not yet been included in the experiments. Please also specify the training resources and inference budget.

5 comments
AuthorsOpenReview ↗

Response #1

We thank the reviewer for the helpful feedback.

W1 & Q1 & Q2: PixGR vs. DiffusionGS

The main methodological differences between PixGR and DiffusionGS are the encoder–decoder architecture and the flow-matching formulation.

A. Methodological Differences

1. Decoder-only vs. Encoder–Decoder Architecture.

DiffusionGS adopts a decoder-only architecture that jointly processes context and target images throughout the entire transformer, leading to computational inefficiencies in three ways:

(1) Across target viewpoints: The same context images must be repeatedly processed when conditioned on different target viewpoints.

(2) Across diffusion sampling steps: Context images must be reprocessed at every sampling step.

(3) With increasing context views: The computational cost grows substantially as the number of context images increases.

In contrast, PixGR adopts an encoder–decoder architecture that encodes clean context images only once into latent representations, while iterative generation is performed entirely within the decoder. This design significantly reduces computational redundancy across diffusion sampling steps and target viewpoints, and improves efficiency when processing larger numbers of context views.

Moreover, since the context encoder is executed only once, PixGR can leverage large pretrained models, such as VGGT, as the context encoder to extract stronger geometric features without substantially increasing the computational cost of iterative generation.

2. DDPM vs. Flow Matching.

DiffusionGS adopts a DDPM formulation, generating target views through a stochastic reverse denoising process. In contrast, PixGR employs flow matching, which learns a continuous transport from noise to clean target images and performs sampling using an ODE solver. Our experiments demonstrate that flow matching achieves higher sampling efficiency than DDPM.

B. Controlled Ablation Studies

To systematically evaluate these design choices, we conduct ablation experiments on RE10K, with all methods trained and evaluated using 2 context views and 3 target views. Since DiffusionGS officially supports only a single context view, we fine-tune it with 2 context views following its official training protocol. The results are summarized in Tables (a) and (b) below.

All inference times are measured on a single A100 GPU. Encoder Time is the one-time context-image encoding cost; Decoder Time covers iterative generation, including intermediate Gaussian renders; and Rendering Time is the final novel-view render from the completed Gaussians.

1. Effect of the encoder–decoder architecture (DiffusionGS → Variant-A). We split the original 24-layer DiT into a 12-layer ViT context encoder and a 12-layer DiT decoder, retaining the DDPM formulation. Variant-A achieves comparable or slightly better reconstruction quality with a 1.75× speedup (1.73s → 0.99s), demonstrating the advantage of separating context encoding from iterative generation.

2. Effect of flow matching (Variant-A → Variant-B). We replace DDPM with flow matching while keeping the architecture unchanged. We use DiffusionGS's official 30 sampling steps for DDPM and 10 steps for flow matching. We find that 10-step sampling produces clean images with flow matching but leaves residual noise with DDPM (Figure 8 of the revised paper). Variant-B achieves better performance with an approximately 3× speedup (0.99s → 0.36s).

3. Effect of the pretrained context encoder (Variant-B → PixGR). Since context images are encoded only once, we can leverage large pretrained models with limited computational overhead. Replacing the 12-layer ViT with VGGT significantly improves reconstruction quality, while only increasing inference time from 0.36s to 0.47s.

4. Efficiency with increasing context views. We compare the inference efficiency of all four methods as the number of context views increases in Table (b) (better viewed in Figure 9 of the revised paper). While DiffusionGS incurs rapidly growing computational costs, Variant-A substantially reduces this overhead by avoiding repeated context encoding. Variant-B further improves efficiency through flow matching. PixGR maintains high efficiency even with the larger VGGT encoder, demonstrating the advantage of our encoder–decoder architecture.

AuthorsOpenReview ↗

Response #2

Table (a). Reconstruction quality and inference-time breakdown on RE10K with two context and three target views.

MethodEncoderDecoderFormulationPSNR↑SSIM↑LPIPS↓FID↓Encoder Time (s)Decoder Time (s)Rendering Time (s)Total Time (s)
DiffusionGS-DiT (24-layer)DDPM23.100.8010.1844.11-1.730.0071.73
Variant-AViT (12-layer)DiT (12-layer)DDPM23.340.8040.1784.010.0090.970.0080.99
Variant-BViT (12-layer)DiT (12-layer)flow23.890.8250.1733.920.0090.330.0080.36
PixGRVGGTDiT (12-layer)flow26.030.8720.1412.750.0890.380.0070.47

Table (b). Inference time for three target views as the number of context views increases.

Method2 context4 context6 context8 context10 context
DiffusionGS1.732.393.214.295.36
Variant-A0.991.391.812.362.91
Variant-B0.360.490.630.831.02
PixGR0.470.650.861.071.32

We have also updated Tables 1 and 2, Figure 5, and Appendix C.3 in the revised paper to include DiffusionGS results.

W1 & Q1: PixGR vs. PixWorld

PixWorld is concurrent work submitted to ICLR 2027, according to OpenReview.

Similar to DiffusionGS, PixWorld adopts a decoder-only architecture and thus suffers from the same computational inefficiencies discussed above in our comparison with DiffusionGS.

Furthermore, PixWorld predicts RGB images at high noise levels and switches to 3D Gaussian rendering only at low noise levels, resembling the generation-then-reconstruction pipeline discussed in Figure 1, which may lead to cross-view inconsistencies during generation. In contrast, PixGR directly generates a unified 3D Gaussian representation at each sampling step, preserving multi-view consistency throughout the generation process. Table (c) compares PixGR and PixWorld on RE10K one-view generation and shows that PixGR outperforms PixWorld with significantly higher efficiency.

Table (c). RE10K generation quality and inference time under a single-context-view setting.

MethodPSNR ↑SSIM ↑LPIPS ↓FID ↓Time (s) ↓
PixWorld17.660.6510.40722.4530.88
PixGR19.530.6550.28415.000.37

W2 & Q2: Pixel-space vs. latent-space generation

We use SD-VAE to encode target images into latents and train a latent-space DiT to denoise them conditioned on VGGT-encoded context tokens and target camera poses. The frozen VAE decoder converts the denoised latents back into target images, and a Gaussian head takes both context and target images to construct a shared 3D Gaussian representation. This implementation uses the same context images and target poses as PixGR, enabling the pixel-space versus latent-space comparison in Table (d) under matched settings.

Table (d). Pixel-space versus latent-space generation.

PSNR↑SSIM↑LPIPS↓FID↓
latent-space23.380.8260.1793.93
pixel-space26.030.8720.1412.75

As shown in Table (d), the pixel-space model achieves significantly better performance than the latent-space variant, highlighting the advantages of direct pixel-space generation.

W2 & Q2: Iterative Computation

As shown in Table (e), PixGR (one-step) still outperforms the regressor (one-step), demonstrating that the benefits of pixel-space generative decoding do not rely on iterative computation.

To further evaluate the effect of iterative computation, we implement an iterative regressor starting with two context images and three blank target inputs, feeding rendered target views back into the decoder at each subsequent step. As shown in Table (e), increasing the number of iterations degrades the regressor's performance by amplifying blurry artifacts in underconstrained regions, whereas PixGR's performance consistently improves with more iterations.

Table (e). One-step and ten-step reconstruction results for the regressor and PixGR.

MethodIterationsPSNR↑SSIM↑LPIPS↓FID↓
Regressor124.340.8210.1665.42
PixGR125.780.8670.1453.12
Regressor1023.940.8170.1705.72
PixGR1026.030.8720.1412.75

Q2: Training resources and inference budget

We use 8 A100 GPUs for training and a single A100 GPU for inference. Inference requires 13 GB of GPU memory per scene (2 context views and 3 target views).

Dear Reviewer YbGc,

As the discussion phase draws to a close, we would like to sincerely thank you for your time and thoughtful feedback. We hope that our rebuttal and additional experimental results have adequately addressed your concerns and provided further clarity on our contributions.

If you find that your previous concerns have been sufficiently resolved, we would be grateful if you could kindly consider reflecting these clarifications in your final evaluation. Of course, we would also be happy to address any remaining questions.

Thank you again for your valuable time and consideration.

Best regards,

The Authors

Reviewer_gzKzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RixGR, a pipeline that combines pixel-space generation with feed-forward 3d reconstruction. PixGR uses a pretrained scene encoder with feed-forward 3d reconstruciton.

Critical strengths
  1. The idea of combining generative modelling with the feed-forward encoder is interesting. But i don't think this paper belongs to JOINT reconstruction and generation. Pls refer to weaknesses sec.

  2. The exp results shows the improvement over several baselines.

Critical weaknesses
  1. The claim of joint reconstruction and generation is not convincing. The scene encoder produces fixed feature tokens before denoising, w/o any updates from the denoising generative process. The method is closer to Reconstruction-then-Generation, as illustrated in Fig 1, rather than joint reconstruction and generation.

  2. The lightweight DiT model is trained from scratch. It is not clear whether the model can generalize to diverse scenes and can learn strong generative prior.

  3. This method simply concatenates scene tokens with noisy target tokens. Other conditioning designs, eg, cross-attn, is actually more convincing, and not fully evaluated and discussed.

  4. Camera-conditioned video generation is also a relevant solution but this paper does not discuss about that.

Questions for authors
  1. Typo: PixG3R in the caption of Fig 1. It should be PixGR.

  2. How can a lightweight diffusion model learn the strong generative prior?

  3. Can author better isolate the benefit of generative modelling from simply additional params and iterative refinement?

  4. How sensitive is the reconstruction quality to the number of context views?

4 comments

We thank the reviewer for the thoughtful feedback.

W1: Joint reconstruction and generation

We would like to clarify that PixGR performs JOINT reconstruction and generation, NOT the reconstruction-then-generation pipeline discussed in Figure 1. The key differences are:

(1) Two-stage vs. End-to-end Training. The reconstruction-then-generation pipeline adopts a two-stage training procedure, where the stage-1 reconstruction model is frozen during stage-2 training. In contrast, PixGR is trained end-to-end in a single stage, with the context encoder jointly optimized rather than frozen.

(2) Separate vs. Joint Reconstruction and Generation. In the reconstruction-then-generation pipeline, stage-1 produces explicit 3D Gaussians that remain unchanged during stage-2 generation. In contrast, PixGR's context encoder produces cached latent representations, which are further updated through global self-attention with target tokens in the decoder. The explicit 3D Gaussians are produced only by the generative decoder, supporting both context-view reconstruction and target-view generation.

W2 & Q2: Generalization of the lightweight DiT

Our core paradigm is an encoder–decoder framework that combines a cached scene representation with a pixel-space flow decoder. We adopt a lightweight DiT to match the model size and computational efficiency of regression-based methods, while isolating the benefits of pixel-space generative decoding from large pretrained generative priors for a fairer comparison. Even without such priors, PixGR outperforms regression-based methods and some methods relying on pretrained generative models, while generalizing to out-of-domain ACID scenes (Table 5).

The core advantages of PixGR stem from its paradigm design rather than decoder size (see Appendix C.3 and Sec. 5.2.3). Motivated by the scaling trends observed in prior work [1], scaling up the decoder and leveraging larger training datasets could further strengthen its generative priors.

[1] Esser, Patrick, et al. "Scaling rectified flow transformers for high-resolution image synthesis." Forty-first international conference on machine learning. 2024.

W3: Conditioning Design (Concatenation vs. Cross-Attention)

We adopt concatenation because PixGR predicts consistent pixel-aligned Gaussian primitives for both context and target views. Global self-attention over the concatenated tokens enables information exchange across all views, allowing the model to predict consistent Gaussians for each view.

In contrast, cross-attention, where target tokens query context features, only updates target tokens while leaving context tokens fixed. This limits bidirectional information exchange and may lead to inconsistent Gaussian prediction across views.

W4: Camera-conditioned video generation

In Sec. 2.2 (Lines 132–143), we discuss several methods that rely on camera-conditioned video generation models. In Tables 1 and 2 and Figure 4, we also compare against MVSplat360, which builds upon the pretrained Stable Video Diffusion model with camera conditioning.

Q3: Ablation on iterative refinement

As shown in Table (a), PixGR (one-step) still outperforms the regressor (one-step), demonstrating that the benefits of pixel-space generative decoding do not rely on iterative computation.

To further evaluate the effect of iterative computation, we implement an iterative regressor starting with two context images and three blank target inputs, feeding rendered target views back into the decoder at each subsequent step. As shown in Table (a), increasing the number of iterations degrades the regressor's performance by amplifying blurry artifacts in underconstrained regions, whereas PixGR's performance consistently improves with more iterations.

Table (a). One-step and ten-step reconstruction results for the regressor and PixGR.

MethodIterationsPSNR↑SSIM↑LPIPS↓FID↓
Regressor124.340.8210.1665.42
PixGR125.780.8670.1453.12
Regressor1023.940.8170.1705.72
PixGR1026.030.8720.1412.75

Q4: Reconstruction quality with different numbers of context views

Table (b). DL3DV reconstruction PSNR (dB) with different numbers of context views.

Method2 views4 views6 views
DepthSplat19.6523.1924.18
PixGR20.9724.1825.27

As shown in Table (b), both methods improve as more context views provide additional scene information. PixGR maintains an advantage at every setting, showing that its reconstruction gains persist across context-view counts.

Thank authors for the detailed response. I still have the following concerns, opening for discussion:

  1. Joint reconstruction vs reconstruction then generation.

a) two stage vs end2end training. No matter the pipeline is optimized jointly or not, both this paper and methods in fig 1c follows the same reconstruction then generation pattern. From my perspective, joint reconstruction would be those pipeline that scene representation (img or feature tokens) and the generated novel views are updated jointly, and reconstruct scene based on the feedback from one module to another. If authors explains from the training perspective, i would say this method is end2end training rather than joint reconstruction, other methods are two-stage training.

b) Separate vs joint reconstruction and generation: No matter what presentation we use for scenes (3dgs or scene tokens), the scene representation is fixed, rather than optimized in the loop with generation. If authors claim that the token are further updated through global self attn, I would also claim that other method's 3dgs representation are further updated through latent diffusion. This explanation is actually not convincing for me.

  1. Lightweight DiT. Thank you. If authors claim that the lightweight DiT is only used to match the model size and computational efficiency, actually removing the lightweight DiT can prove the capacity of the proposed method even STRONGER since the model size would be less than other baselines and the inference speed would be faster.

  2. Conditioning design: why we need to update context tokens since the context won't change anyway?

  3. Camera-conditioned video generation: thanks for the explanation. But I would say that MVSplat360 is not a PURE video generation model. What I mean are methods like CameraCtrl, MotionCtrl, and the one authors mentioned: Stable Video Diffusion. Those kind of methods learn the embedding camera representation and the novel views are generated conditioned on the camera feature representation.

  4. Q4: given more context views while keeping the number of target views fixed, the total number of tokens fed into the dit decoder increases. how can the dit effectively handle this larger number of tokens, given that such input lengths are OOD relative to the training setting?

Thanks authors for detailed response. I'm open for discussion!

Thank you for your further feedback and openness to discussion.

W1 & W3: Joint Reconstruction and Context Token Updates

To further clarify, PixGR distinguishes three types of representations:

RepresentationStageViews IncludedUpdated during Sampling?Role
Cached context tokensEncodingContextNoFixed conditioning; not the final reconstruction
Unified scene representationsDecodingContext + TargetYesUpdated with evolving target views by integrating context and evolving target tokens through global self-attn.
Shared 3D GaussiansDecodingContext + TargetYesLinearly decoded from unified scene representations at each step; renderings drive subsequent flow updates

In W1, you argue that joint reconstruction requires the scene representation and generated views to evolve jointly. The scene representation we refer to is the unified scene representation in the decoder, comprising both context and evolving target-view tokens. Jointly updating this representation with evolving target views is precisely what PixGR does in the decoder. The cached encoder tokens are merely fixed conditioning signals, not the scene representation used for final reconstruction.

Regarding your statement that "other methods' 3DGS representations are further updated through latent diffusion," we would like to clarify that in the reconstruction-then-generation methods illustrated in Fig. 1(c), latent diffusion only refines the 3DGS-rendered images, while the underlying 3DGS representation remains FIXED.

Therefore, according to your definition, PixGR indeed performs joint reconstruction and generation, with both the unified scene representation and generated novel views evolving throughout sampling.

In W3, you question why context tokens should be updated when the observed context images remain unchanged. Since the unified scene representation encompasses both context and target views, updating context-side representations enables bidirectional information exchange with evolving target views, encouraging geometric consistency across the reconstructed scene. To further validate this design, we compare global self-attention with cross-attention:

Attention DesignPSNR ↑SSIM ↑LPIPS ↓FID ↓
Cross-Attention24.890.8130.1723.45
Global Self-Attention (Ours)26.030.8720.1412.75

W2: Lightweight DiT

We would like to clarify that removing the lightweight DiT would eliminate the generative modeling capability that PixGR aims to introduce. Our motivation for adopting a lightweight DiT is to investigate whether effective probabilistic modeling can be achieved at minimal computational cost, without relying on large pretrained generative priors. Importantly, our framework is flexible and can accommodate larger pretrained generative decoders without changing the underlying paradigm.

To further validate this flexibility, we replace PixGR's lightweight DiT decoder with a pixel-space variant of Wan2.2-TI2V-5B [1], adapting its pretrained latent priors to pixel space following the L2P alignment strategy [2], while keeping the overall PixGR framework unchanged. Both variants are evaluated on RE10K under the one-view generation setting.

MethodPSNR ↑SSIM ↑LPIPS ↓FID ↓Time (s) ↓
PixGR (Lightweight DiT)19.530.6550.28415.000.37
PixGR (Wan2.2-5B)19.890.6890.25610.1231.22

While the pretrained Wan-based decoder achieves better reconstruction quality, our lightweight DiT achieves approximately 84× faster inference with competitive reconstruction performance. These results demonstrate that PixGR supports different generative decoder capacities within the same paradigm, offering flexibility in balancing reconstruction quality and computational efficiency.

W4: Camera-Conditioned Video Generation

Thank you for the clarification. To address this concern, we have additionally evaluated CameraCtrl [3], GEN3C [4], and Geometry Forcing [5], three camera-conditioned video generation methods, on the RE10K one-view generation benchmark.

MethodPSNR ↑SSIM ↑LPIPS ↓FID ↓Time (s) ↓
CameraCtrl17.260.5480.36162.9637.88
GEN3C18.320.5950.3089.91745.21
Geometry Forcing16.790.5530.34634.08118.23
PixGR (Ours)19.530.6550.28415.000.37

PixGR achieves the best PSNR, SSIM, LPIPS, and inference efficiency, while GEN3C achieves a lower FID. Unlike these video generation approaches, PixGR reconstructs a shared explicit 3D Gaussian representation, enabling arbitrary novel-view rendering without repeated generative sampling.

Q4: Generalization to More Context Views

PixGR employs global self-attention with shared Transformer weights to handle variable-length inputs and is trained with varying numbers of context views to adapt to different input view counts.

To further evaluate OOD generalization, we test the same checkpoint, trained with up to 6 context views, using up to 20 context views while keeping the number of target views fixed, without any fine-tuning.

Context ViewsPSNR ↑SSIM ↑LPIPS ↓
220.9710.69470.2713
424.1850.80890.1730
625.2970.84060.1480
10 (OOD)26.0730.86020.1343
15 (OOD)26.1980.86710.1304
20 (OOD)26.2380.86870.1303

Despite exceeding the maximum training-time view count, PixGR continues to improve reconstruction quality with additional context views, demonstrating effective generalization to longer input sequences.

[1] Wan, Team, et al. "Wan: Open and advanced large-scale video generative models." arXiv preprint arXiv:2503.20314 (2025). [2] Chen, Zhennan, et al. "L2p: Unlocking latent potential for pixel generation." arXiv preprint arXiv:2605.12013 (2026). [3] He, Hao, et al. "Cameractrl: Enabling camera control for text-to-video generation." arXiv preprint arXiv:2404.02101 (2024). [4] Ren, Xuanchi, et al. "Gen3c: 3d-informed world-consistent video generation with precise camera control." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025. [5] Wu, Haoyu, et al. "Geometry forcing: Marrying video diffusion and 3d representation for consistent world modeling." International Conference on Learning Representations. Vol. 2026. 2026.

Comments on the paper

7 comments
AuthorsOpenReview ↗

Response #1

We thank the reviewer for the insightful feedback.

W1 & W2 & Q1 & Q2: PixGR vs. DiffusionGS

The main methodological differences between PixGR and DiffusionGS are the encoder–decoder architecture and the flow-matching formulation.

A. Methodological Differences

1. Decoder-only vs. Encoder–Decoder Architecture.

DiffusionGS adopts a decoder-only architecture that jointly processes context and target images throughout the entire transformer, leading to computational inefficiencies in three ways:

(1) Across target viewpoints: The same context images must be repeatedly processed when conditioned on different target viewpoints.

(2) Across diffusion sampling steps: Context images must be reprocessed at every sampling step.

(3) With increasing context views: The computational cost grows substantially as the number of context images increases.

In contrast, PixGR adopts an encoder–decoder architecture that encodes clean context images only once into latent representations, while iterative generation is performed entirely within the decoder. This design significantly reduces computational redundancy across diffusion sampling steps and target viewpoints, and improves efficiency when processing larger numbers of context views.

Moreover, since the context encoder is executed only once, PixGR can leverage large pretrained models, such as VGGT, as the context encoder to extract stronger geometric features without substantially increasing the computational cost of iterative generation.

2. DDPM vs. Flow Matching.

DiffusionGS adopts a DDPM formulation, generating target views through a stochastic reverse denoising process. In contrast, PixGR employs flow matching, which learns a continuous transport from noise to clean target images and performs sampling using an ODE solver. Our experiments demonstrate that flow matching achieves higher sampling efficiency than DDPM.

B. Controlled Ablation Studies

To systematically evaluate these design choices, we conduct ablation experiments on RE10K, with all methods trained and evaluated using 2 context views and 3 target views. The results are summarized in Tables (a) and (b) below. All inference times are measured on a single A100 GPU. Encoder Time is the one-time context-image encoding cost; Decoder Time covers iterative generation, including intermediate Gaussian renders; and Rendering Time is the final novel-view render from the completed Gaussians.

1. Effect of the encoder–decoder architecture (DiffusionGS → Variant-A). We split the original 24-layer DiT into a 12-layer ViT context encoder and a 12-layer DiT decoder, retaining the DDPM formulation. Variant-A achieves comparable or slightly better reconstruction quality with a 1.75× speedup (1.73s → 0.99s), demonstrating the advantage of separating context encoding from iterative generation.

2. Effect of flow matching (Variant-A → Variant-B). We replace DDPM with flow matching while keeping the architecture unchanged. We use DiffusionGS's official 30 sampling steps for DDPM and 10 steps for flow matching. We find that 10-step sampling produces clean images with flow matching but leaves residual noise with DDPM (Figure 8 of the revised paper). Variant-B achieves better performance with an approximately 3× speedup (0.99s → 0.36s).

3. Effect of the pretrained context encoder (Variant-B → PixGR). Since context images are encoded only once, we can leverage large pretrained models with limited computational overhead. Replacing the 12-layer ViT with VGGT significantly improves reconstruction quality, while only increasing inference time from 0.36s to 0.47s.

4. Efficiency with increasing context views. We compare the inference efficiency of all four methods as the number of context views increases in Table (b) (better viewed in Figure 9 of the revised paper). While DiffusionGS incurs rapidly growing computational costs, Variant-A substantially reduces this overhead by avoiding repeated context encoding. Variant-B further improves efficiency through flow matching. PixGR maintains high efficiency even with the larger VGGT encoder, demonstrating the advantage of our encoder–decoder architecture.

AuthorsOpenReview ↗

Response #2

Table (a). Reconstruction quality and inference-time breakdown on RE10K with two context and three target views.

MethodEncoderDecoderFormulationPSNR↑SSIM↑LPIPS↓FID↓Encoder Time (s)Decoder Time (s)Rendering Time (s)Total Time (s)
DiffusionGS-DiT (24-layer)DDPM23.100.8010.1844.11-1.730.0071.73
Variant-AViT (12-layer)DiT (12-layer)DDPM23.340.8040.1784.010.0090.970.0080.99
Variant-BViT (12-layer)DiT (12-layer)flow23.890.8250.1733.920.0090.330.0080.36
PixGRVGGTDiT (12-layer)flow26.030.8720.1412.750.0890.380.0070.47

Table (b). Inference time for three target views as the number of context views increases.

Method2 context4 context6 context8 context10 context
DiffusionGS1.732.393.214.295.36
Variant-A0.991.391.812.362.91
Variant-B0.360.490.630.831.02
PixGR0.470.650.861.071.32

We have also updated Tables 1 and 2, Figure 5, and Appendix C.3 in the revised paper to include DiffusionGS results.

W1 & W2 & Q1: PixGR vs. LagerNVS

Although LagerNVS also adopts a geometric encoder with a lightweight pixel-space diffusion decoder, PixGR differs in two key aspects:

(1) Joint multi-view generation: LagerNVS processes and generates each target view independently, potentially leading to cross-view inconsistencies. In contrast, PixGR jointly processes multiple target viewpoints and generates a unified 3D Gaussian representation at each sampling step, improving multi-view consistency. As shown in Figure 7, incorporating more target keyframes further improves reconstruction quality.

(2) Explicit 3D reconstruction: LagerNVS does not produce an explicit 3D scene representation, requiring repeated diffusion sampling for additional novel views. In contrast, PixGR generates a 3D Gaussian representation, enabling efficient rendering of arbitrary novel views without repeated generation.

Q2: Controlled Comparison

In Table 7, DiffusionGS is evaluated under matched training data (RE10K), image resolution (256×256), and target-view configuration (3 views). We further isolate the contributions of our proposed components through controlled ablations in Table (a) above.

Q3: Efficiency Clarification

We use 10 flow-matching sampling steps in all experiments. In Figure 5, PixGR's reported inference time includes context-image resizing, VGGT encoding at 448×448 resolution, all decoding and intermediate rendering steps, and final novel-view rendering.

Table (a) provides a detailed time breakdown:

  • Encoder time: Context-image resizing and VGGT encoding at 448×448 resolution.
  • Decoder time: All decoding and intermediate rendering steps.
  • Rendering time: Final novel-view rendering.

Thanks for providing the supplementary ablation experiments and timing details. These additions explain for the design choices and answer my questions regarding missing sampling and runtime details. While my remaining concern is that the methodological differences compared to the closest previous work are limited. DiffusionGS already predicts a shared Gaussian representation during denoising and uses differentiable rendering to obtain sharp image predictions. LagerNVS has separated scene encoding from repetitive generation by combining a VGGT-based scene encoder with lightweight pixel-space diffusion decoding. I acknowledge that PixGR integrates these features through joint multi-view Gaussian decoding and employs flow matching, improving efficiency. However, I believe this is primarily an architectural integration and improvement of existing designs, lacking additional methodological insights. Therefore, I have decided to maintain my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.