acceptodds
Under review as a conference paper at ICLR 2027

Rendering-Optimal Is Not Mesh-Optimal: Composing Multi-Budget Gaussians for Compact Surface Extraction

Abstract

3D Gaussian Splatting (3DGS) achieves high-quality rendering, but its Gaussians do not form an explicit mesh. Gaussian-based surface reconstruction methods fit a surface to the learned Gaussians and mesh it on a spatial structure built around them, yielding an explicit mesh. In these methods, mesh complexity is largely determined by the spatial distribution of Gaussians, with denser Gaussians tending to yield denser triangulation. Obtaining a compact yet detailed mesh therefore depends on allocating Gaussians densely where the geometry is complex and sparsely elsewhere. However, the placement and density of these Gaussians are driven by photometric supervision and may not align with the geometry required for compact mesh reconstruction. We address this mismatch by composing Gaussians trained at different budgets, i.e. Gaussian counts, into a single set of Gaussians, drawing each region from the budget its geometry needs. Starting from the lowest budget, we keep its Gaussians where they already capture the geometry and replace them with higher-budget Gaussians only where rendered depth indicates a sufficient geometric gain. The resulting composite is then meshed with the underlying reconstruction method's own extraction procedure, yielding more compact meshes while preserving geometric quality. Across multiple Gaussian-based surface reconstruction methods and datasets, our composites improve the geometry–mesh-size trade-off, reducing mesh size by 26–59% relative to extraction from the finest budget while maintaining comparable or better geometric accuracy and rendering quality.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Gaussian Splatting, Surface Reconstruction, Mesh Extraction, Compact Meshes, Gaussian Composition

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SYikOpenReview ↗
Rating
2
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper composes Gaussian representations trained at different primitive budgets to obtain compact meshes. Starting from a low-budget representation, it replaces Gaussians where higher-budget candidates improve agreement with depth rendered from a high-budget reference. The composite is optimized and meshed using the underlying reconstruction method. Evaluation includes Gaussian Wrapping, MILo, and GGGS.

Critical strengths
  1. A relevant distinction between rendering and meshing. Allocating Gaussians for photometric reconstruction need not yield an efficient spatial structure for mesh extraction. Local budget allocation is a reasonable response.
  2. Evaluation includes geometric metrics. The paper reports Chamfer distance and F1 in addition to mesh-rendering quality, and tests multiple reconstruction backbones.
  3. Important limitations are disclosed. The appendix includes QEM comparisons, a matched-optimization single-budget control, full dataset results including Box, and discussion of reference-depth errors and input preparation cost.
Critical weaknesses
  1. The incremental value of multi-budget composition is small in the strongest control. At the default gamma = 0.2, Table 6 retains 95.57–99.83% of the original G4 Gaussians across backbones. For MILo, Table 12 compares both representations after the same 2K optimization: G4 gives 18.11 MB and CD 0.1342, versus 17.78 MB and 0.1341 for the composite. This is approximately 1.8% additional size reduction with nearly unchanged accuracy. Much of the headline reduction relative to G5 may therefore be obtained by selecting G4 and optimizing it. Equivalent controls across the other backbones are needed.
  2. The advantage over direct mesh simplification is conditional. The paper does compare against QEM at matched face counts; this baseline is not absent. Appendix H reports better CD for QEM throughout the tested MILo range, with additional reversals at smaller mesh sizes for other settings. The proposed method may avoid extracting the largest mesh, but peak extraction memory and end-to-end timing are not provided to establish that advantage.
  3. Depth-reference agreement does not guarantee geometric improvement. Selection is supervised by the highest-budget model, which need not have the most accurate geometry. On Box with MILo, Table 7 reports CD 1.93717 for G3, 4.42256 for the G5 reference, and 4.34694 for the default composite. The failure is acknowledged, but it limits the interpretation of the selection score as geometric gain. The criterion also does not directly account for the mesh-complexity cost of a replacement.
  4. Scope and computational cost need clearer contextualization. The main DeMapGS geometric aggregate excludes Box, although full results are provided in the appendix. On Tanks and Temples, Gaussian Wrapping’s default composite reduces size but also lowers mean F1 from 0.498 to 0.455 relative to G5. Moreover, the approximately 41–91 second composition-stage timings exclude multi-budget input preparation and extraction. These are conditional quality–cost trade-offs, not evidence of uniformly preserved geometry at negligible extra cost.
Questions for authors

Q1. Could the authors clarify how much improvement comes from multi-budget composition compared with directly optimizing a lower-budget representation? Extending the matched-optimization comparison in Table 12 to other backbones would help.

Q2. Could the authors report the total processing time and peak memory, including preparation of the multi-budget representations? This would clarify the practical trade-off relative to mesh extraction followed by simplification.

Q3. How sensitive is the method to the quality of the highest-budget reference depth? It would be helpful to discuss when this reference is reliable and how its errors affect the final mesh.

Q4. Could the authors evaluate the method on a wider range of challenging scenes where the baseline reconstruction quality is lower? Since the baseline PSNR is already high in several reported settings, it would be helpful to understand whether the method can provide more substantial quality improvements in these cases, alongside the gains in mesh compactness.

12 comments

W1 & Q1. Independent Contribution of Multi-Budget Composition

We thank the reviewer for raising this important concern. We agree that isolating the contribution of multi-budget composition from the subsequent Gaussian optimization is essential.

To address this, we extended the matched-optimization experiment in Table 12 to Gaussian Wrapping and GGGS, in addition to MILo. Specifically, we compare direct extraction from G4 and our composed representations, both with and without identical 2K Gaussian optimization.

Experimental setup. The following results are averaged over five scenes from the DeMapGS dataset (Book, Buddha, Bust, Cat, and Tree), following the same evaluation protocol as our main experiments. ✓ and ✗ indicate whether the identical 2K Gaussian optimization is applied or not, respectively.

MethodRepresentationGS optim. (2K)Mesh (MB) ↓CD ↓
Gaussian WrappingG4✗10.510.1548
G4✓10.460.1544
Ours (γ=0.2\gamma=0.2)✗10.470.1545
Ours (γ=0.2\gamma=0.2)✓10.430.1542
MILoG4✗18.290.1381
G4✓18.110.1342
Ours (γ=0.2\gamma=0.2)✗17.960.1370
Ours (γ=0.2\gamma=0.2)✓17.780.1341
GGGSG4✗77.150.1180
G4✓83.240.1132
Ours (γ=0.2\gamma=0.2)✗76.310.1153
Ours (γ=0.2\gamma=0.2)✓82.100.1116

Even without the 2K optimization, our composites produce smaller meshes with lower CD than G4 across all three backbones. These improvements remain after applying identical optimization to both G4 and Ours. In particular, for GGGS, composition reduces CD from 0.1180 to 0.1153 before optimization. After identical 2K optimization, Ours achieves a lower CD (0.1116 vs. 0.1132) with a smaller mesh (82.10 vs. 83.24 MB). These results demonstrate that multi-budget composition contributes to the geometry–mesh-size trade-off independently of the final Gaussian optimization, although the improvements at γ=0.2\gamma=0.2 are relatively modest.

As shown in Appendix B (Table 6) on NeRF Synthetic, our composites retain most G4 Gaussians at γ=0.2\gamma=0.2, consistent with the relatively small improvements over G4. At higher thresholds, Gaussians from different budgets are more substantially mixed, making the contribution of multi-budget composition more apparent.

To demonstrate this, we additionally compare our results at γ=0.6\gamma=0.6 against the same G4 baseline with identical 2K optimization.

MethodRepresentationMesh (MB) ↓CD ↓
Gaussian WrappingG4 + 2K optim.10.460.1544
Ours (γ=0.6\gamma=0.6)8.8550.1552
GGGSG4 + 2K optim.83.240.1132
Ours (γ=0.6\gamma=0.6)59.070.1137

For Gaussian Wrapping, Ours (γ=0.6\gamma=0.6) reduces mesh size by 15.3% (10.46 to 8.855 MB), with only a 0.0008 increase in CD (0.1544 to 0.1552). For GGGS, Ours (γ=0.6\gamma=0.6) achieves a 29.0% reduction in mesh size (83.24 to 59.07 MB), while CD increases only slightly from 0.1132 to 0.1137 (approximately 0.44%).

Overall, these results demonstrate that multi-budget composition provides improvements beyond those achieved by Gaussian optimization alone. While the improvements are modest at γ=0.2\gamma=0.2, composition enables more substantial mesh-size reductions with only minor changes in geometric accuracy at γ=0.6\gamma=0.6.

W2 & Q2. Mesh Simplification, Extraction Memory, and Computational Cost

We acknowledge that QEM achieves better geometric accuracy in several settings, including the evaluated MILo configurations, as reported in Appendix H. However, the two approaches operate at different stages: QEM simplifies an already extracted mesh, whereas our method reduces Gaussian representation complexity before mesh extraction. This distinction can be particularly important when extracting a large intermediate mesh is limited by GPU memory.

1. Extraction Memory on Mip-NeRF 360

As reported in Sec. 4.5, extracting meshes from the original full-budget G5 representations of Gaussian Wrapping and GGGS on Mip-NeRF 360 results in out-of-memory (OOM) errors. Consequently, we constructed subsampled G5 representations for these two backbones, which were used for baseline mesh extraction and reference-depth supervision in Table 4.

To quantify the extraction-memory benefit, we additionally measured peak GPU memory during mesh extraction across all nine scenes of Mip-NeRF 360. The table reports the average peak GPU memory across scenes, with the maximum value in parentheses. Memory is measured in GiB. For Gaussian Wrapping and GGGS, G5 denotes the subsampled representation used in Table 4.

MethodG5 (GiB) ↓G4 (GiB) ↓Ours (γ=0.2\gamma=0.2) (GiB) ↓Reduction vs. G5
Gaussian Wrapping2.708 (3.900)2.017 (2.928)2.026 (2.841)25.2%
MILo8.498 (9.918)7.753 (9.385)7.372 (9.209)13.2%
GGGS2.927 (4.506)2.474 (3.672)2.324 (3.496)20.6%

Compared with the evaluated G5 representations, our method reduces average peak extraction GPU memory by 25.2% for Gaussian Wrapping, 13.2% for MILo, and 20.6% for GGGS. We also observe a consistent trend on DeMapGS, where peak extraction GPU memory decreases as γ\gamma increases across all three backbones (see our response to Reviewer ERz1, W1). This further supports the extraction-memory benefits of reducing representation complexity before meshing.

Compared with G4, extraction memory is further reduced for MILo and GGGS, while remaining nearly unchanged for Gaussian Wrapping. Nevertheless, as shown in Table 4, our method produces smaller meshes than G4 across all three backbones while improving mesh-rendering PSNR:

  • Gaussian Wrapping: 156.48 → 144.38 MB (PSNR: 23.413 → 23.455 dB).
  • MILo: 208.37 → 180.99 MB (PSNR: 22.867 → 23.510 dB).
  • GGGS: 293.94 → 278.77 MB (PSNR: 22.903 → 23.106 dB).

These results demonstrate that our approach reduces extraction-stage memory requirements while maintaining competitive mesh-rendering quality. The OOM cases also highlight a practical advantage of reducing representation complexity before extraction: when the original full-budget G5 mesh cannot be extracted due to memory constraints, post-extraction simplification cannot be applied to that mesh.

2. Total Computational Cost

We also measured the total processing time and peak memory consumption, including multi-budget preparation and mesh extraction, to characterize the additional computational overhead of our approach.

The following results are averaged over five DeMapGS scenes (Book, Buddha, Bust, Cat, and Tree). Total processing time includes G5 preparation, additional G1–G4 preparation (for Ours), composition, final refinement, and mesh extraction.

MeasurementGGGS (G5 / Ours)Gaussian Wrapping (G5 / Ours)
Total processing time (min) ↓29.10 / 44.3816.77 / 35.58
Overall peak GPU VRAM (GiB) ↓15.941 / 15.9412.561 / 3.380
Overall peak CPU RSS (GiB) ↓6.691 / 5.3144.033 / 4.120
Mesh extraction time (s) ↓255.01 / 59.7019.04 / 18.57

Our method incurs an additional 15.27 minutes for GGGS and 18.81 minutes for Gaussian Wrapping. For GGGS, the overall peak GPU memory remains unchanged at 15.941 GiB because the highest memory consumption occurs during G5 training, which is shared by both pipelines. In contrast, for Gaussian Wrapping, the overall peak increases from 2.561 to 3.380 GiB because the final Gaussian refinement stage requires more memory than the original G5 training stage. Nevertheless, mesh extraction becomes substantially faster for GGGS, with a 76.6% reduction in extraction time.

Overall, these results clarify the practical trade-off of our approach: additional offline preparation costs in exchange for improved geometry–mesh-size trade-offs and reduced extraction-stage resource requirements in the evaluated settings. While our method does not aim to minimize end-to-end reconstruction time, reducing representation complexity before mesh extraction can provide practical benefits, particularly when extracting the original full-budget mesh is memory-intensive or infeasible.

W3 & Q3. Reliability of Reference Depth and Its Impact on Mesh Reconstruction

We agree that the highest-budget representation does not necessarily provide the most accurate geometry, and that agreement with its depth does not guarantee improved ground-truth geometry. As described in Sec. 3.2, G5 depth serves as a geometric proxy rather than ground-truth geometry. Severely inaccurate reference geometry, as observed in the MILo Box scene, can therefore limit the effectiveness of our method. This limitation is already discussed in Appendix J.

1. Sensitivity to Reference-Depth Errors

To evaluate how reference-depth errors affect the complete pipeline, we applied spatially coherent 3D perturbations to G5 and used the resulting reference for both Gaussian selection and the subsequent 2K refinement. This extends our selection-only sensitivity analysis (see our response to Reviewer ERz1, Q1), in which the original G5 reference was retained during refinement.

Experimental setup. We evaluated MILo on five DeMapGS scenes (Book, Buddha, Bust, Cat, and Tree), using γ=0.2\gamma=0.2 and 0.60.6. Perturbation magnitudes were calibrated relative to the original G5-to-GT depth RMSE, following the same protocol as our selection-only experiment. Geometric deformations become visually noticeable at 0.25 × RMSE and substantial at 1.0 × RMSE.

Reference perturbationCD ↓ (γ=0.2\gamma=0.2)Mesh (MB) ↓CD ↓ (γ=0.6\gamma=0.6)Mesh (MB) ↓
Original G50.134117.780.15799.16
0.1 × RMSE0.134817.730.15858.76
0.25 × RMSE0.135717.430.16377.70
0.5 × RMSE0.140716.630.17756.22
1.0 × RMSE0.165314.820.22064.28

The results show that our method maintains relatively stable mesh accuracy under mild reference perturbations, even when the perturbed reference is used for both selection and refinement. At 0.1 × RMSE, CD increases by approximately 0.5% for γ=0.2\gamma=0.2 (0.1341 → 0.1348) and 0.4% for γ=0.6\gamma=0.6 (0.1579 → 0.1585). At 0.25 × RMSE, where geometric deformations become visually noticeable, CD at γ=0.2\gamma=0.2 increases only slightly from 0.1341 to 0.1357.

Under stronger perturbations, performance degradation becomes more pronounced, particularly at γ=0.6\gamma=0.6. As the perturbed reference deviates from the original G1–G4 candidates, fewer regions may satisfy the relative-depth-gain threshold, leading to fewer finer Gaussians being selected. Consistent with this, we observed a substantial decrease in selected G4 Gaussians as the perturbation magnitude increased. At 1.0 × RMSE, mesh size decreases from 9.16 to 4.28 MB at γ=0.6\gamma=0.6, accompanied by an increase in CD from 0.1579 to 0.2206. Thus, the accuracy degradation should be considered together with the substantial change in mesh complexity, rather than interpreted solely as degradation at a fixed mesh budget.

Overall, these results indicate that our method is relatively robust to mild reference-depth errors, while stronger perturbations can affect both Gaussian selection and final mesh accuracy. These findings further characterize the reference-quality limitation discussed in Appendix J.

2. Relationship Between Depth Gain and Mesh Complexity

We acknowledge that our relative-depth-gain criterion does not explicitly account for the mesh-complexity cost of individual Gaussian replacements. Instead, the threshold γ\gamma controls the extent of finer-Gaussian selection and the resulting geometry–mesh-size trade-off. Incorporating mesh-complexity cost directly into the selection criterion is a potential direction for future work, as already discussed in Appendix K.

Reviewer_ERz1OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper discusses Compact surface extraction with Gaussian representation. It adaptively refines details of a coarse set according to the difference to finer representations. Applied to Gaussian Wrapping, MILo, and GGGS, the approach reduces extracted mesh size by approximately 26%–59% relative to the finest-budget baseline while preserving or improving geometry and mesh-rendering quality on several datasets.

Critical strengths
  1. A well-supported geometry–mesh-size improvement across multiple extractors. At γ=0.2\gamma=0.2, the method reduces mesh size relative to the finest-budget baseline by about 29% for Gaussian Wrapping, 32% for MILo, and 55% for GGGS on DeMapGS, with comparable or lower Chamfer distance. The trend also holds for mesh rendering and extends to NeRF Synthetic, Mip-NeRF 360, DTU, and Tanks and Temples.
Critical weaknesses
  1. The compact output hides a heavy multi-representation input requirement. The method assumes G1,…,G4G_1,\ldots,G_4 plus a finer G5/GRG_5/G_R. For MILo these require five separate training runs; for GGGS and Gaussian Wrapping a base model is sampled into multiple budgets and each candidate is optimized for 10K iterations. Table 5's 40.5–91.2 seconds explicitly excludes pretraining these representations and mesh extraction. Therefore, the paper demonstrates compact meshes, but not an efficient end-to-end compact-surface pipeline. A comparison against training one fine representation followed by modern mesh simplification should include total GPU-hours, peak disk/memory, and all intermediate representations.
  2. Missing comparison with Gaussian LoD methods. The proposed multi-budget composition closely resembles a spatially adaptive LoD hierarchy, yet the paper only briefly cites existing LoD methods without examining whether their coarse-to-fine representations could achieve a similar mesh-size–quality trade-off. This comparison is particularly important because LoD methods can also control representation detail in a single training process, avoiding the costly preparation of several separately optimized Gaussian sets required here.
Questions for authors

Beyond weakness, there is an additional question:

How sensitive is selection to errors in D(⋅;GR)D(\cdot;G_R)? Please report a diagnostic using GT depth where available, and/or perturb the reference depth to quantify robustness.

7 comments

W1. Computational Cost of Multi-Budget Preparation

We thank the reviewer for raising concerns about the computational and storage costs associated with preparing multiple Gaussian representations.

We would like to clarify that our primary objective is to improve the geometry–mesh-size trade-off of extracted meshes, rather than to develop a computationally efficient end-to-end reconstruction pipeline. Nevertheless, we agree that the additional overhead should be explicitly characterized. We therefore measured total processing time and peak memory consumption, including multi-budget preparation and mesh extraction.

1. Total Computational Cost

We measured the processing time for GGGS and Gaussian Wrapping across five DeMapGS scenes (Book, Buddha, Bust, Cat, and Tree). Total times are calculated by summing the measured preparation, composition/refinement, and extraction stages.

MeasurementGGGSGaussian Wrapping
G5 preparation (min)24.8516.45
Additional G1–G4 preparation (min)17.2818.08
Composition + final refinement (min)1.250.74
G5 extraction (sec)255.0119.04
Ours extraction (sec)59.7018.57
G5 total time (min)29.1016.77
Ours total time (min)44.3835.58
Additional time (min)+15.28+18.81

Our method introduces an additional 15.28 minutes for GGGS and 18.81 minutes for Gaussian Wrapping per scene. Unlike Table 5, these measurements include multi-budget preparation and mesh extraction. As shown in Table 1, at γ=0.2\gamma=0.2, this additional preparation yields 55.4% and 28.6% smaller meshes for GGGS and Gaussian Wrapping, respectively, with improved CD compared with G5. Notably, GGGS mesh extraction time also decreases by 76.6% (255.01 to 59.70 seconds). This suggests that reducing Gaussian complexity before extraction can provide additional efficiency benefits when mesh extraction is computationally expensive, although the overall processing time remains higher.

Our framework does not inherently require five independent training runs: GGGS and Gaussian Wrapping construct multi-budget representations by sampling a single pretrained model and refining each subset for 10K iterations. Nevertheless, multi-budget preparation introduces additional computational and storage costs for intermediate representations.

2. Peak Memory Consumption

We measured overall peak GPU and CPU memory, as well as peak GPU memory during mesh extraction.

MeasurementGGGS (G5 / Ours)Gaussian Wrapping (G5 / Ours)
Overall peak GPU VRAM (GiB)15.941 / 15.9412.561 / 3.380
Overall peak CPU RSS (GiB)6.691 / 5.3144.033 / 4.120

For GGGS, overall peak GPU memory remains unchanged because the highest consumption occurs during G5 training, which both pipelines share. For Gaussian Wrapping, it increases from 2.561 to 3.380 GiB because final refinement requires more memory than G5 training.

We further measured peak GPU memory during mesh extraction across different thresholds γ\gamma:

SettingGGGS (GiB) ↓Gaussian Wrapping (GiB) ↓MILo (GiB) ↓
G54.5090.7333.004
Ours (γ=0.2\gamma=0.2)3.2770.7082.928
Ours (γ=0.4\gamma=0.4)3.2250.7022.907
Ours (γ=0.6\gamma=0.6)3.1200.6932.842
Ours (γ=0.8\gamma=0.8)2.8230.6732.789

At γ=0.2\gamma=0.2, peak extraction memory decreases by 27.3% for GGGS, 3.4% for Gaussian Wrapping, and 2.5% for MILo. At γ=0.8\gamma=0.8, these reductions reach 37.4%, 8.2%, and 7.2%, respectively.

Unlike post-extraction mesh simplification, our method reduces representation complexity before extraction, allowing the underlying extraction procedure to operate with fewer Gaussians and lower memory requirements in the evaluated settings. This is particularly relevant for memory-intensive methods: as reported in Sec. 4.5, extracting meshes from the original full-budget GGGS and Gaussian Wrapping representations on Mip-NeRF 360 results in out-of-memory errors.

W2. Comparison with Gaussian LoD Methods

We agree that existing Gaussian LoD methods can efficiently provide multi-resolution Gaussian representations within a single training process, avoiding the additional preparation costs associated with our current implementation. However, their rendering-oriented objectives do not necessarily guarantee favorable geometry–mesh-size trade-offs for surface extraction.

To investigate this, we additionally evaluated two rendering-oriented Gaussian methods, FLoD (Seo et al., 2025), a multi-level LoD approach, and GNS (Deng et al., 2025), a budget-controlled pruning approach, by extracting meshes from their representations at different budgets.

Experimental setup. We evaluated five DeMapGS scenes (Book, Buddha, Bust, Cat, and Tree), following the same evaluation protocol as our main experiments. We report Chamfer distance (CD), Gaussian PSNR, and mesh size. For GNS, K denotes thousands of Gaussians (e.g., 5K = 5,000 Gaussians), not training iterations.

MethodMesh (MB) ↓Gaussian PSNR (dB) ↑CD ↓
FLoD LoD20.84631.830.5611
FLoD LoD32.58134.110.3630
FLoD LoD47.83336.830.2765
FLoD LoD541.99041.340.2370
GNS 5K3.98238.280.2911
GNS 10K7.72040.100.3752
GNS 15K11.42341.190.3332
GNS 20K15.09341.880.2201
GNS 30K21.52242.820.1850
Gaussian Wrapping14.60843.270.1548
Gaussian Wrapping-Ours (γ=0.2\gamma=0.2)10.43342.910.1542
Gaussian Wrapping-Ours (γ=0.4\gamma=0.4)10.17042.820.1545
Gaussian Wrapping-Ours (γ=0.6\gamma=0.6)8.85542.250.1552
Gaussian Wrapping-Ours (γ=0.8\gamma=0.8)4.91040.160.1637

The results show that the evaluated rendering-oriented LoD representations do not achieve comparable mesh-size–accuracy trade-offs to the surface-reconstruction-oriented representations used in our framework. For example, GNS 30K and Gaussian Wrapping-Ours (γ=0.4\gamma=0.4) achieve the same Gaussian rendering PSNR (42.82 dB), but our method produces a 52.7% smaller mesh (10.17 vs. 21.52 MB) with 16.5% lower CD (0.1545 vs. 0.1850). Similarly, FLoD LoD5 produces a 41.99 MB mesh with CD 0.2370, whereas Gaussian Wrapping-Ours (γ=0.6\gamma=0.6) achieves a smaller mesh (8.86 MB) with lower CD (0.1552). These comparisons reflect differences in the underlying representation objectives as well as the resulting mesh quality, rather than isolating the effect of composition alone.

We also observe that increasing the LoD rendering budget does not necessarily improve extracted mesh geometry. For GNS, increasing the Gaussian count from 5K to 10K improves Gaussian PSNR from 38.28 to 40.10 dB, while mesh CD deteriorates from 0.2911 to 0.3752. Similarly, although FLoD LoD5 achieves lower average CD than LoD4, it yields worse CD in three of the five evaluated scenes (Book, Buddha, and Bust), despite increasing the average mesh size from 7.83 to 41.99 MB. These observations further support that improvements in Gaussian rendering quality do not necessarily translate into improvements in extracted mesh geometry.

Overall, these results suggest that efficiently obtaining multiple Gaussian budgets through rendering-oriented LoD methods does not necessarily translate into favorable mesh-size–accuracy trade-offs. While single-training LoD hierarchies could help reduce preparation costs, integrating them into our framework would involve challenges in maintaining their coarse-to-fine relationships during selective Gaussian replacement and ensuring suitable surface alignment for mesh extraction. We consider this direction for future work.

Q1. Robustness to Reference-Depth Errors

We agree that understanding the sensitivity of Gaussian selection to reference-depth errors is important for assessing the reliability of our approach. To investigate this, we conducted an additional experiment using spatially coherent 3D perturbations of the G5 representation.

Rather than adding independent noise to individual depth maps, we applied a smooth 3D displacement field to the Gaussian centers of G5 and rendered the resulting reference depths across all training views. This ensures that the perturbations originate from a shared 3D representation rather than being introduced independently across views.

Experimental setup. We evaluated MILo on five DeMapGS scenes (Book, Buddha, Bust, Cat, and Tree), using γ=0.2\gamma=0.2 and 0.60.6. We calibrated the perturbation magnitude using the RMSE between the original G5 depth and ground-truth depth. Specifically, a×a \times RMSE indicates that the RMS depth deviation between the perturbed and original G5 representations is approximately aa times the original G5-to-GT depth RMSE. In our visual inspection, geometric deformations become noticeable at 0.25 × RMSE and are substantial at 1.0 × RMSE.

To isolate the sensitivity of Gaussian selection, we perturbed only the selection reference, while keeping the G1–G4 candidate representations and the original G5 reference used for subsequent 2K Gaussian refinement unchanged across all conditions. The following results are averaged over the five DeMapGS scenes, with CD and mesh size measured after identical 2K refinement.

Selection referenceCD ↓ (γ=0.2\gamma=0.2)Mesh (MB) ↓CD ↓ (γ=0.6\gamma=0.6)Mesh (MB) ↓
Original G50.134117.780.15799.16
0.1 × RMSE0.134217.750.15838.77
0.25 × RMSE0.134517.560.16297.73
0.5 × RMSE0.134617.020.17106.28
1.0 × RMSE0.135615.620.18774.42

The results show that the final mesh accuracy remains relatively stable under mild reference-depth perturbations at both thresholds, with greater sensitivity at γ=0.6\gamma=0.6. At 0.1 × RMSE, CD increases by approximately 0.1% (0.1341 → 0.1342) for γ=0.2\gamma=0.2 and 0.3% (0.1579 → 0.1583) for γ=0.6\gamma=0.6. At γ=0.2\gamma=0.2, CD changes only slightly even under stronger perturbations, reaching 0.1356 at 1.0 × RMSE, although mesh size also decreases from 17.78 to 15.62 MB. These results indicate relatively stable final mesh accuracy in this setting when the subsequent refinement reference remains unchanged.

The greater sensitivity at γ=0.6\gamma=0.6 is consistent with our threshold-based selection mechanism. As the perturbed reference deviates from the original candidate representations, fewer regions may satisfy the stricter relative-depth-gain threshold, leading to fewer finer Gaussians being selected. Consistent with this explanation, we observed a substantial decrease in the number of selected G4 Gaussians as the perturbation magnitude increased. Consequently, mesh size decreases from 9.16 to 4.42 MB at 1.0 × RMSE, accompanied by an increase in CD from 0.1579 to 0.1877. This accuracy change should therefore be interpreted together with the substantial reduction in mesh complexity, rather than solely as degradation at a fixed mesh budget.

Overall, these results indicate that Gaussian selection is relatively insensitive to mild reference-depth perturbations, while stronger perturbations can alter the selected Gaussian budget and resulting mesh accuracy, particularly at higher γ\gamma. These findings further characterize the sensitivity to reference quality discussed in Appendix J.


We sincerely thank the reviewer for the thoughtful comments and suggestions. We would be happy to address any further questions or concerns.

Reviewer_ywKgOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The proposed method constructs representations at five Gaussian budgets. It progressively combines candidates G1,...,G4G_1, ... ,G_4, using the largest representation G5G_5 only to provide reference depth maps. At each stage, it measures how much the next candidate reduces depth error relative to this reference, compared with the current composite.

The evaluation covers Gaussian Wrapping, MILo, and GGGS on DeMapGS, NeRF Synthetic, and Mip-NeRF 360, with supplementary DTU and Tanks and Temples experiments.

The results establish that the complete pipeline can improve trade-offs relative to direct extraction from individual budgets without the additional 2K reference-depth refinement.

Critical strengths
  1. Rendering-Optimal Is Not Mesh-Optimal is a relevant and important question. The paper evaluates the size and quality of an extracted mesh, which are useful objectives beyond Gaussian rendering PSNR.
  2. Several gains in the proposed method are substantive.
  3. The method is straightforward to integrate.
Critical weaknesses
  1. The advantage over mesh simplification is conditional, and the independent contribution of composition is not convincingly isolated.
  2. The depth-gain rule is plausible but only indirectly related to the mesh objective.
  3. The novelty should be stated more narrowly relative to the cited backbones.
Questions for authors
  1. How often does a predicted positive depth gain translate into an actual reduction after subset replacement?
  2. At matched face counts, how does the method compare with optimized G4G_4 followed by QEM? Can G4G_4-only importance sampling or pruning followed by identical refinement match the full pipeline?
  3. Which Gaussian Wrapping extraction variant and depth definition are used?
  4. How does the current method compare to GoDe (Di Sario et al., 2025), which already supports multiple Gaussian budgets through a progressive hierarchy?
4 comments

Q1. Actual Depth Improvement After Gaussian Subset Replacement

We sincerely thank the reviewer for the careful assessment and constructive feedback. To address the question of whether predicted depth improvements are realized after Gaussian subset replacement, we conducted an additional analysis of our two-stage relative-depth-gain criterion (Sec. 3.2–3.3), which first identifies regions where finer Gaussians improve depth accuracy and then selects Gaussians for replacement.

Experimental setup. We evaluated MILo on five DeMapGS scenes (Book, Buddha, Bust, Cat, and Tree), using γ=0.2\gamma=0.2 and 0.60.6. At each composition stage (G2–G4), we measured the percentage of predicted improvement pixels whose reference-depth error actually decreased after subset replacement, excluding coverage addition and subsequent 2K Gaussian optimization to isolate the effect of replacement.

Composition stageγ=0.2\gamma=0.2 ↑γ=0.6\gamma=0.6 ↑
G295.55%22.68%
G397.46%51.71%
G491.40%57.48%
Overall95.59%48.13%

At γ=0.2\gamma=0.2, 95.59% of the identified pixels show an actual reduction in reference-depth error after subset replacement. This result supports the effectiveness of our criterion in identifying regions where replacement leads to improvement.

At γ=0.6\gamma=0.6, this percentage decreases to 48.13%, while 43.54% of the identified pixels remain unchanged after subset replacement. This behavior is consistent with the conservative Gaussian-level selection described in Sec. 3.2–3.3. As discussed in the paper, even when some pixels would benefit from finer Gaussians, our criterion intentionally avoids replacing a Gaussian if it also contributes substantially to regions where refinement is unnecessary.

Overall, these results show that the predicted depth improvements are largely realized after subset replacement at γ=0.2\gamma=0.2, while the stricter γ=0.6\gamma=0.6 threshold leads to more selective replacements. We will include this analysis to further clarify the behavior of our selection criterion in the revised paper.


We are also conducting additional experiments to address W1 and Q2 and will share the results once they are completed.

Q3. Gaussian Wrapping Extraction Variant and Depth Definition

We use the same two-pivot mesh extraction with learned normals and identical extraction settings as the Gaussian Wrapping baseline (sdf_mode=ours). For depth supervision during Gaussian refinement, we use Gaussian Wrapping's median-depth definition, corresponding to the transmittance threshold T(t)=0.5T(t)=0.5.

For our Gaussian selection criterion (Sec. 3.2), we instead use opacity-normalized expected camera-space z-depth, defined as

D=∑iTiαizi∑iTiαi.D = \frac{\sum_i T_i \alpha_i z_i}{\sum_i T_i \alpha_i}.

We observed that using median depth for Gaussian selection can lead to abrupt depth changes when comparing different Gaussian subsets. This is because the transmittance threshold-crossing location can shift sharply as the Gaussian configuration changes. In contrast, opacity-normalized expected depth avoids this threshold-crossing dependency, providing a more stable signal for relative-depth-gain estimation.

We will clarify the distinction between the depth definitions used for selection and refinement in the revised manuscript.


Q4. Comparison with GoDe

We thank the reviewer for drawing our attention to GoDe (Di Sario et al., 2025). While GoDe efficiently constructs progressive Gaussian hierarchies for rendering and compression, our method focuses on improving the geometry–mesh-size trade-off of extracted meshes.

The methodological distinction lies in how Gaussian budgets are organized and selected. GoDe constructs an importance-based progressive hierarchy to support multiple rendering budgets, whereas our method spatially composes Gaussians from different budgets using a relative-depth-gain criterion. Rather than selecting Gaussians based on rendering importance, our approach selectively replaces lower-budget Gaussians where finer representations provide sufficient geometric improvement before mesh extraction.

Our additional comparisons with rendering-oriented Gaussian methods, FLoD and GNS (please see our response to Reviewer ERz1, W2), further illustrate that high Gaussian rendering quality does not necessarily translate into favorable mesh-size–accuracy trade-offs. For example, GNS with 30K Gaussians achieves a Gaussian PSNR of 42.82 dB but produces a 21.52 MB mesh with CD 0.1850, whereas Gaussian Wrapping-Ours (γ=0.4\gamma=0.4) achieves the same PSNR with a smaller mesh (10.17 MB) and lower CD (0.1545).

These results highlight the distinction between rendering-oriented Gaussian budget allocation and our geometry-aware composition for compact surface extraction. We will include GoDe and clarify this methodological distinction in the Related Work section of the revised manuscript.

W2. Indirect Relationship to Mesh Objective

While our relative-depth-gain criterion does not directly optimize the final mesh objective, it provides effective geometry-aware guidance for Gaussian selection before mesh extraction. As shown in Fig. 5, depth-guided selection achieves a better CD–mesh-size trade-off than RGB-based selection, and Gaussian replacement alone improves the trade-off over G4 even without subsequent optimization. These results demonstrate that our depth-guided criterion contributes to improving mesh compactness and geometric accuracy despite its indirect relationship to the final mesh objective.

As discussed in Appendix K, incorporating the mesh-complexity cost of individual Gaussian replacements directly into the selection criterion could provide more explicit control over the geometry–mesh-size trade-off. We consider this a direction for future work.


W3. Novelty Relative to Existing Backbones

Our approach builds upon existing Gaussian-based surface reconstruction methods, including Gaussian Wrapping, MILo, and GGGS, whose Gaussian distributions influence the complexity of extracted meshes.

Our methodological contribution lies in geometry-aware composition across Gaussian budgets. Specifically, we introduce a relative-depth-gain criterion to selectively replace lower-budget Gaussians with finer ones where they provide sufficient geometric improvement. This composition is performed before mesh extraction, without modifying the underlying reconstruction or extraction algorithms.

W1 & Q2. Independent Contribution of Multi-Budget Composition and Comparison with Single-Budget Alternatives

We apologize for the delay in responding to W1 and Q2, as we have been conducting additional experiments. We sincerely appreciate the reviewer ywKg's patience and constructive feedback. We provide the results of our additional comparisons with identically optimized G4, optimized G4 followed by QEM simplification, and G4-only importance pruning below.

1. Independent contribution of multi-budget composition.

We extended the matched-optimization experiment in Table 12 to Gaussian Wrapping and GGGS, in addition to MILo. Specifically, we applied identical 2K Gaussian optimization to both G4 and our composed representations.

The following results are averaged over five DeMapGS scenes (Book, Buddha, Bust, Cat, and Tree).

MethodRepresentationMesh (MB) ↓CD ↓
Gaussian WrappingG4 + 2K optim.10.460.1544
Ours (γ=0.2)10.430.1542
MILoG4 + 2K optim.18.110.1342
Ours (γ=0.2)17.780.1341
GGGSG4 + 2K optim.83.240.1132
Ours (γ=0.2)82.100.1116

Our method achieves smaller meshes with lower CD across all three backbones under identical refinement. Importantly, these improvements are also observed before the final 2K optimization. For example, with GGGS, composition reduces mesh size from 77.15 to 76.31 MB while improving CD from 0.1180 to 0.1153, without additional optimization. Similar improvements are observed for Gaussian Wrapping and MILo (please see our response to Reviewer SYik, W1/Q1, for the detailed comparison). These results support the independent contribution of composition beyond the subsequent Gaussian optimization.

2. Comparison with optimized G4 followed by QEM.

We additionally applied QEM simplification to meshes extracted from identically optimized G4 representations, matching the face counts of our results at γ=0.2 and γ=0.4, resulting in nearly identical mesh sizes.

The following table reports changes in CD relative to optimized G4 before QEM simplification, averaged over the same five DeMapGS scenes.

MethodΔCD (γ=0.2)ΔCD (γ=0.4)
MILo−0.000012+0.000015
GGGS+0.000039+0.000018
Gaussian Wrapping−0.000029+0.000035

QEM largely preserves the geometric accuracy of optimized G4, with CD changes smaller than 0.00004 across all evaluated settings. Consequently, the CD values before and after QEM remain nearly identical at the reported precision.

At γ=0.2, our method achieves lower CD than optimized G4+QEM across all three backbones. At γ=0.4, our method retains lower CD for GGGS, whereas optimized G4+QEM achieves lower CD for MILo and Gaussian Wrapping.

These findings are consistent with the conditional effectiveness of post-extraction simplification discussed in Appendix H. We acknowledge that QEM can achieve favorable geometry–mesh-size trade-offs when an accurate intermediate mesh is available. Our approach instead reorganizes Gaussian representations before mesh extraction, allowing compact meshes to be obtained directly without post-extraction simplification.

3. Comparison with G4-only importance pruning.

To further examine whether single-budget pruning can reproduce the benefits of cross-budget composition, we additionally evaluated G4-only importance pruning followed by identical 2K Gaussian optimization on Mip-NeRF 360, which contains unbounded scenes with spatially varying geometric complexity. For this baseline, we followed the importance-based Gaussian sampling strategy used in MILo to select Gaussians from G4.

The following results are averaged over all nine scenes of the Mip-NeRF 360 dataset.

MethodRepresentationMesh PSNR (dB) ↑Mesh (MB) ↓
MILoOurs (γ=0.4)23.49118.54
G4 pruning + 2K optim.23.25118.93
GGGSOurs (γ=0.4)23.02213.06
G4 pruning + 2K optim.23.00235.97

For MILo, our method achieves 0.24 dB higher mesh-rendering PSNR with nearly identical mesh size (118.54 vs. 118.93 MB). For GGGS, our method produces a 9.7% smaller mesh (213.06 vs. 235.97 MB) while maintaining comparable mesh-rendering PSNR (23.02 vs. 23.00 dB).

These results suggest that, in the evaluated unbounded scenes, G4-only importance pruning followed by identical refinement does not necessarily reproduce the benefits of our geometry-aware cross-budget composition.


We sincerely thank the reviewer again for their patience and thoughtful consideration of our additional experiments. We would be grateful for any further questions or suggestions and would be happy to provide additional clarification if helpful.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.