acceptodds
Under review as a conference paper at ICLR 2027

DeltaMerger: Learning Visual Merge Costs for Zero-Shot Gaussian Splat Simplification

Abstract

We introduce DeltaMerger, which simplifies pretrained 3D Gaussian Splatting representations by recursive pairwise merging with a learned merge cost, followed by a shared terminal refiner. Analytic merge costs measure the attribute discrepancy between two Gaussians and their proposed parent, but the visual effect of a merge also depends on visibility, compositing, and the distortions left by earlier merges. We supervise a compact shared predictor with rendered interventions at intermediate simplification states, measuring the immediate image change caused by replacing a candidate pair with its analytic parent and the signed change in error relative to the original rendering. The requested budget selects between two supervision regimes: the immediate change at 5–25% retention, and the change in fidelity to the original rendering, which can favor merges that partially repair earlier distortion, at 0.1–1%. Trained once on 64 DL3DV scenes, the frozen predictor reads only descriptors of the pair, its neighborhood, and its proposed parent, together with simplification progress and the requested budget, and transfers zero-shot to held-out DL3DV scenes and to four datasets never used for training, without images, cameras, rendering, or per-scene optimization. On 46 held-out scenes, DeltaMerger without refinement improves mean PSNR over NanoGS with the same candidate neighborhood at every retention budget, by 0.35 dB on average. Its shared refiner, trained once through differentiable rendering, corrects appearance and scale at fixed primitive count; the complete DeltaMerger improves on unrefined analytic selection by 1.12 dB, and learned selection keeps a 0.14 dB advantage when analytic selection receives its own refiner.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Gaussian Splatting, Gaussian simplification, primitive merging, learned merge cost, rendered interventions, zero-shot transfer, 3D scene compression

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gnwzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a learned cost for selecting Gaussian pairs during recursive simplification. It retains an analytic parent-construction rule and trains a predictor using rendered interventions at intermediate simplification states. The supervision distinguishes immediate rendering changes from changes in fidelity to the original representation. The idea is interesting.

Critical strengths

The rendered-intervention supervision is well motivated.

The evaluation covers 46 scenes from five datasets and three training seeds, with per-scene results and paired confidence intervals.

The matched-refinement experiment is particularly useful. Training separate refiners for analytic and learned selection helps isolate the contribution of the learned merge cost.

Critical weaknesses

Some evaluation scenes were leaked during MLP training. The paper should identify these scenes and report results separately on the test scenes.

I have a concern about the supervision. The MLP is trained on individual pair merges, while each simplification pass merges several disjoint pairs. Their visual effects may interact through occlusion and alpha compositing, so the individual scores may not fully capture the joint effect. I need some explanations.

Questions for authors

What is the main difference compared to other importance-based score pruning methods?

How much does it cost to get all training datasets?

Could a MLP determine the parameters of the merged Gaussians?

6 comments
AuthorsOpenReview ↗

Response to Reviewer gnwz (1/3): W1 and Q1

We thank the reviewer for the positive assessment, for finding the supervision well motivated and the matched-refinement experiment useful, and for the questions. We answer each weakness and question below; Tables A and B are in “General response (2/3)”, and the revisions are in blue in the revised PDF.

W1. Evaluation scenes and training

“Some evaluation scenes were leaked during MLP training. The paper should identify these scenes and report results separately on the test scenes.”

None of the original 46 scenes is among the training scenes of the submitted networks: the predictor and the refiners were trained on 64 DL3DV scenes only.

After fixing the submitted method, hyperparameters, and checkpoints, we checked cross-dataset transfer during development on nightsnow and playroom (Deep Blending), chair and drums (NeRF Synthetic), and train and truck (Tanks and Temples). This check did not inform those choices. Table A and manuscript Table 16 report the other 40 scenes from the submitted panel, which form the primary evaluation. Appendix D.7 and Table 31 report the six separately.

Scenes outside the 46. With the submitted networks unchanged, Table B reports 97 scenes outside the 46, from nine datasets; six of these datasets appear neither in training nor among the 46. They range from close-up objects to city districts seen from the air, in handheld and DSLR photos, videos and rendered aerial views, whereas all training scenes come from DL3DV.

Over the 97 scenes, learned − analytic selection (PSNR, each with its own refiner) is above zero from 1% to 25% and below zero at 0.1% (−0.08 dB); at 5–15% it is above zero in 7 of the 8 groups of four or more scenes, while on LLFF (8) the interval contains zero at every budget. Our full method is +0.8 to +1.5 dB above official NanoGS in these groups (mean of the five paper budgets).

Revision: Table B is in Appendix D.1 (Table 17).

Q1. Difference from importance-based pruning

“What is the main difference compared to other importance-based score pruning methods?”

Pruning scores single Gaussians and deletes them; we score pairs and replace each selected pair by one parent that still covers both. Regions covered only by deleted Gaussians lose their coverage, unless the survivors are re-optimized with photographs; our parent keeps the mass-weighted moments (position, covariance, color) of its two children, and the costs are recomputed after each pass.

Our score is learned from the rendered effect of the merge itself, with the target budget as an input, and needs no camera at deployment, whereas the scores of LightGaussian, PUP and REFINE are computed from the training cameras. Our pipeline does contain one pruning step, the opacity prefilter.

Original 46-scene panel, PSNR in dB, without fine-tuning (pruning step only for LightGaussian and PUP). We ran this comparison in the campaign for the submitted Table 1 and report it in Appendix D.5 of the revision.

Method0.1%1%5%10%25%
LightGaussian11.6915.3018.8220.8724.55
PUP8.5611.0615.3018.2523.19
REFINE11.3314.6417.6919.5523.59
RAP (submitted Table 1)10.4914.9719.1521.4125.00
Ours, without refiner17.0119.9822.3823.6525.67
Ours, full method17.6220.8923.3524.5226.14

Without refiner, our selection is ahead of LightGaussian in PSNR, the best of the three pruning methods we added, by +5.3 dB at 0.1% down to +1.1 dB at 25%, in 45 or 46 of the 46 scenes from 0.1% to 10% and in 31 at 25%.

Revision: Appendix D.5.

AuthorsOpenReview ↗

Response to Reviewer gnwz (2/3): W2, interactions between the merges of a pass

W2. Single-merge labels and the joint effect of a pass

“Their visual effects may interact through occlusion and alpha compositing, so the individual scores may not fully capture the joint effect.”

The concern is correct, and the paper did not quantify it; we now measure the gap and test one of its consequences. A training label is the rendered effect of one merge in an otherwise unchanged state; a pass applies up to 0.5N disjoint merges (N: active Gaussians), and the labels miss their joint effect. Every reported quality result includes it, since quality is measured after the complete simplification.

How large is the gap? On 14 DL3DV development scenes (one seed), as a fraction of the image change of the same merges: sets of 4 or 16 scattered merges differ from the sum of their single effects by less than 1% on average, but their joint effect equals the sum exactly in 86% of the sets of 4 and 49% of the sets of 16 (the merges do not overlap in the image); a whole pass differs from ten times a random tenth of its merges by +17% at 0.1% and by −3% to +9% at 1–25% (three of five intervals exclude zero).

The order changes more: 21–33% of the pairs of probed merges change order when rendered inside their pass instead of alone, with either cost.

A tested consequence. If interactions inside a pass matter, a cost trained on single merges should do better with smaller passes.

After the reviews, we capped the passes of learned selection at 0.25N (size chosen among three on six development scenes); the submitted method keeps 0.5N. Analytic selection keeps 0.5N, its best measured setting (capped at 0.25N, it loses 0.3 to 2.1 dB on the 33 scenes below). Learned − analytic, each with its refiner, in dB; in parentheses, the submitted 0.5N on the same scenes:

Scenes0.1%1%5%10%25%
Original 46-scene panel−0.33 (−0.17)+0.29 (+0.12)+0.59 (+0.29)+0.62 (+0.30)+0.39 (+0.18)
15 DL3DV scenes of Table B−0.25 (−0.25)+0.40 (+0.16)+0.66 (+0.35)+0.76 (+0.35)+0.64 (+0.22)
25 other DL3DV scenes−0.27 (−0.15)+0.27 (+0.13)+0.57 (+0.38)+0.72 (+0.37)+0.60 (+0.19)
Tanks and Temples (5), one seed and one analytic refiner−0.13 (−0.05)+0.38 (−0.08)+0.55 (+0.23)+0.46 (+0.21)+0.31 (+0.06)
33 scenes of five other datasets−0.11 (0.00)+0.25 (+0.06)+0.55 (+0.28)+0.50 (+0.24)+0.26 (+0.13)

Overall, at 1–25% smaller passes raise the gain to +0.25 to +0.76 dB on all five sets (every interval above zero), consistent with the reviewer's concern; the loss at 0.1% is not reduced, and is larger on the 46 scenes and the 25 DL3DV scenes.

  • The effect of the pass size on our own selection is not uniform: below zero at 0.1% on the 33 scenes (−0.10 dB), positive on ETH3D, Deep Blending and LLFF, close to zero on Nerfstudio, and negative on the single Ref-NeRF real scene (−0.26 to −0.43 dB at 5–25%). Retraining our refiner on 0.25N states changes little.

  • Time: 1.37 times the end-to-end time of the submitted setting (95% interval 1.33–1.40); this is 0.78 times the wall-clock time of the official optimized NanoGS (k = 4), and 1.48 times its CPU time.

This pass-size sensitivity analysis uses the original 46-scene panel, including the six scenes inspected to check transfer during development.

Limits: this analysis does not isolate the mechanism, since the two sides do not use the same pass size and smaller passes also recompute the costs twice as often.

Revision: Section 3.3 states the single-merge approximation; Appendix D.6 reports this sensitivity analysis and its run time.

AuthorsOpenReview ↗

Response to Reviewer gnwz (3/3): Q2 and Q3

Q2. Cost of the training data

“How much does it cost to get all training datasets?”

A one-time cost: under an hour of one workstation (24 CPU cores; the collection is CPU-bound) for the predictor labels, and about 5 CPU job-hours per predictor seed for the refiner's training states. The data are generated automatically from 64 public DL3DV scenes and their released 3DGS models, without annotation.

ItemCost
Predictor labels (1.1 GB)about 48 min of one workstation (estimated from six of the 128 collection jobs, timed and extrapolated)
Refiner training states (12.7 GB), per predictor seedabout 5 CPU job-hours (13.7 when we repeated one seed on a shared machine)
Training the predictor25–30 s in the logs of the submitted checkpoints
Training the refiner18–20 min of optimization on one RTX A6000, measured after the submission on three re-trainings with the same recipe (one for the submitted predictor) (43–58 min for the whole process)
Deployment, per assetour full method: 7.3 s on four CPU cores; official NanoGS (k = 4): 12.6 s (Table 8)

For scale, collecting the predictor labels takes about as long as fitting a single 3DGS scene, and the two networks (62 kB and 38 kB) are then used unchanged on every new asset.

Q3. A network for the parameters of merged Gaussians

“Could a MLP determine the parameters of the merged Gaussians?”

Yes for the final Gaussians: this is what our refiner does. For the parents built during the recursion, we keep the analytic moments; two small prototypes of learned parents were not conclusive.

  • Final Gaussians. The refiner is an MLP (7,751 parameters) shared by all scenes. It uses no photograph and no camera at deployment, and applies bounded corrections to the color, opacity and three scales of every merged final Gaussian: +0.75 dB after learned selection and +0.96 dB after analytic selection (Table 2). At the six budgets from 0.5% to 20% that it was not trained on, it adds +0.54 to +0.99 dB on the 40 primary evaluation scenes.

  • Parents of the recursion: why analytic. The moments keep every covariance valid by construction, whereas a predicted parent needs constraints or a fallback. A parent becomes a child in later passes: at 0.1%, a final Gaussian results from up to about ten levels of pairwise merges, so errors of a learned parent would compound, whereas an error of the cost changes which pairs are merged but leaves every parent a moment match of its two children. Our labels, descriptors and training states also depend on the parent rule, so learning the parent means training parent and cost together.

  • Prototypes. A learned correction of the parent color (before the submission, earlier version of the system): the best of 16 variants, a linear model, was 0.014 dB below the analytic parent. Learned parents in the last passes (analytic selection): +0.47 / +0.17 dB at 0.1 / 1% without refiner, but +0.10 / −0.03 dB with a refiner for each variant (+0.02 dB at 0.1% without the scene that had served to choose the checkpoints).

Limits: we did not test larger models or more training data for the parents, and we do not conclude that learned parents cannot help.

Revision: the revised paper distinguishes the two cases (final Gaussians; parents of the recursion).

Thank you for these questions. The revised manuscript reports the primary evaluation on 40 scenes, the six transfer checks in Appendix D.7, and results on 97 additional scenes in Appendix D.1. Our reply to W2 measures how merges interact within a pass, and Appendix D.6 reports how changing the pass size affects quality and runtime. The revision also includes the pruning comparison in Appendix D.5 and the training costs in Table 10. We would be happy to discuss any of these results further.

Reviewer_QYJ4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DeltaMerger simplifies pretrained 3DGS models through recursive merging guided by a learned cost predictor, followed by a shared terminal refiner. The predictor uses local Gaussian descriptors and budget-dependent supervision from rendered interventions. Deployment requires only the Gaussian model and target count, without images, cameras, or per-scene optimization.

Critical strengths
  1. Practical deployment: Compact networks enable Gaussian-only simplification and produce standard Gaussian assets.

  2. Motivated supervision: Distinguishing immediate image changes from changes in accumulated error provides a useful perspective on merge selection.

  3. Controlled experiments: Matched selection and refinement comparisons, multiple datasets, and three training seeds support the empirical results. Learned selection alone improves mean PSNR by approximately 0.35 dB.

Critical weaknesses
  1. Insufficient input information: The predictor uses local statistics but lacks explicit information about viewpoints, occlusion, and accumulated rendering errors. For example, the same Gaussian pair may be visible or hidden by distant objects, giving different true merge costs despite identical network inputs. Full-scene supervision cannot resolve this missing information at deployment. The reported gains show that the predictor is useful on average, but direct tests of ranking accuracy and sensitivity to occlusion changes are needed to assess its reliability.

  2. Modest benefit with matched refinement: The learned selector’s advantage decreases from approximately 0.35 dB to 0.14 dB when both planners receive their own refiners, and reverses to −0.17 dB at 0.1% retention. Further analysis is needed to establish when learned selection provides benefits beyond refinement.

  3. Generalization remains incompletely tested: Some evaluation scenes were inspected during development, and training and evaluation share the same five retention budgets. An untouched test set and unseen-budget evaluation would strengthen the transfer claims.

Questions for authors

Please refer to the weakness.

7 comments
AuthorsOpenReview ↗

Response to Reviewer QYJ4 (1/4): W1, test 1 (ranking accuracy)

We thank the reviewer for the precise diagnosis, and for finding the deployment practical, the supervision motivated and the controlled experiments supportive. We answer each weakness below; Tables A and B are in “General response (2/3)”, and the revisions are in blue in the revised PDF.

W1. Ranking accuracy

“The predictor uses local statistics but lacks explicit information about viewpoints, occlusion, and accumulated rendering errors. … direct tests of ranking accuracy and sensitivity to occlusion changes are needed to assess its reliability.”

We agree with the premise, and we ran both tests. None of the 35 inputs encodes cameras, visibility, occlusion or accumulated error. Without cameras, the predictor can only estimate a cost averaged over views and over the configurations that share the same local descriptors; the rendered labels, pooled over eight cameras, provide such an average in training. The tests measure how reliable this average is.

Test 1: ranking accuracy. In each state of a simplification run, we take 24 candidate pairs, render the cost of merging each one alone on eight cameras, as in training, and compare this order with the order of the predicted costs (within-state Spearman; the rendered cost is our predictor's own training target).

Dataset (scenes)OursAnalyticOurs − analytic (95% CI)Ours higher
DL3DV, development (15)0.3600.208+0.152 (+0.124, +0.180)15 of 15
DL3DV, Table B (15)0.3630.199+0.164 (+0.137, +0.196)15 of 15
Nerfstudio (9)0.2960.136+0.160 (+0.120, +0.199)9 of 9
Deep Blending, Table B (3)0.2850.163+0.123 (no interval)3 of 3
LLFF (2)0.3230.182+0.141 (no interval)2 of 2
Ref-NeRF real (1)0.2750.179+0.097 (no interval)1 of 1
ETH3D (8)0.2370.148+0.090 (+0.054, +0.126)7 of 8
BungeeNeRF (4)0.2710.182+0.089 (+0.076, +0.102)4 of 4
Mip-NeRF 360, original panel (5)0.2230.153+0.070 (+0.039, +0.100)5 of 5
Deep Blending, original panel (4)0.1520.047+0.105 (+0.074, +0.129)4 of 4
NeRF Synthetic, original panel (4)0.3120.203+0.109 (+0.059, +0.179)4 of 4

All scenes are outside training (64 DL3DV scenes). Table B uses additional scenes. Table A reports the 40 primary scenes from the original panel.

For scale: a random order scores 0 in expectation, the analytic cost 0.05–0.21, ours 0.15–0.36, and the camera-informed reference of Part 3 0.58–0.65.

Overall, our cost ranks merges closer to their rendered cost than the analytic cost in every dataset, and in 69 of the 70 scenes; every interval is above zero.

The difference between the two costs varies with the budget. It is largest at 5–25%: there, the interval is above zero for every dataset with four or more scenes, except Mip-NeRF 360 at 10% and Deep Blending and NeRF Synthetic (original panel) at 25%.

At 1%, by contrast, the interval is above zero only on DL3DV and, barely, on NeRF Synthetic; at 0.1%, it contains zero on ETH3D, Nerfstudio and NeRF Synthetic. At these two budgets, the part of the target that depends on the error already accumulated accounts for most of its within-state variance (0.70 at 0.1% and 0.65 at 1%, DL3DV development). Neither cost observes this error, and both rank poorly there. The 0.1% budget is also the one at which learned selection is behind after refinement (W2).

Revision: Section 3.3 (limit of the inputs); Appendix D.4.

AuthorsOpenReview ↗

Response to Reviewer QYJ4 (2/4): W1 test 2 (sensitivity to occlusion changes)

W1 (continued). Sensitivity to occlusion changes

“For example, the same Gaussian pair may be visible or hidden by distant objects, giving different true merge costs despite identical network inputs.”

The premise holds for the cost values, much less for the ranking: 10–39% of the costs change by a factor of two or more, but the rankings with and without the content in front agree at 0.82–0.97 (Spearman).

We built this test with the same merges, rendered with and without the content in front of them: for each candidate merge and each of the eight cameras, we removed the Gaussians that lie in front of the pair without touching it, and rendered the merge again. Both costs keep their scores on the unmodified state, so only the reference ranking changes. Gap: Spearman of ours − analytic (95% interval), against the observed ranking and against the ranking without that content.

Dataset (scenes)Cost changed ×2 or moreRankings agreeGap, observedGap, without
DL3DV, development (9)32%0.87+0.117 (+0.066, +0.158)+0.095 (+0.045, +0.139)
DL3DV, Table B (15)29%0.85+0.178 (+0.134, +0.223)+0.111 (+0.080, +0.145)
Nerfstudio (9)39%0.82+0.157 (+0.103, +0.215)+0.100 (+0.058, +0.143)
Deep Blending, Table B (3)24%0.88+0.096+0.056
LLFF (2)21%0.90+0.100+0.080
Ref-NeRF real (1)10%0.95+0.020+0.048
BungeeNeRF (4)16%0.92+0.100 (+0.068, +0.133)+0.094 (+0.078, +0.118)
Deep Blending, original panel (4)11%0.97+0.090 (+0.064, +0.118)+0.088 (+0.058, +0.129)
NeRF Synthetic, original panel (4)11%0.97+0.021 (−0.045, +0.105)+0.023 (−0.052, +0.110)

Overall, our cost stays ahead of the analytic cost under both rankings on the three larger scene sets: DL3DV development, DL3DV of Table B, and the 15 scenes of Nerfstudio, Deep Blending, LLFF and Ref-NeRF real taken together, for which the gap is +0.128 observed and +0.085 without (both intervals above zero).

This holds on average over the budgets and at 5%, 10% and 25% taken separately. Without the content in front, the gap is nonetheless smaller on DL3DV of Table B and on these 15 scenes (by 0.067 and 0.043), because the analytic cost gains there. At 0.1%, the analytic cost is higher than ours against the ranking without the content in front (−0.09, −0.12 and −0.14 on the three sets). On the two remaining sets, BungeeNeRF and the 8 original-panel scenes (Deep Blending and NeRF Synthetic together), the gap is positive, but we state no direction: with four scenes, the exact test over the scenes cannot go below p = 0.125, and it gives p = 0.08 and 0.09 on these 8 scenes.

The reviewer's example in our data (descriptive). Among the 13,560 merges of Test 1 on DL3DV development, 90 have their parent inside a label view yet change no pixel in any of the eight views (occluded, sub-pixel or invisible change).

Their rendered cost is low, with a mean rank of 0.22 on a 0 to 1 scale, but both costs rank them near the middle: ours at 0.44 and the analytic cost at 0.47. Neither cost identifies this case, as the reviewer expected.

Revision: Appendix D.4.

AuthorsOpenReview ↗

Response to Reviewer QYJ4 (3/4): W1 tests 3 and 4 (what the missing information costs)

W1 (continued). The price of using no camera

“Full-scene supervision cannot resolve this missing information at deployment.”

We agree, and we measured this price. We retrained the same network with inputs that need the cameras and the original renderings: the visibility of the pair in each view and the error already accumulated there. This reference is not deployable; it shows how well a cost can rank merges with this information. It is outside our setting: our method needs only the PLY file, which carries no camera or photograph, and runs on four CPU cores (7.3 s per asset, Table 8).

This reference would also need a rasterizer at every pass, and rendering the cost of each candidate merge, as for our training labels, is far costlier: about 16 hours per million candidates on an RTX 5090, while a pass on a large scene scores millions.

States, candidates and target are those of Test 1 (ours here is the mean of the three predictor seeds, 0.365 on DL3DV of Table B, against 0.363 for seed 0 alone in Test 1).

Test 3: a camera-informed reference.

Dataset (scenes)AnalyticOursCamera-informedDistance covered by ours
DL3DV, Table B (15)0.1990.3650.62739% (33–45%)
Nerfstudio (9)0.1360.2980.62933%
Deep Blending, Table B (3)0.1630.2920.63527%
LLFF (2)0.1820.3350.58138%
Ref-NeRF real (1)0.1790.2970.59329%
BungeeNeRF (4)0.1820.2780.64521% (17–24%)
Deep Blending, original panel (4)0.0470.1500.63617%
NeRF Synthetic, original panel (4)0.2030.3060.64823%

Distance covered: (ours − analytic) / (camera-informed − analytic); its interval is given where the scene set is a single dataset.

Overall, our camera-free cost covers 17–39% of the distance from the analytic cost to this reference, depending on the dataset; the rest is the measured price of using no camera for these inputs, which the analytic cost pays too.

  • The informed network is ahead of ours in every scene, and visibility brings almost all of its gain (the accumulated error adds +0.003 to +0.015).

  • It reaches 0.796–0.873 at 5–25% and 0.304–0.395 at 0.1% and 1%. At 1%, the share covered by our cost is not separated from zero on BungeeNeRF (10%), on the 15 scenes of Nerfstudio, Deep Blending, LLFF and Ref-NeRF real together (9%) and on the 8 original-panel scenes together (11%).

Test 4: does the ranking error change which merges a pass selects? Both costs go through the same selector, which accepts pairs by increasing cost and skips any pair that touches a Gaussian already taken. On the 15 DL3DV scenes, no non-final pass stopped at the 0.5N cap (88–91% of the active Gaussians were merged), so a pass is fixed by the order of the costs among the candidate partners of each Gaussian.

On the states of Test 1, our cost orders these partners as rendered in 62.2% of cases against 59.1% for the analytic cost (+3.2 points, 95% CI +2.0 to +4.4; 15 DL3DV scenes) and in 61.4% against 58.4% (+3.1, +1.1 to +5.0; 4 BungeeNeRF scenes), where chance is 50%; on DL3DV there is no difference at 0.1% and 1%. It finds the cheapest partner in 36.5% against 33.2% (exact test p = 0.07) and in 35.3% against 31.3% (interval contains zero).

Revision: Section 3.3; Appendix D.4.

Reviewer_KUhXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes DeltaMerger for zero-shot simplification of pretrained 3D Gaussian Splatting representations. The method replaces the analytic merge cost of NanoGS with a learned predictor. A refiner further adjusts appearance, opacity, and scale after simplification. Both networks are trained once and applied to new Gaussian assets without per-scene optimization. Experiments show improvements over NanoGS.

Critical strengths
  1. The paper addresses a useful practical problem.

  2. The rendered-intervention supervision is interesting.

Critical weaknesses
  1. The technical contribution is somewhat incremental. The main framework comes from NanoGS (recursive merging, analytic parent construction). The analytic merge cost is replaced with a learned predictor and a refiner is added. Related learned post-hoc simplification methods such as CVT-GS are discussed, but no direct quantitative comparison is providec.

  2. The improvement from the learned merge cost becomes small once refinement is applied, from 0.35dB to 0.14dB. At 0.1% retention, learned selection with refinement is actually 0.17 dB worse. It seems most improvements come from the refiner rather than the learned merge cost.

  3. Ablation of the budget-dependent supervision is only reported without refinement. Since refinement seems to be critical for the improvements, the proposed supervision should be evaluated with refinement.

  4. Both networks are trained and evaluated only at the same five retention ratios. Since the predictor is explicitly conditioned on the requested budget, it is important to show how the model generalizes across budgets.

Questions for authors
  1. Why is there no quantitative comparison with CVT-GS or another closely related learned post-hoc Gaussian simplification method?

  2. Given that learned selection provides only a 0.14 dB average gain after matched refinement and loses at 0.1% retention, how do we justify the practical importance of the learned merge-cost predictor?

  3. Does the budget-dependent target schedule remain better than the alternative supervision targets when each variant has its own separately trained refiner?

  4. How does the method perform at retention ratios not used during training, for example 2%, 7%, or 15%?

6 comments
AuthorsOpenReview ↗

Response to Reviewer KUhX (1/3): W1, what is new relative to NanoGS

We thank the reviewer for the careful review, and for finding the problem useful and the rendered-intervention supervision interesting. We answer each weakness and question below; Tables A and B are in “General response (2/3)”, and the revisions are in blue in the revised PDF.

W1. What is new relative to NanoGS

“The technical contribution is somewhat incremental. The main framework comes from NanoGS (recursive merging, analytic parent construction).”

We agree on the framework: the recursion and the analytic parents come from NanoGS, and the paper credits them. Our contribution is the merge decision, a cost learned from rendered interventions on intermediate states with the budget as an input, and a shared refiner that uses no image at deployment.

The predictor trained with rendered supervision outperforms analytic-cost imitation in PSNR at the tested retentions from 1% to 25%.

We trained the same network, with the same 35 inputs, to imitate the analytic cost, and gave it a refiner trained by the same recipe (three seeds). The imitation control derives both the training targets and the pair weights from the analytic cost.

PSNR difference (dB), each with its refiner0.1%1%5%10%25%
40 scenes: imitation − analytic+0.03+0.03+0.01+0.03−0.02
40 scenes: ours − imitation−0.19+0.09+0.29+0.29+0.21
15 DL3DV scenes of Table B: imitation − analytic−0.00+0.02+0.02−0.02−0.08
15 DL3DV scenes of Table B: ours − imitation−0.25+0.14+0.33+0.37+0.29

Bold / italic: interval above / below zero.

  • Imitating the analytic cost brings no measurable gain: the imitation is not separated from refined analytic selection (it is behind at 25% on the DL3DV scenes, and slightly behind in LPIPS at 5–25% on the 40 scenes).

  • Comparison with analytic-cost imitation: ours is ahead of the imitation in 29 of the 40 scenes at 1% and in 39–40 at 5–25%; on the 40 scenes, SSIM follows the PSNR results and LPIPS favors our method at 5–25% (it favors the imitation at 1%, by 0.002). At 0.1%, ours is behind by 0.19 dB on the 40 scenes and 0.25 dB on the 15 DL3DV scenes.

  • The decision differs, not the implementation: our analytic selection without refiner is within 0.007 dB in mean PSNR of NanoGS (k = 4) at every budget on the 40 scenes (Table 2).

In summary, while the merging framework is that of NanoGS, the merge decision is new, and this control shows that rendered supervision improves PSNR over analytic-cost imitation at the tested retentions from 1% to 25%.

The table above uses the 40 primary evaluation scenes. Appendix D.2 documents the control on the original panel and describes its limitations.

AuthorsOpenReview ↗

Response to Reviewer KUhX (2/3): W2 and Q2, then Q1

W2 & Q2. The value of learned selection once both sides are refined

“The improvement from the learned merge cost becomes small once refinement is applied, from 0.35dB to 0.14dB. At 0.1% retention, learned selection with refinement is actually 0.17 dB worse.”

“… how do we justify the practical importance of the learned merge-cost predictor?”

We agree that the refiner brings the larger part of the gain (+0.75 dB after learned selection, +0.96 dB after analytic selection on the 40 primary evaluation scenes, Table 2), and that learned selection is behind at 0.1%.

From 1% to 25%, what remains after refinement lets our assets reach the same PSNR with 8–18% fewer Gaussians, for about 1.5 s more per asset. The mean gap after refinement over the five paper budgets is +0.15 dB on the 40 scenes (+0.14 dB in the submitted paper). At 1%, 5% and 10%, 60–67% of the selection gain survives matched refinement (40% at 25%):

Learned − analytic, 40 scenes of Table A0.1%1%5%10%25%
Without refiner (dB)+0.140+0.175+0.453+0.536+0.471
Each with its own refiner (dB)−0.164+0.117+0.299+0.320+0.191
Share of the gain kept after refinementreversed67%66%60%40%
Gaussians analytic selection needs for our PSNR0.87×1.09×1.21×1.20×1.11×

Bold / italic: interval above / below zero. Last row: interpolated; every interval above 1× from 1% to 25%.

  • What 8–18% means in practice: file size and GPU memory grow with the number of Gaussians. On bicycle (Mip-NeRF 360; 6,131,954 Gaussians), our 10% asset keeps 613,195; to match its PSNR, analytic selection with its own refiner needs 1.18× as many (best of three seeds; 1.14–1.18× across seeds): a 180 MB PLY instead of 152 MB, and 277 MB of GPU memory to render one 1237×822 view instead of 237 MB (RTX A6000, gsplat renderer).

  • Cost: 7.26 s per asset for our full method against 5.75 s for analytic selection with its refiner (four CPU cores). Official NanoGS (k = 4) takes 12.56 s.

  • Beyond these budgets and scenes: the gain after refinement is above zero at all ten budgets from 1% to 40% (Table A) and, over the 97 scenes outside the original 46-scene panel, from 1% to 25% (Table B). These come from nine datasets, from close-up objects to city districts seen from the air, in handheld and DSLR photos, videos and rendered aerial views; at 5–15%, it is above zero in 7 of the 8 datasets with four or more scenes.

  • Not an artifact of one refiner: with two more refiner seeds per predictor seed (learned side only), the mean PSNR on the 40 primary scenes changes by at most 0.07 dB per budget, and the learned-minus-analytic difference keeps its sign at all five budgets.

  • Where it does not help: at 0.1% (ahead in 8 of the 40 scenes) and at 0.05% (−0.23 dB) learned selection is behind, and at 0.5% there is no clear difference. At 0.1%, smaller passes do not reduce the loss (reply to Reviewer gnwz, W2). Analytic selection with its own refiner has higher PSNR at 0.1% (Table 1) and 0.05% (Table A).

Revision: Table A in Appendix D, Table B in D.1, and the refiner-seed analysis on the original panel in D.3. See also the abstract, limitations, and Table 1.

Q1 (and W1, second point). CVT-GS

“Why is there no quantitative comparison with CVT-GS or another closely related learned post-hoc Gaussian simplification method?”

We could not run CVT-GS: it is on arXiv only (v1, 8 September 2026, less than three weeks before the deadline), and we found no public code or weights (checked on 11 November). A controlled comparison is therefore missing, and the revised paper says so.

For context only, on the 21 scenes both papers evaluate, our full method − CVT-GS is −0.41 / −0.18 / +0.20 dB in PSNR and −0.021 / −0.021 / −0.019 in SSIM at 0.1 / 1 / 10%. Three differences limit this comparison: the source assets (ours already 0.08–0.22 dB lower in PSNR per dataset before simplification); the NanoGS runs and evaluators (the two papers' NanoGS differ by +0.17 dB at 10% with the default k = 16, and in LPIPS by 12–15%, so LPIPS is not comparable); and the training data (LLFF, ShapeNet and ScanNet for CVT-GS; 64 DL3DV scenes for us). With these caveats, CVT-GS is ahead at 0.1% and 1% by more than the NanoGS offset, and in SSIM (sources within 0.003) at all three ratios; our +0.20 dB at 10% is about the NanoGS offset, so we do not read it as superiority.

The decisions also differ: CVT-GS groups spatial cells and predicts a representative with a network; we learn which pairs to merge and keep analytic parents.

We do compare with methods that have released code: our full method is +1.1 dB (25%) to +7.3 dB (0.1%) above RAP, a learned post-hoc pruning method (Table 1, 40 primary scenes). Three pruning methods are compared in our reply to Reviewer gnwz, Q1.

AuthorsOpenReview ↗

Response to Reviewer KUhX (3/3): W3 and Q3, W4 and Q4

W3 & Q3. The supervision targets, each with its own refiner

“Does the budget-dependent target schedule remain better than the alternative supervision targets when each variant has its own separately trained refiner?”

Yes against the fidelity-only target, no against the image-change-only target. We trained three predictor seeds per target, each seed with its own refiner (same recipe, architecture and inputs as in the paper), and report the results on the 40 primary scenes:

Variant − submitted target (dB), own refiners0.1%1%5%10%25%
Fidelity only−0.045−0.244−0.260−0.277−0.253
Image change only+0.280+0.041−0.018−0.009+0.005
  • The benefit of the fidelity term at 0.1–1%, shown without refinement in Table 3, does not persist after refinement: at 0.1%, the refiner adds +1.13 dB after the image-change-only target and +0.60 dB after ours. We therefore restrict this claim to selection without refinement; the submitted method is unchanged.

  • Post hoc, the image-change target alone stays ahead of analytic selection after refinement: the image-change-only predictor with its refiner is ahead at all five budgets on the 40 primary scenes (+0.12 to +0.31 dB) and from 1% to 25% on the 40 DL3DV scenes of Table B. The fidelity-only predictor has no clear advantage at any budget on the 40 primary scenes (−0.21 to +0.04 dB). LPIPS favors analytic selection at 1% on the 40 primary scenes, and at 0.1% and 1% on the DL3DV scenes.

Revision: Section 4.3 reports key results on the 40 primary scenes. Appendix D.2 describes the protocol and reports results on the original 46-scene panel. The claim on the fidelity term is restricted to selection without refinement.

W4 & Q4. Budgets not used in training

“How does the method perform at retention ratios not used during training, for example 2%, 7%, or 15%?”

At 2 / 7 / 15%, learned selection is ahead of analytic selection by +0.183 / +0.337 / +0.265 dB, each with its own refiner, in 35, 39 and 38 of the 40 scenes of Table A. We ran the unchanged predictor and refiners at eight budgets that were not training targets (three seeds); the predictor receives the new budget as an input, and the refiner runs at budgets it was not trained on.

Budget not used in training0.05% ‡0.5%2%3%7%15%20%40% ‡
Each with its own refiner (dB)−0.233−0.001+0.183+0.244+0.337+0.265+0.233+0.100
Without refiner (dB)+0.169+0.178+0.211+0.313+0.425+0.504+0.557+0.315
Scenes ahead with refiner (of 40)718353539383632

‡ Outside the 0.1–25% training range.

  • The gain is largest at 7%, a budget that was not a training target. Without refiner, learned selection is ahead at all eight new budgets.

  • The loss at 0.1% extends to 0.05%; at 0.5% there is no clear difference.

Revision: Table A in Appendix D; Table B in D.1.

We hope these results answer the reviewer's concerns: the merge decision is what is new, its rendered supervision is what makes it better than the analytic cost, and this gain survives matched refinement at every budget from 1% to 40%, including six budgets that were not training targets, while the paper now states where it does not help.

If so, we would be grateful if the reviewer took them into account in the final assessment, and we would be glad to answer any further question during the discussion period.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Rebuttal Summary

General response (1/3): strengths and discussion

Dear AC and reviewers,

We sincerely thank all reviewers for their insightful comments and constructive feedback.

Strengths

  • A useful practical problem: compact networks, trained once, simplify a pretrained 3DGS asset from its Gaussians and a target count alone, without images, cameras or per-scene optimization, and output standard Gaussian assets (mentioned by reviewers KUhX, QYJ4).
  • Supervision from rendered interventions at intermediate simplification states, which separates the immediate image change of a merge from its change in error relative to the original rendering (mentioned by all reviewers).
  • Controlled experiments: matched selection and refinement, where separate refiners for analytic and learned selection isolate the contribution of the learned cost; the original panel of 46 scenes from five datasets, three training seeds, per-scene results and paired confidence intervals (mentioned by reviewers QYJ4, gnwz).

Discussion

We have strengthened our work based on the reviews as follows:

DiscussionR-KUhXR-QYJ4R-gnwzOur response
Evaluation scenes; scenes outside them✓✓1. / Table B, App. D.1
Budgets not used in training✓✓2. / Table A
Gain after matched refinement✓✓3. / Table A
What the learned cost adds to NanoGS✓4. / App. D.2, D.3
Camera and occlusion information✓5. / Sec. 3.3, App. D.4
Joint effect of the merges of a pass✓6. / Sec. 3.3, App. D.6
CVT-GS, pruning, cost of the data, parents✓✓7. / App. D.5, Table 10
  1. Evaluation scenes and cross-dataset transfer [R-QYJ4, R-gnwz]. The predictor and refiners were trained on 64 DL3DV scenes disjoint from the original 46-scene panel. After fixing the submitted method, hyperparameters, and checkpoints, we inspected six scenes during development to check cross-dataset transfer: nightsnow and playroom (Deep Blending), chair and drums (NeRF Synthetic), and train and truck (Tanks and Temples). This check did not inform those choices. The submitted paper included the six scenes in its 46-scene panel and disclosed that some evaluation scenes had been inspected during development. Table A reports the 40 primary evaluation scenes of the revised manuscript. Appendix D.7 reports the six development-time transfer scenes separately.

    On 97 scenes from nine datasets outside the 46, learned selection is ahead from 1% to 25% (+0.09 to +0.31 dB, every interval above zero) and behind at 0.1% (−0.08 dB). The full method is +0.8 to +1.5 dB above official NanoGS (Table B).

  2. Budgets not used in training [R-KUhX, R-QYJ4]. On the 40 scenes of Table A, learned selection is ahead at six of the eight budgets that were not training targets: +0.18 / +0.24 / +0.34 / +0.27 / +0.23 / +0.10 dB at 2 / 3 / 7 / 15 / 20 / 40%; there is no clear difference at 0.5%, and it is behind at 0.05% (−0.23 dB).

  3. Gain after matched refinement [R-KUhX, R-QYJ4]. On the same scenes, each with its own refiner, learned selection is ahead of analytic selection at all ten budgets from 1% to 40%: +0.10 to +0.34 dB, every interval above zero; from 1% to 25%, our assets reach the same PSNR with 8–18% fewer Gaussians. It is behind at 0.1% (−0.16 dB).

  4. What the learned cost adds to NanoGS [R-KUhX]. On the 40 primary evaluation scenes, the same network trained to imitate the analytic cost, with its own refiner, is not separated from refined analytic selection in PSNR. Relative to this control, our predictor gains +0.09 / +0.29 / +0.29 / +0.21 dB at 1 / 5 / 10 / 25% and loses 0.19 dB at 0.1%. With refiners, the benefit of the fidelity term at 0.1–1% does not persist.

  5. Camera and occlusion information [R-QYJ4]. On 70 scenes from nine datasets, our cost orders candidate merges closer to their rendered order than the analytic cost (within-state Spearman 0.15–0.36 against 0.05–0.21 by dataset; higher in 69 of 70 scenes). Removing the Gaussians in front of each pair changes 10–39% of the costs by a factor of two or more; our cost stays ahead under both rankings on the three larger scene sets (no direction stated on the two smaller ones). Our camera-free cost covers 17–39% of the distance from the analytic cost to a non-deployable reference with camera-derived inputs.

  6. Joint effect of the merges of a pass [R-gnwz]. In a sensitivity analysis, passes half as large (0.25N instead of 0.5N) raise the gain to +0.25 to +0.76 dB at 1–25% on five scene sets, at 1.37 times the run time.

  7. Comparisons and costs [R-KUhX, R-gnwz]. We add three pruning methods, the cost of the training data and two prototypes of learned parents. A controlled comparison with CVT-GS is missing: we found no public code or weights.

AuthorsOpenReview ↗

Rebuttal Summary

General response (2/3): Tables A and B

How to read the numbers. “Learned − analytic” is the PSNR of learned selection minus that of analytic selection, each followed by its own refiner, in dB. Three predictor seeds, each with its own refiner; 95% intervals over scenes, without multiplicity correction. The submitted networks are unchanged.

Table A. Thirteen budgets on the 40 evaluation scenes. † Budget not used as a training target. ‡ Outside the 0.1–25% training range. Last column: number of Gaussians that analytic selection with its refiner needs to reach our PSNR, relative to ours.

BudgetLearned − analytic (95% interval)Scenes ahead (of 40)Without refinerGaussians at equal PSNR
0.05% †‡−0.233 (−0.333, −0.136)7+0.169n/a
0.1%−0.164 (−0.259, −0.063)8+0.1400.87×
0.5% †−0.001 (−0.074, +0.080)18+0.1781.00×
1%+0.117 (+0.059, +0.182)33+0.1751.09×
2% †+0.183 (+0.129, +0.237)35+0.2111.14×
3% †+0.244 (+0.181, +0.306)35+0.3131.18×
5%+0.299 (+0.225, +0.380)36+0.4531.21×
7% †+0.337 (+0.264, +0.430)39+0.4251.21×
10%+0.320 (+0.228, +0.432)37+0.5361.20×
15% †+0.265 (+0.190, +0.352)38+0.5041.13×
20% †+0.233 (+0.173, +0.297)36+0.5571.13×
25%+0.191 (+0.144, +0.245)38+0.4711.11×
40% †‡+0.100 (+0.055, +0.145)32+0.315n/a
  • From 1% to 40%, learned selection is ahead at all ten budgets (every interval above zero); from 2% to 25%, no scene loses more than 0.2 dB. It is behind at 0.05% and 0.1%, with no clear difference at 0.5%. Without refiner, it is ahead at all 13 budgets.
  • At 2 / 7 / 15%, the refiner adds +0.96 / +0.95 / +0.78 dB on top of learned selection, and our full method − official NanoGS (k = 4) is +1.18 / +1.38 / +1.28 dB.

Table B. 97 scenes outside the 46 (three seeds). Six of the nine datasets appear neither in training nor among the 46. Bold: interval above zero. Italic: below zero. Plain: the interval contains zero. Last column: our full method − official NanoGS (k = 4), mean of the five paper budgets.

Group (scenes)0.1%1%2%5%7%10%15%25%Full − NanoGS
DL3DV (40)−0.19+0.14+0.27+0.37+0.33+0.36+0.18+0.20+1.52
Tanks and Temples (5)−0.01+0.01+0.04+0.28+0.22+0.23+0.05+0.10+1.42
Deep Blending (4)+0.13+0.13+0.20+0.42+0.38+0.36+0.35+0.11+1.26
LLFF (8)−0.13−0.09+0.13+0.22+0.23+0.19+0.09−0.03+0.81
Nerfstudio (10)+0.06+0.03+0.14+0.27+0.25+0.25+0.21+0.16+1.09
Ref-NeRF real (3)+0.05+0.27+0.28+0.32+0.33+0.30+0.26+0.22+1.10
ETH3D (14)−0.03+0.06+0.19+0.24+0.32+0.17+0.10+0.06+1.40
BungeeNeRF (8)+0.10+0.16+0.20+0.32+0.22+0.37+0.13+0.12+0.82
Free (5)−0.22+0.06+0.16+0.24+0.26+0.26+0.17+0.19+1.03
All 97−0.08+0.09+0.21+0.31+0.29+0.30+0.16+0.14+1.29
  • Over all 97 scenes, learned selection is ahead from 1% to 25% (every interval above zero) and behind at 0.1%.
  • At 5%, 7%, 10% and 15%, the interval is above zero in 7 of the 8 groups of four or more scenes (+0.17 to +0.42 dB at 5–10%); at 25% in four, at 2% in five, at 1% in two (DL3DV and BungeeNeRF).
  • LLFF (8): the interval contains zero at every budget; one scene (room) loses 0.69 to 1.18 dB from 1% to 25%.
AuthorsOpenReview ↗

Rebuttal Summary

General response (3/3): revisions

The revised PDF shows added and rewritten text in blue:

  • Abstract and limitations: the loss at the lowest budgets; Table 1: “analytic selection + refiner” [R-KUhX, R-QYJ4].
  • Section 3.3: the limit of the inputs [R-QYJ4] and the single-merge approximation [R-gnwz]; a corrected sentence on compositing in the introduction.
  • Fidelity term: claim restricted to selection without refinement; CVT-GS: comparison stated as missing [R-KUhX].
  • Table 10: corrected offline costs [R-gnwz]. New Appendix D: Tables A and B and all new experiments.

Best regards, The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.