DeltaMerger: Learning Visual Merge Costs for Zero-Shot Gaussian Splat Simplification

We introduce DeltaMerger, which simplifies pretrained 3D Gaussian Splatting representations by recursive pairwise merging with a learned merge cost, followed by a shared terminal refiner. Analytic merge costs measure the attribute discrepancy between two Gaussians and their proposed parent, but the visual effect of a merge also depends on visibility, compositing, and the distortions left by earlier merges. We supervise a compact shared predictor with rendered interventions at intermediate simplification states, measuring the immediate image change caused by replacing a candidate pair with its analytic parent and the signed change in error relative to the original rendering. The requested budget selects between two supervision regimes: the immediate change at 5–25% retention, and the change in fidelity to the original rendering, which can favor merges that partially repair earlier distortion, at 0.1–1%. Trained once on 64 DL3DV scenes, the frozen predictor reads only descriptors of the pair, its neighborhood, and its proposed parent, together with simplification progress and the requested budget, and transfers zero-shot to held-out DL3DV scenes and to four datasets never used for training, without images, cameras, rendering, or per-scene optimization. On 46 held-out scenes, DeltaMerger without refinement improves mean PSNR over NanoGS with the same candidate neighborhood at every retention budget, by 0.35 dB on average. Its shared refiner, trained once through differentiable rendering, corrects appearance and scale at fixed primitive count; the complete DeltaMerger improves on unrefined analytic selection by 1.12 dB, and learned selection keeps a 0.14 dB advantage when analytic selection receives its own refiner.