MarginalGS: Representation-Conditioned Approximation in Native High-Resolution 3D Gaussian Reconstruction

Native high-resolution multi-view imagery makes fine text, boundaries, and surface appearance observable, but this evidence creates spatially non-uniform approximation demand. After establishing a global Gaussian representation, equal capacity increments may reduce error in some regions but contribute little in others. We define this representation-conditioned mismatch between approximation demand and allocated capacity as region-wise representation imbalance. Full-resolution optimization with unrestricted refinement can allocate capacity indiscriminately, causing excessive primitive growth and substantial representation cost. MarginalGS resolves this imbalance through representation-conditioned capacity allocation and hybrid support design. It estimates where added capacity can reduce error and concentrates native-resolution supervision there rather than expanding the representation uniformly. For refined primitives, the Eckart–Young–Mirsky (EYM) error quantifies covariance distortion under rank-2 restriction, while resolution and reliability select support. The hybrid representation controls refinement-induced growth while preserving reconstruction fidelity. Across four high-resolution real-world datasets, MarginalGS achieves the highest SSIM and ROI PSNR on every dataset, competitive full-image PSNR, and the lowest total memory.