acceptodds
Under review as a conference paper at ICLR 2027

Optimizing Sampling Pattern Correlations for Neural Field Training

Abstract

Neural fields are optimized from samples of a source signal whose locations can often be chosen freely. While existing approaches exploit this freedom by adapting individual sample locations to the signal, we investigate a complementary question: how should the samples be spatially correlated? We characterize these correlations through sampling-pattern power spectra and explore this space to identify patterns that optimize the outcome of neural field fitting. By evaluating sampling patterns through the resulting neural field, our approach accounts for the complete fitting process rather than integration accuracy alone. The optimized patterns outperform IID uniform sampling as well as correlated patterns designed for numerical integration, with particularly pronounced benefits under limited sampling budgets. Moreover, the patterns transfer across source signals, enabling inexpensive proxy signals to drive their optimization for more costly settings. We analyze the resulting sampling structures and characterize how they depend on the neural field fitting problem. Together, our results establish optimized sampling correlations as an effective tool for improving neural field fitting.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
neural fields

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_cTu1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a framework for optimizing sampling correlations for neural field fitting. Unlike prior sampling approaches designed for numerical integration, the proposed method optimizes sampling patterns based on the reconstruction quality of the resulting neural field. To do so, sampling correlations are compactly parameterized via radial power spectra and explored using Bayesian optimization. The results show improvements over several sampling baselines, particularly under limited sampling budgets. The authors further demonstrate that patterns optimized on inexpensive proxy signals can transfer to more computationally expensive tasks.

Critical strengths
  1. The paper investigates a relatively underexplored aspect of neural field training, establishing a meaningful connection between sampling theory and neural field optimization. The formulation is well-motivated and the proposed spectral parameterization provides a compact search space for optimizing sampling correlations.

  2. The method improves neural field reconstruction performance over existing sampling strategies across 2D and 3D tasks. The authors also demonstrate that optimized sampling patterns transfer across signals, including to computationally expensive rendering and geometry processing tasks.

Critical weaknesses
  1. Although the proposed method generally improves performance, its gains over strong baselines such as GBN are often relatively modest. Given the additional cost of Bayesian optimization and pattern realization, it remains unclear whether these improvements justify the computational overhead compared to existing sampling strategies that require no task-specific optimization.
Questions for authors
  1. Could the authors provide a breakdown of the computational cost of each stage (e.g. BO, realization, and neural field fitting) and quantify the additional overhead relative to standard neural fitting?

  2. Appendix G shows that different realization losses can produce different anisotropic patterns despite matching similar radial spectra. How sensitive is downstream reconstruction performance to these choices, and does the realization optimizer also affect the resulting patterns?

  3. How would the comparison with GBN change if the neural field training hyperparameters were tuned separately for each sampling strategy, rather than using a shared configuration?

3 comments

Weakness/ Question 1

  • Weakness: Modest improvement over GBN

    • Please refer to the 1 - Why the improvements are perceived as modest section in the common concerns/ questions.
  • Question: Cost breakdown

    • Please refer to the 2 - Computational cost section in the common concerns/ questions.

Question 2

We evaluate how patterns realized from different losses affect the reconstruction quality (LDR, AAPE).

Loss | Samples1K4K16K65K262K1M
L216.6619.1522.8228.1934.5235.67
L116.7119.2422.9428.4134.7035.73
Weighted L116.7519.2822.9728.4934.7635.72

L1 outperforms L2 consistently, and weighted L1 achieves the best PSNR in 5 out of 6 budgets. The results indicate that different realization losses lead to different sampling patterns (as reflected in their full-dimensional spectra), hence different reconstruction performance.

We evaluate how patterns realized from different optimizers affect the reconstruction quality (LDR, AAPE, weighted L1 loss).

Optimizer | Samples1K4K16K65K262K1M
SGD - momentum: 016.7419.2722.9628.4834.7435.71
SGD - momentum: 0.516.7519.2722.9628.4834.7635.73
SGD - momentum: 0.916.7419.2522.9528.4734.7335.72
Adam16.7519.2822.9728.4934.7635.72

We notice that different optimization algorithms produce nearly identical reconstruction performance, indicating robustness with respect to the choice of the optimizer.

Question 3

We swept the learning rate, the most sensitive training hyperparameter, independently for both methods (LDR, AAPE, 65K samples).

Lr1e-23e-32e-31.5e-31e-37e-45e-41e-4
Optimized Pattern12.6726.2026.5526.7326.8526.8226.6326.11
GBN12.6726.0126.3626.5126.6226.6026.4325.91

Both methods peak at the same learning rate, the one used in the paper, so tuning each method separately leaves our comparison unchanged. More importantly, our advantage is nearly constant (0.19–0.23 dB) at every learning rate where training converges (at 1e-2, both diverge). The gain comes from the sampling pattern itself, not from a particular training configuration. This holds even though the pattern was optimized at a single learning rate.

I thank the authors for their detailed rebuttal and additional experiments.

However, my main concern remains regarding the overall significance of the contribution. While the proposed framework is interesting and the improvements over GBN appear statistically consistent, their magnitude is relatively modest. Given that existing sampling strategies already achieve competitive performance without neural field fitting-aware optimization, I remain unconvinced whether the proposed approach represents a sufficiently substantial advance over these methods.

While I find the work technically sound, I am not fully convinced that the demonstrated benefits justify the additional optimization framework. Therefore, I will maintain my original rating.

Thank you for engaging with our rebuttal, and for confirming that the work is technically sound and that the improvements are statistically consistent. We would add that these gains come at no cost to the user. The optimized patterns are fixed point sets, used exactly like GBN, and in 10 of the 11 tested settings they improve on GBN for 13 or 14 of the 14 test signals. Together with the finding that integration-optimal patterns are not fitting-optimal, we believe this makes optimized sampling correlations a useful tool for neural field fitting.

Reviewer_dFvDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors propose a method to optimize the spatial correlation of sampling points for neural field training. They represent the sampling pattern through its power spectrum and using a small number of B-spline control points. They use Bayesian optimization to find the best spectrum based on neural field reconstruction quality, and show better results than random and other sampling methods, especially with limited samples.

Critical strengths

1- The idea of optimizing spatial correlations of sampling points rather than individual sample locations is interesting and novel.

2- The method shows improvements over random sampling and other common sampling strategies, specially with limited sampling budgets.

3- The optimized sampling patterns show good transferability across different signals and neural field architectures.

Critical weaknesses

1- My main concern is that the method only optimizes the radial power spectrum, which removes directional information. The experiments in Appendix B.2 show that anisotropy and the orientation of the sampling pattern have a significant effect on reconstruction quality. This makes me wonder why the authors choose to optimize only radial correlations rather than also considering directional information, specially since the resulting anisotropy seems to be an important part of the performance improvement.

2- Most of the experimental results show relatively small imporvement over GBN, why?

3- The optimization seems expensive, and I would like to see the total computational cost compared to the improvements it provides.

Questions for authors

N/A, already posed above.

3 comments

Weakness/ Question 1

We agree that anisotropy matters. In fact, our framework already exploits it. Realization maps each radial spectrum to a consistent anisotropic pattern (App. G), so BO effectively searches over anisotropic structures and selects the ones that benefit fitting. The anisotropy in B.2 was discovered by our pipeline. We parameterize only the radial profile for two reasons. First, compactness: our radial spline has 7 free parameters, while a full 2D spectrum has thousands of degrees of freedom (far more in 3D), which is beyond the range where BO is sample-efficient. Second, realizability: arbitrary anisotropic spectra are often not realizable as point patterns, and parameterizing realizable ones is an open problem.

Weakness/ Question 2

Please refer to the 1 - Why the improvements are perceived as modest section in the common concerns/ questions.

Weakness/ Question 3

Please refer to the 2 - Computational Cost section in the common concerns/ questions.

thanks for the clarification on anisotropy and the arguments provided. After reading the other reviews and the discussion, I agree that I still don't see the need for (expensive) optimization if the gains are not significant compared to hand-designed heuristic methods that seems to work almost as well. Therefore I retain my initial score.

We thank the reviewer for engaging with our rebuttal. We would like to clarify three points.

Significance The gains over GBN are statistically significant: 10 of 11 settings at p < 0.01, with improvements on nearly every individual test signal and no setting in which GBN is significantly better. These are consistent improvements over the strongest baseline in the field.

Cost The search is a one-time cost of about 70 GPU-hours per task family, run on a cheap proxy task. That is comparable to a standard hyperparameter search, and the result transfers across signals and even tasks (Figs. 6b, d), adding no overhead to any later fit.

Why optimize? Beyond the performance gains, our framework shows something a hand-designed heuristic cannot: integration-optimal sampling is not fitting-optimal. GBN happens to be the strongest baseline here, but nothing tells a practitioner this in advance. Other designs fail badly in some settings, as Regular does in Figs. 4 and 6b. Our method finds a pattern that matches or beats the best heuristic automatically, and for new tasks without an established heuristic, it provides one.

Reviewer_w4niOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed a novel method for sampling in neural field training. Specifically, the author uses power spectrum to define a sampling strategy which models the sampling pattern correlations and minimizes the average loss over the whole field, and then align target sample points to this power spectrum. To reduce the parameters of frequencies of $ N \times d $, an alternative parameterized radial power spectrum with MM B-spline functions is adopted. The training process optimizes the network and power spectrum through two alternating inner and outer loops. The inner loop uses the power spectrum to generate sample points and train the networks over them. While the outer loop adopts the Bayesian Optimization to search for a better spectrum power parameters setting which minimizes the integral of reconstruction loss. The authors conducted comprehensive experiments on six tasks, including 2D image fitting, 3D geometric reconstruction, and expensive Monte Carlo queries. They demonstrated the effectiveness of the proposed sampling framework through cross-signal and cross-architecture transfer, anisotropic ablation, and repeated searches.

Critical strengths
  1. The author uses the correlation between sampling points as a design variable for neural field training. This is a valuable complement to existing adaptive signal sampling methods.
  2. The experiments are comprehensive, and the transfer results are valuable. The paper evaluates six tasks, including 2D image fitting, 3D geometric reconstruction, and expensive Monte Carlo queries. In particular, the transfer of patterns obtained from inexpensive proxy tasks to expensive query tasks directly supports the paper's main application motivation. The appendix also includes cross-architecture cross-evaluation, anisotropic ablation, and repeated Bayesian optimization experiments.
Critical weaknesses
  1. The experimental results are not significant, and according to the experiment part, in some cases GBN achieves comparable results as the proposed method (figure 4, figure 6(a), figure 6(d), figure 8 and table 1). The author does not provide a complexity analysis against DBN, so I cannot determine the practical value of the method.
Questions for authors
  1. Could the author provide a complexity analysis against comparison methods?
  2. According to the definition of power spectrum (equation 3), PX(0)=NP_{\mathcal{X}}(0)=N. However the spline parameters c0,c1,...,csc_0,c_1,...,c_s, which determine the radial average power spetrum as radial close to 00, are set to 00. Could the author explain this setting?
  3. The radial average power spectrum used in this paper is heavily compressed. In my opinion, the sampling points generated using this method are still insufficient to capture the strongly asymmetric (or randomly distributed) high-frequency information in the signal field. Could the authors conduct module ablation experiments to verify whether the effectiveness of the method mainly depends on the knowledge gained from continuous-domain loss optimization or the power spectrum modeling capability?
3 comments

Weakness 1

Please refer to the 1 - Why the improvements are perceived as modest section in the common concerns/ questions.

Question 1

Please refer to the 2 - Computational Cost section in the common concerns/ questions.

Question 2

The reviewer is correct that PX(0)=NP_\mathcal{X}(0) = N for any pattern. This DC term is fixed by the number of points and carries no information about correlations, so we exclude it from the realization loss (Eq. 5) – we made this exclusion explicit in the revision. The first control point serves only to shape the spectrum at low nonzero frequencies. Setting c1,⋯ ,c4=0c_1, \cdots, c_4 = 0 is an empirical choice for search efficiency: in preliminary runs, these coefficients consistently converged to zero, i.e., high-performing patterns strongly suppress low frequencies, much like blue noise. Fixing them lets BO spend its budget where the spectra actually differ.

Question 3

Note that our patterns are deliberately signal-agnostic: they are not meant to capture the high-frequency content of a specific signal, which is the domain of adaptive methods and complementary to our approach. It is exactly this signal independence that lets patterns optimized on cheap proxies transfer to expensive tasks (Figs. 6b, d). Further, the compact radial parameterization does not restrict us to isotropic patterns. Our realization consistently produces anisotropic structure (App. G), and this structure matters (App. B.2).

The requested ablation is largely already in the paper, and it shows that both components matter. Spectral modeling: restricting the realizable spectra to isotropic ones degrades performance (App. B.2). Fitting-in-the-loop objective: GBN is a spectrally optimized pattern built from an integration-oriented objective without fitting in the loop, and our patterns significantly outperform it (Fig. 4). We will make this ablation explicit in the revision.

Thank the authors for detailed rebuttal. Most of my concerns have been addressed. However, regarding the transferability of the sampling patterns, I remain uncertain as to why patterns optimized on proxy tasks can be applied to complex tasks, and how the authors can demonstrate that the learned sampling patterns generalize to other tasks, given that the pattern search process involves task-dependent components. Therefore, I will maintain my rating.

We thank the reviewer for engaging with our rebuttal and raising two new questions. We would like to answer as follows:

  • Why transfer is possible: We hypothesize that the proxy and expensive tasks both share similar signal characteristics; for example, LDR/HDR image fitting and MCR involve 2D image-like signals.

  • How generalization can be demonstrated: The protocol described in Sec. 4.1 already explained how we evaluate the transferability. For tasks beyond those evaluated in our paper, the same protocol can be reused to test the transfer. We emphasize that we do not claim universal transfer to arbitrary tasks; rather, our experiments demonstrate transfer across the related signals and related tasks. Answering whether this holds true for every task is an interesting future direction.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Common concerns/ questions

We thank the reviewers for insightful feedback. In the following, we provide responses to raised concerns, which all will be thoroughly integrated into the revision.

1- Why the improvements are perceived as modest

[Concern # 1] - The experimental results are not significant

We agree that mean curves alone do not establish significance, and have added paired statistical tests against the strongest baseline, GBN, on one 2D task (LDR images) and one 3D task (occupancy fields), with correction for testing multiple budgets. Our patterns are significantly better in 10 of 11 settings, all with p < 0.01, and they improve results on 13 or 14 of the 14 test signals in each of these settings. GBN is not significantly better in any setting. The one tie is LDR at the largest budget (1M samples), where dense sampling leaves little room for correlation to help. The revised paper describes the full protocol and results.

[Concern # 2] - Improvements over GBN are often relatively modest, and GBN achieves comparable results in some cases

GBN is indeed on par in some settings, and this is expected. It is a strong hand-designed pattern whose spectrum lies within our search space, so matching it is our method’s floor. The reviewer’s observation leads to an important practical question: which pattern should one use? Here the results are unambiguous. No baseline is reliably best: Regular, for example, ties for best in Fig. 8 but is among the worst in Figs. 4 and 6b. Ours is the only pattern that is consistently at or near the top, it is never significantly worse than the strongest alternative, and it is found automatically.

[Question # 1] - Why most of the results show relatively small improvement over GBN

GBN is the state of the art in spectrally optimized sampling. Fitting a neural field minimizes a loss integral, so an integration-optimized pattern already captures much of the available gain. Our method targets what remains: how sample correlations shape the fitting dynamics. Measured against that headroom, the gains are substantial (e.g., over 45% more than GBN’s improvement over random sampling for LDR at 4K). They also cost nothing to use: no extra samples, no extra training, no model changes. Starting without any knowledge of blue noise, our method discovers patterns that outperform the strongest integration-optimized design. This shows, for the first time, that integration-optimal sampling is not fitting-optimal.

2 - Computational Cost

The pattern search is a one-time cost of roughly 70 GPU-hours (9 h on 8 GPUs for LDR). Each candidate evaluation is dominated by neural field fitting (~8 min) rather than realization (~2 min). This cost is paid once, on a cheap proxy task, and then amortized: the resulting patterns are fixed, reusable, and transfer to new signals and even to new tasks (Figs. 6b, d). Downstream, they are used exactly like GBN, as a precomputed sample set, so they add zero overhead to neural field fitting. In contrast, signal-adaptive methods pay their cost again for every new signal. The savings are largest exactly where sampling is expensive: in Monte Carlo rendering, a single 1M-sample acquisition takes ~20 min, so even a small quality gain at a fixed budget outweighs a one-time search. The revised paper includes a full per-stage cost breakdown in Appendix A.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.