acceptodds
Withdrawn from ICLR 2027

Sobol Indices for Parameter-Conditioned Neural Networks.

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Networks conditioned on a continuous parameter, such as surrogates for parametric partial differential equations, replace many trained models with one. Two questions follow: what has such a network learned about its parameter, and which simpler architectures could match it? We address both with the Sobol (functional ANOVA) decomposition of the trained network over inputs and parameter, estimated by standard Monte-Carlo sampling. On a loss-conditional physics-informed network for parametric 2D Helmholtz, the indices match those of the exact solution to 0.002 (0.005 across three training seeds) and show that 43% of the output depends jointly on space and the parameter, which first-order attribution methods cannot express. The indices of one pilot network also predict the lowest error that any restricted architecture can reach. Networks trained by regression reach the bound read from the pilot to within 0.003 on the headline target, and the exact bound to within 0.001 on two further targets. Networks trained with the physics-informed loss stay above it. Finally, a neural HDMR estimator that fits the decomposition directly learns a nearly correct function (three-way index within 0.02), but its standard readout at the default resolution overstates that index by 0.14, six times the seed spread. The reported indices still sum to one, so this error does not appear in them. Computing the indices from the fitted function removes the error; at this problem size Monte-Carlo estimation is also about 500 times cheaper.

Primary area:
interpretability and explainable AI
Keywords:
Sobol indices, functional ANOVA, global sensitivity analysis, PINNs, physics-informed neural networks, parametric surrogates, interpretability, neural architecture design, HDMR

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_m8yCOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper shows that Sobol indices computed from a single trained parametric surrogate (e.g., an LC-PINN) provide a tight, achievable lower bound—the "architectural floor"—on the relative L2L^2 error of any architecturally restricted neural function class, and uses this to select architectures, verify that trained restricted networks reach their predicted floors, and diagnose a biased readout in a Fourier-HDMR estimator.

Critical strengths

The empirical verification is honest.

Critical weaknesses

The verification (Table 2) uses the L2L^2 projection computed from the same decomposition, so it validates the estimation pipeline rather than the operational claim.

Table 7 shows regression reaches the floor on three targets, but the physics-informed case leaves floors loose.

The estimator section (Section 7) diagnoses a bias but offers little guidance on when the Fourier-HDMR estimator is worth using.

Questions for authors

Could the authors show a concrete case where the floor rules out an architecture that a reasonable practitioner would otherwise have tried, and where training that architecture confirms the floor was informative?

I found this paper extremely difficult to follow. Can you add more background/related work sections, e.g., in the appendix, to ease understanding? It would be better to have more illustrations as well.

Reviewer_eKMpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper uses Sobol indices to analyze input variable interactions in parameter-conditioned neural networks and uses them to predict theoretical lower bounds on the L2L^2 error for various restricted architectures. First, drawing on the orthogonal decomposition of classical functional ANOVA, the authors establish a relationship between the variance of excluded interaction components and the best approximation error (Theorem 4). Subsequently, using a parameterized 2D Helmholtz equation as a case study, the authors observe that trained PINNs exhibit Sobol indices closely matching those of the analytical solution and utilize these indices to predict error lower bounds for different restricted networks. They further find that networks trained via supervised regression and those obtained through distillation from a pilot network can approach these predicted lower bounds, whereas physics-informed training typically results in larger errors. Finally, the authors employ Fourier-feature HDMR to fit function terms organized by variable subsets and analyze the estimation bias inherent in the purification and readout processes.

Critical strengths
  1. Sobol-based error floors provide a straightforward way to screen restricted architectures before training them. When the target indices are sufficiently accurate, these bounds can rule out architectures whose approximation errors necessarily exceed a given tolerance.

  2. The experiments on the considered manufactured benchmarks are carefully controlled. The authors compare supervised regression, distillation, and physics-informed training, and investigate the sources of estimation bias in Fourier-HDMR.

Critical weaknesses
  1. The central theoretical result is essentially a direct application of classical functional ANOVA orthogonal projection theory, with limited theoretical novelty.

For independent input variables, classical functional ANOVA decomposes a function into mutually orthogonal components, u=∑SuSu=\sum_S u_S. If a function class admits only the interactions indexed by a family F\mathcal F, its best L2L^2 approximation is the corresponding orthogonal projection. When F\mathcal F is nonempty and downward closed, we have

\|u-v\|_{L^2(p)}^2 \= \sum_{S\notin\mathcal F} \|u_S\|_{L^2(p)}^2.$$ For $\operatorname{Var}_p(u)>0$, normalization gives $$\inf_{v\in\mathcal H_{\mathcal F}} \frac{\|u-v\|_{L^2(p)}^2}{\operatorname{Var}_p(u)} \= \sum_{S\notin\mathcal F}S_S(u).$$ For a finite-width restricted neural architecture, the representable function class is generally only a subset of the ideal space. Therefore, the result provides a lower bound rather than guaranteeing that the network can attain the corresponding projection error. These conclusions follow directly from ANOVA orthogonality and the Pythagorean identity, without exploiting any specific mathematical properties of neural networks or parameterized PDEs. Closely related results have long been established. Wang and Fang (2003, *The Effective Dimension and Quasi-Monte Carlo Integration*) studied approximation errors arising from truncating higher-order ANOVA components. Hooker (2004, *Discovering Additive Structure in Black Box Functions*) quantified interactions through the approximation loss incurred by projection onto additive function spaces. Hart and Gremaud (2018, *An Approximation Theoretic Perspective of Sobol' Indices with Dependent Variables*) explicitly connected Sobol indices to optimal $L^2$ approximation errors and discussed their use in surrogate model construction. The authors already acknowledge and cite the relevant classical theory. The remaining contribution is to apply this idea to parameter-conditioned neural networks and consider different families of allowed interactions. However, this extension follows directly from the same orthogonal projection principle and does not establish a new approximation-theoretic result. In particular, the agreement between predicted floors and orthogonal projection errors in Table 2 is a numerical verification of this identity, as the authors themselves acknowledge. Therefore, while Theorem 4 provides a valid mathematical foundation, it is not a substantial theoretical contribution. The novelty of the paper must instead rest on whether this application provides new insights or practical advantages beyond existing Sobol-based approximation and surrogate construction methods. 2. The experiments do not yet establish a clear advantage for practical PINN architecture selection or ANOVA estimation. First, the authors use manufactured Helmholtz problems with known analytic solutions and train restricted networks by supervised regression. They find that these networks approach the optimal $L^2$ projection floors predicted by Sobol indices. Distillation experiments further show that an accurate pilot network can serve as a substitute for the analytic target. These are valid empirical results, but they mainly demonstrate that the particular network parameterizations and supervised training procedures can approximate the ideal projections. It remains unclear whether this tightness extends to actual physics-informed optimization. In the physics-informed setting considered here, the training objective is based on the PDE residual rather than the $L^2$ error relative to the exact solution. These objectives generally have different minimizers within a restricted function class. Even if the optimizer finds a global optimum, there is no reason to expect the resulting network to attain the Sobol $L^2$ projection floor. Thus, a gap between physics-informed training errors and the theoretical floors is not surprising in itself. The authors also report architecture-ranking results, with Kendall $\tau$ values of 0.83, 0.87, and 0.41 across three targets. However, among the six nonconstant architectures in Table 7, 12 of the 15 pairwise comparisons involve nested function classes, for which the ordering of the optimal $L^2$ approximation errors already follows from set inclusion. Although this does not guarantee the ordering of actual physics-informed training errors, the overall Kendall $\tau$ does not fully establish the predictive value of Sobol-based screening for non-nested architectures. A more direct example is the near-additive target. The $f_{xy}+f_k$ architecture has a predicted floor of 0.085 and a physics-informed error of 0.104. The more expressive all-pairs architecture has a lower floor of 0.073 but a higher actual error of 0.142. The full-order architecture has a theoretical floor of zero but still produces an error of 0.113. Thus, a lower Sobol floor does not necessarily lead to better performance under actual physics-informed training. Another important issue concerns the pilot network. Theorem 4 characterizes the best approximation error of the function being decomposed, whereas practical screening uses Sobol indices computed from a trained pilot. Although Section 3.3 discusses how pilot approximation error affects the predicted floor, it remains unclear how to obtain a reliable and computable error bound when the true PDE solution is unavailable. Without controlling the discrepancy between the pilot and the true solution, the pilot-based floor cannot automatically serve as a rigorous lower bound for the true solution. The paper also does not demonstrate a clear computational advantage over existing architecture selection methods after accounting for the cost of training the full pilot network. Finally, the Fourier-HDMR component has limited independent methodological novelty. The identifiability of interaction components and the need for purification have already been studied, for example by Lengerich et al. (2020, *Purifying Interaction Effects with the Functional ANOVA*). The reported readout bias primarily arises because the fitted components are not orthogonal, while their variances are nevertheless normalized as if they formed an orthogonal ANOVA decomposition. The authors provide a useful numerical characterization of this bias and show that recomputing Monte Carlo Sobol indices on the fitted global function substantially improves the estimates. However, this correction relies on an existing standard estimator. The paper also includes comparisons with Monte Carlo, a tanh estimator, and polynomial chaos. Nevertheless, Fourier-HDMR does not demonstrate a clear advantage in accuracy or computational efficiency in the main experiments. For the expensive-query or sample-only settings proposed by the authors, the existing comparisons do not yet establish an advantage over applicable sample-based ANOVA estimation methods under matched budgets. Furthermore, since the estimator also aims to recover individual component functions, comparisons based primarily on Sobol index errors do not fully establish its value. Overall, the theoretical foundation of this work largely comes from established functional ANOVA approximation theory. While the experiments are detailed and provide some useful empirical observations, they do not convincingly demonstrate that applying these existing tools to parameter-conditioned neural networks yields sufficiently new theoretical insights, algorithmic improvements, or practical advantages.
Questions for authors
  1. What are the substantive differences between Theorem 4 and the results of Wang and Fang (2003), Hooker (2004), and Hart and Gremaud (2018)? If the primary contribution is an application rather than new approximation theory, what capabilities does the proposed Sobol-based architecture screening provide beyond existing surrogate construction methods?

  2. When the true PDE solution is unavailable, how can a reliable error bound between the pilot network and the true solution be obtained? Can the error propagation analysis in Section 3.3 be converted into a computable screening criterion that also accounts for uncertainty in Sobol index estimation?

  3. How effective is Sobol-based screening for selecting non-nested architectures? Can the authors report the corresponding ranking results separately and compare against direct architecture search under the same total computational budget, including pilot training, Sobol estimation, and candidate-network training?

  4. What independent advantages does the Fourier-HDMR estimator offer over existing ANOVA purification methods, polynomial chaos, and sample-based sensitivity estimators? Can the authors compare both Sobol index accuracy and component-function recovery under matched sample and computational budgets?

Reviewer_qmCVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies networks that take a physical parameter as an extra input. It applies the Sobol (functional ANOVA) decomposition to the trained network over space and the parameter. The indices show which inputs act alone and which act together. The main result is an error floor. If an architecture cannot use some input interactions then its error cannot go below a bound set by the Sobol indices of those interactions. One trained pilot network gives this floor for many simpler architectures. Restricted networks trained by regression reach the predicted floors. Networks trained with the physics-informed loss stay above them. The paper also studies a neural HDMR estimator. Its standard readout gives biased indices because the fitted terms are not orthogonal. The bias is hidden because the indices still sum to one. Computing the indices directly from the fitted function removes it.

Critical strengths

The experiments are careful. The authors use fixed seeds and paired comparisons. They check the floors against analytic references. They also train restricted networks from scratch to test them. This makes the main claims believable. The paper is open about its limits.

Critical weaknesses

Limited novelty. The main theorem is not new math. It follows from the standard ANOVA orthogonality and the Pythagorean identity. The same link between Sobol indices and best approximation error is already known in sensitivity analysis. The paper says this itself. The new part is applying it to parametric networks and using it to pick architectures. On its own this is a small contribution for ICLR.

Practical relevance. The practical value is not yet clear. The floor is tight only when restricted networks are trained by regression on the true target. In the physics-informed setting that motivates the paper the trained networks stay well above the floor. The main example also has only three inputs. In that case it is cheap to just train every restricted architecture. The paper does not show a case where the floor saves real training cost. It also does not show that the chosen architecture is faster at inference. This limits the impact for practitioners.

Questions for authors

Is there a higher-dimensional case (six or more inputs) where the floor rules out architectures that would be costly to train? Is the selected restricted architecture actually cheaper at inference?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.