acceptodds
Under review as a conference paper at ICLR 2027

Generalized Functional ANOVA: A Complete Theoretical Framework

Abstract

The functional ANOVA provides a fundamental representation of square-integrable multivariate functions into main effects and higher-order interactions. For independent inputs, the components belong to mutually orthogonal Hilbert subspaces and admit an explicit representation. For dependent inputs, however, the components are only hierarchically orthogonal: although existence and uniqueness results are available, the Hilbert subspaces underlying the generalized decomposition have remained implicit. We resolve this representation problem for continuous inputs supported on a bounded hyperrectangle whose joint density is bounded above and away from zero. We introduce a distribution-adapted family of functions and prove that it forms a Riesz basis of the space, thereby guaranteeing a unique, stable, and unconditionally convergent representation. We then show that, for every coalition of variables, the corresponding block of this basis exactly characterizes the Hilbert subspace containing the functional ANOVA component. Our construction recovers the classical orthogonal decomposition under input independence. As a direct consequence, computing the generalized functional ANOVA reduces to estimating coefficients in an explicit, distribution-adapted basis. Finally, as a proof of concept, we introduce an elementary, fast and model-agnostic estimator based on our theoretical results. Experiments on synthetic and real-world datasets illustrate its connections with established tabular explanation methods and show that low order components often capture most of the signal in the model output.

Primary area:
interpretability and explainable AI
Keywords:
Generalized Functional ANOVA, Nonparametric Interaction Models, Feature Dependence, Hilbert Spaces

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YKq9OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies generalized functional ANOVA for dependent inputs. It proposes a density-weighted Legendre-polynomial basis and proves that it forms a Riesz basis, allowing a unique representation of the hierarchically orthogonal components. Based on the proposed approach, the authors introduce an estimator using truncated polynomial expansions and estimated marginal densities. Experiments on synthetic and real-world datasets study reconstruction accuracy and compare the estimated effects with existing explanation methods.

Critical strengths

Addressing the problem of efficiently estimating functional ANOVA decompositions in settings with dependent features is an interesting and relevant research direction.

Critical weaknesses

1. Limited novelty and insufficient discussion of related work. The main contribution in Section 3 is a density-weighted Legendre-polynomial basis for generalized functional ANOVA. However, polynomial-based decompositions for dependent inputs have already been studied by Rahman [1], while Chastaing et al. [2, 3] developed closely related theoretical and numerical approaches. It is not clear how the proposed approach differs from these methods or what additional benefits it provides. In particular, the significance of the Riesz-basis result relative to existing work is insufficiently explained. I therefore find it difficult to identify a substantial new contribution. I also find the theoretical presentation unclear. *Sections 1 and 2 do not clearly distinguish marginal effects, pure main effects, and interaction effects. The discussion of conditional-expectation-based decompositions is also imprecise: Möbius inversion remains valid under dependence, but generally does not yield hierarchically orthogonal components.

2. Inadequate experiments and missing relevant comparisons. The experiments in Section 5 mainly compare the proposed method with SHAP, TreeHFD, NAM, and EBM. I do not find these comparisons particularly meaningful without further justification, since these methods do not necessarily estimate the same quantities. In particular, SHAP attributions are not pure main effects. More importantly, comparisons with existing estimators of generalized functional ANOVA [1, 3] are missing. Related approaches such as interaction purification [4] and Random Planted Forests [5] should also be compared agains and discussed. The experiments focus almost entirely on main effects and pairwise interactions, which have already been studied extensively. I would expect experiments involving higher-order interactions and a more direct evaluation of the recovered component functions. Good reconstruction accuracy alone does not demonstrate that the individual effects are correctly estimated. Finally, Section 4 provides no statistical guarantees for the estimator, and the experiments do not systematically investigate the influence of polynomial truncation or marginal-density estimation. I also could not find a code repository for reproducing the results.

[1] Rahman (2014). A Generalized ANOVA Dimensional Decomposition for Dependent Probability Measures.

[2] Chastaing, Gamboa & Prieur (2012). Generalized Hoeffding–Sobol Decomposition for Dependent Variables—Application to Sensitivity Analysis.

[3] Chastaing, Gamboa & Prieur (2015). Generalized Sobol Sensitivity Indices for Dependent Variables: Numerical Methods.

[4] Lengerich et al. (2020). Purifying Interaction Effects with the Functional ANOVA: An Efficient Algorithm for Recovering Identifiable Additive Models.

[5] Hiabu, Mammen & Meyer (2020). Random Planted Forest: A Directly Interpretable Tree Ensemble.

Questions for authors

see weaknesses

Reviewer_LnTpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies generalized functional ANOVA for dependent continuous inputs. The authors construct a distribution-adapted family of functions and show that it forms a Riesz basis, providing an explicit representation of the Hilbert subspaces associated with the ANOVA components. They also propose a truncated series estimator based on the construction. Experiments on synthetic and real-world datasets examine reconstruction accuracy and agreement with existing explanation methods, including SHAP and TreeHFD.

Critical strengths
  1. The paper addresses a relevant theoretical problem in generalized functional ANOVA. The inverse-density construction provides an explicit representation of the component subspaces under dependent inputs. The Riesz-basis result is useful, as it connects the decomposition to nonparametric series estimation.

  2. The proposed estimator offers a relatively simple way to apply the theoretical construction to black-box models. The experiments illustrate its relationship with existing explanation methods, and the reported computational efficiency is a practical advantage.

Critical weaknesses
  1. My main concern is the gap between the theoretical decomposition and the proposed estimator. Under dependent inputs, least squares restricted to ∣S∣≤K|S|\leq K does not necessarily recover the corresponding low-order ANOVA components, since omitted higher-order interactions need not be orthogonal to the fitted space. The density estimation step raises a related issue: hierarchical orthogonality relies on the true marginal densities, while the implementation uses estimated and clipped densities. It is unclear how these approximations affect the recovered components.

  2. I am also not fully convinced by the experimental validation. The analytical example is particularly favourable to the method, as the true components lie within the chosen basis and order. On real data, reconstruction R2R^2 is evaluated on the fitted samples, while agreement with SHAP does not establish that the estimated components are correct under dependence.

Questions for authors
  1. Under dependent inputs, does the truncated least-squares estimator recover the true low-order generalized functional ANOVA components when higher-order interactions are present? Under what conditions would this hold? A synthetic example with known higher-order components would help clarify this.

  2. Appendix C states that the pointwise quality of the density estimate is largely irrelevant to hierarchical orthogonality. I am not sure this holds, since the argument in Lemma 3.3 relies on the exact marginal densities. Could the authors explain what happens when pSp_S is replaced by p^S\hat{p}_S, particularly when the ratio pS/p^Sp_S/\hat{p}_S differs from one?

  3. How does the estimator handle discrete or categorical features, given that the construction assumes continuous inputs? The same question applies to the tanh-transformed inputs, whose densities may not satisfy Assumption 2.1.

  4. Could the authors report held-out reconstruction R2R^2 and component-wise errors on synthetic data with known ANOVA components?Although Appendix D.6 reports the hyperparameter values, could the authors clarify how KK, MM, ddensityd_{\text{density}} and ε\varepsilon were selected for each dataset?

  5. Could the authors clarify which aspects of the proposed continuous-input construction go beyond the finite-dimensional inverse-likelihood framework of Ferrere et al. (2026)?

  6. In Appendix D.8, are the theoretical components square-integrable under the transformed Gaussian distribution? If not, how should the empirical cosine values in Table 8 be interpreted?

Reviewer_k9BEOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies explicit representations of generalized functional ANOVA for dependent continuous inputs. Under the assumption that the joint density on a bounded hyperrectangle is bounded above and bounded away from zero, the authors construct a distribution-adapted family by dividing tensorized Legendre polynomials by the corresponding marginal densities. They prove that this family is a Riesz basis of L2(PX)L^2(P_X) and that each block indexed by a variable subset spans the space containing the corresponding generalized fANOVA component. They then propose a practical estimator based on marginal density estimation, series truncation, and regression, and report experiments on synthetic and tabular datasets. I see potential value in the central representation result. In particular, the block-triangular operator argument and the coercivity of its diagonal blocks provide a coherent route to the Riesz-basis property.

Critical strengths
  1. An explicit representation with rigorous infinite-dimensional stability guarantees.

    Under the assumption that the joint density on a bounded hyperrectangle is bounded above and away from zero, the authors construct distribution-adapted basis functions using Legendre polynomials and inverse marginal densities. Theorem 3.2 establishes that this family forms a Riesz basis of the space of square-integrable functions. This guarantees completeness, uniqueness of the coefficient representation, unconditional convergence, and norm stability.

    These properties are substantive in an infinite-dimensional setting and do not follow merely from linear independence or hierarchical orthogonality. The proof through a bounded, invertible operator provides a coherent mathematical foundation for the construction. This result is a major strength because it supplies a stable representation on which further approximation and estimation theory can be developed.

  2. A precise connection between the basis blocks and the generalized fANOVA component spaces.

    Lemma 3.3 and Theorem 3.4 show that the proposed representation respects the identifying structure of generalized fANOVA: each basis block generates the component space associated with its variable subset. This is important because a generic function expansion does not automatically distinguish main effects from pure higher-order interactions. Corollary 3.5 also establishes consistency with the classical decomposition under input independence.

    The explicit block structure provides a concrete route to coefficient estimation and finite approximations controlled by interaction order and polynomial degree. The proof-of-concept implementation illustrates this route using samples and model evaluations without requiring access to the predictor’s internal architecture. This connection makes the theoretical representation computationally approachable and provides a useful starting point for subsequent statistical analysis.

Critical weaknesses

1. The practical estimator and its evaluation do not establish faithful recovery of the intended generalized fANOVA components.

Relevant locations: Section 4, particularly Eqs. (16)–(18); Appendix C; and the reconstruction and orthogonality evaluations in Section 5 and Appendix D.7.

The representation theorem concerns a complete expansion using the true marginal densities. The practical method instead fits a truncated, data-dependent dictionary. Under dependence, components associated with incomparable variable subsets need not be orthogonal. Consequently, refitting a truncated expansion can transfer omitted interactions into retained main effects, even with known densities and infinitely many observations.

For example, consider

\qquad x\in[-1,1]^3,\qquad 0<|a|<1,$$ and $$f(x)=2x_1+x_1x_2.$$ This distribution satisfies Assumption 2.1, and all univariate and bivariate marginals are uniform. The complete generalized fANOVA has $$f_{\{1\}}=2x_1,\qquad f_{\{1,2\}}=x_1x_2,$$ with every other component equal to zero, including $f_{\{3\}}=0$. However, population least squares with $K=1,M=1$ gives $$\widehat f(x)=2x_1+\frac a9x_3,$$ because $$\mathbb{E}[X_1X_2X_3]=\frac a{27}, \qquad \mathbb{E}[X_3^2]=\frac13.$$ At $a=0.9$, the reconstruction score is approximately $0.9254$, despite assigning a nonzero contribution to a variable absent from the target function. Thus, accurate reconstruction does not establish accurate recovery of the original function’s components. The orthogonality diagnostic is also weaker than the identifying constraint. Equation (3) requires orthogonality against every square-integrable function of a strict subset, whereas Appendix D.7 checks correlations only between the estimated components themselves. For $K=1$, zero values are largely ensured by explicit recentering. Moreover, Table 7 evaluates theoretical components rather than recovery errors of the estimated components. With estimated and clipped densities, the exact cancellation underlying the theoretical constraint is no longer automatic either. **Why this affects my assessment:** I recognize that the estimator is presented as a proof of concept and that a complete statistical theory is outside the stated scope. The concern is that the current evidence supports prediction approximation more directly than recovery of the claimed decomposition. The estimator’s population target and the interpretation of its recovered blocks need to be clarified, and the empirical evidence should directly assess component recovery. **2. The supplementary robustness example in Appendix D.8 violates the square-integrability requirement of the framework.** **Relevant locations:** Appendix D.8, Eqs. (108)–(109), and the component definition reused from Eq. (99). Appendix D.8 sets $X_j=\tanh Z_j$, where each $Z_j$ has a standard-normal marginal, and reuses the earlier component definitions with only their density denominators changed. Simplifying Eq. (99) gives $$f_2(x)=\frac{q(x)}{p_2(x)}, \qquad q(x)=P_1(x)+P_2(x),$$ where $q(1)=2$. Let $\phi_N$ denote the standard-normal density. Since $$p_2(\tanh z)=\frac{\phi_N(z)}{\operatorname{sech}^2z},$$ the second moment is $$\begin{aligned} \mathbb{E}[f_2(X_2)^2] &=\int_{-1}^{1}\frac{q(x)^2}{p_2(x)}\,dx\\ &=\int_{\mathbb{R}} \frac{q(\tanh z)^2\operatorname{sech}^4z}{\phi_N(z)}\,dz =\infty. \end{aligned}$$ Indeed, as $z\to+\infty$, the integrand grows on the order of $\exp(z^2/2-4z)$. Therefore, the proposed component does not belong to the required $L^2$ space. Small empirical cosine values cannot validate the example within this framework, because finite samples may miss the extreme tails responsible for divergence. **Why this affects my assessment:** This calculation does not invalidate Theorems 3.2 and 3.4 under Assumption 2.1. It does invalidate the supplementary example as evidence that the construction remains valid beyond those assumptions. Its mathematical validity must be established before its numerical behavior can support the associated robustness conclusions.
Questions for authors
  1. Population target of the truncated estimator.
    Is Eq. (18) intended to estimate the retained components of the complete generalized fANOVA, or the best population approximation within the truncated dictionary? How should the counterexample above affect the interpretation of the recovered main effects and interactions? If the two targets are claimed to coincide, what additional assumptions ensure this?

  2. Correspondence between the defined and implemented estimators.
    Do the reported experiments solve the full least-squares problem in Eq. (18), or use LARS/BIC selection followed by SVD refitting, as described in Appendix C? If selection is used, how is it incorporated into the estimator’s definition, and how many terms are retained in the reported experiments?

  3. Density estimation and hierarchical orthogonality.
    Is the statement in Appendix C, lines 1135–1137, that capturing the density’s global trend is sufficient to guarantee hierarchical orthogonality intended as an exact guarantee or an empirical observation? What argument supports it after density estimation and clipping, given that the cancellation leaves the factor pS/p^Sp_S/\widehat p_S? Please identify any conditions needed for the claimed conclusion.

  4. Independent evaluation and componentwise accuracy.
    Were reconstruction and orthogonality evaluated on observations independent of density estimation, dictionary selection, and coefficient fitting? Which reported results quantify errors of the estimated components against the known synthetic main effects and bivariate interaction? Please distinguish these from reconstruction scores and the theoretical-component cosines in Table 7.

  5. Validity of Appendix D.8.
    Is there a missing assumption or modification to the component definitions that makes E[f2(X2)2]\mathbb{E}[f_2(X_2)^2] finite? If the stated definitions are intended, do the authors agree with the divergence calculation above? If so, how should the example and the conclusions drawn from it be revised?

  6. Precise theoretical novelty.
    How does the proposed component-space characterization differ from the constructive polynomial expansions in Rahman (2014), A Generalized ANOVA Dimensional Decomposition for Dependent Probability Measures, and from the inverse-likelihood construction of Ferrere et al. (2026)? Please identify the precise new representation, completeness, or Riesz stability guarantee and distinguish the assumptions under which the respective results hold.

1 comment
Reviewer_k9BEOpenReview ↗

Official Comment by Reviewer k9BE

Thank you for the detailed response; I have increased the score to 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.