acceptodds
Under review as a conference paper at ICLR 2027

SLICE: Building Models from Structured Single-Feature Perturbation Effects

Abstract

Single-feature perturbation analysis is a fundamental method for investigating how a predictive model behaves when a single feature is varied while the remaining features are held fixed. Existing visualization approaches analyze these effects after a model has been fitted, typically by aggregating responses across dataset samples or by visualizing individual response curves. However, aggregation can obscure heterogeneous responses across samples, while the individual curves may be numerous, complex, and heavily interleaving, making the overall perturbation behavior difficult to characterize globally. Moreover, these approaches provide no structural guarantee on behavior under other unseen feature values. We reverse this workflow: rather than first fitting a model and subsequently attempting to understand its perturbation effects, we explicitly constrain the form these effects may take and build the model around those constraints. We introduce SLICE (Structured Laminar Individual Conditional Effects), a class of models whose centered single-feature perturbation curves form structured, laminar families. In such families, perturbation curves form an ordered family rather than crossing and interleaving arbitrarily, making the full range of possible single-feature responses easy to characterize globally while still allowing the underlying predictor to capture interactions of arbitrary order. We characterize the prediction functions compatible with these perturbation requirements and develop a differentiable parameterization that satisfies them by construction. Across benchmark datasets, SLICE outperforms additive models and often approaches or matches fully flexible baselines, while preserving globally comprehensible single-feature perturbation behavior. SLICE thus shifts single-feature perturbation analysis from merely post-training inspection to explicit control during model construction.

Primary area:
interpretability and explainable AI
Keywords:
single-feature perturbations, model analysis, individual conditional expectation, partial dependence

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_jQafOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SLICE builds predictive models around structured single-feature perturbation behavior instead of attempting to summarize arbitrary perturbation curves after training. Its affine form constrains centered curves to a laminar family while retaining feature interactions. The paper characterizes the compatible functional form, develops differentiable parameterizations, and evaluates predictive performance and perturbation representations.

Critical strengths
  1. The motivation is clear: ICE curves can be illegible, while averaging conceals heterogeneous responses. Constraining the predictor to have globally describable perturbation behavior directly addresses this tension.

  2. Connecting that requirement to a functional characterization and trainable model constructions is a substantive methodological contribution. The experiments show that the resulting model class can retain useful interactions.

Critical weaknesses
  1. Interaction-enabled baseline. Appendix C.1.3 disables EBM interactions, whereas A-SLICE includes pairwise interactions. An interaction-enabled EBM/GA2M under comparable tuning would better establish the accuracy-interpretability trade-off. This comparison does not affect the validity of the functional characterization.

  2. Expressivity cost. Shared modulation functions restrict interaction shapes. A representative nonconforming pattern and its predictive cost would clarify when the structural guarantee is useful and when it is restrictive.

Questions for authors
  1. How does A-SLICE compare with an interaction-enabled EBM/GA2M under matched tuning?
  2. Which interaction patterns expose the main cost of the laminar restriction?
3 comments

We thank the reviewer for their constructive assessment and for recognizing the motivation and methodological contributions of SLICE. We agree that stronger interaction-aware comparisons and a concrete illustration of the expressivity limitations would strengthen the paper. We have addressed both points in the revised manuscript, as detailed below.

Weakness 1

We agree that comparing A-SLICE only against additive EBM does not fully characterize the trade-off between predictive flexibility and perturbation structure. The original EBM configuration intentionally disabled interactions to serve as an additive baseline. However, we have now added two interaction-enabled EBM variants: one with up to pp pairwise interactions and another with all (p2)\binom{p}{2} possible pairwise interactions, where pp is the number of features. Their results are reported in Tables 3–4.

These comparisons reveal a dataset-dependent trade-off. For example, on Churn, A-SLICE (spline) and EBM with all pairwise interactions achieve nearly identical ROC-AUC (0.925 versus 0.926). On Bike Sharing, interaction-enabled EBM achieves higher predictive performance (R2R^2 of 0.926 versus 0.873). However, our new rank-order consistency analysis (Appendix D.3, Table 13) reveals substantial differences in perturbation structure: on Bike Sharing, A-SLICE achieves a consistency score of 1.000, compared with 0.748 for EBM with all pairwise interactions.

These results illustrate the distinction between interaction-capable models with and without guaranteed laminar perturbation families. We also provide centered ICE visualizations of the additional baselines (Appendix D.2, Figures 4–13), allowing direct qualitative comparison.

Regarding matched tuning, we acknowledge that the newly added EBM variants have not yet undergone hyperparameter optimization due to time constraints. Their results are therefore preliminary, and we plan to complete tuning for the camera-ready version.

Weakness 2

We agree that the shared modulation functions restrict the interaction patterns A-SLICE can represent. Although A-SLICE supports interactions of arbitrary order, all interactions involving a given feature must share the same univariate modulation function. This is a deliberate restriction that guarantees globally structured perturbation families, but it can also limit predictive expressivity.

To illustrate this limitation concretely, we have added an analytical example in Appendix E.2. Consider

f(x1,x2,x3)=x1x2+x12x3,f(x_1,x_2,x_3)=x_1x_2+x_1^2x_3,

on [−1,1]3[-1,1]^3. This function contains two interactions involving x1x_1, one with linear dependence on x1x_1 and another with quadratic dependence.

For the reference value x1=0x_1=0, the perturbation curves corresponding to contexts (x2,x3)=(0,0),(1,0),(0,1)(x_2,x_3)=(0,0),(1,0),(0,1) are respectively 00, tt, and t2t^2. In A-SLICE, differences between any two perturbation curves for the same feature must be scalar multiples of a shared width function. Exact representation would therefore require tt and t2t^2 to be proportional, which is impossible. Hence, this function cannot be represented exactly by A-SLICE, even when higher-order interactions are permitted.

This example identifies a concrete interaction pattern excluded by the laminar restriction. The resulting predictive cost depends on the data distribution and is illustrated empirically by the performance differences between A-SLICE and the interaction-enabled baselines. We have also clarified this expressivity limitation in the discussion.

Actions taken

The revised manuscript includes the following changes, highlighted in blue in the updated PDF:

  • Interaction-aware benchmarks: Added two interaction-enabled EBM variants, CALM, shallow decision trees, and TabPFN-3.5 (Tables 3–4).

  • Perturbation analysis: Expanded the centered ICE visualizations (Figures 4–13) and introduced a quantitative rank-order consistency metric (Appendix D.3, Table 13).

  • Expressivity limitations: Added a concrete counterexample demonstrating an interaction pattern that A-SLICE cannot represent exactly (Appendix E.2).

Note on preliminary results. The newly added baselines (except decision trees) remain untuned, and some Table 13 results are based on a single fold rather than the full five-fold evaluation. We plan to complete these evaluations for the camera-ready version.


We thank the reviewer again for their thoughtful and constructive feedback. We believe the additional interaction-aware benchmarks and explicit expressivity example substantially strengthen the paper by clarifying both the benefits and limitations of enforcing structured perturbation behavior. We hope these revisions address the reviewer's concerns and welcome any further questions.

Dear Reviewer jQaf,

Thank you again for your time and valuable suggestions. We wanted to briefly highlight a few additional revisions that further address your concerns regarding the comparison with interaction-aware EBMs.

Following further discussion, we have expanded Section 6.3 to explicitly discuss the predictive trade-off between A-SLICE and interaction-enabled EBMs. We acknowledge that EBM with all pairwise interactions achieves stronger performance on most real-world benchmarks, while highlighting the more competitive results against EBM (1x) and the complementary findings on the Feynman datasets.

We have also extended Appendix D.2 with native EBM explanations (Figure 14), including univariate shape functions and two-dimensional interaction plots. These help illustrate an important distinction between the models' explanatory objectives.

For an interaction-aware GAM, the complete perturbation effect of feature ii is

g_i(x_i)-g_i(c_i) +\sum_{j\neq i} \left[h_{ij}(x_i,x_j)-h_{ij}(c_i,x_j)\right].$$ Understanding this effect therefore requires combining its univariate shape function with all relevant pairwise interactions. For example, EBM (all pairs) on Bike Sharing contains 66 pairwise interaction functions, of which 11 involve any given feature. These are typically represented through two-dimensional plots. In contrast, A-SLICE represents the complete perturbation family as $$\Delta f_i^{c_i}(x_i,x_{-i}) = m_i(x_i)+\delta_i(x_{-i})w_i(x_i).$$ This enables a compact characterization of all possible perturbation curves through representative curves and globally valid bounds. While the structural restrictions underlying this representation can limit predictive flexibility, they provide a specific capability that the native GA\$^2\$M explanations do not directly offer. Finally, we have refined our wording throughout the manuscript to emphasize this specific objective rather than broader claims about interpretability. We hope these additions further strengthen the comparison you requested, and we would be very happy to address any remaining questions or concerns.

Dear Reviewer jQaf,

We would like to share one further addition to the manuscript that directly addresses your concern regarding the predictive cost of A-SLICE's structural restrictions.

In our initial response, we provided a concrete example of a function that A-SLICE cannot represent exactly. We have now strengthened this analysis by deriving an exact, nonzero approximation gap, rather than demonstrating non-representability alone.

Actions taken

We have added Appendix E.3 with a new theoretical result. Specifically, we consider the smooth target function f∗(x1,x2)=cos⁡(x1−x2)f^*(x_1,x_2)=\cos(x_1-x_2), where X1X_1 and X2X_2 are independent and uniformly distributed on [−π,π][-\pi,\pi].

We first prove that this function cannot be represented by A-SLICE, using a simple argument based directly on the structure of its perturbation curves. We then derive the exact minimum population MSE achievable by A-SLICE, showing that it equals 1/41/4, compared with 1/21/2 for GAMs and 00 for unrestricted models. These correspond to optimal R2R^2 values of 0.50.5, 00, and 11, respectively.

The result demonstrates that A-SLICE can recover half of the predictive improvement over an additive model while still incurring a strictly positive approximation error due to its structural constraints. Importantly, the result is established analytically, independently of model parameterization or optimization.

We believe this addition provides a more precise answer to your second question by identifying both a concrete interaction pattern that violates the A-SLICE constraints and the exact predictive cost of that restriction.


We hope that this new theoretical result, together with the interaction-aware benchmarks and expanded perturbation analyses presented in our earlier responses, addresses the main concerns raised in your review. Given these substantial additions, we would be grateful if you would consider revisiting your assessment and score in light of the revised manuscript. We would also welcome any further questions or clarification on concerns that remain unresolved.

Thank you again for your constructive feedback.

Reviewer_P29YOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

feature perturbation is a post-hoc method used to analyze trained black-box models. Results are usually aggregated to make visualization easier, but there are also approaches that visualize individual effects. the authors propose using this analysis to build more interpretable models. They claim that their method outperform other methods while preserving globally comprehensive single-feature perturbation behavior.

Critical strengths

Despite minor typos, I find the notation and problem formulation precise.

Critical weaknesses

1.The main contribution is not supported by the experiments. You list this as your first contribution "First, we formulate global single-feature perturbation structure as a property that can be imposed during model construction rather than analyzed only post-hoc", and in your abstract you mentioned "We reverse the work flow". however, during training the perturbation structure is imposed only through the parameterization. You mention it in lines 366-370, I expected to see the main contribution in the experiment: how exactly it worked on the real-world data.

  1. the experiments do not support the claims of the paper. All experiments in section 6 evaluate only predictive accuracy. Comparing GAM in table 4 and A-SLICE(1) in Table 2 indicates that the gains come from interaction terms.

  2. Writing and clarity. The writing is very informal in general.

Questions for authors

Between Table 2 and Table 4, the results of A-SLICE(2) on bike sharing data doesn't match, why?

Minor comments:

Line 123: alyzed → analyzed

Line 123: post hoc vs. post-hoc; pick one and be consistent.

Line 197: X−i\mathcal{X}_{-i} is not defined.

Line 232: "Let us unpack this definition." → delete.

Line 282: ! → .

Equation 1: You introduce (t,x−i)(t, x_{-i}) in the notation but use xix_i in Eq. 1: Δfici(xi,x−i)→Δfici(t,x−i)\Delta f_i^{c_i}(x_i, x_{-i}) \rightarrow \Delta f_i^{c_i}(t, x_{-i}).

First paragraph of Section 6: change the list structure to plain text.

Tables 3 and 4: bold marks the best result within each category, not overall.

2 comments

We thank the reviewer for their assessment and feedback. However, we respectfully disagree with the two main criticisms, which appear to rest on incorrect interpretations of our methodological contribution and experimental objectives. In particular, enforcing perturbation structure through the model parameterization is precisely the central contribution of SLICE, while the predictive improvements arising from interaction terms are expected and explicitly investigated in our experiments. Moreover, the original submission already included real-world demonstrations of the learned perturbation structure. Below, we clarify these points and present substantially expanded experimental evidence addressing the reviewer's concerns.

Weakness 1

First, enforcing the perturbation structure through the model architecture and parameterization is precisely the main contribution of SLICE, not a limitation of the approach.

We define the general SLICE class through its perturbation structure (Section 3.2) and characterize the functional form of its affine subclass in Theorem 1 (Section 3.3). Our differentiable parameterization guarantees the required structure by construction, for all admissible inputs and independently of the learned parameter values. Thus, no additional perturbation-specific loss, constraint penalty, or empirical verification during training is necessary.

We also clarify that lines 366–370, cited by the reviewer, describe an optional post-training model-editing procedure, not the SLICE training algorithm. No manual intervention is required during training.

We also disagree that the main contribution is not demonstrated on real-world data. The original submission already included experiments on real datasets (Table 2 and 4), illustrated the learned perturbation structure on the Churn dataset (Figure 2) and compared SLICE's perturbation curves with conventional ICE plots on the Bike Sharing dataset (Appendix D.2). These visualizations demonstrate the structural properties of fitted models, complementing the theoretical guarantees established in Section 3.

Nevertheless, we recognize the value of a more extensive empirical comparison. We have therefore expanded Appendix D.2 with centered ICE visualizations for additional baselines (Figures 4–13) and introduced a quantitative rank-order consistency metric (Appendix D.3, Table 13). This metric measures how consistently perturbation curves preserve their relative ordering across feature values, providing direct empirical evidence of the structural differences between SLICE and competing models.

Weakness 2

We respectfully disagree with the reviewer's statement that the experiments do not support the paper's claims. The introduction makes the following empirical claim:

across benchmark datasets, A-SLICE consistently improves over additive models and often approaches or matches fully flexible black-box baselines, while retaining globally comprehensible single-feature perturbation behavior.

Tables 3 and 4 directly evaluate the predictive performance claim, while the globally structured perturbation behavior follows from the theoretical results in Section 3 and is illustrated on real-world data in Figure 2 and Appendix D.2.

The reviewer's observation that predictive improvements arise from interaction terms is correct, but this is precisely the intended result of the experiment, not a limitation or an alternative explanation of our findings. Section 6.1 explicitly investigates whether introducing structured interactions improves predictive performance, using a controlled comparison with fixed architectural settings. Our contribution is not the observation that interactions improve performance, but the construction of models that can capture these interactions while preserving globally structured single-feature perturbation behavior. This is the central distinction between SLICE and unconstrained interaction-capable models.

To address the reviewer's concern more directly, we have substantially expanded the empirical evaluation in two directions.

First, we introduce stronger interaction-aware baselines, including two EBM variants: one with up to pp pairwise interactions and another with all (p2)\binom{p}{2} possible pairwise interactions (where pp is the number of features). We also add CALM (Gkolemis et al., 2026) and shallow decision trees. Unlike SLICE, these models can capture interactions without guaranteeing laminar perturbation families. Their inclusion therefore allows a more direct comparison of the predictive performance of interaction-capable models with and without SLICE's structural restrictions (Tables 3–4). We additionally include TabPFN-3.5 as a modern black-box baseline.

Second, we introduce a quantitative evaluation of perturbation structure. Our new rank-order consistency metric measures how consistently the relative ordering of individual perturbation curves is preserved across feature values. Specifically, for each pair of feature values, we compare the rankings of perturbation effects across contexts using the absolute Kendall rank correlation, allowing for global reversals of the ordering. The resulting score ranges from 0 to 1, with lower values indicating more inconsistent curve ordering. Every laminar perturbation family achieves a score of 1 by construction, including the degenerate additive case where all curves coincide. Unlike the theoretical guarantees, the metric is evaluated empirically on sampled contexts and feature values and can be applied to any predictive model.

We evaluate this metric across the benchmark datasets and report the results in a new Table 13 (Appendix D.3). For example, on Bike Sharing, A-SLICE achieves 1.000, compared with 0.748 for EBM with all pairwise interactions, 0.903 for CALM, and 0.700 for XGBoost. These results provide a quantitative comparison of the structural simplicity of learned perturbation families, complementing our qualitative visualizations and predictive evaluations.

We emphasize that the two central properties of our method - guaranteed laminar perturbation structure and the ability to capture interactions of arbitrary order - are established mathematically in Section 3. The experiments assess the predictive flexibility of this constrained model class, while the additional analyses further quantify its perturbation structure. We therefore believe that the concerns raised do not indicate a flaw in the soundness of the proposed method.

Weakness 3

We have revised the manuscript to improve the formality and consistency of the writing, including the passages identified by the reviewer.

Question

The results differ because Tables 2 and 4 use different hyperparameter-tuning protocols. Table 2 is a controlled interaction-order ablation, where all architectural hyperparameters are fixed and only the learning rate is tuned using 10 trials. Table 4 reports the full benchmark, where the complete hyperparameter space is optimized using 50 trials. These protocols are described in Appendix C. We have now clarified this distinction in Section 6.3 to avoid confusion.

Minor comments

Line 123 typo The word is spelled correctly in the manuscript. The apparent typo results from automatic LaTeX hyphenation across two lines.

Line 123 post hoc We have standardized the usage where appropriate.

Line 197 We agree that X−i\mathcal{X}_{-i} was only implicitly defined, analogously to x−ix_{-i}. We have now added its explicit definition for clarity.

Line 232 We have removed the phrase "Let us unpack this definition" as suggested.

Line 282 We have replaced the exclamation mark with a period to maintain a more formal writing style.

Equation 1 The notation (t,x−i)(t,x_{-i}) defines the operation of replacing the ii-th feature with an arbitrary value tt. Equation 1 uses xix_i to express the perturbation as a function of the original feature variable x\mathbf{x}. We consider both notations consistent.

List in Section 6 We prefer to retain the numbered research questions, which clearly outline the objectives of Section 6.

Table 3 and 4 The captions explicitly state that bold indicates the best result within each model category. This highlights comparisons among models with similar interpretability and predictive flexibility. We therefore prefer to retain this formatting.

Actions taken

The revised manuscript includes the following changes, highlighted in blue in the updated PDF:

  • Expanded benchmarks: We have added CALM, shallow decision trees, TabPFN-3.5, and two interaction-enabled EBM variants (Tables 3–4).
  • Perturbation analysis: We have added extensive centered ICE visualizations (Figures 4–13) and introduced a quantitative rank-order consistency metric, evaluated across benchmark datasets (Appendix D.3, Table 13).
  • Clarity and style: We addressed the reviewer's stylistic comments.

Note on preliminary results. Due to time constraints, the newly added baselines (except decision trees) remain untuned, and some Table 13 results use one fold rather than five. These results are preliminary; we plan to complete tuning and evaluation for the camera-ready version.


We hope these clarifications address concerns about our contribution and experimental objectives. We emphasize that SLICE's structural guarantees were already established in the original submission, while the new experiments further demonstrate their practical relevance. We thank the reviewer and hope they will reconsider their assessment. We welcome further questions or concerns and would be happy to address them.

Reviewer_bgc3OpenReview ↗
Rating
3(+2)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper proposes training a machine learning model whose single-feature changes (effects) are easier to understand. The approach provides visualizations showing how much changing one feature can raise or lower the prediction across different values of the other features. Its performance is compared with the XGBoost and random forests black-box models; as well as the interpretable GAM, NAM, and EBM models, using popular Feynman function datasets and classification/regression benchmarks. Results show that the proposed approach generally outperforms the interpretable baselines and sometimes matches XGBoost.

Critical strengths
  1. The paper's text is well written and easy for a human to follow. All Figures and Tables in the main text are of high quality.
  2. The general motivation and research idea are interesting to study: whether we can train interpretable models based on the feature effects and their interaction structure.
Critical weaknesses
  1. The related work cited and discussed is very limited, undermining novelty. It omits a large body of relevant research on feature effects and interpretable models to the point where I am not sure that the authors know the topic of their research.
    • A. Work on approximating feature effects with gradients (DALE: Differential Accumulated Local Effects for efficient and accurate global explanations, ACL 2023) and quantifying their heterogeneity (RHALE: Robust and Heterogeneity-Aware Accumulated Local Effects, ECAI 2023).
    • B. Work on similar interpretable models that stem from GAMs (Exploring and Interacting with the Set of Good Sparse Generalized Additive Models, NeurIPS 2023) and feature effects (Regionally Additive Models: Explainable-by-Design Models Minimizing Feature Interactions, ECML 2023).
    • C. Work on other interpretable models such as trees and the Rashomon effect (only for example: Exploring the Whole Rashomon Set of Sparse Decision Trees, NeurIPS 2022)
    • D. Work on visualizing feature effects (Visualizing the Feature Importance for Black Box Models, ECML 2018)
    • E. Work on the variance of feature effects (Relating the Partial Dependence Plot and Permutation Feature Importance to the Data Generating Process, xAI 2023) and grouping conditional feature effects (Model-agnostic feature importance and effects with dependent features: a conditional subgroup approach, DAMI 2023)
    • F. Work on the robustness of feature effects to data and model perturbations (On the Robustness of Global Feature Effect Explanations, ECML 2024; Fooling Partial Dependence via Data Poisoning, ECML 2022)
    • G. Work on the disagreement of such explanations due to feature interactions (GRANITE: A Generalized Regional Framework for Identifying Agreement in Feature-Based Explanations, AISTATS 2026; Unifying Feature-Based Explanations with Functional ANOVA and Cooperative Game Theory, AISTATS 2025)

Thus, it is unclear which parts of the methodology are informative and how they relate to state-of-the-art, e.g. "discovering heterogeneous subgroups" (A, E, G), "optimization" (A), "editing the model" (B, C), "certified bounds" (F).

  1. Experiments have critical issues:
    • "Can SLICE capture complex high-order interactions? (Sec. 6.2)" As the paper is written, my understanding is that Table 3 measures model performance and not the accuracy of capturing interactions. Answering the question requires measuring the error/accuracy of interaction approximation/recognition instead.
    • "Does SLICE outperform other methods on common benchmarks? (Sec. 6.3)" The contribution of empirically outperforming baselines (interpretable models) from up to year 2021 is rather weak. The paper should also clearly mention what are the "commonly used regression and classification dataset" with references in the main text.

Other comments:

  • The words "perturbation" and "context" seem to be new, unnecessary jargon introduced for the purpose of this work. It would be best to stick to the established notions of feature removal, feature marginalization, probability distributions and their empirical estimates instead.
  • Abstract sentence "In such families," is too long.
  • Figure 1 should be referenced sooner (or appear later).
  • Table 9 includes code-specific model names such as "ebm_additive".
Questions for authors
  1. Can you compare to Regionally Additive Models, Rashomon GAMs (B) and trees (C) empirically?
  2. How does Theorem 1 relate to the fANOVA decomposition (G)?
  3. How do Equations 10 & 11 relate to the bounds given in (F)?
  4. What is the novelty of Proposition 1? It essentially says "a function is additive iff it has no interactions". The proof can be simplified to

Replace the coordinates of xx by those of cc, one at a time. Each replacement subtracts fi(xi)f_i(x_i), since the hypothesis holds regardless of the other coordinates, so f(x)=f(c)+∑i=1Nfi(xi).f(x)=f(c)+\sum_{i=1}^{N}f_i(x_i). Conversely, if f(x)=b+∑i=1Ngi(xi)f(x)=b+\sum_{i=1}^{N}g_i(x_i), then f(x)−f(ci,x−i)=gi(xi)−gi(ci).f(x)-f(c_i,x_{-i})=g_i(x_i)-g_i(c_i). Thus fi=gi−gi(ci)f_i=g_i-g_i(c_i), also proving equality up to a constant.

And I think the other proofs can be simplified as well.

  1. Can you compare to TabPFN since XGBoost might not be a well-tuned baseline?

After rebuttal: The paper has been substantially revised during the rebuttal period to address several issues. Overall, I think it's borderline and could also use another round of reviews.

3 comments

We thank the reviewer for their assessment and constructive feedback. While we acknowledge that the paper's positioning relative to existing work could be strengthened, we believe that SLICE makes a distinct methodological contribution that is not addressed by the cited approaches. Below, we clarify these distinctions, present substantially expanded empirical comparisons, and outline the corresponding revisions to the manuscript.

Weakness 1

We agree that our related-work discussion should have covered these research directions more thoroughly, and we will expand it accordingly. Our central question is whether we can characterize the entire family of single-feature perturbation curves (including their shapes, ranges, and variation) over all admissible contexts. For an unrestricted predictor, evaluating finitely many contexts cannot establish such global properties. SLICE instead constrains these families during model construction, yielding a compact representation with globally valid bounds.

Below, we clarify how the cited approaches relate to this objective.

A. DALE and RHALE. These methods analyze Accumulated Local Effects (ALE), which integrate conditionally averaged local derivatives, rather than the complete family of fixed-context perturbation curves. DALE estimates ALE efficiently, while RHALE additionally quantifies heterogeneity through the conditional variance of local derivatives. However, these statistics do not determine the possible shapes, ranges, or relationships of individual perturbation curves, nor establish bounds over all admissible contexts. SLICE instead constrains the complete perturbation family by construction.

B. Regionally Additive Models (RAM) and Rashomon GAMs. RAM and its closely related architecture form the same research line, CALM (Gkolemis et al., 2026), are particularly relevant as they capture interactions through region-specific univariate functions. However, these functions do not necessarily describe the complete perturbation curves: changing xix_i can also change the regional assignments and contributions of other features. For CALM,

g_i^{r_i(x_{-i})}(t)-g_i^{r_i(x_{-i})}(c_i)\ +\sum_{j\ne i}\Big[ g_j^{r_j(t,x_{-{i,j}})}(x_j) -g_j^{r_j(c_i,x_{-{i,j}})}(x_j) \Big].$$ The second term captures regional switches (CALM's _hidden jumps_). Consequently, complete perturbation curves may exhibit discontinuities and arbitrary crossings. Therefore, although CALM provides interpretable region-specific shape functions, these functions alone do not characterize how the complete prediction changes when a feature is varied. In contrast, SLICE directly constrains these complete perturbation curves, ensuring that their possible shapes and variation across contexts admit a compact global representation. We have added CALM as a baseline (Tables 3-4) and visualize its actual centered perturbation curves (Figure 13). Rashomon GAMs, in contrast, explore sets of near-optimal additive predictors for model selection and editing. Their variability concerns *differences between candidate models*, rather than heterogeneous perturbation effects across contexts within one predictor. They do not introduce context-dependent interactions. **C. Decision Trees and their Rashomon Sets.** Decision trees provide interpretable rules, while Rashomon methods explore alternative near-optimal trees. However, individual trees can exhibit discontinuous perturbation curves with arbitrary crossings, without guarantees of a compact, ordered family. Rashomon methods study variability *between models*, rather than SLICE's within-model structural guarantees. We have added shallow decision trees as baselines (Tables 3-4) and compare their centered perturbation curves with SLICE (Figure 12). **D. Visualizing Feature Importance for Black-Box Models.** Partial Importance (PI) and Individual Conditional Importance (ICI) plots visualize how varying a feature affects *predictive loss rather than the model's predictions*. PI aggregates across observations, while ICI shows individual loss profiles. Neither characterizes or constrains the possible shapes, ranges, or relationships of prediction perturbation curves across all admissible contexts. **E. Variance of Feature Effects and Conditional Subgroups.** The first work studies statistical properties of Partial Dependence (PD) and Permutation Feature Importance (PFI), rather than the range of individual perturbation curves. The second is closely related to REPID and GADGET, already discussed in Section 2 All construct regional explanations through interpretable partitions, although the cited approach primarily addresses feature dependencies, whereas REPID and GADGET target interaction-related heterogeneity. The resulting regional PD curves still average individual effects within each subgroup and therefore cannot characterize the full family of perturbation curves over all admissible contexts.

F. Robustness of Global Feature Effect Explanations. These works investigate how post-hoc explanations, particularly PD and ALE, change when the data or fitted model changes, including through data poisoning. Their bounds concern the stability of an aggregated explanation, not the range of possible perturbation effects across contexts of a fixed predictor. They therefore do not establish the possible shapes, ranges, or relationships of individual perturbation curves over all admissible contexts.

G. GRANITE and Functional ANOVA. GRANITE extends regional explanation methods such as REPID and GADGET (already discussed in Section 2) by identifying regions where different feature-based explanations agree. However, regional agreement does not characterize the complete perturbation family or guarantee its structure.

Functional ANOVA provides a general framework for decomposing prediction functions into feature-subset components, but does not restrict their functional forms. We clarify its precise relationship to Theorem 1 in our response to Question 2.

Clarification of methodological contributions

We do not claim subgroup discovery, gradient-based optimization, or model editing as independent methodological novelties. Section 5 illustrates applications enabled by SLICE's representation. Its scalar context parameter δi(x−i)\delta_i(x_{-i}) facilitates grouping observations with similar perturbation behavior; its differentiable parameterization enables conventional gradient-based training; and its perturbation functions can be edited while preserving the model's structural form.

Likewise, our certified bounds characterize possible perturbation effects across contexts of a fixed predictor, not robustness to changes in the data or model.

Our central methodological contributions remain the characterization of globally structured perturbation families, the derivation of compatible prediction functions (Theorem 1), and their trainable parameterization with guarantees by construction.

Weakness 2

Evaluation of high-order interactions. We agree that Table 3 evaluates the accuracy of approximating target functions, rather than recovering their individual interaction components. Our intention was to investigate whether SLICE's structural constraints limit its ability to approximate complex non-additive functions. While Theorem 1 establishes that Affine-SLICE supports structured interactions of arbitrary order, Table 3 provides empirical evidence of its approximation capabilities on nonlinear analytical functions from the Feynman benchmark. We will therefore rename Section 6.2 to “Can SLICE approximate complex non-additive functions?” and clarify that the experiment does not establish interaction recovery.

Benchmark selection and dataset descriptions. Our primary objective is to evaluate the predictive cost of enforcing globally structured perturbation effects, rather than claim superiority over all competing models. Established GAMs remain relevant baselines despite their publication dates. For example, a recent comprehensive study by Kruschel et al. (2026) evaluated 14 models across 20 datasets and ranked EBM as the strongest interpretable model and second-best overall. Importantly, the EBM considered in that study supports pairwise interactions, unlike our additive-only EBM baseline; we acknowledge this distinction.

Nevertheless, we agree that comparisons with newer and stronger methods strengthen the evaluation. We have therefore added CALM, shallow decision trees, and TabPFN-3.5 to our benchmarks. We also extend our EBM comparisons to include two interaction-enabled variants: one with up to pp pairwise interactions and another with all (p2)\binom{p}{2} possible pairwise interactions, where pp is the number of features. We retain additive EBM as a reference for context-independent feature effects. These additional baselines allow us to better assess the predictive cost of enforcing SLICE's laminar perturbation constraints (Tables 3–4). Accordingly, we will rename Section 6.3 to "How does SLICE compare with additive, interaction-aware, and black-box models?", better reflecting the scope of the expanded evaluation.

Finally, the datasets, their sources, and preprocessing details are already documented in the appendix. We agree that the main text should be more informative and will explicitly name all eight datasets, provide their references, and direct readers to the appendix for further details.

Other comments

  • Terminology. Perturbation and context are established terms in related literature. In particular, CALM (Gkolemis et al., 2026) uses both to describe varying one feature while holding the remaining features (the context) fixed. Our terminology is also consistent with the ICE framework. Feature removal and marginalization describe different operations.
  • Abstract. We have shortened the sentence beginning “In such families.”
  • Figure 1. We have referenced the figure earlier in the introduction.
  • Table 9. We have replaced code-specific identifiers with descriptive model names.

Questions

  1. We have added CALM, a recent closely related work to RAM, and shallow decision trees to our benchmarks (Tables 3–4), together with visualizations of their centered perturbation curves (Figures 12–13). We also include two interaction-enabled EBM variants. Rashomon methods instead study variation between near-optimal predictors, whereas SLICE constrains heterogeneous perturbation effects across contexts within a single predictor. We therefore view these approaches as complementary rather than direct methodological competitors.
  2. Functional ANOVA decomposes a prediction function into feature-subset components. In particular, under the anchored fANOVA decomposition at reference point cc, the A-SLICE representation (with gi(ci)=ui(ci)=0g_i(c_i)=u_i(c_i)=0) yields higher-order interaction components of the form fS(c)(xS)=αS∏i∈Sui(xi), ∣S∣≥2f_S^{(c)}(x_S)=\alpha_S\prod_{i\in S}u_i(x_i), \ |S|\geq 2. Unlike general fANOVA, which does not restrict the functional form of these components, A-SLICE requires factorized interactions with the same univariate function uiu_i shared across all interactions involving feature ii. Theorem 1 establishes that this shared-factor structure characterizes the Affine-SLICE model class and guarantees its laminar perturbation families.
  3. The bounds concern different quantities. The cited work studies the robustness of global explanations, such as PD and ALE, to changes in the data or predictive model. Our Equations 10–11 instead bound individual prediction changes across all admissible contexts of a fixed predictor. These guarantees follow from SLICE's structural constraints rather than the stability of post-hoc explanations.
  4. We agree that Proposition 1 establishes a standard characterization of additive functions. We do not claim it as a novel contribution; its purpose is to show that context-independent perturbation curves necessarily imply an additive predictor, motivating SLICE. We appreciate the suggested coordinate-by-coordinate argument and will simplify the proof using telescoping while retaining explicit intermediate steps for clarity. We will also review the remaining proofs for possible simplifications.
  5. We have added TabPFN-3.5 as a modern tabular baseline (Table 3, Table 4). We also clarify that XGBoost was hyperparameter-tuned using 50 Optuna/TPE trials per outer cross-validation fold, with early stopping and the search space documented in Appendix C.

Actions taken

The revised manuscript includes the following changes, highlighted in blue in the updated PDF:

  • Expanded experiments: We have added CALM, shallow decision trees, TabPFN-3.5, and two interaction-enabled EBM variants to the benchmarks (Tables 3–4), along with their perturbation-curve visualizations for all baselines (Figures 5–13).
  • Related works: We have expanded the discussion of the related works and clarified SLICE's novelty relative to existing approaches (Section 2).
  • Experimental framing: We have revised the titles and descriptions of Sections 6.2–6.3, clarified that the Feynman experiment evaluates function approximation rather than interaction recovery, and explicitly identify and reference the benchmark datasets in the main text.
  • Presentation and clarity: We will simplify the proof of Proposition 1. We have addressed the minor stylistic comments.

Note on hyperparameter tuning. Due to time constraints, the newly added baselines (except decision trees) have not yet undergone hyperparameter tuning. Their results in Tables 3–4 should therefore be regarded as preliminary. We plan to complete hyperparameter tuning and report the corresponding results in the camera-ready version.


We hope these clarifications and additional experiments address the reviewer's concerns and better establish SLICE's methodological novelty and empirical contributions. We thank the reviewer again for their constructive feedback and hope they will reconsider their assessment in light of these revisions. We would be happy to address any additional questions or concerns and welcome further discussion.


References

Kruschel, S., et al. "Challenging the Performance-Interpretability Trade-Off: An Evaluation of Interpretable Machine Learning Models" (2026)

Gkolemis, V., et al. "Interpretability-by-Design with Accurate Locally Additive Models and Conditional Feature Effects" (2026)

Comments on the paper

6 comments

Thank you for taking the time to carefully consider our responses and the revised manuscript. We are very pleased that the additional clarifications and experiments have addressed your main concerns, and we appreciate your willingness to raise your rating in light of these revisions. Your constructive feedback has substantially improved the paper, for which we are grateful. Should you have any remaining questions or concerns, we would be happy to address them.

Thank you very much for engaging with our rebuttal, clarifying your remaining concerns, and reconsidering your assessment. We sincerely appreciate your thoughtful and constructive feedback.

Clarifying our objective

We agree that the rank-order consistency metric primarily confirms the structural property guaranteed by SLICE, rather than independently establishing improved interpretability.

However, we would like to clarify that our goal is not to establish that SLICE is more interpretable than existing models in general. As emphasized throughout the manuscript, we focus on a narrower objective: obtaining a compact, globally valid characterization of single-feature perturbation effects, including their variation across contexts.

This allows us to address questions such as: Does increasing a feature always increase the prediction? What is the maximum possible change in prediction when varying a feature between two values? How much can this effect differ across contexts? Our objective is to make such questions directly answerable, with guarantees extending to all admissible contexts.

Comparison with interaction-aware GAMs

In this respect, SLICE and interaction-aware GAMs (GA$^2$M) serve different explanatory objectives. GA$^2$M provides univariate shape functions and pairwise interaction functions, which are valuable for examining individual contributions. However, understanding the complete perturbation effect of feature ii requires combining all relevant components:

g_i(x_i)-g_i(c_i) +\sum_{j\neq i} \left[h_{ij}(x_i,x_j)-h_{ij}(c_i,x_j)\right].$$ In contrast, A-SLICE represents the entire perturbation family as $$\Delta f_i^{c_i}(x_i,x_{-i}) = m_i(x_i)+\delta_i(x_{-i})w_i(x_i).$$ Thus, every possible perturbation curve is characterized by two univariate functions and a context-dependent number, enabling compact visualization through representative curves and globally valid bounds. This distinction becomes particularly important when many interactions are included. For example, on Bike Sharing, EBM (all pairs) contains 66 pairwise components alongside 12 univariate functions. Understanding the complete perturbation effect of just one feature may involve examining up to 11 two-dimensional interaction heatmaps and combining their contributions. While these plots are useful for understanding individual interactions, synthesizing their combined effect across contexts is considerably less straightforward. ### Predictive performance and trade-offs We agree that the predictive trade-off deserves more explicit discussion. EBM (all pairs) achieves higher mean performance than A-SLICE on seven of eight real-world benchmarks (Table 4), sometimes by substantial margins. However, the results are more nuanced with fewer interactions. Compared with EBM (1x), which includes up to as many pairwise interactions as features, A-SLICE performs comparably on Churn (AUC 0.925 vs. 0.924) and MAGIC (0.926 for both), and better on Abalone (R\$^2\$ 0.571 vs. 0.550). The Feynman experiments (Table 3) offer another perspective: A-SLICE achieves higher mean performance than EBM (all pairs) on seven of eight complex non-additive functions, including I.32.5 (R\$^2\$ 0.968 vs. 0.837). Although A-SLICE uses higher-order interactions in these experiments, the results demonstrate that its structural constraints do not preclude strong predictive performance on complex functions. Overall, SLICE improves substantially over additive models while retaining structured perturbation behavior. More flexible interaction-aware models can achieve better performance, but they offer different explanatory representations. The performance differences also reflect architectural and optimization differences, not solely the laminar constraint. ### Actions taken We have incorporated the following changes into the revised manuscript: 1. **Appendix D.2**: Added native EBM explanations (Figure 14), including univariate shape functions and two-dimensional interaction heatmaps, complementing the existing c-ICE comparisons. 2. **Section 6.3**: Expanded the discussion of predictive trade-offs, including comparisons with EBM (1x) and EBM (all pairs), and clarified the differences in how they represent complete perturbation effects. 3. **Throughout the manuscript**: Refined potentially broad claims such as ''globally comprehensible perturbation behavior'' to emphasize compact, globally valid characterizations designed to facilitate global understanding. --- We hope these additions clarify both the specific objective of SLICE and the trade-offs associated with its structural guarantees. Thank you again for your valuable feedback and constructive engagement. Please let us know if you have any remaining concerns or questions; we would be very happy to address them.

Thank you again for your positive reassessment of our work. As the discussion period approaches its end, we wanted to briefly follow up on your comment about increasing your rating, as we noticed the numerical score still appears unchanged.

If you have any remaining reservations that affect your overall assessment, we would be grateful for the opportunity to address them while the discussion remains open.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.