acceptodds
Under review as a conference paper at ICLR 2027

Non-Parametric Rehearsal Learning via Conditional Mean Embeddings

Abstract

In machine learning, a critical class of decision-related problems concerns preventing predicted undesirable outcomes, referred to as the *avoiding undesired future* (AUF) problem. To address this, the *rehearsal learning* framework has been proposed to model influence relations for effective decisions. However, existing rehearsal methods rely on restrictive parametric assumptions such as linear systems or additive noise, limiting their practical applicability. In this paper, we propose the first non-parametric rehearsal learning approach for AUF without assuming specific functional forms of data generation processes. Specifically, we use kernel machinery to reformulate the AUF objective into a unified representation that disentangles desirability modeling from action-induced distributional changes. To retain constraint-margin information beyond binary success labels, we present a smooth Probit surrogate with an approximation error bound. We estimate the smoothed AUF objective by scalar response regression and CME-based adjustment, with pointwise consistency at a fixed query and fixed smoothing parameter under stated spectral conditions. Such a formulation naturally accommodates nonlinear systems and non-additive noise; empirical results on synthetic and real-data-derived semi-synthetic benchmarks demonstrate effectiveness and flexibility of our approach.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
probabilistic inference

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_qDrsOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

The paper proposes a non-parametric approach to rehearsal learning for selecting optimal interventions on actionable variables. The method uses a Probit smoothed approximation of the binary success indicator and combines kernel ridge regression (KRR) with CME to estimate the expected desirability of different interventions from observational data. The authors provide theoretical guarantees on the smoothing approximation error, estimation consistency, and decision consistency under stated assumptions. They also present experiments on synthetic and semi-synthetic benchmarks to demonstrate the effectiveness of the proposed approach.

Critical strengths

As I see it the main contribution is the new modeling flexibility. The proposed approach extends rehearsal learning to nonlinear systems with non-additive noise, without relying on restrictive assumptions about the data-generating mechanisms. By directly estimating the expected desirability of interventions, it avoids the need to construct a complete generative simulator and repeatedly simulate possible outcomes when evaluating candidate actions. Second is theoretical guarantees. The paper provides a theoretical analysis of the Probit approximation error and the statistical consistency of the nested estimator, alongside conditional guarantees for decision consistency. Third: Empirical evaluation. The authors evaluate their approach on several synthetic ans semi-synthetic benchmarks, demonstrating competitive performance in linear settings and improvements over the reported baselines in nonlinear settings. The additional experiments comparing smooth and hard-indicator responses provide evidence supporting the proposed smoothing technique.

Critical weaknesses

Limited methodological novelty: as I see it, the AUF problem considered in this paper can be naturally formulated as an offline contextual bandit problem with causal adjustment, where the context corresponds to the observed state, the actionable variables represent the action, and the reward is the indicator of reaching the desired outcome region. The paper does not sufficiently clarify what distinguishes its rehearsal learning formulation from these established frameworks or what advantages it offers over them. Furthermore, the proposed nested KRR and CME estimator closely follows existing causal response estimation methods, particularly Singh et al. (2024). The authors explicitly acknowledge that their estimator coincides with the existing heterogeneous response estimator under matched kernels and regularization parameters. Consequently, the main methodological novelty appears to be the Probit-smoothed desirability function, its theoretical analysis, and its application to rehearsal learning. It is unclear whether these contributions constitute a sufficiently substantial methodological advancement.

Insufficient comparison with established alternative approaches: As stated above, it is unclear how the proposed method differs from established contextual bandit approaches. Nevertheless, the experimental evaluation focuses primarily on previous rehearsal learning algorithms, without comparisons against offline contextual bandit or causal policy optimization methods, despite the close mathematical relationship between these problems. Furthermore, although Grad-Rh uses a generative modeling approach, it optimizes a different heuristic objective. Therefore, the experiments do not isolate the advantages of directly estimating the intervention objective over learning a generative model and optimizing the same AUF probability.

Lack of empirical analysis of estimation accuracy and convergence: Although the authors establish theoretical convergence guarantees, the experiments primarily report the success probability achieved by the selected interventions. Since the synthetic benchmarks provide access to the true data-generating mechanisms, the authors could directly evaluate estimation errors and empirical convergence rates as functions of the training-set size. Such experiments would help assess the practical accuracy and sample efficiency of the estimator and compare them against alternative methods.

Dependence on unverified causal identification assumptions: The proposed approach relies on the conditional exchangeability assumption introduced in Proposition 1, which requires that all relevant confounders are sufficiently accounted for by the observed variables. However, the authors do not establish that this assumption holds in realistic observational settings or investigate the consequences of violating it. For example, in the banking scenario, an unobserved internal risk score could influence both the historically assigned interest rate and the repayment probability, violating conditional exchangeability. Since this assumption is fundamental to identifying the intervention objective, its violation could introduce systematic errors that would not disappear even with unlimited training data. Moreover, the semi-synthetic experiments rely on assumed data-generating mechanisms and do not demonstrate robustness to unobserved confounding.

Limitations in optimization and generality: The estimated objective is generally non-concave, and the proposed multi-start gradient ascent algorithm provides no guarantee of reaching the global optimum. Although the theoretical decision-consistency result accounts for optimization error, it assumes this error vanishes without demonstrating that the implemented algorithm achieves this condition.

A less inherent weakness: the proposed Probit construction restricts the desired outcome region to convex polyhedra, limiting its direct applicability to more general or disconnected success regions.

Questions for authors
  1. How does the AUF formulation considered in this paper fundamentally differ from an offline contextual bandit problem with causal adjustment? What advantages does the proposed rehearsal learning approach offer?

  2. What exactly is the novelty of the proposed nested KRR/CME estimator given that it coincides with the heterogeneous response estimator of Singh et al. (2024) under matched conditions? Does the proposed method offer any theoretical or empirical advantages over the original estimator?

  3. Could the authors provide comparisons against established offline contextual bandit or causal policy optimization methods? Additionally, could they compare against a generative simulator-based approachs?

  4. How sensitive is the proposed method to violations of conditional exchangeability, particularly when relevant confounders are unobserved? Could the authors provide experiments examining the robustness of the selected interventions under such violations?

  5. Since the objective is generally non-concave, how close does the proposed multi-start gradient ascent algorithm get to the global optimum in compare to a brute force algorithm that provides the global minima?

  6. Less critical yet worth clarifying: Is the restriction to convex polyhedral desired regions fundamental to the proposed method?

Reviewer_mTz3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the Avoiding Undesired Future (AUF) problem, where the goal is to identify interventions that increase the probability of achieving desired outcomes using observational data. To relax the parametric assumptions made by existing rehearsal learning methods, the authors propose a non-parametric approach based on kernel methods. The method replaces binary success indicators with a Probit-smoothed response to retain information about the distance to the target boundary. Kernel ridge regression (KRR) is then combined with conditional mean embeddings (CME) to estimate the intervention objective, which is optimized using multi-start projected gradient ascent. The paper provides theoretical results on the smoothing approximation error and pointwise consistency of the estimator, along with experiments on synthetic and semi-synthetic benchmarks, including NHANES.

Critical strengths

The theoretical analysis is fairly thorough, but I would like to see more evidence for the benefit of CME-based confounder adjustment. Figure 3 shows only one example, making it difficult to judge how well the adjustment works under different confounding conditions.

Critical weaknesses

The core estimator is largely the same as that of Singh et al. (2024). The main change is the use of Probit smoothing to retain information about how close outcomes are to the target boundary, which is lost with binary success labels. While this improves the empirical results, the underlying estimation method remains unchanged. The paper also only compares Probit smoothing with the hard indicator, without considering other common smoothing functions. It is therefore unclear whether Probit offers any particular advantage over alternative choices. Overall, I find the methodological contribution rather incremental.

Questions for authors

Could the authors clarify what advantages Probit smoothing has over other common smoothing functions in this setting?

Reviewer_HR89OpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the Avoiding Undesired Future problem, which seeks context-specific alterations on actionable intermediate variables so that downstream outcomes fall within a desired polyhedral region with maximum probability. To avoid restrictive linear-Gaussian structural assumptions, the authors replace the binary success indicator with a smooth Probit product surrogate to retain continuous constraint-margin information. They then estimate the confounder-adjusted expectation via two-stage nested Kernel Ridge Regression and Conditional Mean Embeddings under decomposable RBF kernels, optimizing the resulting closed-form objective via multi-start projected gradient ascent. The paper provides approximation and pointwise consistency bounds and evaluates the method on linear, nonlinear synthetic, and semi-synthetic benchmarks.

Critical strengths
  • Motivation Against Distance Heuristics: The paper illustrates why prior nonlinear rehearsal heuristics that minimize distance to the target region's center can misalign with maximizing target probability, and shows how Probit smoothing preserves continuous constraint-margin gradients when successful observational samples are scarce.

  • Positioning and Theoretical Completeness: The authors extend the theoretical consistency analysis to regression targets lying outside the RKHS via fractional source conditions.

Critical weaknesses
  • Inactive Confounder Adjustment on Both Real-Data-Derived Benchmarks: A central premise of the paper is using Stage-2 Conditional Mean Embeddings to adjust for pre-alteration confounders UU that simultaneously influence actions AA and outcomes YY. However, Table 2 and Appendix D.1 reveal that for both real-data-derived benchmarks—Bermuda and NHANES—the pre-alteration confounder set is empty, setting the adjustment vector identically to 1. Consequently, on both real-world-derived tasks, Stage 2 of the algorithm is completely inactive and the nested estimator degenerates into standard single-stage Kernel Ridge Regression on context and action. The two-stage CME adjustment is only ever tested on synthetic tasks with a single scalar confounder.

  • Simplified Action Spaces and Discrepancies in Implemented Generators:

  1. BankExp: While Figure 1 and the introduction motivate the lending problem as a multivariate trade-off jointly altering loan amount A1A_1 and interest rate A2A_2, Appendix D.1 and D.3 reveal that A1A_1 is excluded from the implemented generator altogether, reducing BankExp to a 1D scalar optimization over A2A_2.

  2. Non-Syn2: Although the learner optimizes over a 2D action (A1,A2)(A_1, A_2), the structural equation for the outcome Y1Y_1 in Appendix D.3 does not depend on A1A_1 at all, making A1A_1 a dummy variable with zero causal effect on the target.

  • Missing Non-Parametric Machine Learning Baselines: Since the core two-stage estimator is directly adapted from Singh et al. (2024), the empirical comparison relies heavily on outperforming parametric linear baselines and a distance-based flow heuristic. Aside from the hard-indicator variant, the experiments omit standard non-parametric regression or classification baselines—such as Gaussian Processes, Gradient Boosted Trees, or standard MLPs trained to predict desirability from (X,U,A)(X, U, A) coupled with Monte Carlo marginalization over U∣XU \mid X—which is especially relevant on Bermuda and NHANES where UU is empty.

  • Untested Sensitivity to Partial-Order Misspecification: The identification in Proposition 1 requires prior knowledge of the exact causal partition separating pre-alteration confounders UU from post-alteration mediators UpostU^{\text{post}}. While the paper notes that adjusting for UpostU^{\text{post}} induces post-treatment bias, there is no empirical ablation evaluating how the method degrades when intermediate variables are misclassified between UU and UpostU^{\text{post}}.

Questions for authors
  1. Evaluating Active CME Adjustment on Real-World Benchmarks: Since U=∅U = \emptyset in both Bermuda and NHANES reduces the method to single-stage KRR, can the authors evaluate a setting on Bermuda or NHANES with a non-empty pre-alteration confounder set UU—for instance, by intervening on a subset of actionable variables while treating the remaining baseline intermediate variables as confounders in UU?

  2. Comparison with Standard Non-Parametric Predictors: How does the proposed kernel approach compare against standard non-parametric estimators, such as an MLP or Gaussian Process predicting wη(Y)w_\eta(Y) from (X,U,A)(X, U, A), particularly on Bermuda and NHANES where U=∅U = \emptyset?

  3. Multivariate BankExp and Non-Syn2 Equations: Why was A1A_1 omitted from the BankExp generator despite being featured in Figure 1, and was A1A_1 intentionally excluded from the outcome equation of Y1Y_1 in Non-Syn2?

  4. Robustness to Partition Errors: How sensitive is the empirical AUF probability if a post-alteration mediator in UpostU^{\text{post}} is mistakenly included in the adjustment set UU?

Reviewer_LxEpOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a non-parametric framework for rehearsal learning, which aims to identify effective actions to prevent undesirable future outcomes using observational data. Specifically, the authors introduce a Probit-based smooth surrogate for the indicator function of a desired outcome region. The conditional expectation of this surrogate is estimated using Kernel Ridge Regression (KRR), and Conditional Mean Embeddings (CMEs) are employed to marginalize over pre-alteration covariates, enabling the estimation of the post-intervention objective from observational data. The resulting objective is then optimized using multi-start projected gradient ascent to identify feasible actions. On the theoretical side, the paper analyzes the approximation error introduced by Probit smoothing, the estimation error of the nested KRR estimator, and decision consistency under additional assumptions. Empirically, the proposed method demonstrates competitive performance against existing rehearsal learning approaches on synthetic and real-data-derived semi-synthetic benchmarks.

Critical strengths
  • Clear motivation and presentation.
    • The paper addresses an important and practically relevant problem, with clear connections to real-world decision-making. The motivation is compelling, and the paper is generally well-written and easy to follow.
  • An objective aligned with the AUF problem.
    • The use of Probit smoothing provides a principled surrogate that closely approximates the original AUF probability while retaining information about constraint margins. This is particularly appealing compared to heuristic surrogate objectives that may not align with the actual decision-making goal.
  • Theoretical analysis.
    • The paper provides theoretical guarantees covering surrogate approximation and estimation errors, as well as decision consistency under additional conditions. These results strengthen the theoretical foundation of the proposed framework.
  • Detailed supplementary experiments.
    • The appendix includes useful additional analyses, including ablation studies, hyperparameter sensitivity analyses, and scalability experiments. These provide further insight into the behavior and practical characteristics of the proposed method.
Critical weaknesses
  • Limited methodological novelty.
    • As acknowledged by the authors, the proposed nested KRR estimator coincides with the heterogeneous response estimator of Singh et al. (2024) under matched settings.
    • Since Probit smoothing is also an established technique, the proposed framework appears to be largely a combination of existing methods, although its application to AUF and the additional theoretical analysis are valuable.
  • Limited evaluation of sample efficiency and scalability.
    • All main experiments use 1,000 observational samples. Although Appendix D.4 examines computational scalability, the evaluation of statistical performance across different sample sizes is limited.
    • Since the proposed approach relies on non-parametric estimation, a systematic evaluation across sample sizes and increasing input dimensions would help clarify its practical advantages and limitations relative to parametric methods.
Questions for authors
  • Regarding post-alteration variables.

    • My understanding is that (U_{\mathrm{post}}) is not explicitly included in either stage of the proposed estimator, but is instead implicitly marginalized out through the outcome regression. Is this interpretation correct? If so, could these observed post-alteration variables be explicitly leveraged, for example through mediator modeling or sequential conditional mean embeddings, to improve estimation accuracy or sample efficiency?
  • Regarding the experimental comparisons.

    • Could the authors discuss the potential advantages and limitations of the proposed nested KRR approach compared to flexible conditional generative models that optimize the same Probit-smoothed AUF objective? Such a comparison would help clarify the practical trade-offs between direct non-parametric estimation and generative modeling.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.