- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2(+1)
Read review
Paper summary
The paper proposes a non-parametric approach to rehearsal learning for selecting optimal interventions on actionable variables. The method uses a Probit smoothed approximation of the binary success indicator and combines kernel ridge regression (KRR) with CME to estimate the expected desirability of different interventions from observational data. The authors provide theoretical guarantees on the smoothing approximation error, estimation consistency, and decision consistency under stated assumptions. They also present experiments on synthetic and semi-synthetic benchmarks to demonstrate the effectiveness of the proposed approach.
Critical strengths
As I see it the main contribution is the new modeling flexibility. The proposed approach extends rehearsal learning to nonlinear systems with non-additive noise, without relying on restrictive assumptions about the data-generating mechanisms. By directly estimating the expected desirability of interventions, it avoids the need to construct a complete generative simulator and repeatedly simulate possible outcomes when evaluating candidate actions. Second is theoretical guarantees. The paper provides a theoretical analysis of the Probit approximation error and the statistical consistency of the nested estimator, alongside conditional guarantees for decision consistency. Third: Empirical evaluation. The authors evaluate their approach on several synthetic ans semi-synthetic benchmarks, demonstrating competitive performance in linear settings and improvements over the reported baselines in nonlinear settings. The additional experiments comparing smooth and hard-indicator responses provide evidence supporting the proposed smoothing technique.
Critical weaknesses
Limited methodological novelty: as I see it, the AUF problem considered in this paper can be naturally formulated as an offline contextual bandit problem with causal adjustment, where the context corresponds to the observed state, the actionable variables represent the action, and the reward is the indicator of reaching the desired outcome region. The paper does not sufficiently clarify what distinguishes its rehearsal learning formulation from these established frameworks or what advantages it offers over them. Furthermore, the proposed nested KRR and CME estimator closely follows existing causal response estimation methods, particularly Singh et al. (2024). The authors explicitly acknowledge that their estimator coincides with the existing heterogeneous response estimator under matched kernels and regularization parameters. Consequently, the main methodological novelty appears to be the Probit-smoothed desirability function, its theoretical analysis, and its application to rehearsal learning. It is unclear whether these contributions constitute a sufficiently substantial methodological advancement.
Insufficient comparison with established alternative approaches: As stated above, it is unclear how the proposed method differs from established contextual bandit approaches. Nevertheless, the experimental evaluation focuses primarily on previous rehearsal learning algorithms, without comparisons against offline contextual bandit or causal policy optimization methods, despite the close mathematical relationship between these problems. Furthermore, although Grad-Rh uses a generative modeling approach, it optimizes a different heuristic objective. Therefore, the experiments do not isolate the advantages of directly estimating the intervention objective over learning a generative model and optimizing the same AUF probability.
Lack of empirical analysis of estimation accuracy and convergence: Although the authors establish theoretical convergence guarantees, the experiments primarily report the success probability achieved by the selected interventions. Since the synthetic benchmarks provide access to the true data-generating mechanisms, the authors could directly evaluate estimation errors and empirical convergence rates as functions of the training-set size. Such experiments would help assess the practical accuracy and sample efficiency of the estimator and compare them against alternative methods.
Dependence on unverified causal identification assumptions: The proposed approach relies on the conditional exchangeability assumption introduced in Proposition 1, which requires that all relevant confounders are sufficiently accounted for by the observed variables. However, the authors do not establish that this assumption holds in realistic observational settings or investigate the consequences of violating it. For example, in the banking scenario, an unobserved internal risk score could influence both the historically assigned interest rate and the repayment probability, violating conditional exchangeability. Since this assumption is fundamental to identifying the intervention objective, its violation could introduce systematic errors that would not disappear even with unlimited training data. Moreover, the semi-synthetic experiments rely on assumed data-generating mechanisms and do not demonstrate robustness to unobserved confounding.
Limitations in optimization and generality: The estimated objective is generally non-concave, and the proposed multi-start gradient ascent algorithm provides no guarantee of reaching the global optimum. Although the theoretical decision-consistency result accounts for optimization error, it assumes this error vanishes without demonstrating that the implemented algorithm achieves this condition.
A less inherent weakness: the proposed Probit construction restricts the desired outcome region to convex polyhedra, limiting its direct applicability to more general or disconnected success regions.
Questions for authors
-
How does the AUF formulation considered in this paper fundamentally differ from an offline contextual bandit problem with causal adjustment? What advantages does the proposed rehearsal learning approach offer?
-
What exactly is the novelty of the proposed nested KRR/CME estimator given that it coincides with the heterogeneous response estimator of Singh et al. (2024) under matched conditions? Does the proposed method offer any theoretical or empirical advantages over the original estimator?
-
Could the authors provide comparisons against established offline contextual bandit or causal policy optimization methods? Additionally, could they compare against a generative simulator-based approachs?
-
How sensitive is the proposed method to violations of conditional exchangeability, particularly when relevant confounders are unobserved? Could the authors provide experiments examining the robustness of the selected interventions under such violations?
-
Since the objective is generally non-concave, how close does the proposed multi-start gradient ascent algorithm get to the global optimum in compare to a brute force algorithm that provides the global minima?
-
Less critical yet worth clarifying: Is the restriction to convex polyhedral desired regions fundamental to the proposed method?