- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies estimator selection and aggregation for off-policy evaluation (OPE). It is well known that the relative performance of OPE estimators depends on the underlying data-generating distribution, and no single estimator is uniformly best across all settings. The paper proposes Personalized Estimator Aggregation for Off-Policy Evaluation (PerOPE), a data-driven framework that uses contextual information to select or aggregate estimators, in contrast to previous methods that employ a single estimator or context-independent aggregation weights. The authors conduct extensive experiments on synthetic and real-world data, demonstrating the empirical benefits of incorporating contextual information into estimator aggregation.
Critical strengths
I find the problem practically relevant, and the proposed framework is simple and intuitive. The formulation of context-dependent estimator aggregation is particularly straightforward. The empirical evaluation is also fairly extensive, covering various experimental settings and including a real-world production dataset, which helps demonstrate the practical value of the approach.
Critical weaknesses
My main concern is that the paper does not adequately address the fundamental theoretical difficulty of data-driven estimator selection. Although PerOPE is presented as an adaptive approach that selects or aggregates estimators based on observed data, the resulting procedure is itself a single fixed estimator, i.e., a mapping from the logged data to a policy-value estimate. Therefore, adaptivity alone does not circumvent the fundamental limitations of OPE, including the impossibility of uniform dominance in general settings and existing minimax lower bounds (e.g., Wang et al., 2017).
This does not mean that data-driven estimator selection is inherently ineffective. Rather, its theoretical justification requires some structural assumptions under which the selection rule can be learned reliably. For example, Su et al. (2020) exploit a structured bias-variance trade-off to establish an oracle-type guarantee for adaptive estimator selection. In contrast, PerOPE does not appear to identify comparable structural assumptions or provide a theoretical guarantee that accounts for the statistical cost of learning its context-dependent aggregation rule.
In particular, Theorem 2 (Non-negative Oracle Improvement) is somewhat misleading as a theoretical justification for the proposed method. It merely observes that the class of context-dependent aggregations contains the class of context-independent aggregations, so the optimal MSE over the former cannot be larger than that over the latter. However, this is an oracle-level comparison that assumes knowledge of the true MSE-minimizing aggregation rule, which is precisely what the proposed method attempts to learn from data. The theorem therefore provides no guarantee that the learned estimator actually achieves a lower MSE, nor does it quantify the additional estimation error introduced by personalization.
While the experimental results are encouraging, they cannot establish a general improvement across data-generating distributions. Without a theoretical characterization of when personalization is beneficial, or an oracle inequality accounting for the cost of learning the aggregation rule, I find the central claim insufficiently justified. The paper would be substantially stronger if it clarified the assumptions under which personalization provides a genuine statistical advantage, rather than relying primarily on empirical improvements.
Questions for authors
See the weaknesses above.
2 comments
We would like to kindly inform the reviewer that we have submitted a revised manuscript, with all changes highlighted in red, directly addressing the key theoretical concerns. In particular, we have clarified the oracle-level interpretation of Theorem 2, explicitly distinguished it from finite-sample performance, and strengthened the discussion connecting our theoretical results to the empirical evaluation.
We would sincerely appreciate any remaining theoretical questions or objections before the discussion deadline. If no further concerns are raised, we believe it is reasonable to conclude that the central theoretical issues identified in the review have been adequately addressed, and we hope this will be reflected in the final evaluation.
We sincerely thank the reviewer for the detailed theoretical feedback and for recognizing the motivation and potential value of personalized estimator aggregation. We appreciate the opportunity to further clarify the scope of our theoretical guarantees and the empirical evidence supporting our approach.
1. What does the theory establish?
We appreciate this theoretical comment and agree with its central distinction.
Theorem 2 is an oracle-level hypothesis-class comparison, not a statement that the learned PerOPE estimator uniformly dominates conventional OPE methods. Its role is to identify the representational advantage of allowing context-dependent weights. It does not account for the additional error incurred when learning a more flexible aggregation rule from finite data.
A more informative analysis separates oracle approximation error from estimation and optimization errors. For example, writing R(T) for true MSE and R^(T) for estimated MSE, suppose a personalized estimator class TK satisfies, with high probability,
supT∈TKR(T)−R(T)≤ε(n,K)
If the learned estimator approximately minimizes R^(T) with optimization error δopt, then
R(TK)≤infT∈TKR(T)+2ε(n,K)+δopt
This inequality illustrates the relevant trade-off. A larger tree class may reduce the best achievable MSE but increase the statistical error associated with selecting an estimator. It does not imply improvement over the best global estimator, and establishing an explicit bound for ε(n,K) requires additional assumptions on the MSE-estimation module, data dependence, and tree complexity.
2. When should personalization be beneficial?
We thank the reviewer for raising this question. PerOPE is particularly beneficial when the relative accuracy of candidate estimators varies across contexts, allowing personalized aggregation to exploit heterogeneity that global methods cannot capture. Theorem 2 theoretically supports this advantage at the oracle level.
Our experiments already provide empirical evidence of when personalization is effective. The synthetic experiments demonstrate that PerOPE's advantage generally increases with contextual heterogeneity, while Appendix E.1 and E.2 further examine robustness to tree depth and sample size. Importantly, these improvements are achieved without careful hyperparameter tuning.
Furthermore, our real-world experiments also demonstrate the practical benefits of personalization, with PerOPE consistently outperforming its context-independent counterparts across multiple evaluation scenarios. These findings provide empirical evidence that meaningful contextual heterogeneity exists in real-world OPE settings and that exploiting such heterogeneity through personalization can indeed improve estimation accuracy.
Our contribution does not require universal improvement under every possible data-generating distribution. Rather, our theoretical analysis and extensive experiments demonstrate both the potential and practical benefits of contextual personalization, which we believe sufficiently support our central claims.