acceptodds
Under review as a conference paper at ICLR 2027

Personalized Estimator Aggregation for Off-Policy Evaluation

Abstract

Off-policy evaluation enables accurate assessment of counterfactual policies using only logged interaction data. Although many estimators have been proposed, their estimation accuracy depends strongly on factors such as the logging policy, the evaluation policy, and the available sample size. This sensitivity makes it difficult to deliver an estimator that performs well across diverse practical settings. Although existing approaches to estimator selection or aggregation attempt to address this challenge, they critically overlook contextual information. In practical applications, the most effective estimator can differ substantially across distinct contexts or user segments. We thus propose Personalized estimator aggregation for Off-Policy Evaluation (PerOPE), a novel framework that adaptively aggregates multiple base estimators to deliver the most suitable one for each individual or context. By accounting for contextual variation, this approach captures the heterogeneity of estimator accuracy across context groups and enables personalized aggregation of estimators based on user features. We conduct extensive experiments on synthetic and real-world production datasets, and the results show that PerOPE consistently outperforms existing estimator selection or aggregation methods across a broad range of experiment conditions.

Primary area:
causal reasoning
Keywords:
off-policy evaluation, estimator selection, estimator aggregation, hyperparameter tuning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_SXUFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies estimator selection and aggregation for off-policy evaluation (OPE). It is well known that the relative performance of OPE estimators depends on the underlying data-generating distribution, and no single estimator is uniformly best across all settings. The paper proposes Personalized Estimator Aggregation for Off-Policy Evaluation (PerOPE), a data-driven framework that uses contextual information to select or aggregate estimators, in contrast to previous methods that employ a single estimator or context-independent aggregation weights. The authors conduct extensive experiments on synthetic and real-world data, demonstrating the empirical benefits of incorporating contextual information into estimator aggregation.

Critical strengths

I find the problem practically relevant, and the proposed framework is simple and intuitive. The formulation of context-dependent estimator aggregation is particularly straightforward. The empirical evaluation is also fairly extensive, covering various experimental settings and including a real-world production dataset, which helps demonstrate the practical value of the approach.

Critical weaknesses

My main concern is that the paper does not adequately address the fundamental theoretical difficulty of data-driven estimator selection. Although PerOPE is presented as an adaptive approach that selects or aggregates estimators based on observed data, the resulting procedure is itself a single fixed estimator, i.e., a mapping from the logged data to a policy-value estimate. Therefore, adaptivity alone does not circumvent the fundamental limitations of OPE, including the impossibility of uniform dominance in general settings and existing minimax lower bounds (e.g., Wang et al., 2017).

This does not mean that data-driven estimator selection is inherently ineffective. Rather, its theoretical justification requires some structural assumptions under which the selection rule can be learned reliably. For example, Su et al. (2020) exploit a structured bias-variance trade-off to establish an oracle-type guarantee for adaptive estimator selection. In contrast, PerOPE does not appear to identify comparable structural assumptions or provide a theoretical guarantee that accounts for the statistical cost of learning its context-dependent aggregation rule.

In particular, Theorem 2 (Non-negative Oracle Improvement) is somewhat misleading as a theoretical justification for the proposed method. It merely observes that the class of context-dependent aggregations contains the class of context-independent aggregations, so the optimal MSE over the former cannot be larger than that over the latter. However, this is an oracle-level comparison that assumes knowledge of the true MSE-minimizing aggregation rule, which is precisely what the proposed method attempts to learn from data. The theorem therefore provides no guarantee that the learned estimator actually achieves a lower MSE, nor does it quantify the additional estimation error introduced by personalization.

While the experimental results are encouraging, they cannot establish a general improvement across data-generating distributions. Without a theoretical characterization of when personalization is beneficial, or an oracle inequality accounting for the cost of learning the aggregation rule, I find the central claim insufficiently justified. The paper would be substantially stronger if it clarified the assumptions under which personalization provides a genuine statistical advantage, rather than relying primarily on empirical improvements.

Questions for authors

See the weaknesses above.

2 comments

We sincerely thank the reviewer for the detailed theoretical feedback and for recognizing the motivation and potential value of personalized estimator aggregation. We appreciate the opportunity to further clarify the scope of our theoretical guarantees and the empirical evidence supporting our approach.

1. What does the theory establish?

We appreciate this theoretical comment and agree with its central distinction.

Theorem 2 is an oracle-level hypothesis-class comparison, not a statement that the learned PerOPE estimator uniformly dominates conventional OPE methods. Its role is to identify the representational advantage of allowing context-dependent weights. It does not account for the additional error incurred when learning a more flexible aggregation rule from finite data.

A more informative analysis separates oracle approximation error from estimation and optimization errors. For example, writing R(T)R(T) for true MSE and R^(T)\hat{R}(T) for estimated MSE, suppose a personalized estimator class TKT_K satisfies, with high probability,

sup⁡T∈TK∣R^(T)−R(T)∣≤ε(n,K)\sup_{T \in \mathcal{T}_K} \left| \widehat{R}(T) - R(T) \right| \leq \varepsilon(n,K)

If the learned estimator approximately minimizes R^(T)\hat{R}(T) with optimization error δopt\delta_{\mathrm{opt}}, then

R(T^K)≤inf⁡T∈TKR(T)+2ε(n,K)+δoptR(\widehat{T}_K) \leq \inf_{T \in \mathcal{T}_K} R(T) + 2\varepsilon(n,K) + \delta_{\mathrm{opt}}

This inequality illustrates the relevant trade-off. A larger tree class may reduce the best achievable MSE but increase the statistical error associated with selecting an estimator. It does not imply improvement over the best global estimator, and establishing an explicit bound for ε(n,K) requires additional assumptions on the MSE-estimation module, data dependence, and tree complexity.

2. When should personalization be beneficial?

We thank the reviewer for raising this question. PerOPE is particularly beneficial when the relative accuracy of candidate estimators varies across contexts, allowing personalized aggregation to exploit heterogeneity that global methods cannot capture. Theorem 2 theoretically supports this advantage at the oracle level.

Our experiments already provide empirical evidence of when personalization is effective. The synthetic experiments demonstrate that PerOPE's advantage generally increases with contextual heterogeneity, while Appendix E.1 and E.2 further examine robustness to tree depth and sample size. Importantly, these improvements are achieved without careful hyperparameter tuning.

Furthermore, our real-world experiments also demonstrate the practical benefits of personalization, with PerOPE consistently outperforming its context-independent counterparts across multiple evaluation scenarios. These findings provide empirical evidence that meaningful contextual heterogeneity exists in real-world OPE settings and that exploiting such heterogeneity through personalization can indeed improve estimation accuracy.

Our contribution does not require universal improvement under every possible data-generating distribution. Rather, our theoretical analysis and extensive experiments demonstrate both the potential and practical benefits of contextual personalization, which we believe sufficiently support our central claims.

We would like to kindly inform the reviewer that we have submitted a revised manuscript, with all changes highlighted in red, directly addressing the key theoretical concerns. In particular, we have clarified the oracle-level interpretation of Theorem 2, explicitly distinguished it from finite-sample performance, and strengthened the discussion connecting our theoretical results to the empirical evaluation.

We would sincerely appreciate any remaining theoretical questions or objections before the discussion deadline. If no further concerns are raised, we believe it is reasonable to conclude that the central theoretical issues identified in the review have been adequately addressed, and we hope this will be reflected in the final evaluation.

Reviewer_xUGKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a weighting scheme for aggregating multiple estimators at a context level. The aggregation relies on a tree-based learning algorithm that assigns contexts to a weighting scheme of candidate estimators. Candidate aggregation schemes are learned jointly with the context assignments, to minimise an estimate of the total MSE.

Critical strengths
  • The idea is simple, interesting and practically valuable.
  • The paper is generally well-written and understandable
Critical weaknesses
  • The learning algorithm deserves a more in-depth explanation in the main text. Even the appendix does not clarify enough how the learning is performed.
  • The paper does not report experiments on a publicly available real-world dataset.
  • The method relies on some learning components.
  • The paper does not demonstrate that the proposed method is better than single accurate estimators.
  • A hyperparameter tuning procedure is employed for PerOPE, but the paper does not clarify which metric is monitored and on which data is computed.
  • There is still the problem of selecting a good MSE estimation algorithm, which is again an estimator selection problem.

Minor. Missing citation: Automated Off-Policy Estimator Selection via Supervised Learning - Felicioni et al.

Questions for authors
  • Can you add a formal proof of Theorem 2?
  • Why did you write "Our candidate set, however, contains biased estimators such as DM, SNIPS, and SNDR"?
  • How does the aggregated estimator compare to the single estimators DM, SNIPS, SNDR, DR, IPS?
  • How does the aggregated estimator compare to other baselines such as PASIF and SLOPE applied to general estimator selection (see Cief et al. (2025))?
  • How would you select a good MSE estimation algorithm in a real scenario?
  • Does the hyperparameter search involve a validation set?
4 comments

We thank the reviewer for the careful evaluation of our work and for recognizing the simplicity and flexibility of the proposed framework. We appreciate the opportunity to clarify our methodological contributions and highlight the theoretical and empirical evidence already provided in the paper.

1. Clarifying the learning algorithm

We thank the reviewer for the suggestion to clarify the learning procedure. We would like to emphasize that the learning procedure of PerOPE is already described in Appendix C (Algorithm 1), with the corresponding implementation settings provided in Section 5.1 and Appendix D.2. Specifically, Algorithm 1 describes the recursive tree construction, including context partitioning, MSE-based split evaluation, and the optimization of estimator aggregation weights within each segment. Section 5.1 further specifies that we use Optuna with 50 trials to search for the best split at each node, while Appendix D.2 provides the configurations of the underlying MSE-estimation procedures, including 10-fold cross-validation for OCV-based variants and bootstrap sampling for OPERA-based variants.

Given the strict space constraints of the main text, we deliberately placed the detailed algorithmic description and implementation settings in the appendix, while keeping the main text focused on the central methodological ideas. Furthermore, we have made the implementation of our synthetic experiments available, allowing the learning procedure and experimental results to be directly reproduced. Therefore, we believe that the algorithm is sufficiently documented and reproducible, and that the reviewer's concern primarily relates to the organization of the presentation rather than a lack of methodological clarity or reproducibility.

2. Why include biased estimators?

The inclusion of DM, SNIPS, and SNDR is intentional. The practical objective is to minimize MSE, not to restrict the candidate class to unbiased estimators. It is widely known that a biased estimator can be preferable when its variance reduction more than compensates for its bias, and mixtures of biased and unbiased contributions can further improve the resulting trade-off.

It is important to distinguish between the theoretical guarantees and the practical applicability of PerOPE. The primary purpose of Theorem 1 is to demonstrate that introducing contextual personalization does not compromise the unbiasedness of the base estimators, provided that each base contribution is conditionally unbiased and the aggregation rule satisfies the stated independence assumptions. In other words, PerOPE preserves unbiasedness when aggregating conditionally unbiased estimators under these conditions. While these conditions do not necessarily hold for all candidate estimators, including DM, SNIPS, and SNDR, it is important to emphasize that PerOPE is designed to minimize MSE rather than guarantee unbiasedness for arbitrary candidate sets. Incorporating potentially biased estimators can improve the bias–variance trade-off and ultimately reduce estimation error.

3. Comparison with individual estimators and additional selection methods

We thank the reviewer for the suggestion. However, we would like to emphasize that the primary contribution of PerOPE is not a new OPE estimator, but a general framework for improving existing estimator selection and aggregation methods through contextual personalization. Therefore, the most relevant baselines for evaluating our contribution are existing selection and aggregation methods, rather than individual estimators such as IPS, DM, or DR.

In particular, OCV and OPERA already operate on a candidate set of individual estimators, selecting or aggregating them in a data-driven manner. By comparing each PerOPE variant with its corresponding context-independent baseline using the same candidate estimators and underlying MSE-estimation approach, our experiments directly isolate the benefit of personalization. We believe this controlled comparison provides more meaningful evidence for our central contribution than comparisons with individual estimators alone.

Moreover, the motivation for estimator selection and aggregation is precisely that no single estimator is universally optimal across different OPE settings. Comparing against the best individual estimator selected retrospectively using the true MSE would introduce an oracle advantage unavailable in practical OPE. Our experiments instead evaluate whether personalization improves upon established methods that automatically select or aggregate estimators from logged data.

Regarding additional baselines, PAS-IF was already compared with OCV in the original OCV paper, where OCV demonstrated superior performance, motivating our focus on the more recent OCV baseline. Meanwhile, SLOPE is already included in our hyperparameter-selection experiments in the appendix, where its comparison is most relevant.

We therefore believe that our comparisons with OCV and OPERA are well aligned with the paper's research objective and provide direct empirical evidence for the practical value of contextual personalization. While further comparisons may offer additional context, our existing evaluations already cover representative estimator selection, aggregation, and hyperparameter-selection methods, sufficiently supporting our central contribution.

4. Proof of Theorem 2

The proof follows from set inclusion. Any global aggregation can be represented as a personalized aggregation with constant weights across all contexts. Therefore,

Vno-per⊆VPerOPE\mathcal{V}_{\mathrm{no\text{-}per}} \subseteq \mathcal{V}_{\mathrm{PerOPE}}

and minimizing true MSE over the larger class cannot yield a worse oracle minimum.

We agree that the theorem is elementary and should not be interpreted as a guarantee for the data-driven learning algorithm. We have empirically demonstrated the finite-sample performance of the proposed algorithm in the experimental sections.

5. Real-world MSE estimation, validation, and reproducibility

We thank the reviewer for raising these points. We would like to emphasize that our real-world experiment is carefully designed to provide a meaningful and controlled evaluation using a substantial dataset of 204,191 observations from an actual production A/B test. The dataset contains observations collected under three distinct policies, with sample sizes of 87,878, 89,193, and 27,120. This large-scale dataset provides a substantial empirical basis for evaluation, while the three distinct policies allow us to assess PerOPE across diverse combinations of logging and evaluation policies, rather than relying on a single evaluation setting. Specifically, for each scenario, we use the on-policy estimate from one policy as a reference and perform OPE using data collected under the other two policies. Importantly, all compared methods are evaluated against the same on-policy reference under identical experimental conditions, ensuring a controlled and fair comparison of their relative estimation accuracy. We also report 95% confidence intervals to quantify estimation uncertainty.

Regarding validation, the underlying MSE-estimation procedures already incorporate cross-validation or bootstrap-based estimation, as detailed in Appendix D.2. Moreover, we deliberately use a fixed tree depth rather than tuning it separately for each experimental scenario, demonstrating the practical effectiveness of PerOPE without extensive hyperparameter optimization.

Regarding reproducibility, the production dataset contains proprietary user-interaction data and cannot be publicly released due to confidentiality constraints. This is a common practical restriction in research involving real-world production systems and should not be confused with a methodological limitation. Importantly, we have already shared the code for our synthetic experiments, which uses the same core implementation of PerOPE and the baseline methods as our real-world experiments. Thus, although the production data cannot be disclosed, the underlying algorithms and experimental procedures can be independently inspected and reproduced using the publicly available synthetic implementation. This provides an important level of transparency and reproducibility beyond merely reporting results on a proprietary dataset.

We therefore believe that the carefully controlled real-world comparisons, together with the shared implementation used across synthetic and production experiments, provide sufficient empirical evidence and methodological transparency to support our central claim: contextual personalization can improve existing OPE selection and aggregation methods in both synthetic and real-world settings.

We would like to kindly remind the reviewer that we have submitted a revised manuscript, with all changes highlighted in red, addressing the key comments in the review. These revisions further clarify our learning algorithm, theoretical guarantees, baseline comparisons, and real-world evaluation, including the rationale for PAS-IF and the existing SLOPE comparisons in the appendix.

We would appreciate any remaining questions or specific objections before the discussion deadline, so that we have an opportunity to address them. Otherwise, we believe the substantive concerns have been satisfactorily resolved and hope that the reviewer will reflect these clarifications and improvements in their final assessment.

Reviewer_EHFvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses estimator selection for off-policy evaluation in contextual bandits. Existing methods make one choice for the whole dataset: OCV selects a single estimator by cross-validation, and OPERA learns one set of weights for averaging several estimators. The authors argue that the best estimator can differ across users, for example because the evaluation policy diverges from the logging policy in some user segments and not others.

Their method, PerOPE, uses the fact that standard estimators (IPS, DM, DR, SNIPS, SNDR) are averages of per-sample terms, so each sample's term can be computed with a different blend of estimators. A decision tree on user features partitions the data, and each leaf learns its own blend. Since the true error is unknown, splits and blends are chosen by minimising the error estimate of an existing method (OCV or OPERA), so PerOPE wraps a baseline rather than replacing it.

The theory shows that the blend is unbiased when its inputs are unbiased and the weights are fit independently of the evaluation data, and that with known errors the best per-leaf blend is never worse than the best global one. In the synthetic experiments the evaluation policy's inverse temperature depends on user features; at the strongest dependence, PerOPE reduces MSE from about 2.8e-3 to 1.8e-3 over OCV-IPS and from 9.5e-3 to 2.2e-3 over OPERA (Fig. 6). On a three-arm email A/B test with about 204,000 users, each arm evaluated from the other two, PerOPE has lower point-estimate MSE than its baseline in all three cases. The authors conclude that personalized aggregation consistently improves on existing selection and aggregation methods.

Critical strengths

Flexible framework for OPE. Standard OPE estimators are averages of per-sample terms, so the choice or blend of estimators does not have to be the same for every user. PerOPE generalizes both single-estimator selection (OCV) and global aggregation (OPERA), which are recovered as a tree of depth zero. I find the most useful way to read this is as a control knob for the bias-variance trade-off in the selection step itself: depth zero recovers OCV or OPERA, and deeper trees allow increasingly local choices at the cost of judging each choice on less data.

Modular and interpretable. PerOPE wraps any existing error estimator rather than replacing it, and each variant is compared with the baseline that uses the same estimator, which makes the comparisons controlled. The learned tree can be inspected, and the case study in Fig. 4 shows segments that line up with how far the evaluation policy departs from the logging policy.

Critical weaknesses

1. No systematic way to set the depth. Tree depth controls the trade-off between local adaptation and the reliability of each local choice, but the paper gives no rule or analysis for choosing it. In OPE this is hard, because there is no ground truth to validate against, so the depth has to be chosen with the same noisy error estimate that deeper trees overfit to. For the OCV variants, error is lowest at K = 1-2 and rises with depth while the main experiments use K = 4 (Fig. 7), and the gain over OCV is gone by n = 2000 (Fig. 8). At K = 4 and n = 1000 each leaf holds roughly 60 samples, which are still split 10 ways for cross-validation.

2. The experiments do not yet establish a practical benefit. The synthetic experiments are a useful illustration but a toy case. The context dependence comes from an evaluation policy whose inverse temperature goes strongly negative for many users (about -73 in Fig. 4), so it deliberately picks the worst actions, which is where per-segment switching pays off most. The largest synthetic gains are also over OPERA, which performs poorly in this setup. The real-world experiment is meant to close this gap, but its results are directional rather than definitive. In two of three scenarios the differences between methods are around 1e-6 with overlapping error bars, while the A/B reference has noise of a similar size. Key details (click rate, CI construction, propensities) are missing, and the data and code are not available. In the one scenario with a large gap (j = 3), the choice of reference estimator in OCV changes the error about 400-fold, against about 2-3-fold from personalization, which suggests the method's practical impact is modest next to other configuration choices.

3. The theory does not cover the method as run. Theorem 1 requires unbiased base estimators and weights chosen independently of the evaluation data, but three of the five candidates are biased and Algorithm 1 fits and evaluates on the same data. Theorem 2 is an oracle statement that holds because global blends are a special case of per-leaf ones. It does not address the finite-sample trade-off in weakness 1.

Questions for authors
  1. Real-world details. What is the click rate per arm? How are the 95% CIs in Fig. 5 computed from a single dataset, how noisy is the on-policy reference, and how are propensities and support handled when the logged data mixes two arms?
  2. Release. Will the code, the experimental details (including the split-search module and the Optuna search space) and the data, or an anonymised or synthetic version of it, be released?
  3. The gain at zero context dependence. The largest OPERA-variant gain appears at βcons=5\beta_{cons} = 5, βcoef=0\beta_{coef} = 0, where the evaluation policy does not depend on the context, and it nearly disappears at βcoef=2.5\beta_{coef} = 2.5. How do the authors explain this, given that the gains are attributed to context heterogeneity?
  4. The error estimate inside small leaves. How reliable is the OCV estimate on about 60 samples per leaf, and what is the effect of replacing OCV's per-estimator train/validation ratio with a single averaged ratio (App. D.2)?
6 comments

We sincerely thank the reviewer for the thoughtful and constructive feedback, and for recognizing the practical relevance and potential benefits of context-dependent estimator aggregation. We appreciate the opportunity to further clarify the robustness and practical effectiveness of PerOPE.

1. Choosing tree depth and avoiding overfitting

We respectfully disagree that the choice of tree depth constitutes a significant limitation of PerOPE. We have already investigated the sensitivity to tree depth in Appendix E.1 (Figure 7), where we evaluate depths K = 1,...,6. The results demonstrate that PerOPE achieves substantial improvements across multiple depth choices, including relatively shallow trees, indicating that its effectiveness does not critically depend on selecting a particular depth. Although the optimal depth may vary depending on the underlying MSE-estimation procedure, this is a natural characteristic of tree-based methods rather than a fundamental weakness of PerOPE.

More importantly, we deliberately use a fixed depth K throughout the main experiments, rather than tuning it separately for each experimental setting. Our objective is not to maximize performance through extensive hyperparameter tuning, but to demonstrate that PerOPE can consistently outperform existing estimator selection and aggregation methods even with a simple, fixed configuration. We believe this is a stronger demonstration of practical utility, as it shows that the benefits of personalized aggregation can be realized without introducing an additional hyperparameter optimization burden.

Therefore, the existing depth-sensitivity analysis, together with the consistent improvements under a fixed configuration, provides empirical evidence that PerOPE is practically robust to the choice of tree depth. While data-driven depth selection could potentially yield further improvements, such tuning is not necessary to obtain the substantial gains already demonstrated in our experiments.

2. Reliability of the real-world experiment

We thank the reviewer for the suggestion to provide additional details on the real-world evaluation. As described in Section 6, our experiment uses a production email-marketing A/B test with a substantial dataset of 204,191 observations collected under three distinct policies, with sample sizes of 87,878, 89,193, and 27,120. The large sample size provides a substantial empirical basis for evaluation, while the three distinct policies enable us to assess PerOPE across diverse combinations of logging and evaluation policies. For each evaluation scenario, one policy provides an on-policy reference, while the data collected under the other two policies are used for OPE. Importantly, all methods are evaluated using the same logged data and on-policy reference within each scenario, ensuring a controlled and fair comparison of their relative estimation accuracy. We also report 95% confidence intervals to quantify uncertainty.

We appreciate the reviewer's interest in additional details regarding policy assignment, logging propensities, and the construction of the on-policy reference, and will clarify these aspects where possible. However, we emphasize that these requests primarily concern supplementary experimental details rather than fundamental weaknesses in our methodology or empirical findings. The consistent improvements observed across multiple evaluation policies provide meaningful evidence of the practical effectiveness of PerOPE. We therefore believe that these concerns do not substantially weaken our central contribution or the conclusions supported by our experiments.

3. Why does personalization help when \beta_coef = 0?

This is an important point that deserves clarification. Setting βcoef=0\beta_coef = 0 makes the evaluation policy's inverse-temperature parameter constant, but it does not make the evaluation policy itself context-independent. Specifically, πe(a∣x)∝exp⁡(βconsq(x,a)),\pi_e(a \mid x) \propto \exp\left(\beta_{\mathrm{cons}} q(x,a)\right),, and q(x,a) varies with the context through the context and context–action interaction terms.

Thus, even when βcoef=0\beta_{coef} = 0, the action distribution, logging/evaluation overlap, and reward-estimation difficulty may differ across contexts. Consequently, PerOPE can retain its advantage by exploiting this contextual heterogeneity, even in this setting.

4. Reliability of MSE estimation in small leaves

We thank the reviewer for raising this interesting point. In principle, smaller leaves may lead to higher variance in local MSE estimation. However, this issue can be readily mitigated by controlling standard tree hyperparameters, such as min_samples_leaf and maximum tree depth, which prevent excessively small partitions and help maintain reliable estimation within each leaf.

More importantly, as emphasized above, our experiments already demonstrate that PerOPE consistently outperforms its non-personalized counterparts across a wide range of settings without requiring careful tuning of these hyperparameters. In particular, we deliberately use a fixed tree depth throughout the main experiments, and our additional depth-sensitivity analysis in Appendix E.1 (Figure 7) further demonstrates that the improvements are not critically dependent on a particular depth choice. These results suggest that the potential increase in estimation variance due to partitioning does not undermine the practical benefits of personalized aggregation.

Therefore, while further tuning of tree complexity could improve estimation stability, our empirical findings demonstrate that PerOPE achieves substantial gains even without such additional hyper-parameter tuning, highlighting its practical robustness and ease of use.

We would like to kindly remind the reviewer that we have now submitted a revised manuscript, with all changes highlighted in red, addressing the key concerns raised in the review. In particular, we have further clarified the robustness to tree depth, the design and scale of our real-world evaluation, and the role of contextual heterogeneity.

We would greatly appreciate it if the reviewer could examine these revisions and let us know before the discussion deadline if any substantive concerns remain. In the absence of further objections, we believe it is reasonable to regard the key issues raised in the review as satisfactorily addressed, and we hope that this will be reflected in the reviewer's final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.