acceptodds
Withdrawn from ICLR 2027

The Price of Hidden Assignments: Calibration and Minimax Rates in Pooled CATE Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

How accurately can the conditional average treatment effect (CATE) be estimated from noisy pooled treatment reports? We study randomized experiments with fixed-size pools, linked individual covariates and outcomes, hidden treatment assignments, known assignment probabilities, and a common unknown nondifferential reporting channel. Validation of uniformly selected pools reveals their true OR states: whether any member received treatment. Under fixed overlap and channel bounds, pooling with a known channel preserves the usual CATE minimax rate. For an unknown channel, assuming smoothness only of the CATE, we establish matching classwise bounds for expected integrated squared risk, of order , where , is CATE Hölder smoothness, is covariate dimension, and is the class-specified propensity-variance lower bound.The two terms quantify the regression error and the additional calibration cost.Validation and propensity heterogeneity jointly determine the calibration information scale , yielding distinct validation scales for improving on the no-validation minimax rate and recovering the known-channel rate. The lower bound retains treatment-effect information in complete validation records. An upper-bound construction nevertheless attains the same minimax rate using only covariates, true OR states, and noisy reports from a reserved validation subset, without its outcomes, and estimates CATE from the remaining pools. Finite-budget diagnostics show that qualification checks and clipping can suppress calibration gains in this construction. Across all 90 prespecified settings, its mean risk exceeded that of the zero-effect estimator.

Primary area:
causal reasoning
Keywords:
conditional average treatment effect, causal inference, pooled treatment reports, minimax rates, reporting channel calibration, validation data

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ASQVOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies CATE estimation when individual treatment assignments are hidden and only noisy pooled treatment reports are available. The main contribution is a characterization of the minimax rate for CATE estimation, consisting of the usual nonparametric regression rate and an additional term accounting for uncertainty in the reporting mechanism. The authors characterize how validation and propensity score variation affect this rate and construct a rate-optimal estimator. In simulations, the proposed estimator performs worse than the zero-effect estimator across all 90 settings.

Critical strengths

None

Critical weaknesses

The paper is poorly written, and the problem is not clearly explained or motivated. I do not find the quality of the paper suitable for ICLR.

My main concerns are the following:

  1. The observation model is poorly motivated and explained. Near the bottom of page 2, the authors state that each pool provides a report SS on its true OR status T=max⁡iAiT=\max_i A_i. Thus, TT indicates whether anyone in the pool received treatment, while SS is a noisy measurement of this quantity. Why is this particular observation model of interest? The authors should provide a concrete example of when such data would arise. Moreover, sensitivity ss and specificity zz are introduced without explicitly defining them in terms of conditional probabilities. Similarly, the paper repeatedly refers to "channel calibration" without clearly defining what this means. Presumably, it refers to estimating the unknown reporting mechanism, but this should be explicitly stated. Given that these concepts are central to the paper, the lack of clarity in their presentation is particularly concerning.

  2. The main theoretical result has very limited scope. The authors acknowledge in Section 4 that validation can improve the minimax convergence rate only when d=1d=1 and β>1/2\beta>1/2, with sufficiently small propensity variation. For d≥2d\geq 2, the standard CATE regression rate is already attained without validation. This substantially limits the significance of the main result.

  3. The proposed estimator performs worse than the trivial zero estimator in every simulation setting. Section 6 reports that the estimator has higher mean squared risk than simply estimating τ=0\tau=0 across all 90 settings. This seriously undermines the practical relevance of the methodology. The paper also lacks convincing comparisons with simpler estimators.

Questions for authors

None.

Reviewer_q3oTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper establishes matching minimax bounds for the expected integrated squared risk of CATE estimation when individual treatment assignments are hidden, individual covariates and outcomes remain observed, and each group provides a noisy OR report indicating whether any member received treatment. Validation reveals the true OR status for a subset of groups. The bounds separate regression error from the additional cost of an unknown reporting mechanism and show how validation and propensity-score variation jointly reduce this cost. Under suitable conditions, the known-channel regression rate can be recovered by validating only a vanishing fraction of groups. The authors also bound finite-sample minimax risk in a specific scalar model. However, across all tested Monte Carlo settings, their rate-optimal estimator has higher mean integrated squared risk than the zero-effect estimator.

Critical strengths

The paper establishes matching minimax upper and lower bounds for CATE estimation from noisy group-level OR treatment reports. The bounds separate regression error from the additional cost of an unknown reporting mechanism and characterize how validation and propensity-score variation jointly reduce the risk. They also distinguish the validation requirements for improving the convergence rate from those for recovering the known-channel rate. The results require only CATE smoothness, allowing the treatment-specific outcome regression functions to be nonsmooth.

Critical weaknesses
  1. Although Proposition 1 establishes matching minimax rates, it does not provide explicit, practically informative constants for the general CATE model. Consequently, the result gives limited guidance on achievable accuracy or validation sample sizes at realistic budgets. This limitation is particularly relevant because the proposed rate-optimal estimator has higher mean risk than the zero-effect estimator across all tested Monte Carlo settings.
  2. The paper motivates its observation model through restricted data release but does not provide a concrete real-world application demonstrating this combination of conditions: linked individual covariates and outcomes, hidden treatment assignments, noisy group-level OR reports, and access to error-free OR validation. A worked example explaining how these reports arise, why OR summaries are released, and how validation is performed would strengthen the practical motivation.
Questions for authors
  1. Are there any real examples that motivated this study with noisy group-level OR status?
Reviewer_DYBHOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how much accuracy is lost in CATE estimation when treatment reports are pooled and the reporting error is unknown. The authors derive matching upper and lower bounds on the minimax risk and show that: (i) pooling need not slow the convergence rate. (ii) Both validation data and variation in treatment probabilities help correct for reporting errors. The paper also constructs an estimator that attains the minimax rate, although it does not perform well in the experiments.

Critical strengths
  • The theoretical derivations appear to be correct.
  • The authors are transparent about the limitations of their minimax estimator and state them clearly in the abstract.
Critical weaknesses
  • Insufficient practical motivation for the observation model. If the treatment assignment mechanism is known and controlled, it is unclear why the individual treatment assignments would be unavailable and why pooled, error-prone treatment reports would be the only accessible information. The paper does not give a concrete application in which these restrictions arise, nor does it explain why estimating the CATE under them is an important practical task.
  • Weak empirical performance. In all simulation settings, including the stronger-effect scenario, the proposed estimator has higher mean risk than the trivial estimator that always predicts zero. The experiments therefore provide no evidence of practical usefulness.
  • Limited significance of the validation results. As acknowledged in Section 4.2, validation improves the overall minimax rate only when 2β>d2\beta > d. Because the paper restricts β≤1\beta \le 1, this condition can hold only when d=1d = 1 and β>1/2\beta > 1/2. Moreover, a meaningful rate improvement additionally requires the variance of the propensity score to be sufficiently small.
  • Strong assumption on the reporting mechanism. The theoretical guarantees rely on the reporting error depending only on the true treatment assignment and being independent of the outcomes and covariates (i.e., non-differential misclassification). This is a strong assumption, and the paper should justify it and discuss how sensitive the results are to its violation.
  • Presentation. The formatting is inconsistent in several places; for example, the equations in Appendix L are written as plain-text pseudo-code rather than typeset mathematics.
Questions for authors
  • What motivates the setting in which treatment reports are pooled and subject to unknown reporting error (see W1)? Could the authors provide a concrete application in which these constraints arise and the CATE is still worth estimating?
  • Could the authors include more meaningful baselines? The paper derives a validation-only CATE estimator and discusses plug-in calibration approaches; both are natural alternatives under the same observation model and would make the empirical comparison more informative.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.