acceptodds
Under review as a conference paper at ICLR 2027

Targeted Active Learning for Preference-Based Treatment Effects on Multivariate Outcomes

Abstract

Treatment efficacy is traditionally demonstrated on the basis of a single primary outcome. However, clinical decision-making usually requires consideration of multiple outcomes, balancing expected benefits against potential risks. The relative value assigned to these outcomes varies substantially from one patient to another. Given a preference rule over outcome profiles, treatment effects and optimal policies can be defined and estimated. Such a rule is rarely available in practice: it must itself be estimated from pairwise comparisons of outcome profiles, which are costly to collect from clinical experts. We propose an active learning framework that selects which comparisons to query. Standard criteria maximize the information gained on the preference rule itself. We instead target the quantities of interest, and select the query that most reduces uncertainty on the treatment effect and on the optimal policy induced by the learned rule. Under a Gaussian process model of the preference rule, we derive a closed-form approximation of this criterion. On semi-synthetic data built from a Parkinson's disease cohort with 13 clinical outcomes, our criterion achieves lower treatment effect estimation error and lower policy regret than existing criteria at equal query budget.

Primary area:
causal reasoning
Keywords:
active learning, preference learning, causal inference, multivariate outcome, treatment effect

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_xPEPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces CPTE-EIG, an active learning criterion for preference elicitation that targets a causal estimand: the Conditional Preference Treatment Effect (CPTE). The CPTE is the probability that a treated outcome is preferred to a control outcome for a patient with covariate x, where preferences are elicited from clinicians through pairwise comparisons of multivariate outcome profiles. The authors model the latent utility with a Gaussian process, linearise the CPTE around the posterior mean utility to obtain a tractable Gaussian approximation with an error bound, and derive a closed-form expected information gain (up to one-dimensional Gauss–Hermite quadrature). Experiments on a semi-synthetic Parkinson’s disease cohort with 13 outcomes show that CPTE-EIG reduces CPTE estimation error and policy regret compared to BALD, EPIG, and random acquisition, especially under mild and strong treatment effects.

Critical strengths

1.Important and well-motivated problem. Clinical decisions often require trading off multiple outcomes, and eliciting preferences is costly. Targeting the causal estimand directly, rather than the preference function or individual predictions, is a principled use of limited clinician time.

2.Novel methodological contribution. To my knowledge, this is the first active learning method for preference elicitation that targets a causal estimand. The formulation of the CPTE as a posterior random variable and the linearisation with an explicit error bound are technically sound.

3.Computational efficiency. The closed-form CPTE-EIG expression, requiring only Gauss–Hermite quadrature, avoids posterior sampling and model refitting per candidate. The ablation in Appendix F.7 confirms that few quadrature nodes match Monte Carlo with thousands of samples.

Critical weaknesses

1.Candidate pool construction. Queries are restricted to pairs of real PPMI profiles sharing the same query covariate. This is realistic but may limit the diversity of comparisons. The impact of this restriction on the learned preference rule is not analyzed.

2.Outcome dimensionality. The experiments use 13 outcomes. The authors note that profiles with too many dimensions may be hard to compare in a single judgment. Cognitive load and clinician reliability with 13-dimensional profiles are not studied.

3.Baseline scope. BALD, EPIG, and random are reasonable baselines, but the paper does not compare to a method that directly models the CPTE or to a non-active elicitation strategy (e.g., balanced sampling). The advantage of targeting the estimand is clear, but the practical gain over simpler heuristics could be more thoroughly explored.

4.Real-world validation absent. The method is evaluated only on semi-synthetic data with a simulated clinician. While this allows oracle evaluation, it does not demonstrate that the approach works with real clinicians and real preference noise.

Questions for authors

Please see weakness

2 comments

We sincerely thank the reviewer for the careful reading and constructive comments, which we answer below.

  1. Queries are restricted to pairs of real PPMI profiles sharing the same query covariate. This is realistic but may limit the diversity of comparisons. The impact of this restriction on the learned preference rule is not analyzed.

This choice comes from the clinicians we work with: they can only judge reliably profiles that look like real patients (Section 3). We first tried generating queries by continuous optimisation over [0,4]13[0,4]^{13}, but this produced non-integer ordinal scores and clinically unrealistic profiles. A shared query covariate also matches the estimand, since the CPTE only involves contrasts between treated and control profiles at the same x~\tilde x (Eq. 7, Appendix F.2). The pool remains diverse, with 1,369,284 comparisons over all ages and both sexes (Appendix F.1). We will add this discussion to Section 3.

  1. The experiments use 13 outcomes. The authors note that profiles with too many dimensions may be hard to compare in a single judgment. Cognitive load and clinician reliability with 13-dimensional profiles are not studied

The 13 items were chosen with clinicians as the dimensions they weigh in practice when choosing a treatment (Table 1), and removing some would make comparisons less faithful. We agree that cognitive load and rater reliability must be evaluated (Section 6). Several elements help:

  • Real profile pairs often share many item values, so the interface can highlight only the items that differ, grouped by clinical domain.
  • Reliability can be measured with repeated queries (test–retest) and with queries where one profile clearly dominates.
  • The noise scale λ, fitted at every round (Appendix F.3), absorbs part of the inconsistency.
  • CPTE-EIG reduces the number of queries, and hence fatigue.

We added this to Section 6 and build these checks into our elicitation software.

  1. BALD, EPIG, and random are reasonable baselines, but the paper does not compare to a method that directly models the CPTE or to a non-active elicitation strategy (e.g., balanced sampling). The advantage of targeting the estimand is clear, but the practical gain over simpler heuristics could be more thoroughly explored.

We added two baselines that directly target the CPTE: CPTE variance reduction and decision EIG (which targets 1(Ψx>1/2)\mathbb 1(\Psi_x>1/2)), with the same Gaussian approximation as CPTE-EIG (Appendix F.3). They are close to CPTE-EIG (Figures 2–3, Table S1), so most of the gain comes from targeting the CPTE. CPTE-EIG is still significantly better in some small-budget settings. For example, against decision EIG on the strong effect at 40 comparisons, its regret is 26.1 vs. 36.2×10⁻³ (Appendix F.5). It is significantly worse only with the tiny effect at large budgets, where all methods are already accurate (Appendix F.5).

We also tested a non-active balanced sampling strategy, which queries all patient groups (sex × age × disease duration) equally often. It was significantly worse than CPTE-EIG in every setting, and often no better than random. Since it brought no information beyond the random baseline, we did not include it in the paper. Simple heuristics of this kind spread the queries over the whole population, while the gain comes from targeting the CPTE.

  1. The method is evaluated only on semi-synthetic data with a simulated clinician. While this allows oracle evaluation, it does not demonstrate that the approach works with real clinicians and real preference noise.

We fully agree that a study with real clinicians is the natural next step, and it motivated this work: clinician time is scarce, so real elicitation needs a strategy that learns the effect from few questions. Real clinicians cannot provide an oracle CPTE, so controlled experiments are needed first to validate the method.

To make these experiments more realistic, we added LLM clinicians (Section 5.2, Figure 3). Each LLM sees the patients' sex, age group, disease duration and 13 outcomes in words, but never the treatments. We use BioDecision-v2-4B [1], and Qwen3-32B [2] with a neuropsychiatric and a motor persona (prompts in Appendix F.2). Their answers are not generated by our model and are noisy. The Qwen3 answers depend on the presentation order in 31–36% of comparisons, and the two personas disagree on 7.4% of them (Appendix F.2, Figure S1).

CPTE-EIG has the lowest mean CPTE error from 40 comparisons on with BioDecision and the neuropsychiatric persona, and at 40–60 comparisons with the motor persona (Figure 3). Over 18 threshold and budget settings, it also has the lowest regret among CPTE-EIG, EPIG, BALD and random in 17 (Appendix F.6). We plan to evaluate it with real clinicians next.

We thank the reviewer again for these helpful suggestions and would be glad to clarify any remaining point.

[1] https://huggingface.co/lighteternal/biodecision-v2-4b [2] https://huggingface.co/Qwen/Qwen3-32B

We have updated our replies above so that the appendix references match the latest version of the paper. We would be happy to discuss any point that remains unclear.

Reviewer_7Mm9OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CPTE-EIG, an AL method for estimating preference-based treatment effects on multivariate outcomes. Instead of learning the preference function everywhere, the proposed method selects pairwise comparisons that are informative about the target CPTE. The authors use a GP preference model and linearize the CPTE functional to derive an efficient approximation of EIG. Experiments on semi-synthetic Parkinson's disease data show improvements over BALD, EPIG, and random sampling.

Critical strengths
  • The problem formulation is quite well motivated. Active preference elicitation for multivariate treatment effects is a reasonable and real-world setting and the paper is generally well written.

  • The derivation of a tractable EIG under the GP preference model is a useful technical contribution. The authors also provide a bound for the linearization error.

  • The experiments show clear improvements in CPTE estimation and policy regret, especially in the mild and strong treatment effect settings.

Critical weaknesses
  • The core idea is quite straightforward, since estimand-targeted EIG has already been studied in Causal-EPIG. The new preference setting adds some novelty, but overall the contribution isn't very exciting.
  • The paper gives an error bound for the linearized CPTE, but not for the approximated EIG. I would really like to see whether this approximation actually affects query selection in practice.
  • Since the paper mainly evaluates CPTE MSE, I feel like CPTE variance reduction is a very natural baseline. It directly corresponds to Bayes risk reduction under squared loss, whereas EIG targets entropy reduction. It would be great to see a comparison between them.
  • The experiments are only run on a single semi-synthetic dataset with a simulated clinician. I'd like to see a bit more variety, especially to see what happens when the preference model is misspecified.
  • I noticed the authors use the plug-in CPTE estimator instead of its posterior mean. I am not entirely sure why this choice was made, and whether it actually affects the results.
Questions for authors
  • Is it possible to compare the approximated EIG with the exact one via posterior sampling? Maybe just run it on a small toy example. I'm really curious if they end up selecting similar queries.
  • Why not compare with a simple CPTE variance reduction baseline? It feels like a very natural baseline here, and it would really help show if EIG makes a difference.
  • I noticed that activecq (iclr 2026) is not discussed. It also looks at active learning for general causal quantities using information gain and variance reduction. Could the authors clarify the main difference in terms of high-level insights?
  • What happens if the true preference model doesn't match the assumed probit utility model? Could the authors maybe run some tests on model misspecification or try some more complex preference functions?
  • Why use Tx(mt)T_x(m_t) instead of just the posterior mean of CPTE? Would using the posterior mean actually change the MSE or regret results?
  • How sensitive are the results to the numerical approximations? For example, EPIG uses MC but CPTE-EIG uses Gauss-Hermite. Also, the method only samples 20 outcome pairs per target patient. If more samples were used, would the results look much different?
  • One last thing about policy learning: since the final decision just depends on whether CPTE > 0.5, why not optimize directly for the decision instead of the exact CPTE value? I understand that estimating CPTE is the main point, but I feel like this comparison would be interesting to see.
2 comments

We warmly thank the reviewer for these detailed questions, which led to several new experiments in the revised paper.

The true preference model doesn't match the assumed probit utility model

We added an experiment where the clinician does not follow our model (Section 5.2). LLMs act as clinicians, judging which of two outcome profiles is in a better state. We use BioDecision-v2-4B, and Qwen3-32B with a neuropsychiatric and a motor persona (details in Appendices F.1–F.2). These clinicians are inconsistent and disagree with each other (Figure S1). CPTE-EIG's advantage is even larger than with the well-specified clinician (Figure 3): with BioDecision after 100 comparisons, its CPTE MSE is 2.5×10⁻³ vs. 3.9 (EPIG), 3.5 (BALD) and 7.3 (random).

Approximated EIG with the exact one via posterior sampling The paper gives an error bound for the linearized CPTE, but not for the approximated EIG

We ran this comparison (Appendix F.9, Figures S8–S9). The exact criterion estimates I(Ψ_x; V_c) for the non-linearised Ψ_x by posterior sampling. Averaged over horizons, the exact EIG is very similar to our criterion but is about 35 times slower.

CPTE variance reduction baseline

We agree; it is now a baseline in all experiments (Figures 2–3, Appendix F.3), with the same targets and Gaussian approximation. The two are close, showing that targeting the CPTE is the main source of the gain over EPIG and BALD. CPTE-EIG is significantly better with the strong effect at 60 comparisons (Appendix F.5) and with the neuropsychiatric persona at τ = 0.5 with 60 (Appendix F.6). It is significantly worse only with the tiny effect at 100, where all methods are already accurate (Appendix F.5).

I noticed that activecq (iclr 2026) is not discussed.

Thank you for this very relevant reference, which we added to the related work (Section 1.2). Both works target the causal quantity rather than the model, but differ in three ways.

  • What is queried. ActiveCQ acquires outcome labels to learn an outcome regression. Our outcomes are already observed; what is costly is the preference rule defining the effect, learned from noisy pairwise judgments.
  • Structure of the estimand. In ActiveCQ the quantity is a linear functional of a GP, hence Gaussian. Our CPTE averages a probit of utility contrasts over treated–control pairs, with binary answers, so its posterior is not Gaussian, hence our linearisation (Theorem 1) and closed form (Theorem 2). This averaging also means a query can inform every pair and still not the CPTE (Appendix D).
  • Complementarity. We fix the outcome laws and model only preference uncertainty. ActiveCQ could help propagate both.

The authors use the plug-in CPTE estimator instead of its posterior mean

Thank you for this remark. Tx(mt)T_x(m_t) is the mean of the linearised posterior that CPTE-EIG learns about, and ∣Eqt[Tx(U)]−Tx(mt)∣≤Eqt∣Rx(h)∣|\mathbb E_{q_t}[T_x(U)]-T_x(m_t)|\le\mathbb E_{q_t}|R_x(h)|, a remainder that is small (Appendix F.8). We evaluated both on the same runs (Appendix F.10, Figures S10–S11). The posterior mean helps only at the warm start and regret is almost identical.

How sensitive are the results to the numerical approximations?

Integration rule. Replacing Gauss–Hermite (10 or 40 nodes) by Monte Carlo (2,000 or 8,000 samples) gives indistinguishable CPTE MSE at every budget (Appendix F.11, Figure S12), at several times the cost (Figure S13), so the gap with EPIG does not come from the integration rule.

Number of pairs K. We compared the query selected with K sampled pairs to exact scores over all 900 pairs (Appendix F.12). With K = 20, it is in the exact top 1% in 69% of cases (top 5%: 83%), while a random query never is; with K = 100, these become 99% and 100%. Pairs are redrawn each round, so errors do not accumulate.

since the final decision just depends on whether CPTE > 0.5, why not optimize directly for the decision instead of the exact CPTE value?

We added a new decision-EIG baseline (Figures 2–3, Appendix F.3). Its performance is very similar to CPTE-EIG across experiments. However, CPTE-EIG leaves the freedom of choosing any decision threshold at deployment time, where decision-EIG trained for a fixed threshold would be suboptimal (Appendix F.6, Figure S4).

The core idea is quite straightforward, since estimand-targeted EIG has already been studied in Causal-EPIG.

We share Causal-EPIG's principle of targeting the estimand, but Causal-EPIG chooses which patients' outcomes to measure, whereas we choose which pairwise judgments to ask a clinician, and these judgments define the effect itself. Because the CPTE averages a non-linear function of these judgments over pairs of outcomes, applying the principle required our linearisation and closed form (Theorems 1–2), and it beats EPIG, the natural transposition of Causal-EPIG, across experiments.

We thank the reviewer again and would be glad to clarify any remaining point.

We have updated our replies above so that the appendix references match the latest version of the paper. We would be happy to discuss any point that remains unclear.

Reviewer_whFhOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CPTE-EIG, an active preference-learning criterion for multivariate treatment effects. Rather than reducing uncertainty about the preference model itself, it selects pairwise comparisons according to their expected information gain about the conditional preference treatment effect and the induced treatment policy. A Gaussian-process preference model and a linearized approximation make the acquisition criterion tractable. Experiments on a semi-synthetic Parkinson’s disease cohort with 13 outcomes compare CPTE-EIG with BALD, EPIG, and random acquisition under different treatment-effect strengths and query budgets.

Critical strengths

The acquisition objective is directly aligned with the downstream causal estimand rather than general uncertainty in the preference function. The experiments use common candidate pools and preference models across acquisition methods, multiple query budgets, 15 seeds, and bootstrap intervals, making the comparison well controlled.

Critical weaknesses

The evaluation relies on a simulated clinician with a known utility function. It therefore does not assess preference inconsistency or disagreement present in actual elicitation. The analysis conditions on estimated potential-outcome distributions and treats preference uncertainty as the main source of uncertainty. Uncertainty from the outcome models is not propagated into the acquisition criterion.

Questions for authors

Refer to the weakness. And: Is uncertainty in the estimated potential-outcome distributions excluded from CPTE-EIG throughout both training and evaluation? Does the linearization error bound remain small over the query budgets used in the experiments?

2 comments

We thank the reviewer for these questions. They led to a new experiment and to new diagnostics, which are now in the revised paper.

The evaluation relies on a simulated clinician with a known utility function. It therefore does not assess preference inconsistency or disagreement present in actual elicitation.

We added an experiment where the clinician is no longer simulated from our model (Section 5.2, Figure 3). LLMs act as clinicians: each reads the two outcome profiles as text and says which patient is in a better clinical state, without seeing the treatments. We use BioDecision-v2-4B [1], and Qwen3-32B [2] with a neuropsychiatric persona and a motor persona (full prompts in Appendix F.2). The effect is also closer to practice: dopamine agonist vs. levodopa at 5 years, with benefits for younger patients and neuropsychiatric harms for older ones (Appendix F.1).

These clinicians have no utility function, so our probit GP model is misspecified by construction. They are inconsistent: about a third of the Qwen3 answers flip when the two patients are swapped. We ask both orders and draw the answer from the mean probability. This removes position bias but keeps the noise, since the two orders fully cancel on 11–18% of persona comparisons. They also disagree: the agonist is preferred for 15% to 39% of patients depending on the clinician, and the two personas recommend different treatments for 13% of them (Appendix F.2, Figure S1).

Policy regret / CPTE MSE (×10⁻³) after 100 comparisons, mean over 30 seeds:

ClinicianCPTE-EIGEPIGBALDRandom
BioDecision-v2-4B9.5 / 2.516.3 / 3.916.2 / 3.533.0 / 7.3
Neuropsychiatric8.6 / 2.410.6 / 2.811.5 / 3.227.1 / 6.3
Motor8.7 / 1.78.8 / 2.115.5 / 2.816.9 / 4.1

CPTE-EIG has the lowest error and regret with all three clinicians. For the motor persona its regret is on par with EPIG.

We also scored the policies at thresholds τ ∈ {0.4, 0.45, 0.5} (Appendix F.6, Figures S4–S5). This covers 18 settings: 2 personas, 3 thresholds, and 40/60/100 comparisons. Among CPTE-EIG, EPIG, BALD and random, CPTE-EIG has the lowest mean regret in 17 of them, and in the remaining one it is 0.5×10⁻³ behind EPIG.

Is uncertainty in the estimated potential-outcome distributions excluded from CPTE-EIG throughout both training and evaluation?

Yes, at both stages, and by design. The paper separates two sources of data (Section 2). The cohort is fixed before elicitation. The clinician comparisons are the only data the loop collects. Uncertainty in μ^a\hat\mu_a is reduced by observing more patients, not by asking more questions. CPTE-EIG therefore targets the part of the uncertainty on Ψ_x that comparisons can resolve, and draws its pairs from νx\nu_x (Eq. 1) with μ^a\hat\mu_a held fixed.

For evaluation, both potential outcomes of every simulated patient are known. μ^a(⋅∣x)\hat\mu_a(\cdot\mid x) is the empirical law of Y(a)Y(a) over the nearest neighbours of xx (Appendix F.1, "Outcome laws"), so it has no confounding bias. The oracle CPTE uses the true preference rule on the same νx\nu_x (Appendix F.4). The reported errors therefore isolate the cost of learning the preference rule, and any gap between μ^a\hat\mu_a and the true laws is shared by all methods. Propagating this uncertainty, for example by bootstrapping the full pipeline, is a natural extension (Section 6).

Does the linearization error bound remain small over the query budgets used in the experiments?

We acknowledge that the bound Bt(x)B_t(x) of Theorem 1 (proof in Appendix B.2) is not always tight, including in our experiments. On the mild effect it stays between 0.4 and 3.2 at 150 comparisons. It is still a sufficient condition that can be computed at each round, but we agree it does not settle the question on its own.

We therefore measured the remainder directly (Appendix F.8, Figure S7), by Monte Carlo over the GP posterior on the same CPTE-EIG runs. Eqt∣Rx(h)∣\mathbb E_{q_t}|R_x(h)| falls from 0.035–0.083 after the 20 initial comparisons to 0.004–0.010 at 150. At every budget it is several times smaller than the RMSE of the CPTE estimate itself (0.017–0.019 at 150).

We also checked that the linearisation does not change which queries are selected (Appendix F.9, Figures S8–S9). Replacing it with the exact information gain, computed by posterior sampling, gives statistically indistinguishable CPTE MSE and regret at almost every budget, at about 35 times the cost.

We hope these additions address the reviewer's concerns, and we would be glad to clarify any remaining point.

[1] https://huggingface.co/lighteternal/biodecision-v2-4b [2] https://huggingface.co/Qwen/Qwen3-32B

We have updated our replies above so that the appendix references match the latest version of the paper. We would be happy to discuss any point that remains unclear.

Comments on the paper

1 comment

We thank all reviewers for their careful reading and constructive feedback, which helped us improve the paper. Reviewers found the problem well motivated (7Mm9, xPEP), the idea of targeting the causal estimand novel (whFh, xPEP), the derivation useful and efficient (7Mm9, xPEP), and the experiments well controlled (whFh).

Reviewers raised questions about robustness to noisy, disagreeing or misspecified clinicians (whFh, 7Mm9, xPEP), the accuracy of our approximations (whFh, 7Mm9), the choice of baselines (7Mm9, xPEP), the positioning with respect to Causal-EPIG and ActiveCQ (7Mm9), and some modelling choices (whFh, xPEP). We answer each point in detail in the individual responses, and we have uploaded a revised version of the paper. The main changes are:

  • Realistic, misspecified clinicians (whFh, 7Mm9, xPEP). We added a new experiment where LLMs act as clinicians, including two clinician personas with different priorities. Their answers are not generated by our model, are inconsistent, and disagree with each other. The task is also new and grounded in clinical guidelines: dopamine agonists vs. levodopa, with age-dependent benefits and harms. CPTE-EIG keeps its advantage in this setting, and its advantage is even larger than with the simulated clinician.
  • New baselines (7Mm9, xPEP). We added two criteria that directly target the CPTE: CPTE variance reduction and the information gain about the treatment decision. We also added balanced sampling, a non-active elicitation strategy.
  • Robustness to the decision threshold (7Mm9). We evaluate the learned policies at several decision thresholds, and CPTE-EIG remains the best criterion across them.
  • Accuracy of the approximations (whFh, 7Mm9).
    • We measured the linearisation error directly and showed that it is small and decreases with the budget.
    • We compared our closed-form criterion with the exact one computed by posterior sampling: both give the same accuracy, and ours is about 35 times faster.
    • We compared the plug-in CPTE estimate with the posterior mean, Gauss–Hermite quadrature with Monte Carlo, and different numbers of sampled outcome pairs. None of these choices changes our conclusions.
  • Related work (7Mm9). We added ActiveCQ to the related work and clarified how our work differs from Causal-EPIG.
  • Discussion of modelling choices (whFh, xPEP). We extended the discussion of:
    • uncertainty in the potential-outcome laws;
    • the use of real patient profiles as queries;
    • cognitive load and clinician reliability with 13 outcomes;
    • the path towards validation with real clinicians.

We thank the reviewers again for their time and helpful suggestions, and we remain available for any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.