acceptodds
Under review as a conference paper at ICLR 2027

RHOPE: LLM-AUGMENTED ADAPTIVE EXPERIMENTAL DESIGN WITH COMPLEX CONTEXTUAL INFORMATION

Abstract

Experimental design allocates limited measurement effort across candidate treatments to learn their expected outcomes. Adaptive designs refine allocations as evidence accumulates, improving estimation and treatment selection. Contextual information can inform these decisions, but exploiting unstructured or high-dimensional context remains challenging. We use large language models (LLMs) to transform complex context into proxy outcomes and propose Residual-Horizon Optimization for Prediction-Powered Experimentation (RHOPE), a principled Bayesian adaptive experimentation framework that combines true outcomes with LLM-generated proxy outcomes and formulates finite-batch design as a posterior Markov decision process (MDP). Our methodology uses proxy outcomes in two ways: sample augmentation improves estimation precision, while observing proxies before each allocation decision provides advance information. Synthetic experiments and semi-synthetic experiments using Genomics of Drug Sensitivity in Cancer (GDSC) data suggest that estimation precision drives most gains, with smaller advance-information gains, and highlight the importance of proxy informativeness and moment-estimation accuracy.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Adaptive experimental design, Best-arm identification, Prediction-powered inference, Bayesian sequential decision making

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_MqCLOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors investigate how to split a fixed experimental budget across several treatments, run in batches, so that the best treatment can be picked at the end. They propose RHOPE, an LLM-augmented adaptive experimental design method in which an LLM reads each participant's profile before every batch and predicts how that participant would respond to each treatment. The predictions are combined with the real outcomes to estimate each treatment's average effect more precisely, and the next batch is allocated to minimize the expected loss from picking the wrong treatment. The authors prove that adding the LLM predictions cannot make this decision worse, and show that their estimate reduces to the existing PPI++ estimator in a special case. In simulations and in a drug-response benchmark built from cancer cell-line data, RHOPE achieves the lowest simple regret among the compared methods when the outcome variances and correlations are known in advance.

Critical strengths
  1. Adaptive experiments are widely used, and real outcomes are often expensive, so using an LLM to turn participants' rich profiles into predicted outcomes that make each sample count for more is a reasonable idea for an important problem.

  2. The proposed framework is sound: the real outcomes are used to correct the LLM predictions, so a biased LLM does not bias the estimates, and when the outcome variances and correlations are known, an uninformative LLM simply makes the method fall back to standard RHO.

Critical weaknesses
  1. My main concern is the weak evidence that LLM proxies help. The paper is framed around LLMs extracting useful information from complex context, so the first thing to establish is whether the LLM can actually do this. However, the only experiments that use the LLM outputs as they are are the GDSC runs with v = 0. In those runs, the correlation between the LLM predictions and the true outcomes is 0.07-0.10 on average, and below 0.04 for four of the seven drugs (Table 5). At that level, the variance reduction is only around 1%. All the settings where RHOPE clearly wins have informative proxies by construction. The synthetic experiments do not use an LLM. The v > 0 runs add a scaled copy of the simulated patient noise to the LLM output, and no model could predict that noise from the cell-line features. These runs are a useful sensitivity analysis of how the gains grow with proxy quality, but they say nothing about what proxy quality an LLM can actually deliver.

I also do not think the LLM setup was designed carefully. The authors use only DeepSeek V4 Flash with thinking disabled, ask for raw numeric outputs, use one random draw per patient even though 100 draws were generated, and leave out the gene-expression features. I think the authors should at least try a stronger frontier model and put more care into the prompt. Numeric outputs from LLMs are often poorly calibrated, so asking for response levels instead, and averaging over the multiple draws already available, may better reflect what the LLM can do. Finally, there is no non-LLM baseline, such as a ridge model trained on the calibration cell lines, so it is unclear whether the LLM adds anything over a standard predictor.

  1. The methodological novelty is limited. Setting the LLM aside, RHOPE plugs a PPI-style control variate [1] into RHO [2]. Most of the theoretical analysis is incremental: it shows that the estimator is efficient, that the Gaussian approximation still holds, and that more information cannot hurt, all of which are expected for this combination. I would have liked a result on how much simple regret improves as a function of the proxy correlation, or on what happens with estimated moments, since that is where the method loses its advantage in practice. I think the genuinely new piece is observing the proxies before allocation, and it shows no clear effect: RHOPE-Delayed is better than RHOPE in 10 of the 30 table entries, and no standard errors are reported.

Minor:

  • The introduction motivates the method through heterogeneity across participants, but context is never used to decide who gets which treatment. It only reduces noise in the estimates of each treatment's average effect. Framing the motivation around this would make the paper much easier to follow.
  • Ma et al. [3] study best-arm identification with generative proxies, which is very close to this setting. The paper cites it but does not discuss how RHOPE differs from it, and does not compare against it.

References [1] Anastasios N. Angelopoulos, Stephen Bates, Clara Fannjiang, Michael I. Jordan, and Tijana Zrnic. Prediction-powered inference. Science, 382(6671):669–674, 2023.

[2] Ethan Che and Hongseok Namkoong. Adaptive experimentation at scale: A computational framework for flexible batches. arXiv preprint arXiv:2303.11582, 2023.

[3] Tianyi Ma, Hanzhang Qin, Ruihao Zhu, and Jierui Zuo. Best-arm identification with generative proxy. arXiv preprint arXiv:2607.06879, 2026.

Questions for authors

Please see the questions raised in the Weaknesses section.

4 comments
AuthorsOpenReview ↗

Response to Reviewer MqCL (1/4)

We thank the reviewer for a detailed and accurate assessment of both the theory and the experiments. The review identified the two central weaknesses of the submitted version: the theory did not say how much proxies help, and the GDSC experiment did not establish what an LLM can actually contribute. The revision adds three quantitative results and rebuilds the experiment.

Main revisions. Section, table and result numbers refer to the revised PDF.

  1. New semi-synthetic experiment (Section 6.2, Appendix C.2). The GDSC instance now uses 627 cell lines with complete measured responses (the 247 cell lines with imputed entries are excluded, and 22 of the remaining ones form a pool from which 16 in-context examples are drawn at random), patient-noise variance 4, and no synthetic augmentation: every proxy is used exactly as generated. Six proxies are compared: Claude Opus 5.5, GPT-6.1 Sol and DeepSeek V4 Pro, with reasoning enabled and a redesigned 16-shot prompt that includes gene-expression features; DeepSeek V4 Flash with the original prompt and settings, as a weaker LLM; and gradient-boosted trees (GBT) and ridge regression trained on the same 16 labeled cell lines with the same features. Each LLM proxy is the mean of 20 draws per cell line.
  2. Three new theoretical results (Section 5.2, Appendix D.3; proofs in Appendices B.9–B.11). Proposition 2: with known moments, proxies reduce the regret guarantee by the fraction ϱ2(1−1/K)\varrho^2(1-1/K). Proposition 3: the additional value of observing the current batch's proxies before allocation is bounded by a quantity of order Tlog⁡K/t\sqrt{T\log K}/t, so it decays over the horizon. Proposition 4: with an estimated projection coefficient, the gain persists if and only if ϱ2\varrho^2 exceeds a threshold that shrinks with the sample size used for the estimate.
  3. Standard errors. Every entry of the regret tables now reports its standard error.
  4. Presentation. The introduction states that contextual information explains part of the variation in a unit's outcome and can inform experimental design, and the contributions are restated around the quantitative results.
  5. Working model (Appendix E). Appendix E is rewritten. It restates the two covariance restrictions behind the working posterior, explains why we impose them, and verifies in the semi-synthetic experiment that the working posterior stays close to the posterior of a model without these restrictions.

We respond point by point below, starting with novelty and the theory.

AuthorsOpenReview ↗

Response to Reviewer MqCL (2/4)

Novelty and the theory

We agree that the estimator is a PPI-style control variate inside RHO, and that its efficiency, the Gaussian approximation and the fact that more information cannot hurt are the expected consequences of this combination. The revision treats these as the groundwork and puts the weight on the questions the reviewer raises: how much proxies help as a function of the correlation, how much of that comes from observing proxies before allocation, and what remains when the moments are estimated. Three new propositions answer them (Section 5.2 and Appendix D.3; proofs in Appendices B.9–B.11).

Regret as a function of the proxy correlation. Proposition 2: with exchangeable arms, the Bayes simple regret guarantees of RHOPE-Delayed and RHO are the same function RK( ⋅ ;τ)\mathcal R_K(\,\cdot\,;\tau) of their terminal posterior variances, and under a flat prior their ratio converges to 1−ϱ2(1−1/K)1-\varrho^2(1-1/K). Proxies thus act as a variance reduction by the fraction ϱ2(1−1/K)\varrho^2(1-1/K). The synthetic grid over ϱ\varrho (Tables 1 and 3) and the six GDSC proxies, for which the regret of RHOPE increases as ϱ\varrho decreases (Table 2), follow this dependence.

Advance information. We agree that no clear effect is visible. All tables now report standard errors, and none of the paired differences between RHOPE and RHOPE-Delayed is significant. Proposition 3 explains why: under a flat prior, the advance-information gain at batch tt is at most 22log⁡(2K) max⁡a ∣λa∣ (σt,a(2))2bts~a2+(σt,a(2))2bt  ≍  Tlog⁡Kt,2\sqrt{2\log(2K)}\,\max_{a}\,|\lambda_a|\,(\sigma_{t,a}^{(2)})^2\sqrt{\frac{b_t}{\tilde s_a^2+(\sigma_{t,a}^{(2)})^2b_t}}\;\asymp\;\frac{\sqrt{T\log K}}{t}, which decays over the horizon, whereas the estimation benefit removes a fixed fraction of the posterior variance in every batch. Proxies are observed for every unit under every arm, so the proxy means are learned about KK times faster than the residual means, and the current batch's proxies move the posterior little once a few batches have been observed. The abstract, contributions and Section 6 now describe this benefit as small and secondary to estimation precision.

Estimated moments. Proposition 4: when the projection coefficient is replaced by an estimate λ^\hat\lambda from nλn_\lambda pilot pairs, the guarantee of RHOPE contains ϱ2−(s~2/s2) E(λ^−λ)2\varrho^2-(\tilde s^2/s^2)\,\mathbb E(\hat\lambda-\lambda)^2 in place of ϱ2\varrho^2, and it stays below the guarantee of RHO if and only if E(λ^−λ)2<λ2\mathbb E(\hat\lambda-\lambda)^2<\lambda^2, that is, if and only if ϱ2\varrho^2 exceeds a threshold of order 1/nλ1/n_\lambda. This is the pattern of the synthetic ablation with first-batch estimates (Appendix D, Table 6): with small batches RHOPE often falls behind RHO, and with large batches a RHOPE variant is best in every setting. This is the precise sense in which the method loses its advantage when moments are estimated from little data.

AuthorsOpenReview ↗

Response to Reviewer MqCL (3/4)

Evidence that LLM proxies help

We agree with the diagnosis. In the submitted version, the only runs that used the LLM outputs as generated (v=0v=0) had average correlations of 0.07–0.10, and the v>0v>0 runs injected simulated noise that no predictor could recover. We therefore re-diagnosed the original semi-synthetic experiment and rebuilt it (Appendix C.2). Every proxy now enters exactly as generated, with no augmentation, and the setup differs in three further ways.

  1. Data. The re-diagnosis showed that 247 of the 896 GDSC cell lines carry imputed response entries (all imputed entries of a drug share one value), which distort the proxy–outcome correlations. We use the 627 cell lines with complete measured responses that are not used as in-context examples.
  2. Proxies. Claude Opus 5.5, GPT-6.1 Sol and DeepSeek V4 Pro with reasoning enabled, a redesigned prompt (see "The LLM setup" below) and the mean of 20 draws per cell line. DeepSeek V4 Flash with its original prompt and settings, now also averaged over 20 draws instead of one draw per patient, is kept as a control, and ridge regression and gradient-boosted trees (GBT) trained on the same 16 labeled cell lines are the non-LLM references.
  3. Noise. The patient-noise variance is 4 instead of 9. The purpose of the benchmark is to show how proxies of different quality affect the decisions. With variance 9, the proxy–outcome correlations of all models were uniformly low, so proxies of different quality could not be told apart; the smaller noise variance is one component of the redesign. The cell-line responses and the proxies are not affected by this choice.

Results (Table 2; relative simple regret, uniform =100=100, mean ±\pm se over 10,000 replications; RHO, which uses no proxies, is 42.55±1.1742.55\pm1.17):

Proxymean ϱa\varrho_aRHOPE-DelayedRHOPE
Claude Opus 5.50.3138.54±1.1038.54\pm1.1038.70±1.1138.70\pm1.11
GPT-6.1 Sol0.2939.53±1.1239.53\pm1.1238.91±1.1138.91\pm1.11
DeepSeek V4 Pro0.2639.39±1.1139.39\pm1.1139.26±1.1139.26\pm1.11
DeepSeek V4 Flash0.2039.55±1.1239.55\pm1.1240.69±1.1440.69\pm1.14
GBT0.1942.68±1.1542.68\pm1.1541.82±1.1441.82\pm1.14
Ridge0.1654.12±2.2354.12\pm2.2354.39±2.2254.39\pm2.22

With each of the three reasoning LLMs, the lowest regret in Table 2 is attained by RHOPE or RHOPE-Delayed, and both are significantly better than RHO; the improvement shrinks with the weaker Flash proxy. Moreover, with every LLM proxy, the gap between RHOPE and RHOPE-Delayed is much smaller than the gap between RHOPE-Delayed and RHO. This confirms the characterization of estimation precision and advance information in the new theory above: relative to estimation precision, the benefit of advance information is small and secondary. Since every proxy is used as generated, Table 2 now measures what current LLMs deliver on this task.

AuthorsOpenReview ↗

Response to Reviewer MqCL (4/4)

The LLM setup

(a) Stronger models and a more careful prompt. The revised experiment uses Claude Opus 5.5, GPT-6.1 Sol and DeepSeek V4 Pro with reasoning enabled (effort "high" for the first two, "low" for DeepSeek V4 Pro). The prompt (Appendix C.2, with a complete example) lists the drugs' targets and pathways, gives the 16 in-context cell lines with their observed responses, and describes each cell line by cancer type, growth mode, microsatellite status, the status of 18 drug-relevant biomarkers, coding mutations in 300 cancer genes, copy-number gains and losses in 126 regions with the genes they contain, and, as the reviewer suggested, gene-expression features: the expression percentiles of 18 genes related to the drugs' mechanisms.

(b) Raw numeric outputs versus response levels. We kept numeric outputs for a structural reason. RHOPE uses a proxy only through its correlation with the true outcome: the residual correction by the observed outcomes makes the working-posterior mean invariant to any affine transformation of the proxy, so a miscalibrated numeric scale costs nothing, and only the correlation ϱa\varrho_a matters. Coarsening the output into levels would discard the within-level ordering without removing anything harmful. The reviewer is right that the numeric outputs are poorly calibrated (their RMSE against the measured responses is close to that of predicting each drug's mean, although their correlations with the true outcomes can still reach 0.3–0.4); this does not affect RHOPE.

(c) Averaging over draws. Every LLM proxy is now the mean of 20 independent draws per cell line (Appendix C.2).

(d) A non-LLM baseline. Ridge regression and GBT are trained, separately for each drug, on exactly the 16 in-context cell lines, with a numeric encoding of the same features that the prompt shows and hyperparameters chosen by leave-one-out cross-validation within the 16 lines (Appendix C.2). Their proxy–outcome correlations are below those of every LLM, and RHOPE with each reasoning LLM is significantly better than RHOPE with either of them (Table 2). With the same inputs and the same 16 labels, the LLMs extract more usable information than the standard predictors.

Minor 1. Motivation through heterogeneity

Agreed. Context is never used to decide who receives which treatment; the target is the arm with the highest mean outcome. Context explains part of the variation in a unit's outcome, and the LLM turns it into a proxy that reduces the noise in each arm's mean estimate. The introduction now motivates the method in these terms.

Minor 2. Relation to Ma et al. (2026)

Ma et al. study fixed-confidence BAI: an elimination procedure with a δ\delta-PAC guarantee minimizes the number of true samples needed to certify the best arm, and it has no budget parameter. We study fixed-budget BAI with Bayes simple regret, so the two methods address different problems; running theirs under a fixed budget would require choosing an arbitrary δ\delta and truncating it, outside its design and guarantees, which is why we do not compare against it. Appendix A now states this distinction.

We hope these changes address the concerns, and we are glad to answer further questions.

Reviewer_KTXuOpenReview ↗
Rating
1(−2)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces RHOPE (Residual-Horizon Optimization for Prediction-Powered Experimentation), a new framework that uses Large Language Models (LLMs) to handle complex, unstructured background data in adaptive experiments.

Critical strengths

roughly critical topics on LLM, clear expression.

Critical weaknesses

see questions

Questions for authors
  • The built-in knowledge of LLM can provide proxy information for cold start in this framework, but the proxy output predicted by LLM may deviate significantly from the actual results. How does the paper analyze and mitigate the negative impact of proxy prediction bias?
  • This article uses Deepseek to provide proxy information. Does the performance of this method depend on a specific large model? If we switch to other large models, will there be significant changes in the performance of the method and the correlation of the agents?
  • As mentioned in the article, as long as the correlation between the LLM agent's predicted values and the true returns is high enough, the information carried by the agent can compensate for the errors caused by our estimation variance, offset this performance decline, and the overall algorithm performance will not collapse. So when the moment estimation is poor, how to ensure that the correlation between the LLM agent prediction value and the true return is high enough to prevent a significant decrease in algorithm performance? And what are the weights of the impact of the author's quantification of moment estimation, LLM proxy prediction values, and real return correlation on the real algorithm?
6 comments
AuthorsOpenReview ↗

Response to Reviewer KTXu (1/3)

We thank the reviewer for the questions. They concern how the method deals with imperfect LLM predictions and with estimated moments, which the revision now treats more explicitly.

Main revisions. Section, table and result numbers refer to the revised PDF.

  1. New semi-synthetic experiment (Section 6.2, Appendix C.2). The GDSC instance now uses 627 cell lines with complete measured responses (the 247 cell lines with imputed entries are excluded, and 22 of the remaining ones form a pool from which 16 in-context examples are drawn at random), patient-noise variance 4, and no synthetic augmentation: every proxy is used exactly as generated. Six proxies are compared: Claude Opus 5.5, GPT-6.1 Sol and DeepSeek V4 Pro, with reasoning enabled and a redesigned 16-shot prompt that includes gene-expression features; DeepSeek V4 Flash with the original prompt and settings, as a weaker LLM; and gradient-boosted trees (GBT) and ridge regression trained on the same 16 labeled cell lines with the same features. Each LLM proxy is the mean of 20 draws per cell line.
  2. Three new theoretical results (Section 5.2, Appendix D.3; proofs in Appendices B.9–B.11). Proposition 2: with known moments, proxies reduce the regret guarantee by the fraction ϱ2(1−1/K)\varrho^2(1-1/K). Proposition 3: the additional value of observing the current batch's proxies before allocation is bounded by a quantity of order Tlog⁡K/t\sqrt{T\log K}/t, so it decays over the horizon. Proposition 4: with an estimated projection coefficient, the gain persists if and only if ϱ2\varrho^2 exceeds a threshold that shrinks with the sample size used for the estimate.
  3. Standard errors. Every entry of the regret tables now reports its standard error.
  4. Presentation. The introduction states that contextual information explains part of the variation in a unit's outcome and can inform experimental design, and the contributions are restated around the quantitative results.
  5. Working model (Appendix E). Appendix E is rewritten. It restates the two covariance restrictions behind the working posterior, explains why we impose them, and verifies in the semi-synthetic experiment that the working posterior stays close to the posterior of a model without these restrictions.

Point-by-point responses follow.

AuthorsOpenReview ↗

Response to Reviewer KTXu (2/3)

Q1. LLM predictions may deviate from the actual results: how is this bias analyzed and mitigated?

The idea. RHOPE never takes the LLM predictions at face value. It follows the idea of prediction-powered inference (PPI; Angelopoulos et al., 2023). The LLM prediction is available for every unit, whereas the true outcome is observed only for the units assigned to a treatment. The units that have both show how far the predictions are from the truth on average, and this measured gap is used to correct the average prediction. A systematic error of the LLM, however large, enters the average prediction and the measured gap alike, and cancels. A biased LLM therefore does not bias the estimate of a treatment's mean outcome; the quality of the LLM only determines how much precision is gained. A small number of true outcomes is enough to correct imperfect predictions.

In formulas. For a treatment aa, let Y~ˉaall\bar{\tilde Y}_a^{\mathrm{all}} be the average LLM prediction over all units, and let Yˉaobs\bar Y_a^{\mathrm{obs}} and Y~ˉaobs\bar{\tilde Y}_a^{\mathrm{obs}} be the average true outcome and the average LLM prediction over the units assigned to aa. PPI estimates the mean outcome of treatment aa by

θ^aPPI=Y~ˉaall⏟average prediction+Yˉaobs−Y~ˉaobs⏟correction from the true outcomes.\hat\theta_a^{\mathrm{PPI}}=\underbrace{\bar{\tilde Y}_a^{\mathrm{all}}}_{\text{average prediction}}+\underbrace{\bar Y_a^{\mathrm{obs}}-\bar{\tilde Y}_a^{\mathrm{obs}}}_{\text{correction from the true outcomes}}.

If all predictions are off by some amount bb, both Y~ˉaall\bar{\tilde Y}_a^{\mathrm{all}} and Y~ˉaobs\bar{\tilde Y}_a^{\mathrm{obs}} shift by bb, and the shift cancels. More generally, the assigned units are a random subset of all units, so the two prediction averages have the same expectation, and the estimate is centered at the true mean of treatment aa whatever the LLM predicts.

PPI++ and the coefficient λ\lambda. PPI++ (Angelopoulos et al., 2026), a variant of PPI, adds a coefficient λa\lambda_a that adapts how much the predictions are used:

θ^aλ=Yˉaobs−λa(Y~ˉaobs−Y~ˉaall).\hat\theta_a^{\lambda}=\bar Y_a^{\mathrm{obs}}-\lambda_a\bigl(\bar{\tilde Y}_a^{\mathrm{obs}}-\bar{\tilde Y}_a^{\mathrm{all}}\bigr).

With λa=1\lambda_a=1 this is PPI, and with λa=0\lambda_a=0 the predictions are ignored. The coefficient λa\lambda_a in RHOPE borrows this idea: the posterior mean of each treatment in RHOPE has exactly this form under a flat prior (Theorem 2), with λa\lambda_a determined by the correlation between the predictions and the true outcomes, so that informative predictions are used and uninformative ones are down-weighted to zero.

AuthorsOpenReview ↗

Response to Reviewer KTXu (3/3)

Q2. Does the performance depend on a specific LLM?

The method does not depend on a specific LLM: any LLM, or any other predictor, can supply the proxies. What does affect its performance is the quality of the proxies that a model generates, that is, their correlation with the true outcomes.

The new semi-synthetic experiment (Section 6.2) examines this with three stronger LLMs (Claude Opus 5.5, GPT-6.1 Sol, DeepSeek V4 Pro), one weaker LLM (DeepSeek V4 Flash), and two traditional machine-learning techniques (GBT and ridge regression), on the same cell lines. Results (Table 2; relative simple regret, uniform =100=100, mean ±\pm se over 10,000 replications; RHO, which uses no proxies, is 42.55±1.1742.55\pm1.17):

Proxymean correlation with the true outcomeRHOPE-DelayedRHOPE
Claude Opus 5.50.3138.54±1.1038.54\pm1.1038.70±1.1138.70\pm1.11
GPT-6.1 Sol0.2939.53±1.1239.53\pm1.1238.91±1.1138.91\pm1.11
DeepSeek V4 Pro0.2639.39±1.1139.39\pm1.1139.26±1.1139.26\pm1.11
DeepSeek V4 Flash0.2039.55±1.1239.55\pm1.1240.69±1.1440.69\pm1.14
GBT0.1942.68±1.1542.68\pm1.1541.82±1.1441.82\pm1.14
Ridge0.1654.12±2.2354.12\pm2.2354.39±2.2254.39\pm2.22

The correlation does change with the model, and the regret follows it monotonically: the higher the quality of the proxies, the better RHOPE performs.

Q3. Moment estimation and correlation

How to ensure that the correlation is high enough to prevent a significant decrease in performance. The answer has two parts.

  • Before the experiment. The quality of the proxies is largely in the experimenter's hands. As the new semi-synthetic experiment shows, choosing a frontier LLM, designing the prompt carefully, enabling the reasoning (thinking) mode, and averaging repeated queries give a much better chance of obtaining proxies that are highly correlated with the true outcomes, and hence of preventing a significant decrease in performance.
  • After the first batch. Proposition 4 suggests a simple empirical check once the first batch has been collected. Estimate the correlation ϱ\varrho between the LLM predictions and the true outcomes from the first-batch data, and let nλn_\lambda be the number of observations used for the estimate. If ϱ2>1/(nλ−2)\varrho^2>1/(n_\lambda-2), the correlation is high enough; otherwise it is not. This is the form that the condition of Proposition 4 takes for Gaussian outcomes, and it is now stated in the proposition (Appendix D.3).

The weights of moment estimation, LLM predictions, and correlation. Proposition 4 answers this question. The accuracy of moment estimation is governed by the sample size of the first batch and by the size of the moments themselves, and the quality of the LLM predictions is governed by their correlation with the true outcomes. Taking the estimation error of λ\lambda into account, Proposition 4 states when these quantities combine so that the simple-regret bound of RHOPE is below that of RHO, namely if and only if

ϱ2>1nλ(E[(ϵp)2(ϵ~p)2]s2s~2−1)+1nλ−1−(nλ−2)ϱ2nλ(nλ−1),\varrho^2>\frac{1}{n_\lambda}\left(\frac{\mathbb E\bigl[(\epsilon^{\mathrm p})^2(\tilde\epsilon^{\mathrm p})^2\bigr]}{s^2\tilde s^2}-1\right)+\frac{1}{n_\lambda-1}-\frac{(n_\lambda-2)\varrho^2}{n_\lambda(n_\lambda-1)},

where ϵp\epsilon^{\mathrm p} and ϵ~p\tilde\epsilon^{\mathrm p} are the centered true outcome and the centered LLM prediction, with variances s2s^2 and s~2\tilde s^2. The left-hand side measures the quality of the LLM predictions; the right-hand side is the cost of estimating the moments, which decreases with the first-batch sample size nλn_\lambda and depends on the moments of the outcomes.

We hope this answers the questions, and we are glad to clarify further.

Reviewer_QNjjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Contextual information can inform the design of experiments to identify the treatment with the highest expected outcome. The authors use LLMs to convert complex context into proxy outcomes and propose RHOPE. They validate its performance in a synthetic experiment and on semi-synthetic GDSC data.

Critical strengths
  1. RHOPE introduces an adaptive experimentation framework that does not require high-dimensional response calculation.
  2. RHOPE compares proxy-based estimation with advance-information gain while optimizing sequential design.
  3. The authors provide a theoretical analysis under specific conditions.
Critical weaknesses
  1. In Tables 2 and 7, the improvement over RHO is limited when the original LLM outputs are used (v = 0).
  2. The authors do not report prediction errors against the original GDSC response, nor do they compare the LLM proxies with other prediction models.
  3. The authors acknowledge that their assumption of uncorrelated proxies across treatments does not hold in GDSC, but they do not validate the quality of the posterior distributions.
Questions for authors
  1. Can you compare the LLM proxies with a standard prediction model that uses the same input and observation data? Alternatively, can you report the prediction errors against the GDSC data?

  2. How does ignoring correlations between proxies for different treatments affect the posterior distribution? Can you compare the current method with a model that includes these correlations? An intuitive explanation is fine.

5 comments
AuthorsOpenReview ↗

Response to Reviewer QNjj (1/4)

We thank the reviewer for the comments, and we are sorry that the submitted version was difficult to follow. We first list the main revisions, then summarize the paper in brief, and then respond point by point.

Main revisions. Section, table and result numbers refer to the revised PDF.

  1. New semi-synthetic experiment (Section 6.2, Appendix C.2). The GDSC instance now uses 627 cell lines with complete measured responses (the 247 cell lines with imputed entries are excluded, and 22 of the remaining ones form a pool from which 16 in-context examples are drawn at random), patient-noise variance 4, and no synthetic augmentation: every proxy is used exactly as generated. Six proxies are compared: Claude Opus 5.5, GPT-6.1 Sol and DeepSeek V4 Pro, with reasoning enabled and a redesigned 16-shot prompt that includes gene-expression features; DeepSeek V4 Flash with the original prompt and settings, as a weaker LLM; and gradient-boosted trees (GBT) and ridge regression trained on the same 16 labeled cell lines with the same features. Each LLM proxy is the mean of 20 draws per cell line.
  2. Three new theoretical results (Section 5.2, Appendix D.3; proofs in Appendices B.9–B.11). Proposition 2: with known moments, proxies reduce the regret guarantee by the fraction ϱ2(1−1/K)\varrho^2(1-1/K). Proposition 3: the additional value of observing the current batch's proxies before allocation is bounded by a quantity of order Tlog⁡K/t\sqrt{T\log K}/t, so it decays over the horizon. Proposition 4: with an estimated projection coefficient, the gain persists if and only if ϱ2\varrho^2 exceeds a threshold that shrinks with the sample size used for the estimate.
  3. Standard errors. Every entry of the regret tables now reports its standard error.
  4. Presentation. The introduction states that contextual information explains part of the variation in a unit's outcome and can inform experimental design, and the contributions are restated around the quantitative results.
  5. Working model (Appendix E). Appendix E is rewritten. It restates the two covariance restrictions behind the working posterior, explains why we impose them, and verifies in the semi-synthetic experiment that the working posterior stays close to the posterior of a model without these restrictions.
AuthorsOpenReview ↗

Response to Reviewer QNjj (2/4)

The paper in brief. Because the reviewer found the submitted version difficult to follow, we summarize the whole paper here before the point-by-point responses: the problem we study, why we use LLM proxies, our method, and our contributions. We hope that this summary helps the reviewer to follow the paper as a whole more easily.

  • Problem. We want to find the best of KK treatments with a fixed budget of experimental units that arrive in a few batches. Each unit comes with rich context (for example a genomic profile or text), but its true outcome is observed only under the one treatment it receives. After each batch, we can change how the next batch is allocated to the treatments, and at the end we recommend one treatment. The goal is that the recommended treatment is as good as possible (small simple regret).
  • Why LLM proxies. The context explains part of the variation in a unit's outcome, so using it makes the comparison of the treatments less noisy. Rich context (text, images, genomic profiles) is, however, hard to exploit with a model estimated from scarce experimental data. An LLM, as a general-purpose generative model, condenses the context of a unit into a predicted outcome under every treatment (a proxy outcome). These proxies are cheap, are available for all units and all treatments, and need no training on the experimental data. They remove part of the outcome variation from the estimate of each treatment's mean, which leads to a better recommended treatment.
  • Method (RHOPE). The proxies may be wrong, so we combine them with the observed true outcomes in the manner of prediction-powered inference: the true outcomes correct the predictions, so a biased LLM does not bias the estimate of a treatment's mean outcome, while a good LLM makes it more precise. On top of this estimator, we formulate the design problem as a Bayesian dynamic program (a posterior Markov decision process) whose objective is the quality of the final recommendation itself, and we solve it by residual horizon optimization. Each batch is thus allocated by systematically optimizing the final objective rather than by a heuristic rule, and the proxies of the incoming batch are used before it is allocated.
  • Contributions. (i) A systematic optimization framework for adaptive experimentation that uses complex context through LLM proxies, without fitting a high-dimensional response model to scarce experimental data. (ii) Theory: in the Gaussian limit with known moments, proxies cannot make the decision worse, and three new results quantify how much they help, through more precise estimation and through advance information, and what remains when the moments are estimated (Propositions 2–4). (iii) An exact link between our posterior mean and the PPI++ estimator. (iv) Experiments on synthetic data and on a semi-synthetic benchmark built from cancer cell-line data (GDSC).
AuthorsOpenReview ↗

Response to Reviewer QNjj (3/4)

W1. Limited improvement over RHO when the original LLM outputs are used (v=0v=0)

We agree that in the submitted version the improvement was limited when the LLM outputs were used as generated, and we have replaced that experiment. In the new semi-synthetic experiment (Section 6.2, Appendix C.2), every proxy is used exactly as generated, with no augmentation, which corresponds to v=0v=0 throughout. The experiment uses 627 GDSC cell lines with complete measured responses, seven drugs as arms, ten batches, and six proxies: three stronger LLMs with reasoning enabled and a redesigned prompt (Claude Opus 5.5, GPT-6.1 Sol, DeepSeek V4 Pro), one weaker LLM (DeepSeek V4 Flash), and two traditional machine-learning techniques (GBT and ridge regression) trained on the same 16 labeled cell lines.

Results (Table 2; relative simple regret, uniform =100=100, mean ±\pm se over 10,000 replications; RHO, which uses no proxies, is 42.55±1.1742.55\pm1.17):

Proxymean correlation with the true outcomeRHOPE-DelayedRHOPE
Claude Opus 5.50.3138.54±1.1038.54\pm1.1038.70±1.1138.70\pm1.11
GPT-6.1 Sol0.2939.53±1.1239.53\pm1.1238.91±1.1138.91\pm1.11
DeepSeek V4 Pro0.2639.39±1.1139.39\pm1.1139.26±1.1139.26\pm1.11
DeepSeek V4 Flash0.2039.55±1.1239.55\pm1.1240.69±1.1440.69\pm1.14
GBT0.1942.68±1.1542.68\pm1.1541.82±1.1441.82\pm1.14
Ridge0.1654.12±2.2354.12\pm2.2354.39±2.2254.39\pm2.22

With the outputs of the three stronger LLMs used exactly as generated, RHOPE or RHOPE-Delayed attains the lowest regret among all methods, both improve significantly on RHO, and the improvement grows with the quality of the proxies.

W2 / Q1. Prediction errors against GDSC, and comparison with standard prediction models

Prediction errors. For each of the six proxies, the table reports the root mean squared error (RMSE) of the predictions against the measured GDSC responses over the 627 experimental cell lines, averaged over the seven drugs, together with the correlation between the proxy and the true outcome (Table 5).

ProxyRMSEmean correlation with the true outcome
Claude Opus 5.51.320.31
GPT-6.1 Sol1.410.29
DeepSeek V4 Pro1.560.26
DeepSeek V4 Flash1.500.20
GBT1.490.19
Ridge1.530.16

We would like to clarify that the quality of the LLM predictions affects our method mostly through their correlation with the true outcomes, not through the RMSE. Any affine transformation of the predictions (a shift or a rescaling) changes the RMSE, but as long as the correlation is unchanged, the performance of our method is unchanged, because the true outcomes correct the level and the scale of the predictions. For this reason, the paper reports the correlations as the measure of the quality of the LLM predictions (Table 5).

Comparison with standard prediction models. The new semi-synthetic experiment includes two traditional machine-learning techniques that use the same inputs and the same observed data as the LLMs: gradient-boosted trees (GBT), a nonlinear model, and ridge regression, a linear model. Both are trained on the 16 labeled cell lines that the LLMs see as in-context examples, with the same features (Appendix C.2). The proxies they generate are of lower quality than those of the LLMs, and accordingly our method performs worse with their proxies than with the LLM proxies (see the two tables above).

Comments on the paper

1 comment

Dear Reviewers,

We are nearing the end of the discussion phase. If you haven't already done so, please read the authors' responses, reply to acknowledge their feedback, and indicate whether your initial concerns have been addressed (along with any score adjustments, if warranted).

Thank you for your dedicated efforts and valuable insights!

Best, Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.