acceptodds
Under review as a conference paper at ICLR 2027

Not All Exploration Is Informative: Targeted Inference in Adaptive Experiments

Abstract

Adaptive experiments must earn deployment reward while estimating prespecified contrasts. Regret theory asks how much exploration learning requires; adaptive-inference theory recovers valid uncertainty after history-dependent assignment. We ask which exploration informs the requested directions. Equal expected counts can produce arbitrarily different inverse-propensity variance, while equal welfare and scalar information can leave different target contrasts with sharply different precision. We develop clock-separated information flows (\CSIF). A base policy retains its decision clock, while a logged evaluation clock allocates a welfare-priced information portfolio across target directions. Structured acquisition matrices and the predictable quadratic variation for design-based inference remain separate information states. Finite-sample matrix concentration and clock transfer connect deployment to a deterministic reference. An independent Gaussian pilot followed by a frozen evaluation design gives a pilot-to-deployment certificate: under balanced-pilot and leverage conditions, pilot-induced design and reference-information drift scale as . Logged propensities give exact design unbiasedness, a finite-sample interval, and, under score regularity, a studentized limit. At a fixed Gaussian reference instance, a local two-point risk lower bound and rounded fixed-design attainability identify the geometric welfare price up to a universal constant. In -repetition synthetic studies, targeting reduces worst-target standard error by in the bandit model and in the linear model under matched evaluation schedules. Across train/held-out splits of a 64,000-customer randomized e-mail trial, it reduces prospective plug-in worst-contrast width by for a 12,000-customer experiment. Information is therefore a directionally allocated resource, not a pull-count total.

Primary area:
causal reasoning
Keywords:
Causality, Experimental design

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_r4N1OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims to reexamine the alignment strategy within the Regret strategy. The proposed method separates the assignment of the base policy from that of the evaluation to control for the possibility of an imbalance. We provide theoretical guarantees regarding the validity of the proposed method and also evaluate it using synthetic data.

Critical strengths

It is very interesting that the paper identifies issues with conventional regret strategies and proposes countermeasures—including theoretical justifications—for them, and I believe this will be meaningful for future developments.

Critical weaknesses

It seems that this paper focuses on whether the concept behind the proposed method functions appropriately. The fundamental challenge should be to resolve the issues with conventional methods. Regarding the search efficiency of the Regret strategy, the paper does not consider the possibility that the issues it points out may already have been resolved by conventional methods, or that they may not even be bottlenecks in the first place. (This is not an argument that the methods or approaches are identical.)

While I believe it is possible to argue that the problem addressed by the proposed method can coexist with other existing methods, the discussion in this paper makes it difficult to determine whether the problem being tackled is worthwhile. It seems necessary to demonstrate that the presented problem is indeed an issue even in the latest existing methods, or that the proposed method improves upon them. At the very least, a discussion is needed to justify how the adoption of the proposed method adds value compared to existing methods targeting the same subject.

Questions for authors

Could you explain, with supporting evidence, how the proposed method compares to conventional approaches targeting the same subject in terms of the Riclet strategy?

4 comments
AuthorsOpenReview ↗

Thanks for the review and here is the kind response

W1 / Question. What practical value does CS–IF add to current regret–inference methods?

We understand the review’s reference to the “Riclet strategy” as a question about regret-based assignment. For this question, CS–IF improves the precision obtained from a welfare-limited experiment by directing evaluations toward the requested comparisons. We now quantify this value using the complete online algorithm, current target-aware comparators, native MAD inference, and the welfare required to reach a specified accuracy. On the empirical finite-grid frontier of the 600-unit online experiment, reaching worst-target AIPW RMSE .030 costs 21.84 units of welfare under CS–IF, versus 29.88 for uniform evaluation and 27.28 for the MAD-style assignment: savings of 26.9% and 20.0%, including the pilot.

1. The practical bottleneck is the weakest requested comparison.

The scientific question is: how should an adaptive experiment spend its welfare budget so that each prespecified treatment comparison is precise? Reward learning, average arm accuracy, and worst-contrast accuracy place different values on the same observations. This distinction persists even when the amount and welfare cost of exploration are fixed. Proposition 2.1 further shows that matching expected counts and welfare permits arbitrarily different HT variances: allocation timing and logged propensities also matter.

An exact fixed-design Gaussian calculation makes the difference concrete. Suppose an incumbent receives n0n_0 observations, the qq challengers have the same reward gap gg, and their total allocation is ss. Every challenger allocation then costs exactly gsgs. Minimizing the worst challenger-versus-incumbent variance over continuous positive allocations gives V⋆=σ02n0+∑a=1qσa2s,na⋆=sσa2∑bσb2.V^\star=\frac{\sigma_0^2}{n_0}+\frac{\sum_{a=1}^q\sigma_a^2}{s},\qquad n_a^\star=\frac{s\sigma_a^2}{\sum_b\sigma_b^2}. Indeed, if v=max⁡aσa2/nav=\max_a\sigma_a^2/n_a, then na≥σa2/vn_a\ge\sigma_a^2/v and hence v≥∑aσa2/sv\ge\sum_a\sigma_a^2/s; the displayed allocation attains equality. Uniform challenger allocation instead contributes qmax⁡aσa2/sq\max_a\sigma_a^2/s. Thus equal welfare can buy substantially different precision. For two challengers with SDs .15 and .30, targeting reduces this challenger variance term from .18/s.18/s to .1125/s.1125/s, a 37.5% reduction. With n0=400,s=200,σ0=.20n_0=400,s=200,\sigma_0=.20, the resulting worst-contrast SE falls from .03162 to .02574, or 18.6%. A 100,000-draw Gaussian check gives RMSE .03165 and .02580.

This calculation is the classical MV solution for that setting. It identifies when target optimization adds value and why a strong target-aware design should attain the same benchmark. CS–IF uses this geometry inside an experiment that learns the allocation and continues reward-oriented decisions.

2. Complete online comparison at the same total welfare cost.

We ran the repeatedly updated Eq. (3.3), recomputing gaps and variance loads after every observation and solving the positive-KL convex problem before the next assignment. The primary Gaussian experiment has K=3K=3, T=600T=600 including 60 randomized pilot assignments, 500 repetitions, means (.30,.20,.20)(.30,.20,.20), and SDs (.20,.15,.30)(.20,.15,.30). Methods share potential-outcome streams, target contrasts, the UCB base, and the full-horizon logged-propensity AIPW estimator. Uniform/two-history and MAD-style uniform/shared-history assignments isolate allocation and history routing. The strong online MV family uses the same targets and floor, with both zero KL and the same positive-KL grid as CS–IF.

Total mean welfare budgetCS–IFUniform evaluationMAD-style assignmentStrong online MV
25.027565.034745.031756.027552
30.024596.029882.028018.024655
35.022771.026585.026654.023072

Entries are worst-target AIPW RMSE. CS–IF reduces RMSE by 14.3–20.7% relative to uniform and 12.2–14.6% relative to MAD-style assignment across these budgets. At B=30B=30, paired bootstrap intervals for the RMSE difference are [−.00684,−.00367][-.00684,-.00367] against uniform and [−.00506,−.00178][-.00506,-.00178] against MAD-style. CS–IF's marginal coverage is .951–.956 and Bonferroni simultaneous coverage is .958. Strong online MV attains matching precision within Monte Carlo uncertainty: CS–IF retains the efficiency of target-specific design in this setting.

The comparison evaluates the entire declared routing/KL grid. An empirical frontier mixes each target's MSE and total cost over complete configurations selected at trial initialization, then takes the maximum and square root. All assignments, including the pilot, enter both horizon and welfare. At budgets 25 and 30, all four frontier costs equal the budget; at 35 the MAD-style family uses 34.79 and the other families use 35. The paired intervals above resample repetitions conditional on the frontier weights.

The reverse comparison answers the adoption question directly: the minimum total cost on this empirical frontier to reach RMSE .030 is

AuthorsOpenReview ↗

Thanks for the review and here is the kind response(cont'd)

21.8353/29.8752/27.2797/21.8733 for CS–IF/uniform/MAD-style/strong MV. Re-solving the cost-minimization problem in each of 1,000 paired bootstrap draws gives cost-difference intervals [−9.565,−5.763][-9.565,-5.763] against uniform and [−7.300,−2.827][-7.300,-2.827] against MAD-style. Each deployed configuration itself uses observed outcomes alone.

Across K=3,5,10K=3,5,10, CS–IF has lower point RMSE in all 17 matched-budget comparisons against uniform and 15 of 17 against MAD-style. We tested breadth in nine noise/target settings: SDs (.20,.15,.30)(.20,.15,.30), (.20,.10,.40)(.20,.10,.40) and (.20,.20,.20)(.20,.20,.20), crossed with targets B−AB-A, C−AC-A, and both contrasts. All 216 online configurations have 500 repetitions, T=600T=600 including the 60-unit pilot, and the same routing/KL grids for the target-aware methods. Across the nine settings, geometric-mean RMSE reductions over common welfare caps B=25,30,35,40B=25,30,35,40 range from 15.5–19.7% against uniform and 11.0–15.7% against MAD-style assignment. At B=30B=30, the reductions are 17.6% and 13.4%, with favorable paired intervals in eight of nine settings for each comparator. At caps 35 and 40, both observed-error and conditional-moment risk estimates favor CS–IF in all nine settings. Strong online MV retains comparable target efficiency.

We also checked inverse-propensity tails by integrating the exact conditional AIPW score variance along each simulated history. Simulator means/variances enter this evaluation; the online decisions use observed outcomes. For the nine-setting suite at B=30B=30, the resulting root-MSE reductions are 16.6% against uniform and 12.3% against MAD-style. Replaying the primary comparison reproduces every saved estimate and welfare cost exactly; at its original B=30B=30 frontier weights, integrated root-MSE is .024543/.028969/.027594/.024682 for CS–IF/uniform/MAD-style/MV. Both observed-error and conditional-moment evaluations support the allocation gain.

3. How this relates to current methods targeting inference and regret.

MAD supplies general mixture randomization and asymptotic confidence sequences. Lin, Ham and Bojinov's NARP (2026) learns variances, reward gaps and the exploration rate online; its aggregate arm-RMSE objective uses a rooted-Neyman exploration profile. Classical MV design supplies contrast geometry, and Simchi-Levi and Wang characterize the regret–inference Pareto tradeoff. CS–IF optimizes the weakest requested contrast through a priced evaluation update and quantifies how errors in that learned design propagate into deployment information. Its specified reward-history protocol and logged inference complete this construction. The acquisition problem recovers classical contrast-optimal design in the known-information, full-evaluation limit with zero welfare and KL penalties. The online comparison shows that target-design efficiency is retained here.

We implemented NARP's native Algorithm 3, including its square-root minimum-count quota, and evaluated the target-precision gain under moderate and high noise heterogeneity. The complete three-profile grid is retained with the code. Each profile has 500 repetitions per configuration, T=600T=600, the same balanced 20-per-arm warm start, and the same standalone UCB learner. NARP receives a 7×37\times3 exploration-weight/quota grid; CS–IF receives a 6×36\times3 routing/KL grid. Both are evaluated using their full-horizon arm sample means to estimate the same two contrasts; this is a separate common-estimator comparison from the AIPW table above.

Noise SDsTotal budgetCS–IF contrast RMSENative NARP contrast RMSE
(.20,.15,.30)(.20,.15,.30)30.023726.026066
(.20,.10,.40)(.20,.10,.40)30.029888.032455

Across the two heterogeneous profiles and budgets 20,25,30,3520,25,30,35, CS–IF reduces worst-contrast RMSE by 4.5–12.1%. At B=30B=30, the paired RMSE-difference intervals are [−.00409,−.00072][-.00409,-.00072] and [−.00461,−.00061][-.00461,-.00061], conditional on the empirical frontier weights. CS–IF has lower point RMSE in all eight heterogeneous-profile/budget comparisons, with favorable paired intervals in six. The full 138-configuration results connect the gain to noise heterogeneity and the requested contrast criterion.

We also ran MAD's published HT asymptotic confidence sequence on both assignment laws, using bounded Bernoulli outcomes, T=1500T=1500 including 60 randomized pilot units, 500 repetitions, and ρt=min⁡1,1.6t−.24\rho_t=\min\\{1,1.6t^{-.24}\\} after the pilot. Under the sharp null, both methods have zero welfare loss. Targeting reduces mean worst confidence-sequence width from .36926 to .31049 (15.9%), with measured joint path coverage .972 versus .962 over t=100,…,1500t=100,\ldots,1500. Both use the same CS and per-contrast α=.025\alpha=.025. Thus the allocation improvement also benefits MAD's own inferential procedure.

4. Evidence in a marketing-calibrated sequential experiment.

We

AuthorsOpenReview ↗

Thanks for the review and here is the kind response(cont'd)

fitted Bernoulli visit rates from Hillstrom's 64,000 historical observations: .10617 for No E-Mail, .18276 for Men's E-Mail, and .15140 for Women's E-Mail. We then ran a sequential, real-data-calibrated simulation with T=1500T=1500, including 90 randomized pilot assignments. Each method learns from its trial observations; target-aware methods reoptimize online. The 216 final configurations cover the joint two-contrast question and each email-versus-control question separately, with 500 repetitions for the joint question and 300 per single-contrast configuration. Routing, KL and welfare-price grids are fixed before the final fresh-seed evaluation and applied to all relevant design families.

For the prespecified Men's E-Mail versus No E-Mail question at a welfare cap of 40 expected lost visits:

MethodConditional-moment root-MSEMean 95% interval widthRejection probability at the fitted effect
CS–IF.019161.075084.9775
Uniform evaluation.029447.115089.7182
MAD-style assignment.023579.092332.8617

CS–IF reduces root-MSE by 34.9% and 18.7%, respectively. Its measured interval coverage is .9433; the fitted effect is .07659. Reaching root-MSE .025 requires 24.56 expected lost visits for CS–IF, versus 45.35 for uniform and 36.53 for MAD-style: welfare savings of 45.8% and 32.8%. The welfare-priced MV family shares the target-design oracle and attains the same frontier. Full joint and single-contrast curves, achieved costs, interval coverage and conditional-moment checks accompany this example, showing how the scientific query determines the value of targeting.

5. What supports deployment of the learned allocation?

The analysis separates the quantities practitioners must control: pilot and deployment welfare, sensitivity to estimated inputs, and uncertainty in the requested comparisons. For deterministic routing probabilities in a stationary bandit with bounded gaps, an expected pilot-cost bound CpilC_{\rm pil}, and a valid nondecreasing expected-regret bound gbaseg_{\rm base} for the actual initialized decision-history learner, the routing cap Cpil+gbase(T−m)+Δmax⁡∑t>mρt≤BC_{\rm pil}+g_{\rm base}(T-m)+\Delta_{\max}\sum_{t>m}\rho_t\le B certifies the expected total welfare budget. The welfare price determines how the permitted evaluation mass is distributed.

For continuously updated stationary Gaussian contrasts, Corollary E.1 treats a unique best arm, a known noise upper bound, at least two pilot observations per arm, confidence-calibrated decision-only UCB, and fixed routing/design parameters with positive evaluation floor, 0<ρ<10<\rho<1, and τ>0\tau>0. Compare estimated- and true-input updates with the same base proposal on the common floor-constrained simplex. If ete_t bounds information-criterion error uniformly on that simplex and gtg_t bounds gap error, KL strong convexity gives τdt2≤2et+κgtdt,dt=∣ν^t−νt⋆∣1,κ=λhρ.\tau d_t^2\le2e_t+\kappa g_td_t,\qquad d_t=\\|\widehat\nu_t-\nu_t^\star\\|_1,\quad\kappa=\lambda h\rho. Simultaneous Gaussian mean/variance bounds, combined with a deterministic arm-count lower envelope, give deterministic allocation-error bounds Dˉt\bar D_t. A UCB count bound and routing supermartingale give a deterministic wrong-base envelope WˉT=O(log⁡T)\bar W_T=O(\log T). Their cumulative bounds supply Theorems 4.2–4.3 with deterministic drift bˉT=O((log⁡T/T)1/4)\bar b_T=O((\log T/T)^{1/4}) and leverage [(T−m)ρν‾]−1[(T-m)\rho\underline\nu]^{-1}. The matrix-transfer step conditions on the initial pilot log; its inverse-information conclusion applies when the radius is below one. Appendix E.6 supplies the constants and pilot-count probability. Theorem 4.1 gives the independent-pilot, frozen-design specialization. Theorem 4.4 gives exact AIPW moments for predictable online updates. These results explain how target-optimal allocation, reward learning and logged inference fit together.

The computational cost is small in the primary three-arm setting: a separately verified dual solver takes a median 5.3 microseconds per update after compilation across 180 test inputs. Every update in the primary online comparison passes its global objective bound, with maximum excess 4.1×10−104.1\times10^{-10}. Section 5 and Table 5 present the matched-welfare comparison; Appendix E.6 proves the deployment corollary, and Appendix I reports the complete protocols with coverage and solver checks in the code. The practical criterion is explicit: welfare spent per unit of precision on the comparisons the experiment was designed to answer.

Thank you for your polite and detailed response.

I believe your response addresses the concerns I raised. However, it is necessary to clarify these points within the main body of the paper to prevent potential misunderstandings. Along with the points raised by other reviewers, I would like to request improvements in the presentation to make the paper more accessible and easier for readers to follow. Given the limited options for rating, I will maintain a positive-leaning score.

Reviewer_FkFBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies adaptive experiments that need to collect sufficient reward while estimating the mean differences between arms. To address this problem, the authors propose clock-separated information flows, which integrates bandit algorithms with a separate randomized evaluation algorithms that prioritizes arms with the highest variance. The objective is to minimize the variance in the mean differences between arms while achieving a certain regret level. The authors show that such an algorithm can achieve O(T^{2/3}) regret and a standard error of O(T^{-1/3}), and no method with the same regret constraint can do better by more than a constant factor.

Critical strengths

The paper considers an important problem in adaptive experimentation: how to allocate a limited exploration budget to improve the precision of inference. The authors propose a simple yet effective algorithm and deliver technical analysis of regret and standard error in estimation. Moreover, the AIPW estimates are unbiased. In the experiments, the proposed algorithm also achieves good performance compared to various benchmarks. The application to randomized marketing data also helps illustrate the practical value of choosing which arms to sample. I also found the examples in the main text easy to follow and insightful.

Critical weaknesses

The main limitation concerns the analytical results. I find that the idea of the proposed algorithm is closely related to the MAD proposed by Liang, Bojinov (2023). Meanwhile, Theorem 4.1 considers an independent Gaussian pilot followed by a frozen design with a greedy base policy, instead of the Algorithm 1. Another limitation of this work is the organization. I find this paper uses many new terms, such as ledger, clock transfer and certificate, before defining them. This makes it hard to follow the detail of the proofs and appreciate the contribution.

Questions for authors
  1. Can the authors discuss how the proposed algorithm differs from the MAD?
  2. Algorithm 1 replaces the full master problem in Equation (3.0) with the problem in Equation (3.3), which depends on three parameters, namely \lambda_t, h_t, \tau_t. How should these be chosen in practice.
  3. In Appendix E, the authors derive the regret and standard error analysis for three-arm scenario. I wonder whether this result can be extended to a more general setting?
5 comments
AuthorsOpenReview ↗

Thanks for the review and here is the kind response

Q1 / W1. How does CS–IF differ from MAD?

Method and theory. CS–IF chooses the evaluation channel to protect prespecified contrasts and couples it to a reward learner whose decision history is preserved. MAD provides a general uniform-mixture design around an arbitrary adaptive policy, together with asymptotic anytime-valid inference. Our specification adds three concrete objects: a target-dependent optimizer νt\nu_t, a welfare price for its marginal information, and a transfer analysis through the base learner's own update history.

The scientific question is how to spend randomized evaluations so that a welfare-limited experiment resolves its weakest requested comparison. The difficulty is that learning the allocation changes both the information state and subsequent reward decisions. CS–IF sends every observation to the information model, while decision rounds advance the original base learner. This preserves a standalone reward trajectory to which the learned design can be compared. Proposition 2.1 exhibits equal expected counts and welfare with arbitrarily different HT variance, and equal information trace with more than 133-fold different worst-contrast variance. The resulting allocation/transfer/inference decomposition can be reused with different target sets and reward learners.

The evaluation rule purchases marginal target information. For linear information G(a)=xaxa⊤/σa2G(a)=x_ax_a^\top/\sigma_a^2, its marginal value, apart from the common factor htρth_t\rho_t, is ∑jαj(cj⊤M−1xa)2σa2,\sum_j\alpha_j\frac{(c_j^\top M^{-1}x_a)^2}{\sigma_a^2}, where the weights αj\alpha_j identify the active worst targets and MM is the forecast information matrix. Thus the rule is more specific than choosing the highest-variance arm. For example, with current information I2I_2 and target c=e1c=e_1, ten observations of x=e2x=e_2 with noise variance 100 leave target variance at 1; ten observations of x=e1x=e_1 with noise variance 1 reduce it to 1/111/11. The target projection and existing information determine which observation is valuable.

The relation to MAD also has an exact compatibility identity. For νt(a)≥η\nu_t(a)\ge\eta and 0<δt′=Kρtη<10<\delta'_t=K\rho_t\eta<1, set pt′=(πt−ρtη1)/(1−δt′)p'_t=(\pi_t-\rho_t\eta\mathbf1)/(1-\delta'_t). Then πt=(1−δt′)pt′+δt′1/K.\pi_t=(1-\delta'_t)p'_t+\delta'_t\mathbf1/K. This allows the target-directed assignment to use MAD's published HT confidence sequence under its outcome, variance-growth, and routing-rate conditions. The additional result concerns how the evaluation allocation is learned and transferred around the protected reward history. The history rule is an explicit CS–IF specification; MAD permits arbitrary wrapped policies.

Online guarantee, including W1's Algorithm 1 concern. We derive a continuous-update corollary for stationary Gaussian mean contrasts with a unique best arm, known noise upper bound, confidence-calibrated decision-only UCB, constant 0<ρ<10<\rho<1, positive evaluation floor η\eta, and τ>0\tau>0. The pilot supplies at least n0≥2n_0\ge2 observations per arm. Both gap and sample-variance loads are updated after every observation; a variance-projection interval contains the true variances. For estimated- and true-input updates with the same base proposal, let dtd_t be their ℓ1\ell_1 allocation distance. If gtg_t bounds gap error and ete_t uniformly bounds objective error on the common feasible set, KL strong convexity gives \tau d_t^2\le2e_t+\kappa g_td_t,\qquad d_t\le D_t:=\min\\!\left\\{2,\frac{\kappa g_t+\sqrt{\kappa^2g_t^2+8\tau e_t}}{2\tau}\right\\}, with κ=λhρ\kappa=\lambda h\rho. Corollary E.1 makes the transfer inputs deterministic before deployment. With ξ=ρη\xi=\rho\eta, Bernoulli concentration gives the simultaneous count lower envelope n−(s)=n0+ξs1s≥s0/2n_-(s)=n_0+\xi s\mathbf1\\{s\ge s_0\\}/2, where s0=⌈8ξ−1log⁡(K(T−m)/δN)⌉s_0=\lceil8\xi^{-1}\log(K(T-m)/\delta_N)\rceil. Substituting this envelope into the Gaussian mean/variance radii gives deterministic Dˉt\bar D_t. The UCB count bound and routing-supermartingale argument give a deterministic wrong-base bound WˉT=O(log⁡T)\bar W_T=O(\log T). For n=T−mn=T-m, ϵTeval≤ρ∑tDˉt+2ρWˉT,bT≤bˉT:=ρ∑tDˉt+2WˉTnξ=O((log⁡T/T)1/4).\epsilon_T^{\rm eval}\le\rho\sum_t\bar D_t+2\rho\bar W_T,\qquad b_T\le\bar b_T:=\frac{\rho\sum_t\bar D_t+2\bar W_T}{n\xi}=O((\log T/T)^{1/4}). For G(a)=eaea⊤/σa2G(a)=e_ae_a^\top/\sigma_a^2, leverage is at most (nξ)−1(n\xi)^{-1}. Conditioning on the initial pilot log makes MTrefM_T^{\rm ref} measurable; Theorem 4.3 then uses deterministic b=bˉTb=\bar b_T and ℓ=(nξ)−1\ell=(n\xi)^{-1}. Its inverse-information conclusion holds when the resulting radius is below one. Appendix E.6 supplies the explicit radii, constants and pilot-count probability. Theorem 4.1 gives the frozen-deployment specialization; Theorem 4.4 and Appendix F give the exact AIPW moments and interval conditions. Sections 3–4 now map these guarantees to Algorithm 1.

Experiments. We isolated channel allocation and history routing in a 2×22\times2 experiment: T=600T=600 including 60 randomized pilot assignments, 500 repetitions, ρ=.4\rho=.4, h=1h=1, λ=1\lambda=1,

AuthorsOpenReview ↗

Thanks for the review and here is the kind response(cont'd)

τ=.01\tau=.01, and the same full-horizon logged-propensity AIPW estimator. Gaps and loads are recomputed and Eq. (3.3) is solved after every observation.

Evaluation channelBase historyMean total welfare lossWorst-target RMSE
UniformShared: MAD-style UCB comparator19.6750.036544
TargetedShared26.3306.025320
UniformDecision-only23.2292.036416
TargetedDecision-only28.7802.025024

Targeting lowers RMSE by 31.3% with decision-only histories and 30.7% with shared histories. The table records the associated welfare costs; the decision-only protocol supplies the protected trajectory used by the transfer theorem. At common mean welfare budgets B=25,30,35B=25,30,35, the complete online empirical frontier gives CS–IF RMSE .027565/.024596/.022771.027565/.024596/.022771, versus .031756/.028018/.026654.031756/.028018/.026654 for MAD-style assignment: reductions of 13.213.2\\%/12.2\\%/14.6\\%. Pilot costs are included. The frontier mixes each target's MSE and cost over the prespecified configurations before taking worst-target RMSE. At B=30B=30, CS–IF marginal coverage is .951.951–.956.956 and Bonferroni simultaneous coverage is .958.958.

We also implemented MAD's native HT asymptotic confidence sequence on both assignment laws. In 500 bounded-Bernoulli runs with T=1500T=1500, ρt=min⁡1,1.6t−.24\rho_t=\min\\{1,1.6t^{-.24}\\} after the randomized pilot, and monitoring t=100,…,1500t=100,\ldots,1500, the sharp-null comparison has zero welfare loss for both designs. Targeting reduces mean worst CS width from .36926.36926 to .31049.31049 (15.9%); joint path coverage is .972.972 versus .962.962, and false-crossing rates are .028.028 versus .038.038. This uses the same time-uniform inferential procedure and α=.025\alpha=.025 per contrast. Section 5 and Appendix I report the comparison; Appendix B states the relationship to MAD.

Q2. How should λt,ht,τt\lambda_t,h_t,\tau_t be chosen in practice?

Operational rule. Assign the three parameters distinct roles: the welfare price chooses a point on the cost–information tradeoff, the look-ahead specifies the forecast window, and the KL weight controls sensitivity to estimated inputs. The outer routing mass provides the budget guarantee. For a deterministic routing schedule, with pilot charge CpilC_{\rm pil}, an initialized base expected-cost bound gbase(T−m)g_{\rm base}(T-m), and gap bound Δmax⁡\Delta_{\max}, the rule Cpil+gbase(T−m)+Δmax⁡∑t>mρt≤BC_{\rm pil}+g_{\rm base}(T-m)+\Delta_{\max}\sum_{t>m}\rho_t\le B certifies the all-in expected budget. A high-probability version uses a pathwise pilot charge, a high-probability standalone-base bound, and the evaluation-count Bernstein margin. The welfare price then directs the permitted evaluations.

ParameterImplementable choice
λt\lambda_tChoose the effective price κt=λthtρt\kappa_t=\lambda_th_t\rho_t using a held-out calibration model and an estimated welfare ceiling. For fixed current inputs, domain and KL weight, estimated cost is nonincreasing in κt\kappa_t, so a bracketed price search solves the current constrained update.
hth_tSet ht=1h_t=1 in the design-based Eq. (3.2), absorbing it into the effective price. In the acquisition Eq. (3.1), use the remaining horizon or a declared deployment batch: hth_t changes the forecast information matrix.
τt\tau_tEvaluate the sensitivity grid 10−3,10−2,10−110^{-3},10^{-2},10^{-1} in the criterion's stated units; the selection protocol below compares .01/.1.01/.1 using held-out estimated score variance at the chosen welfare level. Record the price, pilot size and normalization with the selected value.

The current-update price rule follows by adding the optimality inequalities at two prices: for κ2>κ1\kappa_2>\kappa_1, (κ2−κ1)⟨ν2−ν1,Δ^t⟩≤0(\kappa_2-\kappa_1)\langle\nu_2-\nu_1,\widehat\Delta_t\rangle\le0. The routing cap above handles the complete experiment's budget as its history evolves.

The KL tradeoff is quantitative. If ff is the unregularized convex objective and ω\omega has full support, 0≤f(ντ)−f(ν0)≤τlog⁡(1/ωmin⁡).0\le f(\nu_\tau)-f(\nu_0)\le\tau\log(1/\omega_{\min}). Together with the simultaneous-input bound above, a desired allocation tolerance d0d_0 and objective-bias allowance ϵreg\epsilon_{\rm reg} can be checked using 2ed02+κgd0≤τ≤ϵreglog⁡(1/ωmin⁡).\frac{2e}{d_0^2}+\frac{\kappa g}{d_0}\le\tau\le\frac{\epsilon_{\rm reg}}{\log(1/\omega_{\min})}. Here e,ge,g are uniform confidence bounds on the actual inputs; the practical calibration uses observable score-variance estimates.

Observed-data calibration and sensitivity. We executed a reproducible selection protocol: fit Gaussian arm means/variances from an independent 3,840-observation logged calibration sample in the Gaussian study, simulate 54 prespecified candidate settings with 64 replications each in that fitted model, and minimize estimated worst score variance among candidates meeting an estimated budget of 30. The candidate grid covers ρ=.25/.4/.55\rho=.25/.4/.55, λ=.1/1/10\lambda=.1/1/10, τ=.01/.1\tau=.01/.1, and pilot sizes 30/60/12030/60/120. The calibration sample is separate from the future trial; the 54×64 candidate trials are

AuthorsOpenReview ↗

Thanks for the review and here is the kind response(cont'd)

computations using that fitted model. Parameters are frozen before a fresh 500-run evaluation in the original environment.

The selected design-based configuration is ρ=.4,λ=.1,h=1,τ=.01\rho=.4,\lambda=.1,h=1,\tau=.01 with 120 randomized pilot units inside T=600T=600. Fresh mean total cost is 28.4878±.0749428.4878\pm.07494 (Monte Carlo SE), worst-target RMSE is .025358.025358, and marginal coverages are .936/.946.936/.946. All these new runs pass the global optimization certificate, with zero solver failures.

This sensitivity scan uses a separate simulation seed from the primary online comparison. At fixed ρ=.4\rho=.4, pilot 60 and τ=.01\tau=.01, the declared price scan gives:

λ\lambdaMean total costWorst-target RMSE
029.0174.026175
.129.0156.026179
128.9984.026192
1024.9452.027762

Across τ=.001/.01/.1\tau=.001/.01/.1, RMSE is .026188/.026192/.026420.026188/.026192/.026420; across pilot sizes 30/60/12030/60/120, it is .025754/.026192/.025271.025754/.026192/.025271, with every pilot charged within the same 600-unit trial. For the design-based branch, (λ,h)=(1,1),(.2,5),(.05,20)(\lambda,h)=(1,1),(.2,5),(.05,20) produces identical actions and propensities at all 500×600500\times600 logged positions. This removes an unnecessary tuning degree of freedom.

We separately tested the acquisition look-ahead. To keep parameter units interpretable, set At=Φt(ω)A_t=\Phi_t(\omega), use a fixed cost scale D0D_0, and optimize Φt(ν)/At+β⟨πt(ν),Δ^t⟩/D0+τˉKL⁡(ν∥ω).\Phi_t(\nu)/A_t+\beta\langle\pi_t(\nu),\widehat\Delta_t\rangle/D_0+\bar\tau\operatorname{KL}(\nu\Vert\omega). This is Eq. (3.3) with λt=βAt/(htD0)\lambda_t=\beta A_t/(h_tD_0) and τt=τˉAt\tau_t=\bar\tau A_t. In 500 online diagonal-Gaussian acquisition runs per setting, with β=.1,τˉ=.01,ρ=.4\beta=.1,\bar\tau=.01,\rho=.4, h=1/25/remainingh=1/25/\text{remaining} gives mean Gaussian target-information SE .03254/.02369/.02341.03254/.02369/.02341, Gaussian arm-mean contrast RMSE .03687/.02370/.02331.03687/.02370/.02331, and total costs 17.9858/27.8248/28.822817.9858/27.8248/28.8228. This measures the forecast-window effect in the acquisition objective. Eq. (3.1) evaluates Gaussian parameter precision; Eq. (3.2) evaluates the propensity-sensitive score criterion used for design-based inference. Section 3 gives the parameter rule beside Eq. (3.3); Appendix I reports calibration and sensitivity, with the full tables in the accompanying code.

Q3. Does the three-arm regret/SE result extend to a general setting?

Theory. The design formulation allows arbitrary finite numbers of arms, target contrasts and parameter dimensions. An explicit extension of the closed-form example uses one incumbent and K−1K-1 equal-gap challengers, common gap g>0g>0, independent Gaussian noise variance σ2\sigma^2, and all challenger-versus-incumbent targets. With continuous fixed-design counts and no prior information, let ss be the total challenger allocation. Equal challenger counts give V_K(s)=\sigma^2\left[\frac1{T-s}+\frac{K-1}{s}\right],\qquad s^\star=\min\\!\left\\{\frac Bg,\frac{\sqrt{K-1}}{1+\sqrt{K-1}}T\right\\}. The proof uses max⁡a1/na≥(K−1)/s\max_a1/n_a\ge(K-1)/s and minimizes the resulting one-dimensional convex function. In the active-budget regime, ΦB=σ2[1T−B/g+(K−1)gB].\Phi_B=\sigma^2\left[\frac1{T-B/g}+\frac{(K-1)g}{B}\right]. Thus B=bT2/3B=bT^{2/3} gives SE⁡∼σ(K−1)g/b,T−1/3\operatorname{SE}\sim\sigma\sqrt{(K-1)g/b}\\,T^{-1/3} for fixed KK. The target count appears in the constant. For a subset of qq challenger targets with fixed nuisance-arm count nfloorn_{\rm floor}, define T′=T−(K−1−q)nfloorT'=T-(K-1-q)n_{\rm floor} and B′=B−g(K−1−q)nfloorB'=B-g(K-1-q)n_{\rm floor}; the same formula uses q,T′,B′q,T',B'.

The online design-based rate also extends directly. If the loaded scores obey ∣ψtj(a)∣≤Uja|\psi_{tj}(a)|\le U_{ja} and evaluation probabilities satisfy νt(a)≥ηa\nu_t(a)\ge\eta_a on the loaded coordinates, then ρt=ρ0t−1/3\rho_t=\rho_0t^{-1/3} gives Var⁡(τ^Tj)≤[34ρ0+o(1)]∑aUja2ηa,T−2/3.\operatorname{Var}(\widehat\tau_{Tj})\le\left[\frac{3}{4\rho_0}+o(1)\right]\sum_a\frac{U_{ja}^2}{\eta_a}\\,T^{-2/3}. The decision-clock regret bound adds Cpil+gbase(T−m)+O(T2/3)C_{\rm pil}+g_{\rm base}(T-m)+O(T^{2/3}). When pilot and base cost are O(T2/3)O(T^{2/3}) or smaller, this yields the stated T2/3T^{2/3} regret and T−1/3T^{-1/3} standard-error exponents for general fixed KK. Theorem 4.5 identifies the matching local Gaussian reference price: at a Gaussian reference with Eθ0RT≤B\mathbb E_{\theta_0}R_T\le B and an admissible constructed local alternative, max⁡i=0,1max⁡jMSE⁡θi(τ^j)≥ΦB/16\max_{i=0,1}\max_j\operatorname{MSE}_{\theta_i}(\widehat\tau_j)\ge\Phi_B/16, while feasible fixed-design OLS attains the reference oracle up to rounding. This is the precise constant-factor comparison underlying the Summary's description.

Experiments. We evaluated K=3,5,10,20K=3,5,10,20 with feasible integer reference allocations and 5,000 independent Gaussian arm-mean samples per design. Across 15 reference cases, mean-per-target empirical/theoretical variance ratios are .990.990–1.0131.013 and marginal coverage is .943.943–.957.957. A five-horizon scan, T=2000,4000,8000,16000,32000T=2000,4000,8000,16000,32000 and B=.2T2/3B=.2T^{2/3}, gives empirical log–log pooled-target RMSE slopes −.348,−.341,−.337,−.340-.348,-.341,-.337,-.340, respectively, tracking the predicted −1/3-1/3.

We additionally executed complete, positive-KL, per-observation

Reviewer_BPwWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to allocate exploration in adaptive experiments when the goal is both to earn reward and to estimate a prespecified set of treatment contrasts. The main point is that exploration counts alone do not tell us how precisely those contrasts can be estimated.

The authors propose clock-separated information flows (CS-IF), which mixes a base policy with an evaluation distribution chosen to improve precision for the requested targets. Evaluation observations are used for inference but do not advance the base learner's decision clock. The method also distinguishes model-based information acquisition from design-based variance under logged assignment probabilities.

The theoretical results include a per-update allocation condition, bounds for transferring a pilot-fitted design to deployment, clock and information-matrix concentration results, AIPW inference, and a local Gaussian lower bound. Experiments cover synthetic bandits, fixed-population inference, linear models, and a train/held-out analysis of the Hillstrom email dataset.

Critical strengths

1. The motivation is concrete and technically relevant.

Proposition 2.1 gives a clear reason to look beyond exploration counts. In the three-arm example, allocating almost all evaluation samples to one challenger leaves the other contrast poorly estimated, despite preserving the same total sample size and welfare cost. The example makes the target-specific objective easy to understand.

I see this as a useful problem formulation, particularly when the inferential targets are fixed before the experiment.

2. There is substantial work on inference and empirical validation.

I appreciate that the authors distinguish the Gaussian information calculations from design-based inference instead of using one variance formula for both.

The fixed-population experiment reports a reduction in worst-contrast interval width from 0.0246 to 0.0223, with 94.0% marginal coverage for the targeted method. The paper also compares against MAD assignment and classical A-/G-optimal designs, and reports pilot-drift diagnostics.

The Hillstrom experiment adds a useful check using independent train/held-out splits. The reported 4.3% improvement is modest, but the analysis is more informative than evaluating the allocation on the same observations used to fit it. I also appreciate that the authors report when the pilot-only certificate becomes nonvacuous.

Critical weaknesses

1. I am not convinced that the main theoretical guarantee covers the algorithm being proposed.

Algorithm 1 repeatedly solves Eq. (3.3), updating the gap estimates and information loads as observations arrive.

Theorem 4.1 analyzes a narrower setting: an independent Gaussian pilot is used to choose a design, and that design is then frozen. Its stability proof compares two optimizers with the same information criterion and different gap estimates. It therefore does not directly address what happens when the information criterion also changes online.

Theorems 4.2 and 4.3 give useful conditional bounds, but the required tracking and drift conditions are not established for the complete repeated-update procedure. I also could not find a guarantee that the offline calibration of the welfare-price parameters enforces the global budget in Eq. (3.0) under those updates.

This distinction matters for interpreting the experiments. The fixed-population study does update the evaluation allocation online, which is useful evidence. But the MAB experiment optimizes the evaluation distribution once, the linear experiment uses a smooth evaluation-only objective, and Hillstrom evaluates prospective allocations on held-out data rather than running a sequential policy.

So I think the paper supports several components of CS-IF, but the evidence for the complete online algorithm is still limited.

2. The connection between the optimized surrogate, the exact inference objective, and existing target-specific design methods needs more work.

Equation (3.2) drops the target-dependent subtraction term from the exact design-based variance before taking the maximum across contrasts.

The subtraction is constant with respect to the allocation for each fixed contrast, but the constants are different across contrasts. Dropping them can therefore change which contrast is worst and can change the optimizer.

For example, fixing the probability of arm A at 0.5, using residuals (0, 1, 2), and targeting B-A and C-A gives an optimum of 0.100 for the probability of B under the surrogate, compared with approximately 0.129 under the exact minimax variance. I checked this numerically.

The paper correctly calls Eq. (3.2) a surrogate, so I am not treating this as a mathematical error. But without an approximation bound or an equivalence condition, it is unclear how closely the optimized quantity tracks the stated inferential objective.

I also think the novelty discussion needs a sharper boundary. Classical c-optimal and MV-optimal designs already address specific contrasts and maximum treatment-contrast variance. Related ideas appear in Fiez et al. (2019), Rosa and Harman (2016), and Connolly et al. (2023). The welfare-inference trade-off has also been studied by Li, Shi, and Simchi-Levi (2024) and Wilder and Welle (2025).

I do not think these works contain the full CS-IF construction. However, they make it important to identify which part of the proposed coupling is genuinely new. Comparisons against A-/G-optimality are useful, but those methods optimize different criteria and are not necessarily the strongest target-aware alternatives.

Questions for authors

Q1. Which guarantee applies to Algorithm 1 with continuously updated gaps and information loads? Can the drift conditions required by Theorems 4.2 and 4.3 be derived for the repeated solution of Eq. (3.3)? I would also like to understand how the global welfare budget is maintained, including the welfare cost of pilot assignments. This would substantially affect my assessment of the theoretical contribution.

Q2. How different can the optimizer of Eq. (3.2) be from that of the exact design-based objective in Eq. (3.0)? Is there a condition under which they agree, or a bound on the worst-contrast variance lost by optimizing the surrogate? I think this matters because the method is motivated by protecting the worst requested contrast.

Q3. What is the contribution relative to classical target-specific optimal design and existing welfare-inference methods? In particular, can the authors clarify what remains new when the model is restricted to the closest c-/MV-optimal design settings? I would also be interested in a matched-budget comparison that runs the complete online Eq. (3.3), rather than only its individual components.

8 comments
AuthorsOpenReview ↗

Thanks for the review and here is the kind response

Q1 / W1. Which guarantees cover repeated online updates, and how is the total welfare budget enforced?

Theory. The continuously updated algorithm has an exact inference guarantee and a learned-design transfer bound. Theorem 4.4 supplies the exact design-based AIPW moments through the logged, predictable propensities. We additionally derive an online corollary for stationary Gaussian mean contrasts with a unique best arm, a known noise upper bound, a decision-only UCB base, constant routing probability 0<ρ<10<\rho<1, evaluation floor νt(a)≥ν‾>0\nu_t(a)\ge\underline\nu>0, and KL weight τ>0\tau>0. The pilot supplies at least n0≥2n_0\ge2 observations per arm. Both gaps and variance loads are re-estimated after every observation; a variance-projection interval contains the true variances. This supplies a continuous-update counterpart to the independent-pilot specialization in Theorem 4.1.

The key is to propagate both input errors. For a common base action and feasible domain, let gt=∣Δ^t−Δ∣∞g_t=\\|\widehat\Delta_t-\Delta\\|_\infty and et=sup⁡ν∣ϕ^t(ν)−ϕt(ν)∣e_t=\sup_\nu|\widehat\phi_t(\nu)-\phi_t(\nu)|. Writing κ=λhρ\kappa=\lambda h\rho, KL strong convexity gives \tau d_t^2\le2e_t+\kappa g_td_t,\qquad d_t\le D_t:=\min\\!\left\\{2,\frac{\kappa g_t+\sqrt{\kappa^2g_t^2+8\tau e_t}}{2\tau}\right\\}. Here dtd_t compares the estimated-input and true-input updates. For loads wja=cja2σa2w_{ja}=c_{ja}^2\sigma_a^2, time-uniform Gaussian mean and sample-variance concentration makes the errors explicit: gt≤2max⁡arμ(Na)g_t\le2\max_a r_\mu(N_a) and et≤(ρν‾)−1max⁡j∑acja2rv(Na)e_t\le(\rho\underline\nu)^{-1}\max_j\sum_a c_{ja}^2r_v(N_a). With x=log⁡(8KT/δs)x=\log(8KT/\delta_s), valid radii are rμ(n)=σˉ2x/nr_\mu(n)=\bar\sigma\sqrt{2x/n} and rv(n)=σˉ2[2x/(n−1)+2x/(n−1)]r_v(n)=\bar\sigma^2[2\sqrt{x/(n-1)}+2x/(n-1)].

For the matrix-transfer step, the bounds are deterministic before deployment. Put n=T−mn=T-m, ξ=ρν‾\xi=\rho\underline\nu, s0=⌈8ξ−1log⁡(Kn/δN)⌉s_0=\lceil8\xi^{-1}\log(Kn/\delta_N)\rceil, and n−(s)=n0+ξs1s≥s0/2n_-(s)=n_0+\xi s\mathbf1\\{s\ge s_0\\}/2. Bernoulli concentration and a union bound give Na(m+s)≥n−(s)N_a(m+s)\ge n_-(s) simultaneously. Define Dˉm+1+s=D!(ξ−1max⁡j∑acja2rv(n−(s)),,2rμ(n−(s))),\bar D_{m+1+s}=\mathcal D\\!\left(\xi^{-1}\max_j\sum_a c_{ja}^2r_v(n_-(s)),\\,2r_\mu(n_-(s))\right), where D(e,g)\mathcal D(e,g) is the displayed stability bound. For confidence-calibrated decision-only UCB, let L_T=\sum_{a\ne a^\star}(1+8\bar\sigma^2x/\Delta_a^2),\qquad \bar W_T=\min\\!\left\\{n,\frac{L_T+\log(1/\delta_c)}{(1-\rho)(1-e^{-1})}\right\\}. The UCB count bound and routing supermartingale control suboptimal base proposals by WˉT\bar W_T. Thus, except with probability δs+δN+δc\delta_s+\delta_N+\delta_c, ϵTeval≤ρ∑t>mDˉt+2ρWˉT,bT≤bˉT:=ρ∑t>mDˉt+2WˉTnξ.\epsilon_T^{\rm eval}\le\rho\sum_{t>m}\bar D_t+2\rho\bar W_T,\qquad b_T\le\bar b_T:=\frac{\rho\sum_{t>m}\bar D_t+2\bar W_T}{n\xi}. For G(a)=eaea⊤/σa2G(a)=e_ae_a^\top/\sigma_a^2, reference-normalized leverage is at most (nξ)−1(n\xi)^{-1}. Condition on the initial pilot log, where MTrefM_T^{\rm ref} is measurable, and apply Theorem 4.3 with deterministic b=bˉTb=\bar b_T and ℓ=(nξ)−1\ell=(n\xi)^{-1}. For fixed parameters and gaps, bˉT=O((log⁡T/T)1/4)\bar b_T=O((\log T/T)^{1/4}); a concentration radius below one yields the inverse-information sandwich. Corollary E.1 and its proof in Appendix E.6 establish this chain. A randomized pilot adds its stated insufficient-count probability.

Budget control. The outer routing mass pays for every assignment; the inner welfare price determines where to spend it. If CpilC_{\rm pil} bounds pilot cost, the initialized standalone base has expected regret bound gbase(n)g_{\rm base}(n), and 0≤Δa≤Δmax⁡0\le\Delta_a\le\Delta_{\max}, choosing deterministic routing probabilities with r=∑t>mρtr=\sum_{t>m}\rho_t gives ERTtotal≤Cpil+gbase(n)+Δmax⁡r.\mathbb E R_T^{\rm total}\le C_{\rm pil}+g_{\rm base}(n)+\Delta_{\max}r. For a high-probability total pseudo-regret budget, use a pathwise pilot charge CpilhpC_{\rm pil}^{hp}, a base bound Gbase(n,δb)G_{\rm base}(n,\delta_b), and choose rr to satisfy Cpilhp+Gbase+Δmax⁡![r+2rlog⁡(1/δe)+23log⁡(1/δe)]≤B.C_{\rm pil}^{hp}+G_{\rm base}+\Delta_{\max}\\!\left[r+\sqrt{2r\log(1/\delta_e)}+\frac23\log(1/\delta_e)\right]\le B. The decision subsequence is a prefix of the standalone base trajectory; evaluation cost is at most Δmax⁡\Delta_{\max} times the number of evaluation rounds. Bernstein's inequality then gives Pr⁡(RTtotal≤B)≥1−δb−δe\Pr(R_T^{\rm total}\le B)\ge1-\delta_b-\delta_e. The implementation reserves the required positive routing floor before allocating the remaining mass. Proposition E.2 states this all-in rule; Algorithm 1 charges the pilot, and Section 2 distinguishes expected, predictable, and realized pseudo-regret.

Experiments. The original fixed-population study updated the allocation online, whereas the original MAB, linear, and Hillstrom studies examined a one-time evaluation design, a smooth evaluation-only objective, and held-out prospective allocations, respectively. The new experiments below evaluate repeated solutions of the complete online Eq. (3.3). We now evaluate repeated, welfare-priced, positive-KL Eq. (3.3), with observed gaps and loads updated every round. The Gaussian comparison uses T=600T=600 total units, including

AuthorsOpenReview ↗

Thanks for the review and here is the kind response(cont'd)

20K20K randomized pilot units, all-versus-control targets, and the fixed grid τ∈.001,.01,.1\tau\in\\{.001,.01,.1\\}. There are 500 repetitions for K=3K=3 and 200 each for K=5,10K=5,10. At every deployment round, the program recomputes gaps and variance loads and solves the full welfare-priced objective with h=1h=1 and welfare price 1. All methods use the same logged-propensity AIPW estimator. At K=3,ρ=.4,τ=.01K=3,\rho=.4,\tau=.01, worst-target RMSE is .025024.025024, mean worst 95% interval width is .100614.100614, and marginal coverages are .960/.962.960/.962. The τ=.001/.01/.1\tau=.001/.01/.1 scan at that routing rate gives RMSE .02504/.02502/.02555.02504/.02502/.02555, showing stable performance across the declared regularization range. The 60 core configurations pass the global optimization certificate, with maximum objective gap 4.07×10−104.07\times10^{-10} and zero solver failures.

An independent continuous-update check using this confidence-calibrated UCB rule at T=300,600,1200T=300,600,1200 (100 repetitions each) records the input errors, base proposals, allocation displacement, and normalized acquisition-information drift. Mean allocation distance to the true-input reference falls from .413.413 to .347.347 to .274.274. These records accompany the derived drift bounds; the matched-budget performance comparison is reported under Q3.

We also executed a separate budget-control stress test with 600 total units, including a balanced 60-unit pilot, and budgets B=12,18,24B=12,18,24. The prespecified Gaussian class has Δmax⁡=.2\Delta_{\max}=.2, minimum best-arm gap .1.1, and σa≤.05\sigma_a\le.05. The pilot-selected greedy base has error probability at most 2e−202e^{-20}; the conservative pilot charge is 8, versus actual pilot cost 6. With δe=.01\delta_e=.01, the routing rule above gives a guarantee of at least 0.99−2e−200.99-2e^{-20}. Across 500 runs per budget, the continuously updated targeted design has mean total costs 7.225,10.502,14.0717.225,10.502,14.071, and zero observed budget exceedances at all three budgets. The guarantee uses model-class bounds specified before simulation; all pilot assignments are charged. The estimand in this stress test uses the fresh deployment units. The main online comparison below includes its randomized pilot in the AIPW estimator.

Q2 / W2. How closely does the surrogate protect the exact worst contrast?

Theory. Equation (3.2) minimizes a conservative bound built from armwise residual loads. Its exact-variance cost is controlled by the spread of the contrast-specific subtraction terms. On a common fixed residual/predictor path and common welfare/positivity domain, write S(π)=max⁡jQj(π),F(π)=max⁡jQj(π)−Dj.S(\pi)=\max_jQ_j(\pi),\qquad F(\pi)=\max_j\\{Q_j(\pi)-D_j\\}. For any ϵ\epsilon-optimal full-horizon surrogate design, 0≤F(π^)−inf⁡πF(π)≤Dmax⁡−Dmin⁡+ϵ.0\le F(\widehat\pi)-\inf_\pi F(\pi)\le D_{\max}-D_{\min}+\epsilon. The proof follows from S−Dmax⁡≤F≤S−Dmin⁡S-D_{\max}\le F\le S-D_{\min}. Equal offsets, including a single target, give identical optimizer sets and worst-target ordering. The coefficient one is sharp: residuals (0,1,ε)(0,1,\varepsilon) on the zero-load domain give excess-to-offset-spread ratio 1/(1+ε)1/(1+\varepsilon), approaching one. Thus offset heterogeneity measures precisely how much the conservative minimax criterion can change the solution.

Your example is reproduced analytically. At pA=.5p_A=.5, the surrogate gives pB=.100000p_B=.100000 and exact worst variance 99; exact minimax design gives pB=(73−7)/12=.128667p_B=(\sqrt{73}-7)/12=.128667 and variance (73+5)/2=6.772002(\sqrt{73}+5)/2=6.772002. Its excess 2.2279982.227998 is below the offset-spread bound 33.

The statistical reason for the armwise criterion is that each unit reveals one potential outcome. Squared residual loads can be estimated armwise; the offsets contain cross-potential-outcome products for the same unit. For bounded residual envelopes ∣rta∣≤Rta|r_{ta}|\le R_{ta}, the bound also has a directly calculable version: Dmax⁡−Dmin⁡≤max⁡jT−2∑t(∑a∣cja∣Rta)2.D_{\max}-D_{\min}\le\max_jT^{-2}\sum_t\left(\sum_a|c_{ja}|R_{ta}\right)^2. Known outcome ranges and logged clipped predictions supply these envelopes. For a horizon surrogate solver with uniform load-objective error EE, penalty oscillation HH, and optimization error ϵs\epsilon_s, the same proof gives excess at most Dmax⁡−Dmin⁡+2E+H+ϵsD_{\max}-D_{\min}+2E+H+\epsilon_s. The horizon result is stated on the common reference path in Eq. (3.0); the per-update learned-design result above governs deployment of Eq. (3.3).

Experiments. We added 216 paired exact/surrogate design problems: K=3,5,10K=3,5,10, all-versus-control/all-pairs/random contrast sets, three positive floors, and four welfare-budget levels. Each pair shares the full residual path, targets, and feasible domain. Every solution passes an independent global primal–dual check, with maximum relative gap

AuthorsOpenReview ↗

Thanks for the review and here is the kind response(cont'd)

9.04×10−79.04\times10^{-7}.

CheckResult
Additive bound satisfied216/216
Median / maximum exact-variance ratio1.0614 / 1.8806
Cases where offsets change the worst active target set191/216
Largest fraction of the offset-spread bound attained93.83%
Allocation difference in equal-offset / single-target controls≤1.6×10−12\le1.6\times10^{-12}

These tests identify both the agreement regime and the magnitude of conservatism when offsets differ. Section 3 links Eq. (3.2) to Proposition E.1; Appendix I reports the paired comparison and identifies the exact surrogate solved in Appendix I.4.

Q3 / W1–W2. What remains new relative to c-/MV-optimal design and welfare–inference methods, and what does the full online method gain at matched budgets?

Theory and positioning. The scientific question is: which evaluations protect every prespecified comparison when both reward gaps and statistical value are learned under a participant-welfare budget? The technical difficulty is that changing the evaluation design changes the information state and the reward learner's subsequent trajectory, while acquisition information and randomization variance are different statistical objects. CS–IF preserves the base's decision history, learns the directional allocation, transfers its input errors into realized information, and performs inference from the actual logged propensity path. The Q1 corollary makes that dependency chain explicit for continuous updates.

The closest classical case is recovered exactly. With known G(a)G(a), pure evaluation, and zero welfare/entropy prices, the acquisition problem is min⁡ν∈ΔKmax⁡jcj⊤[M0+T∑aνaG(a)]−1cj,\min_{\nu\in\Delta_K}\max_j c_j^\top\left[M_0+T\sum_a\nu_aG(a)\right]^{-1}c_j, which is c-optimality for one direction and minimax contrast-variance design for multiple directions. Rosa–Harman, Fiez, and Connolly establish contrast-directed or task-specific design; Li–Shi–Simchi-Levi and Wilder–Welle establish welfare–inference tradeoffs. Our contribution is the deployment coupling and its transfer guarantee around that directional oracle. MAD supplies a related randomized mixture and asymptotic confidence sequence; CS–IF optimizes its evaluation distribution and controls which observations advance the reward learner.

The counterintuitive implication is that equal expected counts and welfare can coexist with arbitrarily different inverse-propensity variance, and a larger scalar acquisition-information measure can leave a requested contrast weaker (Proposition 2.1). These results let a new target set or base learner enter the design through explicit statistical quantities and error bounds, providing a common analysis for contrast estimation and policy-value evaluation.

Matched-budget full-online comparison. We compare the complete positive-KL online procedure with uniform two-clock allocation, a uniform shared-history MAD-style assignment, and online target-aware MV under the same targets, information, positivity domain, and AIPW estimator. The MV comparison includes the same KL grid as CS–IF and an unregularized version.

For each method, we construct the empirical frontier over the prespecified configurations: total cost includes the pilot, and the constraint is mean welfare loss ≤B\le B. A linear program mixes configurations at trial initialization, linearly combines each target's MSE and cost, then minimizes the largest target MSE. This compares full online policies at a common budget; RMSE is computed after combining the target risks. All configurations and mixture weights are retained.

KKWelfare budget BBCS–IFUniformMAD-styleTarget-aware MV
325.02757.03475.03176.02755
330.02460.02988.02802.02466
335.02277.02658.02665.02307
540.03049.03787.03492.03082
1050.04444.05115.04832.04443

Entries are worst-target RMSE. At K=3K=3, the reductions versus uniform are 20.720.7\\%,17.7\\%,14.3\\%; versus MAD-style they are 13.213.2\\%,12.2\\%,14.6\\%. At B=30B=30, the paired bootstrap 95% interval for the CS–IF-minus-uniform RMSE difference is [−.00684,−.00367][-.00684,-.00367]. The corresponding CS–IF-minus-MV interval is [−.00047,.00032][-.00047,.00032], consistent with retaining the strong target-aware design's precision. Bootstrap uncertainty uses 1,000 paired repetition-level resamples conditional on the reported frontier weights. CS–IF marginal coverage across these three K=3K=3 budgets is .947.947–.959.959; Bonferroni simultaneous coverage is .958,.958,.962.958,.958,.962.

All displayed methods satisfy the empirical budget constraint. At K=3,B=25,30K=3,B=25,30, their measured frontier costs equal BB; the full records report actual costs where the optimum leaves budget unused. These experiments evaluate the welfare–precision frontier; the independently certified routing guard under Q1 supplies the deployment budget rule.

As a check on the classical limit, we independently re-solved the 300

Comments on the paper

1 comment
AuthorsOpenReview ↗

Thanks for the review and here is the kind response

Dear Area Chair,

Thank you for coordinating the review and for the reviewers’ constructive feedback. We have revised the manuscript, with changes marked in blue, around five modules:

1. Online guarantees and total welfare control. We distinguish the frozen-design theorem from the continuously updated algorithm. Corollary E.1 derives tracking and information-drift bounds for stationary Gaussian contrasts with a confidence-calibrated decision-only UCB base; Proposition E.2 charges both pilot and deployment welfare. Sections 3–4 and Appendix E.6 state the assumptions and guarantee scopes. See BPwW Q1/W1 and FkFB Q1/W1.

2. Surrogate versus exact variance. Proposition E.1 gives the equal-offset equivalence condition and a sharp additive approximation bound, reproduces the reviewer’s counterexample, and is checked on 216 paired exact/surrogate problems (Section 3, Appendices E.6 and I). See BPwW Q2/W2.

3. Contribution relative to classical design and MAD. Appendix B identifies the classical c-/MV-optimal limit and clarifies the added coupling of learned target allocation, preserved reward history, transfer analysis, and logged-propensity inference. We also test compatibility with MAD’s native confidence sequence. See BPwW Q3 and FkFB Q1.

4. Practical value of the complete online procedure. New experiments repeatedly solve Eq. (3.3) with updated inputs and include pilot costs. Matched-budget comparisons improve precision over uniform and MAD-style assignment while retaining strong target-aware MV efficiency; native NARP, MAD inference, and marketing-calibrated sequential trials provide further evidence (Section 5, Figure 2, Table 5, Appendix I). See r4N1 W1/Question and BPwW Q3.

5. Parameters, generality, and readability. We add observed-data calibration and sensitivity checks, general-K reference results and scale tests through K=20, and definitions before the named information quantities and transfer bounds (Sections 2–4, Proposition E.3, Appendices E.6 and I). See FkFB Q2–Q3/W2 and the continuation replies.

The revised main text is nine pages. Thank you again for your time, careful consideration, and guidance. We warmly welcome further questions and would be happy to discuss any aspect of the responses or revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.