- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper aims to reexamine the alignment strategy within the Regret strategy. The proposed method separates the assignment of the base policy from that of the evaluation to control for the possibility of an imbalance. We provide theoretical guarantees regarding the validity of the proposed method and also evaluate it using synthetic data.
Critical strengths
It is very interesting that the paper identifies issues with conventional regret strategies and proposes countermeasures—including theoretical justifications—for them, and I believe this will be meaningful for future developments.
Critical weaknesses
It seems that this paper focuses on whether the concept behind the proposed method functions appropriately. The fundamental challenge should be to resolve the issues with conventional methods. Regarding the search efficiency of the Regret strategy, the paper does not consider the possibility that the issues it points out may already have been resolved by conventional methods, or that they may not even be bottlenecks in the first place. (This is not an argument that the methods or approaches are identical.)
While I believe it is possible to argue that the problem addressed by the proposed method can coexist with other existing methods, the discussion in this paper makes it difficult to determine whether the problem being tackled is worthwhile. It seems necessary to demonstrate that the presented problem is indeed an issue even in the latest existing methods, or that the proposed method improves upon them. At the very least, a discussion is needed to justify how the adoption of the proposed method adds value compared to existing methods targeting the same subject.
Questions for authors
Could you explain, with supporting evidence, how the proposed method compares to conventional approaches targeting the same subject in terms of the Riclet strategy?
4 comments
Thanks for the review and here is the kind response(cont'd)
21.8353/29.8752/27.2797/21.8733 for CS–IF/uniform/MAD-style/strong MV. Re-solving the cost-minimization problem in each of 1,000 paired bootstrap draws gives cost-difference intervals against uniform and against MAD-style. Each deployed configuration itself uses observed outcomes alone.
Across , CS–IF has lower point RMSE in all 17 matched-budget comparisons against uniform and 15 of 17 against MAD-style. We tested breadth in nine noise/target settings: SDs , and , crossed with targets , , and both contrasts. All 216 online configurations have 500 repetitions, including the 60-unit pilot, and the same routing/KL grids for the target-aware methods. Across the nine settings, geometric-mean RMSE reductions over common welfare caps range from 15.5–19.7% against uniform and 11.0–15.7% against MAD-style assignment. At , the reductions are 17.6% and 13.4%, with favorable paired intervals in eight of nine settings for each comparator. At caps 35 and 40, both observed-error and conditional-moment risk estimates favor CS–IF in all nine settings. Strong online MV retains comparable target efficiency.
We also checked inverse-propensity tails by integrating the exact conditional AIPW score variance along each simulated history. Simulator means/variances enter this evaluation; the online decisions use observed outcomes. For the nine-setting suite at , the resulting root-MSE reductions are 16.6% against uniform and 12.3% against MAD-style. Replaying the primary comparison reproduces every saved estimate and welfare cost exactly; at its original frontier weights, integrated root-MSE is .024543/.028969/.027594/.024682 for CS–IF/uniform/MAD-style/MV. Both observed-error and conditional-moment evaluations support the allocation gain.
3. How this relates to current methods targeting inference and regret.
MAD supplies general mixture randomization and asymptotic confidence sequences. Lin, Ham and Bojinov's NARP (2026) learns variances, reward gaps and the exploration rate online; its aggregate arm-RMSE objective uses a rooted-Neyman exploration profile. Classical MV design supplies contrast geometry, and Simchi-Levi and Wang characterize the regret–inference Pareto tradeoff. CS–IF optimizes the weakest requested contrast through a priced evaluation update and quantifies how errors in that learned design propagate into deployment information. Its specified reward-history protocol and logged inference complete this construction. The acquisition problem recovers classical contrast-optimal design in the known-information, full-evaluation limit with zero welfare and KL penalties. The online comparison shows that target-design efficiency is retained here.
We implemented NARP's native Algorithm 3, including its square-root minimum-count quota, and evaluated the target-precision gain under moderate and high noise heterogeneity. The complete three-profile grid is retained with the code. Each profile has 500 repetitions per configuration, , the same balanced 20-per-arm warm start, and the same standalone UCB learner. NARP receives a exploration-weight/quota grid; CS–IF receives a routing/KL grid. Both are evaluated using their full-horizon arm sample means to estimate the same two contrasts; this is a separate common-estimator comparison from the AIPW table above.
| Noise SDs | Total budget | CS–IF contrast RMSE | Native NARP contrast RMSE |
|---|---|---|---|
| 30 | .023726 | .026066 | |
| 30 | .029888 | .032455 |
Across the two heterogeneous profiles and budgets , CS–IF reduces worst-contrast RMSE by 4.5–12.1%. At , the paired RMSE-difference intervals are and , conditional on the empirical frontier weights. CS–IF has lower point RMSE in all eight heterogeneous-profile/budget comparisons, with favorable paired intervals in six. The full 138-configuration results connect the gain to noise heterogeneity and the requested contrast criterion.
We also ran MAD's published HT asymptotic confidence sequence on both assignment laws, using bounded Bernoulli outcomes, including 60 randomized pilot units, 500 repetitions, and after the pilot. Under the sharp null, both methods have zero welfare loss. Targeting reduces mean worst confidence-sequence width from .36926 to .31049 (15.9%), with measured joint path coverage .972 versus .962 over . Both use the same CS and per-contrast . Thus the allocation improvement also benefits MAD's own inferential procedure.
4. Evidence in a marketing-calibrated sequential experiment.
We
Thanks for the review and here is the kind response(cont'd)
fitted Bernoulli visit rates from Hillstrom's 64,000 historical observations: .10617 for No E-Mail, .18276 for Men's E-Mail, and .15140 for Women's E-Mail. We then ran a sequential, real-data-calibrated simulation with , including 90 randomized pilot assignments. Each method learns from its trial observations; target-aware methods reoptimize online. The 216 final configurations cover the joint two-contrast question and each email-versus-control question separately, with 500 repetitions for the joint question and 300 per single-contrast configuration. Routing, KL and welfare-price grids are fixed before the final fresh-seed evaluation and applied to all relevant design families.
For the prespecified Men's E-Mail versus No E-Mail question at a welfare cap of 40 expected lost visits:
| Method | Conditional-moment root-MSE | Mean 95% interval width | Rejection probability at the fitted effect |
|---|---|---|---|
| CS–IF | .019161 | .075084 | .9775 |
| Uniform evaluation | .029447 | .115089 | .7182 |
| MAD-style assignment | .023579 | .092332 | .8617 |
CS–IF reduces root-MSE by 34.9% and 18.7%, respectively. Its measured interval coverage is .9433; the fitted effect is .07659. Reaching root-MSE .025 requires 24.56 expected lost visits for CS–IF, versus 45.35 for uniform and 36.53 for MAD-style: welfare savings of 45.8% and 32.8%. The welfare-priced MV family shares the target-design oracle and attains the same frontier. Full joint and single-contrast curves, achieved costs, interval coverage and conditional-moment checks accompany this example, showing how the scientific query determines the value of targeting.
5. What supports deployment of the learned allocation?
The analysis separates the quantities practitioners must control: pilot and deployment welfare, sensitivity to estimated inputs, and uncertainty in the requested comparisons. For deterministic routing probabilities in a stationary bandit with bounded gaps, an expected pilot-cost bound , and a valid nondecreasing expected-regret bound for the actual initialized decision-history learner, the routing cap certifies the expected total welfare budget. The welfare price determines how the permitted evaluation mass is distributed.
For continuously updated stationary Gaussian contrasts, Corollary E.1 treats a unique best arm, a known noise upper bound, at least two pilot observations per arm, confidence-calibrated decision-only UCB, and fixed routing/design parameters with positive evaluation floor, , and . Compare estimated- and true-input updates with the same base proposal on the common floor-constrained simplex. If bounds information-criterion error uniformly on that simplex and bounds gap error, KL strong convexity gives Simultaneous Gaussian mean/variance bounds, combined with a deterministic arm-count lower envelope, give deterministic allocation-error bounds . A UCB count bound and routing supermartingale give a deterministic wrong-base envelope . Their cumulative bounds supply Theorems 4.2–4.3 with deterministic drift and leverage . The matrix-transfer step conditions on the initial pilot log; its inverse-information conclusion applies when the radius is below one. Appendix E.6 supplies the constants and pilot-count probability. Theorem 4.1 gives the independent-pilot, frozen-design specialization. Theorem 4.4 gives exact AIPW moments for predictable online updates. These results explain how target-optimal allocation, reward learning and logged inference fit together.
The computational cost is small in the primary three-arm setting: a separately verified dual solver takes a median 5.3 microseconds per update after compilation across 180 test inputs. Every update in the primary online comparison passes its global objective bound, with maximum excess . Section 5 and Table 5 present the matched-welfare comparison; Appendix E.6 proves the deployment corollary, and Appendix I reports the complete protocols with coverage and solver checks in the code. The practical criterion is explicit: welfare spent per unit of precision on the comparisons the experiment was designed to answer.
Thank you for your polite and detailed response.
I believe your response addresses the concerns I raised. However, it is necessary to clarify these points within the main body of the paper to prevent potential misunderstandings. Along with the points raised by other reviewers, I would like to request improvements in the presentation to make the paper more accessible and easier for readers to follow. Given the limited options for rating, I will maintain a positive-leaning score.
Thanks for the review and here is the kind response
W1 / Question. What practical value does CS–IF add to current regret–inference methods?
We understand the review’s reference to the “Riclet strategy” as a question about regret-based assignment. For this question, CS–IF improves the precision obtained from a welfare-limited experiment by directing evaluations toward the requested comparisons. We now quantify this value using the complete online algorithm, current target-aware comparators, native MAD inference, and the welfare required to reach a specified accuracy. On the empirical finite-grid frontier of the 600-unit online experiment, reaching worst-target AIPW RMSE .030 costs 21.84 units of welfare under CS–IF, versus 29.88 for uniform evaluation and 27.28 for the MAD-style assignment: savings of 26.9% and 20.0%, including the pilot.
1. The practical bottleneck is the weakest requested comparison.
The scientific question is: how should an adaptive experiment spend its welfare budget so that each prespecified treatment comparison is precise? Reward learning, average arm accuracy, and worst-contrast accuracy place different values on the same observations. This distinction persists even when the amount and welfare cost of exploration are fixed. Proposition 2.1 further shows that matching expected counts and welfare permits arbitrarily different HT variances: allocation timing and logged propensities also matter.
An exact fixed-design Gaussian calculation makes the difference concrete. Suppose an incumbent receives n0 observations, the q challengers have the same reward gap g, and their total allocation is s. Every challenger allocation then costs exactly gs. Minimizing the worst challenger-versus-incumbent variance over continuous positive allocations gives V⋆=n0σ02+s∑a=1qσa2,na⋆=∑bσb2sσa2. Indeed, if v=maxaσa2/na, then na≥σa2/v and hence v≥∑aσa2/s; the displayed allocation attains equality. Uniform challenger allocation instead contributes qmaxaσa2/s. Thus equal welfare can buy substantially different precision. For two challengers with SDs .15 and .30, targeting reduces this challenger variance term from .18/s to .1125/s, a 37.5% reduction. With n0=400,s=200,σ0=.20, the resulting worst-contrast SE falls from .03162 to .02574, or 18.6%. A 100,000-draw Gaussian check gives RMSE .03165 and .02580.
This calculation is the classical MV solution for that setting. It identifies when target optimization adds value and why a strong target-aware design should attain the same benchmark. CS–IF uses this geometry inside an experiment that learns the allocation and continues reward-oriented decisions.
2. Complete online comparison at the same total welfare cost.
We ran the repeatedly updated Eq. (3.3), recomputing gaps and variance loads after every observation and solving the positive-KL convex problem before the next assignment. The primary Gaussian experiment has K=3, T=600 including 60 randomized pilot assignments, 500 repetitions, means (.30,.20,.20), and SDs (.20,.15,.30). Methods share potential-outcome streams, target contrasts, the UCB base, and the full-horizon logged-propensity AIPW estimator. Uniform/two-history and MAD-style uniform/shared-history assignments isolate allocation and history routing. The strong online MV family uses the same targets and floor, with both zero KL and the same positive-KL grid as CS–IF.
Entries are worst-target AIPW RMSE. CS–IF reduces RMSE by 14.3–20.7% relative to uniform and 12.2–14.6% relative to MAD-style assignment across these budgets. At B=30, paired bootstrap intervals for the RMSE difference are [−.00684,−.00367] against uniform and [−.00506,−.00178] against MAD-style. CS–IF's marginal coverage is .951–.956 and Bonferroni simultaneous coverage is .958. Strong online MV attains matching precision within Monte Carlo uncertainty: CS–IF retains the efficiency of target-specific design in this setting.
The comparison evaluates the entire declared routing/KL grid. An empirical frontier mixes each target's MSE and total cost over complete configurations selected at trial initialization, then takes the maximum and square root. All assignments, including the pilot, enter both horizon and welfare. At budgets 25 and 30, all four frontier costs equal the budget; at 35 the MAD-style family uses 34.79 and the other families use 35. The paired intervals above resample repetitions conditional on the frontier weights.
The reverse comparison answers the adoption question directly: the minimum total cost on this empirical frontier to reach RMSE .030 is