Fast A/B-Testing: Efficient Policy Evaluation via Tree-Coupled Feedback Sharing

Online platforms increasingly compare many adaptive decision policies—ranking systems, recommendation algorithms, pricing rules, and language-model agents—while each reward-bearing interaction can be costly or risky. The standard recipe is a direct A/B/n test, which runs each of $J$ candidate policies over an independent $T$-horizon and therefore uses $JT$ outcomes. We introduce Tree-Coupled A/B Testing (TCAB), an exact feedback-sharing design for reducing the outcome usage while preserving the marginal trajectories. At a high level, the policies are associated to a tree structure and are executed by any topological order. The observed outcome is shared along an edge when the two policies on this edge can be coupled. Every policy retains exactly its standalone finite-horizon trajectory law, even though the policies are deliberately dependent. If $D_{e,t}$ records a mismatch on tree edge $e$ at round $t$, the number of outcomes satisfies the pathwise identity $N(T)=T+{\sum }_{t,e}{D}_{e,t}\le JT$. This cost is conditionally optimal for the selected tree, and we propose a heuristic for empirically finding the optimal tree. For fixed $J$, sublinear pseudo-regret of every policy can imply $\mathbb{E}[N(T)]=T+o(T)$. We also obtain finite-sample variance bounds for pairwise policy contrasts. Experiments on language-model evaluation and adaptive bandit policies demonstrate substantial improvements—around 20-50% outcome save and improved policy evaluation precision.