- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper studies platforms whose decisions change users’ behavior, creating feedback between regulation and strategic adaptation. Users select among executable behavior policies, and the platform routes them using a likelihood ratio test comparing observed actions with benign and harmful reference policies. The platform balances unnecessary intervention against benign users with harmful behavior remaining undetected. The contribution combines this model and an incentive-based objective with regret analysis and existing learning algorithms. Theorem 1 separates platform and user learning errors; Theorems 2-3 strengthen the bounds when user learning satisfies a specific relationship between prediction accuracy, strategy changes, and regret. Experiments use Mahjong and a synthetic service platform.
Critical strengths
- The problem is well posed and motivated. Modeling responses through executable policies captures behavioral differences that aggregate outcomes can conceal, such as genuine beginner play versus an experienced player’s camouflage.
- The theoretical analysis is rigorous under the stated assumptions. Theorem 1 separates platform and user learning errors, clarifying whether poor governance stems from threshold selection or users’ adaptation.
- The experiments isolate the value of behavioral evidence by holding controller families fixed. With PPO, replacing rank predictions with behavioral evidence reduces harmful-user share from 55.99% to 20.77%.
Critical weaknesses
- The stronger theorems cover a restrictive adaptation regime. Assumption 2 requires regret to be bounded by loss-prediction error minus a strategy-movement term, uniformly across runs and horizons (lines 300–305). This structure drives the fast bounds in Theorems 2-3. OMD provides one realization, with full expected-loss feedback and additional learning-rate restrictions. My concern is applicability to external strategic users, whose self-interest does not imply this learning structure.
Minor Concerns
-
The connection to the experimental learners remains incomplete. Harmful users employ PPO or actor–critic. Appendix B.10 extends the analysis by retaining their deviation from predictive mirror updates as an additional error term, but the experiments do not establish that this term is small enough to preserve the fast guarantees.
-
The theoretical predictions are not directly evaluated. Experiments measure harmful-user presence, game placement, and benign welfare, rather than the theoretical loss decomposition. Maintaining improvements from 500 to 1,000 rounds demonstrates persistence, but does not test the overhead.
Questions for authors
-
Could the authors include a small experiment satisfying the theoretical assumptions, using OMD users and reporting the governance bound’s individual terms?
-
Smaller OMD learning rates help satisfy the interaction–stability condition but increase . Which interaction strengths and learning rates yield informative finite-horizon guarantees? How conservative is the interaction estimate in Appendix B.2?
-
Appendix B.13 distinguishes the certificate from actual costs under independently deployed thresholds. What additional response conditions would make optimizing yield near-optimal actual governance loss?
-
Could the framework describe job or dating matching, where recommendations change both sides’ behavior? Which assumptions would need to change, particularly the opponent-independent behavior-window laws in Assumption 1 (lines 170-179)? Such an example would clarify its applicability beyond harmful-user routing.
-
Could the explanations of global versus novice-pool ranks in Figure 3 be made clearer? In panel (c), are the remaining harmful users achieve better placements because removing other harmful users leaves them facing more beginners and fewer skilled competitors? We request the authors to kindly distinguish this composition effect from changes in learned behavior.
Minor Comments
- Table 2 should report false-positive rates or compare recall at matched false-positive rates; recall alone does not establish a better detection tradeoff.
- Display spacing is crowded around lines 300-307, 312-323, and 337-341. Please regularize the vertical spacing.
7 comments
3. Could the authors include a small experiment satisfying the theoretical assumptions, using OMD users and reporting the governance bound’s individual terms?
Thank you for suggesting direct validation. Increasing the horizon twentyfold reduces both OMD/OPGD learning remainders about twentyfold, consistent with (Table R3).
PPO’s induced mode probabilities satisfy the following bound under , , pre-action predictions, and positive analysis-smoothed probabilities:
\mathcal B_i(T)=\mathcal I_i^{\rm act}+\alpha_i^{\rm act}V_i(T)-\sigma_i^{\rm act}P_i(T)+\mathcal D_i(T).$$ Here $V_i(T)$ and $P_i(T)$ are cumulative squared loss-prediction error and strategy movement; $\mathcal D_i(T)$ retains implementation update deviation. Coefficients and full conditions are given in manuscript Appendix B.10, Eqs. (53)–(54). The PPO row uses previous-loss prediction with analysis parameters $\eta=1$ and $\tau=0.01$. **Table R4 reports the fixed-rule governance-bound components, including the zero threshold-trajectory tracking term.** At $T=2000$ and $\beta=\eta=0.5$, the fixed-rule benchmark is $U(\theta)=U^*+Q_T/T=0.194$. The OMD/OPGD predictive bounds exceed measured loss by 0.00121/0.00128, respectively. Initialization decays as $1/T$, and strategy movement provides negative cancellation. The OMD/OPGD results and bound components are included in Appendix B.16 (manuscript Tables 3–4), referenced in §5. **Table R3. User learners and their theoretical and empirical checks** | User learner | Evaluation setting and horizon | Measured quantity and value | Theoretical bound | | --- | --- | --- | --- | | OMD | Fixed rule, $T=100\to2000$ | Average governance loss above $U(\theta)$: 0.00190 → **0.0000948** | Governance-loss bound at $T=2000$: 0.195 | | OPGD | Fixed rule, $T=100\to2000$ | Average governance loss above $U(\theta)$: 0.00241 → **0.000120** | Governance-loss bound at $T=2000$: 0.196 | | PPO | Mahjong, $T=499$ | Average user regret $R/T$: **0.127–0.229** | RVU bound $\mathcal B_i(T)/T$: 32.8–34.8 (simultaneous 95% upper confidence limits) | **Table R4. Fixed-rule governance-bound decomposition** | Average governance loss and bound components | OMD | OPGD | | --- | ---: | ---: | | Measured loss | 0.194 | 0.194 | | $U^*$ | 0.0961 | 0.0961 | | Fixed-rule benchmark gap $Q_T/T$ | 0.0980 | 0.0980 | | Weighted user-regret contribution | 0.00105 | 0.00114 | | Initialization | 0.00131 | 0.00140 | | RVU loss-prediction term | 0.00000127 | 0.00000117 | | Strategy-movement cancellation | −0.0000134 | −0.0000122 | | Threshold-trajectory tracking term | 0 | 0 | | Regret-decomposition bound | 0.195 | 0.195 | | Predictive governance bound | 0.195 | 0.196 | ## 4.1. Which interaction strengths and learning rates yield informative finite-horizon guarantees? Thank you for asking about finite-horizon accuracy. We vary the step size in the same OMD instance. **Increasing $\eta$ from 0.05 to 1 narrows the bound-to-loss gap from 0.0101 to 0.000909 (Table R5).** The table also reports initialization to show how the step size affects this term. These results are included in Appendix B.16 (manuscript Table 5). **Table R5. OMD step size and finite-horizon bounds** | OMD step size $\eta$ | Initialization term | Measured average governance loss | Predictive governance-loss bound | Bound minus measured loss ↓ | | ---: | ---: | ---: | ---: | ---: | | 0.05 | 0.0111 | 0.195 | 0.205 | 0.0101 | | 0.5 | 0.00131 | 0.194 | 0.195 | 0.00121 | | 1 | 0.000955 | 0.194 | **0.195** | **0.000909** | ## 4.2. How conservative is the interaction estimate in Appendix B.2? Thank you for asking how conservative the interaction estimate is. We compute the observed variation as a fraction of its bound in each run to make tightness checkable. **The observed variation reaches 0.967 of its interaction bound.** Across runs satisfying coupling at $\beta=0.1/0.3/0.5$, median per-run maximum ratios are 0.743/0.771/0.756. The ratio is opponent-induced squared loss variation divided by its interaction bound.5. What additional response conditions would make optimizing yield near-optimal actual governance loss?
Thank you for asking how the certificate connects to actual cost. We give sufficient conditions and corresponding grid results specifying when optimizing yields near-optimal actual cost.
Uniform response convergence and controlled variation of certificate slack across thresholds give a near-optimality guarantee. Let and be the finite-horizon and long-run costs of independently deployed thresholds under the same learners and initial law. On a common candidate domain, if
\operatorname{osc}_\theta[U(\theta)-J_\infty(\theta)]\le\varepsilon_{\rm cal},$$ then any $\varepsilon_{\rm opt}$-accurate minimizer $\widehat\theta$ of $U$ satisfies $$J_T(\widehat\theta)-\inf_\theta J_T(\theta) \le\varepsilon_{\rm opt}+\varepsilon_{\rm cal}+2b_T.$$ Uniform contraction, bounded initial displacement, and Lipschitz cost can control $b_T$. On a zero-incentive-gap domain satisfying the fixed-rule theorem with uniform initialization bound $C$, $0\le J_T-U\le C/T$ gives the tighter suboptimality bound $\varepsilon_{\rm opt}+C/T$. **On both grid experiments satisfying these conditions, optimizing $U$ selects the actual-cost minimizer (Table R6).** **Table R6. Thresholds minimizing the certificate and actual governance cost** | Interaction strength $\beta$ | Evaluated threshold domain | Shared minimizer of $U$ and actual governance cost | | ---: | --- | ---: | | 0.1 | [−12,−0.7] | **−0.7** | | 0.5 | [−12,−2] | **−2** | ## 6.1. Could the framework describe job or dating matching, where recommendations change both sides’ behavior? Thank you for suggesting two-sided matching. We give a job-matching example to show how recommendations affect both sides' behavior. **Both sides' policies determine match probabilities.** In job matching, $a_k$ and $b_\ell$ are the response probabilities of the two sides' modes and $q_\theta$ represents recommendation opportunities. Conditional independence gives match probability $q_\theta a_kb_\ell$ and applicant loss $$\ell_A(k;\pi_E,\theta)=c_A(k)-v_Aq_\theta a_k\sum_\ell\pi_E(\ell)b_\ell,$$ The employer has a symmetric loss. ## 6.2. Which assumptions would need to change, particularly the opponent-independent behavior-window laws in Assumption 1 (lines 170–179)? Thank you for pointing specifically to the window laws. We explain the required change to Assumption 1 and the conditions needed for the corresponding guarantee. **Replace Assumption 1 with a fixed family of pair-conditioned window laws $P(W|k,\ell;\theta)$, so each marginal law changes with the other side's mixture.** Retain normalized bounded losses and a zero-loss outside option, check RVU coupling on the bipartite interaction matrix, and use the matching task's own incentive-conversion relation for a cost guarantee.
Thank you for carefully examining the theory and for recognizing the problem formulation, rigor of the analysis, and matched-controller experiments.
1. applicability to external strategic users
Thank you for asking about applicability from external users' perspective. We explain users' own choice of learner and give examples that optimize their own payoffs.
The framework evaluates governance through the regret users actually generate. Under Assumption 1 and the routing/incentive model, Theorem 1 applies to general user-learning processes; sublinear user and platform regrets give a vanishing governance remainder.
Entropic OMD (optimistic multiplicative weights) and OPGD optimize users' own payoffs and provide directly checkable RVU instances. With expected-loss feedback, the corresponding step sizes, and coupling, Theorem 2 gives an O(1/T) remainder for a fixed rule and previous-loss prediction. Theorem 3 retains platform regret and tracking error with pre-action prediction for adaptive thresholds.
2.1. their deviation from predictive mirror updates as an additional error term
Thank you for examining the update deviation in B.10. We give its contribution to the governance bound and the conditions that absorb it into RVU.
Update deviations enter the governance bound explicitly and preserve fast rates under the following absorption conditions. Update error Di(T) contributes (pCFN/T)∑iωiDi(T) to the average-governance-loss bound. Under the theorem's routing/prediction conditions and a uniformly bounded initialization term, suppose fixed nonnegative constants satisfy, uniformly over admissible runs and horizons,
\rho_i < \sigma_i^{\rm act},$$ The RVU prediction-error coefficient then becomes $\alpha_i^{\rm act}+\kappa_i$ and the negative strategy-movement coefficient becomes $\sigma_i^{\rm act}-\rho_i > 0$. Theorems 2–3 apply when the adjusted constants satisfy the corresponding coupling conditions. Here $V_i$ and $P_i$ are cumulative squared loss-prediction error and strategy movement. Details are given in Appendix B.10. ## 2.2. the experiments do not establish that this term is small enough to preserve the fast guarantees. Thank you for suggesting that we quantify update error. **PPO’s measured average user regret $R/T$ is 0.127–0.229 at $T=499$.** The RVU bound $\mathcal B_i(T)/T$ has simultaneous 95% upper confidence limits of 32.8–34.8. **In the governance experiment, Evidence+PPO lowers harmful share and improves benign rank (Table R1); all reward-weight settings have 100% completion over 500 days (Table R2).** We use PPO ([Schulman et al., 2017](https://arxiv.org/abs/1707.06347)) in these governance experiments. The following Response 3 directly checks the OMD/OPGD fast rates under the stated conditions. The governance results are included in Appendix C.16 (manuscript Tables 18–19). **Table R1. Governance under a common platform reward (mean ± SD)** | Platform governance method | Common-period harmful share (0–1) ↓ | Common-period benign expected global rank ↓ | 500-day completion rate (%) ↑ | | --- | ---: | ---: | ---: | | Evidence+PPO | **0.274 ± 0.0275** | **2.55 ± 0.0190** | **100** | | Rank+PPO | 0.450 ± 0.0797 | 2.79 ± 0.101 | 0 | **Table R2. Sensitivity to platform reward weights (mean ± SD)** | Platform reward adjustment | Final-50-day harmful share (0–1) ↓ | Final-50-day benign expected global rank ↓ | 500-day completion rate (%) ↑ | | --- | ---: | ---: | ---: | | Default weights | 0.176 ± 0.0559 | 2.55 ± 0.0338 | **100** | | Rank-feedback term ×0.5 | 0.178 ± 0.0747 | 2.54 ± 0.0152 | **100** | | Rank-feedback term ×2 | 0.239 ± 0.103 | 2.60 ± 0.0788 | **100** | | Harmful-share penalty ×0.5 | 0.259 ± 0.145 | 2.57 ± 0.0489 | **100** | | Harmful-share penalty ×2 | 0.173 ± 0.0326 | 2.54 ± 0.00738 | **100** |