acceptodds
Under review as a conference paper at ICLR 2027

When Decisions Induce Policies: Performative Prediction with Latent Responses

Abstract

When a learned decision rule is deployed in an adaptive environment, future data are shaped by agents who respond to it. Performative prediction captures this feedback through induced distribution shift, but similar observable shifts can arise from different latent policy responses, which may require different interventions. We formulate this problem as a bilevel multi-agent reinforcement learning framework in which strategic users adapt to the incentives induced by a routing rule, while the institution updates the rule from aggregate feedback and behavior-derived evidence. To infer latent responses, we compare behavior windows against reference policies representing benign and targeted behavior. Theoretically, we connect user adaptation and platform learning through a governance-loss bound, and derive finite-time guarantees for predictive, stable user adaptation under explicit interaction and prediction-error conditions. Across competitive Riichi Mahjong and non-competitive platform environments, policy-evidence-based learning better distinguishes latent strategic responses than aggregate response signals, leading to more stable long-run regimes, fewer harmful users, and improved benign-user outcomes.

Primary area:
reinforcement learning
Keywords:
Performative Prediction, Multi-Agent Reinforcement Learning, Strategic Behavior, Online Learning, Adaptive Platform Regulation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_BrcWOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies platforms whose decisions change users’ behavior, creating feedback between regulation and strategic adaptation. Users select among executable behavior policies, and the platform routes them using a likelihood ratio test comparing observed actions with benign and harmful reference policies. The platform balances unnecessary intervention against benign users with harmful behavior remaining undetected. The contribution combines this model and an incentive-based objective with regret analysis and existing learning algorithms. Theorem 1 separates platform and user learning errors; Theorems 2-3 strengthen the bounds when user learning satisfies a specific relationship between prediction accuracy, strategy changes, and regret. Experiments use Mahjong and a synthetic service platform.

Critical strengths
  • The problem is well posed and motivated. Modeling responses through executable policies captures behavioral differences that aggregate outcomes can conceal, such as genuine beginner play versus an experienced player’s camouflage.
  • The theoretical analysis is rigorous under the stated assumptions. Theorem 1 separates platform and user learning errors, clarifying whether poor governance stems from threshold selection or users’ adaptation.
  • The experiments isolate the value of behavioral evidence by holding controller families fixed. With PPO, replacing rank predictions with behavioral evidence reduces harmful-user share from 55.99% to 20.77%.
Critical weaknesses
  1. The stronger theorems cover a restrictive adaptation regime. Assumption 2 requires regret to be bounded by loss-prediction error minus a strategy-movement term, uniformly across runs and horizons (lines 300–305). This structure drives the fast bounds in Theorems 2-3. OMD provides one realization, with full expected-loss feedback and additional learning-rate restrictions. My concern is applicability to external strategic users, whose self-interest does not imply this learning structure.

Minor Concerns

  1. The connection to the experimental learners remains incomplete. Harmful users employ PPO or actor–critic. Appendix B.10 extends the analysis by retaining their deviation from predictive mirror updates as an additional error term, but the experiments do not establish that this term is small enough to preserve the fast guarantees.

  2. The theoretical predictions are not directly evaluated. Experiments measure harmful-user presence, game placement, and benign welfare, rather than the theoretical loss decomposition. Maintaining improvements from 500 to 1,000 rounds demonstrates persistence, but does not test the O(1/T)O(1/T) overhead.

Questions for authors
  1. Could the authors include a small experiment satisfying the theoretical assumptions, using OMD users and reporting the governance bound’s individual terms?

  2. Smaller OMD learning rates help satisfy the interaction–stability condition but increase Dimax⁡/ηiD_i^{\max}/\eta_i. Which interaction strengths and learning rates yield informative finite-horizon guarantees? How conservative is the interaction estimate in Appendix B.2?

  3. Appendix B.13 distinguishes the certificate UU from actual costs under independently deployed thresholds. What additional response conditions would make optimizing UU yield near-optimal actual governance loss?

  4. Could the framework describe job or dating matching, where recommendations change both sides’ behavior? Which assumptions would need to change, particularly the opponent-independent behavior-window laws in Assumption 1 (lines 170-179)? Such an example would clarify its applicability beyond harmful-user routing.

  5. Could the explanations of global versus novice-pool ranks in Figure 3 be made clearer? In panel (c), are the remaining harmful users achieve better placements because removing other harmful users leaves them facing more beginners and fewer skilled competitors? We request the authors to kindly distinguish this composition effect from changes in learned behavior.

Minor Comments

  • Table 2 should report false-positive rates or compare recall at matched false-positive rates; recall alone does not establish a better detection tradeoff.
  • Display spacing is crowded around lines 300-307, 312-323, and 337-341. Please regularize the vertical spacing.
7 comments

Thank you for carefully examining the theory and for recognizing the problem formulation, rigor of the analysis, and matched-controller experiments.

1. applicability to external strategic users

Thank you for asking about applicability from external users' perspective. We explain users' own choice of learner and give examples that optimize their own payoffs.

The framework evaluates governance through the regret users actually generate. Under Assumption 1 and the routing/incentive model, Theorem 1 applies to general user-learning processes; sublinear user and platform regrets give a vanishing governance remainder.

Entropic OMD (optimistic multiplicative weights) and OPGD optimize users' own payoffs and provide directly checkable RVU instances. With expected-loss feedback, the corresponding step sizes, and coupling, Theorem 2 gives an O(1/T)O(1/T) remainder for a fixed rule and previous-loss prediction. Theorem 3 retains platform regret and tracking error with pre-action prediction for adaptive thresholds.

2.1. their deviation from predictive mirror updates as an additional error term

Thank you for examining the update deviation in B.10. We give its contribution to the governance bound and the conditions that absorb it into RVU.

Update deviations enter the governance bound explicitly and preserve fast rates under the following absorption conditions. Update error Di(T)\mathcal D_i(T) contributes (pCFN/T)∑iω~iDi(T)(pC_{\rm FN}/T)\sum_i\widetilde\omega_i\mathcal D_i(T) to the average-governance-loss bound. Under the theorem's routing/prediction conditions and a uniformly bounded initialization term, suppose fixed nonnegative constants satisfy, uniformly over admissible runs and horizons,

\rho_i < \sigma_i^{\rm act},$$ The RVU prediction-error coefficient then becomes $\alpha_i^{\rm act}+\kappa_i$ and the negative strategy-movement coefficient becomes $\sigma_i^{\rm act}-\rho_i > 0$. Theorems 2–3 apply when the adjusted constants satisfy the corresponding coupling conditions. Here $V_i$ and $P_i$ are cumulative squared loss-prediction error and strategy movement. Details are given in Appendix B.10. ## 2.2. the experiments do not establish that this term is small enough to preserve the fast guarantees. Thank you for suggesting that we quantify update error. **PPO’s measured average user regret $R/T$ is 0.127–0.229 at $T=499$.** The RVU bound $\mathcal B_i(T)/T$ has simultaneous 95% upper confidence limits of 32.8–34.8. **In the governance experiment, Evidence+PPO lowers harmful share and improves benign rank (Table R1); all reward-weight settings have 100% completion over 500 days (Table R2).** We use PPO ([Schulman et al., 2017](https://arxiv.org/abs/1707.06347)) in these governance experiments. The following Response 3 directly checks the OMD/OPGD fast rates under the stated conditions. The governance results are included in Appendix C.16 (manuscript Tables 18–19). **Table R1. Governance under a common platform reward (mean ± SD)** | Platform governance method | Common-period harmful share (0–1) ↓ | Common-period benign expected global rank ↓ | 500-day completion rate (%) ↑ | | --- | ---: | ---: | ---: | | Evidence+PPO | **0.274 ± 0.0275** | **2.55 ± 0.0190** | **100** | | Rank+PPO | 0.450 ± 0.0797 | 2.79 ± 0.101 | 0 | **Table R2. Sensitivity to platform reward weights (mean ± SD)** | Platform reward adjustment | Final-50-day harmful share (0–1) ↓ | Final-50-day benign expected global rank ↓ | 500-day completion rate (%) ↑ | | --- | ---: | ---: | ---: | | Default weights | 0.176 ± 0.0559 | 2.55 ± 0.0338 | **100** | | Rank-feedback term ×0.5 | 0.178 ± 0.0747 | 2.54 ± 0.0152 | **100** | | Rank-feedback term ×2 | 0.239 ± 0.103 | 2.60 ± 0.0788 | **100** | | Harmful-share penalty ×0.5 | 0.259 ± 0.145 | 2.57 ± 0.0489 | **100** | | Harmful-share penalty ×2 | 0.173 ± 0.0326 | 2.54 ± 0.00738 | **100** |

3. Could the authors include a small experiment satisfying the theoretical assumptions, using OMD users and reporting the governance bound’s individual terms?

Thank you for suggesting direct validation. Increasing the horizon twentyfold reduces both OMD/OPGD learning remainders about twentyfold, consistent with 1/T1/T (Table R3).

PPO’s induced mode probabilities satisfy the following bound under ∣ℓi,t∣∞,∣vi,t∣∞≤1\\|\ell_{i,t}\\|_\infty,\\|v_{i,t}\\|_\infty\le1, vi,1=0v_{i,1}=0, pre-action predictions, and positive analysis-smoothed probabilities:

\mathcal B_i(T)=\mathcal I_i^{\rm act}+\alpha_i^{\rm act}V_i(T)-\sigma_i^{\rm act}P_i(T)+\mathcal D_i(T).$$ Here $V_i(T)$ and $P_i(T)$ are cumulative squared loss-prediction error and strategy movement; $\mathcal D_i(T)$ retains implementation update deviation. Coefficients and full conditions are given in manuscript Appendix B.10, Eqs. (53)–(54). The PPO row uses previous-loss prediction with analysis parameters $\eta=1$ and $\tau=0.01$. **Table R4 reports the fixed-rule governance-bound components, including the zero threshold-trajectory tracking term.** At $T=2000$ and $\beta=\eta=0.5$, the fixed-rule benchmark is $U(\theta)=U^*+Q_T/T=0.194$. The OMD/OPGD predictive bounds exceed measured loss by 0.00121/0.00128, respectively. Initialization decays as $1/T$, and strategy movement provides negative cancellation. The OMD/OPGD results and bound components are included in Appendix B.16 (manuscript Tables 3–4), referenced in §5. **Table R3. User learners and their theoretical and empirical checks** | User learner | Evaluation setting and horizon | Measured quantity and value | Theoretical bound | | --- | --- | --- | --- | | OMD | Fixed rule, $T=100\to2000$ | Average governance loss above $U(\theta)$: 0.00190 → **0.0000948** | Governance-loss bound at $T=2000$: 0.195 | | OPGD | Fixed rule, $T=100\to2000$ | Average governance loss above $U(\theta)$: 0.00241 → **0.000120** | Governance-loss bound at $T=2000$: 0.196 | | PPO | Mahjong, $T=499$ | Average user regret $R/T$: **0.127–0.229** | RVU bound $\mathcal B_i(T)/T$: 32.8–34.8 (simultaneous 95% upper confidence limits) | **Table R4. Fixed-rule governance-bound decomposition** | Average governance loss and bound components | OMD | OPGD | | --- | ---: | ---: | | Measured loss | 0.194 | 0.194 | | $U^*$ | 0.0961 | 0.0961 | | Fixed-rule benchmark gap $Q_T/T$ | 0.0980 | 0.0980 | | Weighted user-regret contribution | 0.00105 | 0.00114 | | Initialization | 0.00131 | 0.00140 | | RVU loss-prediction term | 0.00000127 | 0.00000117 | | Strategy-movement cancellation | −0.0000134 | −0.0000122 | | Threshold-trajectory tracking term | 0 | 0 | | Regret-decomposition bound | 0.195 | 0.195 | | Predictive governance bound | 0.195 | 0.196 | ## 4.1. Which interaction strengths and learning rates yield informative finite-horizon guarantees? Thank you for asking about finite-horizon accuracy. We vary the step size in the same OMD instance. **Increasing $\eta$ from 0.05 to 1 narrows the bound-to-loss gap from 0.0101 to 0.000909 (Table R5).** The table also reports initialization to show how the step size affects this term. These results are included in Appendix B.16 (manuscript Table 5). **Table R5. OMD step size and finite-horizon bounds** | OMD step size $\eta$ | Initialization term | Measured average governance loss | Predictive governance-loss bound | Bound minus measured loss ↓ | | ---: | ---: | ---: | ---: | ---: | | 0.05 | 0.0111 | 0.195 | 0.205 | 0.0101 | | 0.5 | 0.00131 | 0.194 | 0.195 | 0.00121 | | 1 | 0.000955 | 0.194 | **0.195** | **0.000909** | ## 4.2. How conservative is the interaction estimate in Appendix B.2? Thank you for asking how conservative the interaction estimate is. We compute the observed variation as a fraction of its bound in each run to make tightness checkable. **The observed variation reaches 0.967 of its interaction bound.** Across runs satisfying coupling at $\beta=0.1/0.3/0.5$, median per-run maximum ratios are 0.743/0.771/0.756. The ratio is opponent-induced squared loss variation divided by its interaction bound.

5. What additional response conditions would make optimizing UU yield near-optimal actual governance loss?

Thank you for asking how the certificate connects to actual cost. We give sufficient conditions and corresponding grid results specifying when optimizing UU yields near-optimal actual cost.

Uniform response convergence and controlled variation of certificate slack across thresholds give a near-optimality guarantee. Let JTJ_T and J∞J_\infty be the finite-horizon and long-run costs of independently deployed thresholds under the same learners and initial law. On a common candidate domain, if

\operatorname{osc}_\theta[U(\theta)-J_\infty(\theta)]\le\varepsilon_{\rm cal},$$ then any $\varepsilon_{\rm opt}$-accurate minimizer $\widehat\theta$ of $U$ satisfies $$J_T(\widehat\theta)-\inf_\theta J_T(\theta) \le\varepsilon_{\rm opt}+\varepsilon_{\rm cal}+2b_T.$$ Uniform contraction, bounded initial displacement, and Lipschitz cost can control $b_T$. On a zero-incentive-gap domain satisfying the fixed-rule theorem with uniform initialization bound $C$, $0\le J_T-U\le C/T$ gives the tighter suboptimality bound $\varepsilon_{\rm opt}+C/T$. **On both grid experiments satisfying these conditions, optimizing $U$ selects the actual-cost minimizer (Table R6).** **Table R6. Thresholds minimizing the certificate and actual governance cost** | Interaction strength $\beta$ | Evaluated threshold domain | Shared minimizer of $U$ and actual governance cost | | ---: | --- | ---: | | 0.1 | [−12,−0.7] | **−0.7** | | 0.5 | [−12,−2] | **−2** | ## 6.1. Could the framework describe job or dating matching, where recommendations change both sides’ behavior? Thank you for suggesting two-sided matching. We give a job-matching example to show how recommendations affect both sides' behavior. **Both sides' policies determine match probabilities.** In job matching, $a_k$ and $b_\ell$ are the response probabilities of the two sides' modes and $q_\theta$ represents recommendation opportunities. Conditional independence gives match probability $q_\theta a_kb_\ell$ and applicant loss $$\ell_A(k;\pi_E,\theta)=c_A(k)-v_Aq_\theta a_k\sum_\ell\pi_E(\ell)b_\ell,$$ The employer has a symmetric loss. ## 6.2. Which assumptions would need to change, particularly the opponent-independent behavior-window laws in Assumption 1 (lines 170–179)? Thank you for pointing specifically to the window laws. We explain the required change to Assumption 1 and the conditions needed for the corresponding guarantee. **Replace Assumption 1 with a fixed family of pair-conditioned window laws $P(W|k,\ell;\theta)$, so each marginal law changes with the other side's mixture.** Retain normalized bounded losses and a zero-loss outside option, check RVU coupling on the bipartite interaction matrix, and use the matching task's own incentive-conversion relation for a cost guarantee.
Reviewer_yRjNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies the performative prediction problem under the setting where similar observable shifts can arise from different latent policy response. The authors formulate this problem as a MARL framework and usea behaviour window to infer latent response. They build fixed-rule governance and unified governance guarantee under explicit interaction and prediction-error conditions. They also provide empirical evidence for their policy-evidence based learner in competitive Riichi Mahjong and non-competitive platform environments, which identify the latent response rather than aggregate response signals.

Critical strengths
  • The paper models the case for observing behavior directly when outcome statistics are insufficient for governance. Comparing evidence-based and rank-based routing under matched controller families helps isolate the contribution of the observation signal.
  • The reference policies, behavior modes and predictive learners are modeled separately and clearly.
Critical weaknesses
  • It's a little bit hard to follow the paper, as a lot of the concepts were through out and not well-defined, for example, RVU occurs in line 265 but defined in line 293. Also, poor format as arthors heavily use vspace adjustments. Also the performative prediction and bilevel MARL framework are well-established.
  • The reference policies and behaviors are predefined, and you can use likelihood ratio test to detect which policy the user chooses. However, a lot of the hyperparameters are known in advance, and it's really hard to tell how to identify the latent reponse, or say without those known prior like policy, response or mixture of weights, how to reduce harmful users and improve benigh-user outcomes.
Questions for authors
  • Seems that all the theoretical results are based on fixed mod law and predictive RVU assumption, not sure how large is the gap between the theory and experiment? Can we really vertify predictive RVU in practice? I don't see how we can apply those theorems in real senario. And under what assumptions is that object identifiable?
  • How does the method perform against any baseline like independently trained response agents? As long as they are aware of the 21 executable behavior modes of harmful users.
3 comments

Thank you for recognizing the value of direct behavioral observation and the comparison of observation signals within matched controller families.

1.1. RVU occurs in line 265 but defined in line 293.

Thank you for identifying the definition's placement. We removed the earlier undefined uses and now introduce RVU's full name (Regret bounded by Variation in Utilities) and definition together in §4.2 (Assumption 2).

1.2. authors heavily use vspace adjustments.

Thank you for identifying the formatting details. We have regularized spacing around definitions, equations, and theorems to make adjacent results easier to read.

2. the performative prediction and bilevel MARL framework are well-established.

Thank you for asking us to clarify the relationship to established frameworks. The latent response is users' policies and adaptation process, which the platform cannot directly observe. The platform rule θt\theta_t changes latent policies Πtreal\Pi_t^{\rm real} and hence observed data Dtpol\mathcal D_t^{\rm pol}. Behavioral evidence makes this response observable and connects it to the governance bound.

With the same PPO controller, Evidence+PPO reduces harmful share from 56.0% to 20.8% and improves benign global rank (Table R1). The same comparison is also reported in manuscript Table 1.

Table R1. Main governance comparison with the same PPO controller (mean ± SD)

Platform governance methodHarmful share (%) ↓Benign global mean rank ↓
Evidence+PPO20.8 ± 12.02.60 ± 0.114
Rank+PPO56.0 ± 15.02.69 ± 0.099

3.1. The reference policies and behaviors are predefined

Thank you for asking about prior information. The router uses fixed reference action probabilities and benign calibration data, without requiring a supplied user-response map, mixture weights, or neural parameters.

We also evaluate a strong reference-avoiding policy with 34.2% top-action disagreement with πh\pi_h. Its mean rank is 1.55, and all harmful behavior is detected under unchanged references (Table R2). These results are included in manuscript Table 11 (Appendix C.7), referenced in §5.1.

Table R2. Strong policies under fixed references

User action policyTop-action disagreement with πh\pi_h (%)Mean game rank ↓Harmful-behavior detection (%) ↑
Pretrained strong policy19.61.78100
Strong reference-avoiding policy34.21.55100

3.2. Without those known prior like policy, response or mixture of weights, how to reduce harmful users and improve benign-user outcomes.

Thank you for asking about governance with unknown responses. We compare the routing methods under the same platform reward and population dynamics. Evidence+PPO lowers harmful share, improves benign rank, and completes 500 days (Table R3). This comparison is included in manuscript Table 18 (Appendix C.16) and highlighted in §5.1.

Table R3. Governance under a common platform reward (mean ± SD)

Platform governance methodCommon-period harmful share (0–1) ↓Common-period benign expected global rank ↓500-day completion rate (%) ↑
Evidence+PPO0.274 ± 0.02752.55 ± 0.0190100
Rank+PPO0.450 ± 0.07972.79 ± 0.1010

4.1. How large is the gap between the theory and experiment?

Thank you for asking how theory corresponds to experiment. The OMD/OPGD learning remainders decrease about twentyfold when the horizon increases twentyfold, consistent with 1/T1/T; PPO’s measured average user regret is 0.127–0.229 (Table R4).

For the mode probabilities induced by PPO, ∣ℓi,t∣∞,∣vi,t∣∞≤1\\|\ell_{i,t}\\|_\infty,\\|v_{i,t}\\|_\infty\le1, vi,1=0v_{i,1}=0, pre-action predictions, and positive analysis-smoothed probabilities give the explicit bound

\mathcal B_i(T)=\mathcal I_i^{\rm act}+\alpha_i^{\rm act}V_i(T)-\sigma_i^{\rm act}P_i(T)+\mathcal D_i(T).$$ Here $V_i(T)$ is cumulative squared loss-prediction error, $P_i(T)$ is cumulative squared strategy movement, and $\mathcal D_i(T)$ is the implementation update deviation. Coefficients and full conditions are given in manuscript Appendix B.10, Eqs. (53)–(54). The PPO row uses previous-loss prediction with analysis parameters $\eta=1$ and $\tau=0.01$. The OMD/OPGD results are included in Appendix B.16 (manuscript Table 3), referenced in §5. **Table R4. User learners and their theoretical and empirical checks** | User learner | Evaluation setting and horizon | Measured quantity and value | Theoretical bound | | --- | --- | --- | --- | | OMD | Fixed rule, $T=100\to2000$ | Average governance loss above $U(\theta)$: 0.00190 → **0.0000948** | Governance-loss bound at $T=2000$: 0.195 | | OPGD | Fixed rule, $T=100\to2000$ | Average governance loss above $U(\theta)$: 0.00241 → **0.000120** | Governance-loss bound at $T=2000$: 0.196 | | PPO | Mahjong, $T=499$ | Average user regret $R/T$: **0.127–0.229** | RVU bound $\mathcal B_i(T)/T$: 32.8–34.8 (simultaneous 95% upper confidence limits) | ## 4.2. Can we really verify predictive RVU in practice? Thank you for asking how RVU can be checked. We give the correspondence between the updates and the RVU parameters. For entropy OMD and OPGD with mirror map $K_i\\|x\\|_2^2/2$, summing the prediction/correction optimality inequalities gives $$R_i(T)\le\frac{D_i^{\max}}{\eta_i}+\eta_i+\eta_iV_i(T)-\frac{P_i(T)}{8\eta_i}.$$ Here $V_i(T)$ and $P_i(T)$ sum squared loss-prediction error and strategy movement. Thus $\mathcal I_i=D_i^{\max}/\eta_i+\eta_i$, $\alpha_i=\eta_i$, and $\sigma_i=1/(8\eta_i)$; $D_i^{\max}$ is $\log K_i$ for OMD and $(K_i-1)/2$ for OPGD. At the experimental choice $\eta=\beta=0.5$, the fixed/adaptive coupling left-hand sides are $2\beta^2\eta^2=0.125$ and $4\beta^2\eta^2=0.25$, both at most 1. Appendix B.10 connects neural updates to this analysis through explicit update deviations and their absorption conditions. ## 4.3. I don't see how we can apply those theorems in real scenario. Thank you for asking about practical use. For deployment, user surveys can measure preferences about matching experience, routing costs, participation, and exit. These measurements can inform reward weights, user proportions, and population inflow and exit parameters. **Under Assumption 1 and the routing/incentive model, Theorem 1 converts actual regrets into a governance-loss bound; predictive RVU, the corresponding prediction timing, and coupling give the fast rates in Theorems 2–3.** We use PPO ([Schulman et al., 2017](https://arxiv.org/abs/1707.06347)). Governance results are given in Table R3; **halving or doubling reward weights retains 100% completion over 500 days (Table R5).** The governance and weight-sensitivity results are included in Appendix C.16 (manuscript Tables 18–19). **Table R5. Sensitivity to platform reward weights (mean ± SD)** | Platform reward adjustment | Final-50-day harmful share (0–1) ↓ | Final-50-day benign expected global rank ↓ | 500-day completion rate (%) ↑ | | --- | ---: | ---: | ---: | | Default weights | 0.176 ± 0.0559 | 2.55 ± 0.0338 | **100** | | Rank-feedback term ×0.5 | 0.178 ± 0.0747 | 2.54 ± 0.0152 | **100** | | Rank-feedback term ×2 | 0.239 ± 0.103 | 2.60 ± 0.0788 | **100** | | Harmful-share penalty ×0.5 | 0.259 ± 0.145 | 2.57 ± 0.0489 | **100** | | Harmful-share penalty ×2 | 0.173 ± 0.0326 | 2.54 ± 0.00738 | **100** |

4.4. And under what assumptions is that object identifiable?

Thank you for asking about identifiability assumptions. We give the signature-matrix rank condition to specify when a stable mixture can be uniquely recovered.

A stable latent mixture is uniquely identifiable when the augmented signature matrix has full column rank. Each column of AA contains one mode's detection probabilities, assumed known or calibrated by mode:

Ajk=Pr⁡ke(W)≥ϑj,rq=Aq.A_{jk}=\Pr_k\\{e(W)\ge\vartheta_j\\},\qquad r_q=Aq.

Full column rank of [1⊤;A][\mathbf1^\top;A] makes rq=rq′r_q=r_{q'} imply q=q′q=q'. The smallest singular value on the simplex tangent space controls error amplification; participation rates can separately identify the safe option.

5. How does the method perform against any baseline like independently trained response agents?

Thank you for suggesting independently learning response agents. We keep the feature network fixed and train separate action heads from each user's own game feedback. A second group uses end-to-end PPO to update both the backbone and action head. Both continuously optimize legal-action probabilities over a broader behavioral space than the 21-mode menu.

Independently trained heads and end-to-end PPO both have 100% harmful-behavior detection (Tables R6 and R7). PPO's evidence penalty directly encourages suppression of cumulative detection evidence; ranges span the trained policies' means. These results are included in Appendix C.7 (manuscript Tables 13–14), referenced in §5.1.

Table R6. Detection with independently trained action heads

User action policyMean game rank ↓Harmful-behavior detection (%) ↑
Pretrained strong policy1.50100
Independently trained action heads1.56–1.63100

Table R7. End-to-end action PPO with an evidence penalty

User action policyMean game rank ↓Mean cumulative evidenceHarmful-behavior detection (%) ↑
Pretrained initial policy1.79146100
PPO1.64–1.73133–137100
PPO + evidence penalty1.73–1.89135–144100
Reviewer_pgqBOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses a fundamental limitation in performative prediction, namely that aggregate distribution shifts can obscure distinct underlying behavioral mechanisms, by modeling the response process as a bilevel multi-agent reinforcement learning problem where strategic agents adapt latent sequential policies to institutional routing rules. To distinguish these latent responses without requiring direct policy observation, the authors introduce a behavior-derived evidence interface that scores individual interaction windows via log-likelihood ratios against fixed benign and targeted reference policies. Theoretically, the authors establish a governance-loss decomposition that translates user and platform regrets into cumulative platform performance, proving that under predictive Regret bounded by Variation in Utilities (RVU) conditions and suitable interaction coupling, a fixed decision rule achieves an O(T−1)O(T^{-1}) convergence rate, while adaptive rules boundedly depend on platform regret and user tracking errors. The framework is empirically validated on both a competitive Riichi Mahjong simulator with camouflaged high-skill players and a non-competitive open-service platform, demonstrating that reference-policy evidence substantially outperforms outcome-based and rank-predictive baselines by cutting harmful user share in novice pools by more than half, maintaining stable long-term dynamics, and safeguarding benign user welfare.

Critical strengths
  • The paper addresses an important conceptual gap in performative prediction by modeling distribution shifts as the direct outcome of latent, sequential policy adaptation rather than treating them as monolithic or stateless response maps. Formulating the interaction as a bilevel MARL problem provides a realistic microfoundation for platform governance where strategic users actively adapt their behavior to institutional rules.
  • The authors derive an insightful regret decomposition that directly bounds institutional governance loss by platform learning regret and user regret. Connecting user adaptation to the predictive Regret bounded by Variation in Utilities (RVU) framework provides clear conditions under which mutual strategic adaptation stabilizes at an O(T−1)O(T^{-1}) rate for fixed rules, while also characterizing the impact of threshold tracking errors and inexact neural policy updates.
  • The proposed log-likelihood ratio interface against fixed benign and targeted reference policies offers an elegant, lightweight mechanism to detect latent strategies without requiring intrusive or computationally intractable policy reconstruction. The empirical validation is thorough and creative, evaluating both a complex imperfect-information environment (Riichi Mahjong with diverse policy tiers, camouflaged players, and long horizons) and a non-competitive platform simulator to demonstrate clear, consistent gains over conventional outcome-based and rank-predictive baselines.
Critical weaknesses
  • The core mechanism assumes access to fixed, well-calibrated reference policies (πb\pi_b and πh\pi_h) that reliably represent benign and targeted behavior. In many practical governance settings, obtaining an explicit policy for targeted strategic behavior is difficult, and malicious actors can adopt novel, out-of-distribution strategies that evade detection under fixed reference likelihood ratios.
  • The theoretical guarantees rely heavily on Assumption 1, which posits that agents only adapt their selection probabilities over a fixed, discrete set of executable behavior modes with stationary window laws. If agents continuously evolve the executable policies themselves via open-ended reinforcement learning rather than mixing over a fixed menu of behaviors, the fast-rate guarantees no longer directly apply without substantially enlarging the mode space.
  • While the theory evaluates continuous threshold selection against the clean routing guarantee U(θ)U(\theta), the experimental platforms train the router using heuristic, multi-term shaped rewards with ad-hoc hyperparameters for threshold movement, occupancy, and rank anchors. Similarly, the harmful Mahjong agents adapt along a restricted two-dimensional morphology coordinate decoded by an auxiliary KP predictor rather than learning unrestricted game policies.
Questions for authors
  1. The proposed scoring mechanism relies on fixed reference policies πb\pi_b and πh\pi_h. How robust is the evidence statistic e(W)e(\mathcal{W}) if an adversary employs a high-performing strategy that looks completely unlike πh\pi_h, or explicitly optimizes its policy to minimize the log-likelihood ratio while maintaining high win rates? Have you evaluated the framework against policies trained with such evasion objectives?
  2. In the Mahjong simulator, harmful agents adapt by selecting a continuous 2D morphology parameter mapped to 21 predefined behavior modes via an auxiliary KP predictor. How well does the governance mechanism hold up if harmful users train end-to-end with unconstrained PPO directly over raw game actions, rather than selecting from a discrete menu of fixed modes?
  3. Theorem 3 shows that governance performance degrades with the user tracking error Ei(T)\mathcal{E}_i(T), which grows when the platform moves its threshold faster than users can adapt. Did you observe empirical regimes where rapid platform threshold updates destabilized user learning and worsened overall governance loss, and is there a practical heuristic for tuning platform learning rates relative to user adaptation speeds?
  4. Computing e(W)e(\mathcal{W}) requires evaluating likelihoods under both πh\pi_h and πb\pi_b across behavior windows. What is the computational and memory overhead of performing forward passes through large reference policy networks for every active user in real-time platform operations?
3 comments

Thank you for your careful reading and for recognizing the policy-response model, regret decomposition, and behavioral-evidence experiments in both environments.

1.1. How robust is the evidence statistic e(W)e(\mathcal W) if an adversary employs a high-performing strategy that looks completely unlike πh\pi_h?

Thank you for raising reference mismatch. We compare a pretrained strong policy with a strong reference-avoiding policy. The latter has 34.2% top-action disagreement with πh\pi_h, achieves a mean rank of 1.55, and has 100% harmful-behavior detection under unchanged references (Table R1). These results are included in manuscript Table 11 (Appendix C.7), referenced in §5.1.

Table R1. Strong policies under fixed references

User action policyTop-action disagreement with πh\pi_h (%)Mean game rank ↓Harmful-behavior detection (%) ↑
Pretrained strong policy19.61.78100
Strong reference-avoiding policy34.21.55100

1.2. How robust is the evidence statistic e(W)e(\mathcal W) if an adversary explicitly optimizes its policy to minimize the log-likelihood ratio while maintaining high win rates?

Thank you for raising deliberate evasion. We normalize qα(a∣s)∝π0(a∣s)/(πh(a∣s)+ε)αq_\alpha(a\mid s)\propto\pi_0(a\mid s)/(\pi_h(a\mid s)+\varepsilon)^\alpha over legal actions, with ε=10−10\varepsilon=10^{-10}. Setting α=0\alpha=0 recovers the original policy; increasing it downweights actions favored by the reference.

Stronger evasion raises mean rank to 3.83, with 87.5% harmful-behavior detection (Table R2). This experiment is included in manuscript Table 12 (Appendix C.7), referenced in §5.1.

Table R2. Inverse reweighting and competitive performance

Inverse-reweighting strength α\alphaMean game rank ↓Harmful-behavior detection (%) ↑
01.80100
0.51.77100
13.00100
23.8387.5

1.3. Have you evaluated the framework against policies trained with such evasion objectives?

Thank you for the suggestion. We train PPO with an explicit penalty on detection evidence. The resulting policies have mean ranks of 1.73–1.89 and 100% harmful-behavior detection. The method and results are given in the following Response 2 (Table R3).

2. How well does the governance mechanism hold up if harmful users train end-to-end with unconstrained PPO directly over raw game actions, rather than selecting from a discrete menu of fixed modes?

Thank you for suggesting users who learn game actions directly. We use PPO to update both the backbone and legal-action head, continuously optimizing action probabilities over a broader behavioral space than the 21-mode menu. Adding a penalty on standardized cumulative detection evidence trains policies to evade the detector.

Both trained policy groups have 100% harmful-behavior detection (Table R3). Ranges span the trained policies' means. These results are included in manuscript Table 14 (Appendix C.7), referenced in §5.1.

Table R3. End-to-end action PPO with an evidence penalty

User action policyMean game rank ↓Mean cumulative evidenceHarmful-behavior detection (%) ↑
Pretrained initial policy1.79146100
PPO1.64–1.73133–137100
PPO + evidence penalty1.73–1.89135–144100

3.1. continuous threshold selection against the clean routing guarantee U(θ)U(\theta)

Thank you for asking how reward optimization connects to the theoretical objective. We expanded the explanation of the B.12 transfer bound so that its components can be checked directly.

Under the following approximation and feasible-domain conditions, reward regret transfers to the routing objective. Holding the round's context fixed, let gtg_t be negative platform reward and STgS_T^g its selection regret on the feasible threshold domain ΘA\Theta_A. If a>0a > 0, btb_t is threshold-independent, and

sup⁡θ∈ΘA∣gt(θ)−aU(θ)−bt∣≤dt,\sup_{\theta\in\Theta_A}|g_t(\theta)-aU(\theta)-b_t|\le d_t,

then

a_A=\inf_{\Theta_A}U-\inf_\Theta U.$$ This transfers reward regret, approximation error, and the feasible-domain gap into Theorem 1's governance bound. The derivation is given in Appendix B.12. ## 3.2. multi-term shaped rewards with ad-hoc hyperparameters Thank you for suggesting reward-weight checks. We compare the routing methods under the same platform reward, population dynamics, and user-learning mechanism. **Evidence+PPO lowers harmful share, improves benign rank, and completes 500 days (Table R4).** We then halve or double the rank-feedback term or harmful-share penalty. **Every configuration has a 100% completion rate over 500 days (Table R5).** We added manuscript Tables 18–19 in Appendix C.16 and highlighted the common-reward comparison in §5.1. **Table R4. Governance under a common platform reward (mean ± SD)** | Platform governance method | Common-period harmful share (0–1) ↓ | Common-period benign expected global rank ↓ | 500-day completion rate (%) ↑ | | --- | ---: | ---: | ---: | | Evidence+PPO | **0.274 ± 0.0275** | **2.55 ± 0.0190** | **100** | | Rank+PPO | 0.450 ± 0.0797 | 2.79 ± 0.101 | 0 | **Table R5. Sensitivity to platform reward weights (mean ± SD)** | Platform reward adjustment | Final-50-day harmful share (0–1) ↓ | Final-50-day benign expected global rank ↓ | 500-day completion rate (%) ↑ | | --- | ---: | ---: | ---: | | Default weights | 0.176 ± 0.0559 | 2.55 ± 0.0338 | **100** | | Rank-feedback term ×0.5 | 0.178 ± 0.0747 | 2.54 ± 0.0152 | **100** | | Rank-feedback term ×2 | 0.239 ± 0.103 | 2.60 ± 0.0788 | **100** | | Harmful-share penalty ×0.5 | 0.259 ± 0.145 | 2.57 ± 0.0489 | **100** | | Harmful-share penalty ×2 | 0.173 ± 0.0326 | 2.54 ± 0.00738 | **100** | ## 4.1. Did you observe empirical regimes where rapid platform threshold updates destabilized user learning and worsened overall governance loss? Thank you for suggesting experiments with rapid threshold updates. We compare two switching frequencies in the same OMD instance. **Slower updates reduce the tracking term, user-regret contribution, and measured governance loss (Table R6).** Table R7 also gives the fixed-rule horizon comparison: increasing the horizon from 100 to 2000 reduces the OMD/OPGD learning remainders about twentyfold, consistent with $1/T$. The switching and OMD/OPGD horizon results are included in Appendix B.16 (manuscript Tables 6 and 3), referenced in §5. Table R7 also reports PPO’s measured average user regret. **Table R6. OMD threshold-switching frequency and governance loss** | Threshold-switching interval (rounds) | Tracking term in average governance-loss bound | Weighted user-regret term | Measured average governance loss | | --- | ---: | ---: | ---: | | 100 | **0.00152** | **0.00105** | **0.0999** | | 1 | 0.160 | 0.00114 | 0.100 | **Table R7. User learning in the controlled game and Mahjong** | User learner | Evaluation setting and horizon | Measured quantity and value | | --- | --- | --- | | OMD | Controlled routing game, $T=100\to2000$ | Average governance loss above $U(\theta)$: 0.00190 → **0.0000948** | | OPGD | Controlled routing game, $T=100\to2000$ | Average governance loss above $U(\theta)$: 0.00241 → **0.000120** | | PPO | Mahjong, $T=499$ | Average user regret $R/T$: **0.127–0.229** | ## 4.2. Is there a practical heuristic for tuning platform learning rates relative to user adaptation speeds? Thank you for asking for practical tuning guidance. We use the tracking term and this comparison to explain how to adjust step size and update intervals. **Limit each threshold change and lengthen update intervals to give users time to adapt.** With previous-loss prediction and Lipschitz routing, Theorem 3 bounds tracking by the squared threshold changes $\sum_{t=2}^T|\theta_t-\theta_{t-1}|^2$, so both adjustments reduce this term.

5. What is the computational and memory overhead of performing forward passes through large reference policy networks for every active user in real-time platform operations?

Thank you for asking about reference-network costs. The two networks are shared across users and score behavior in batches. Table R8 reports their scoring latency and throughput on an RTX 4090 in FP32. This setup is now described in §5.1, with measurements in Appendix C.16 (manuscript Table 20).

Table R8. Fixed-reference scoring cost

Batch sizeLatency for scoring with both references (ms)Scoring throughput (observations/s)
125.738.9
51228.91.77×1041.77\times10^4

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

Dear Area Chair,

Thank you for the time and care you have devoted to overseeing our submission. For your convenience, we provide below a concise summary of each reviewer’s post-rebuttal stance and the main points discussed.

Reviewer Stance (Post-Rebuttal)

  1. Reviewer pgqB (initial score 3 → latest recorded score 3) gave a positive initial recommendation. They described the paper as addressing “an important conceptual gap in performative prediction” and highlighted its “insightful regret decomposition.” Their main concern was whether fixed reference policies remain useful when players learn their own strategies or deliberately avoid detection. The fixed-reference detector achieved 100% recall on policies trained directly over legal game actions with a penalty for producing detectable behavior. When action probabilities were reweighted to avoid moves favored by the reference policy, the strongest reweighting reduced recall to 87.5% and worsened the player’s mean finish from 1.80 to 3.83 out of four. Under a common platform reward, evidence-based routing also reduced the harmful-player share and improved benign players’ mean rank. We have not received a further reply from pgqB.

  2. Reviewer yRjN (initial score 2 → latest recorded score 2) followed up on what the platform must know about latent behavior, how the theoretical assumptions can be checked, and whether the detector comparisons establish a benefit for novices. We clarified that the router uses fixed references and benign calibration data without requiring each player’s policy or mixture weights. We also specified when the latent mixture is identifiable and how to assess the stability of behavior within each mode. For players trained to suppress their evidence scores, behavioral evidence achieved 100% recall with a 0.16% benign false-positive rate, compared with 25.16% recall and a 1.25% benign false-positive rate for the RNN. For the recorded PPO runs, the largest upper bound on average regret was 0.2323, below the trivial bound of 2 raised by the reviewer. The reviewer has not replied further.

  3. Reviewer BrcW (initial score 2 → latest recorded score 2) recognized that the regret decomposition applies to the recorded Mahjong runs. Their remaining question concerns the scope of the PPO result: can PPO itself be given a general regret guarantee comparable to Theorems 2 and 3? The revised manuscript proves that optimistic multiplicative weights and optimistic projected gradient descent both satisfy the regret condition used in those theorems. With the platform rule fixed, increasing the horizon from 100 to 2,000 rounds reduced the learning term about twentyfold for both optimizers, as predicted. With PPO as the user optimizer across three 499-day Mahjong runs, the finite-horizon average-regret upper bounds ranged from 0.1306 to 0.2323. Following BrcW’s suggestion, we applied Theorem 1 to those runs to bound harmful exposure in the novice pool. With PPO as the platform controller, switching from rank predictions to behavioral evidence reduced the harmful-player share from 55.99% to 20.77% and improved benign players’ mean global rank from 2.690 to 2.596. In a four-player game, equally matched players have an expected rank of 2.5. The gap between benign players’ mean rank and this benchmark narrowed from 0.190 to 0.096, a 49.5% reduction.

The paper shows how a platform can make better routing decisions when its rules change how users play. A skilled player can finish at the same rank as a novice, yet their sequence of discards can reveal a different strategy. Our theory connects these individual responses to platform outcomes by bounding governance loss in terms of both the platform’s threshold choices and the users’ regret as they adapt. In Mahjong, routing on behavioral evidence more than halved the harmful-player share in the novice pool with both the PPO controller (55.99% to 20.77%) and the linear controller (46.10% to 17.48%). In the non-competitive service simulation, behavioral evidence also improved total benign-user welfare at a fixed threshold, and adapting the threshold improved welfare further.

We appreciate each reviewer’s time and thoughtful questions. We addressed their concerns point by point in the revised manuscript, where the proofs and controlled experiments establish the results for the two optimistic updates and the Mahjong experiments show how the framework works with PPO in practice.

Thank you again for your time and for guiding our submission through this review cycle. We hope the above summary helps streamline your decision process.

Warm regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.