- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The authors study how counterfactual actions reveal differences in sellers’ behavior in learning markets that may be hidden by similar observed prices. They propose measuring these effects through matched intervention and control rollouts, which produce a counterfactual response surface. They provide theoretical estimation guarantees and evaluate the method on scripted and learned pricing policies.
Critical strengths
None
Critical weaknesses
- The assumption that we can snapshot and replay the market seems restrictive. Although this is achievable in controlled simulators, it is unclear in which realistic applications we could restore every seller’s learner state and replay the same future market shocks. A clearer description of the intended users and the access they would require would help establish the method’s practical applicability.
- Although the paper demonstrates several uses for the response surface, I am not yet convinced that these experiments establish sufficient practical value. In particular, the targeting experiment measures recovery of labels assigned to scripted policy types, rather than the consequences of an actual market intervention. It would be helpful to explain more concretely how the response surface would inform a decision and what benefit that decision would produce.
- Overall, I remain unconvinced that the paper provides substantial enough additional insight or value. The theoretical results appear relatively elementary, so the contribution seems to depend mainly on the diagnostic method and its empirical benefits. A clearer explanation of what the full response surface adds beyond existing deviation-based diagnostics, and why that added value justifies the required access and evaluation effort, would strengthen the contribution.
Questions for authors
- In which realistic settings would we have the ability to restore seller learner states and replay market shocks?
- Could you give a concrete example of a decision informed by the response surface, particularly in the targeting application, and explain how its benefit would be evaluated beyond recovering scripted policy labels?
6 comments
Response to Reviewer Aekw (Part 2/3)
W2 & Q2: A Concrete Decision and Its Benefit
Thank you for this question. To turn the surface into a decision with consequences, we ran a remedy experiment with welfare outcomes on learned sellers, and targeting by the surface nearly doubles the consumer-surplus gain of targeting by PCI (Sec. 5.2 and App. B.10).
1. The Decision
A monitor or a vendor can apply a remedy to only of its seller pairs, since each remedy must be set up and checked. The remedy delays rival-price information, so that rivals' prices enter the observation with a 10-period lag. The data-age limits used in practice work in the same way. The RealPage settlement, for example, lets the models train on nonpublic competitor data only when the data are at least 12 months old [3]. The question is which pairs to treat.
2. How We Evaluate It
We apply the remedy to all 60 learned pairs (3 algorithms and 20 seeds) and continue Adaptive learning for steps. We then score each ranking rule at a budget of , which means . Since every pair is treated, we can compute the best possible choice of 12 pairs.
The surface score is averaged over the 6 cells of the full grid, and it uses no labels. For each pair, ΔPCI is the mean PCI over the last periods minus the same window of a matched run from the same snapshot with the same shocks but no remedy. The 95% intervals come from 2,000 bootstrap resamples of pairs, with the ranking recomputed in each resample.
3. Results
In this logit market, lower prices under the same demand raise consumer surplus, . We report the change in CS for the treated pairs, measured against the matched runs.
| Ranking rule | Mean ΔPCI, treated pairs | ΔCS, treated pairs |
|---|---|---|
| Random | −0.06 [−0.08, −0.04] | +4.3% [+2.9, +5.7] |
| PCI | −0.09 [−0.13, −0.05] | +7.5% [+4.2, +10.9] |
| Static deviation test | −0.11 [−0.15, −0.07] | +8.7% [+5.5, +11.9] |
| Response surface (no labels) | −0.17 [−0.21, −0.13] | +13.8% [+10.6, +17.0] |
| Best possible 12 pairs | −0.21 | +17.2% |
The same 12 treatments bring almost twice the gain in consumer surplus when the pairs are chosen by the response surface. Over PCI, the paired gain is +6.3 percentage points [+2.4, +10.1] in ΔCS and −0.08 [−0.13, −0.03] in ΔPCI. Over the static deviation test, it is +5.1 [+1.0, +9.3] and −0.06 [−0.11, −0.01]. The surface reaches 81% of the best possible effect. Total surplus rises under every rule, most under the surface.
4. What the Decision Maker Learns
High prices are a poor guide to where a remedy works. PCI spends part of the budget on high-price pairs that do not react to their rivals, and these pairs hardly move under the remedy. Over all 60 pairs, the Spearman correlation between the score and ΔPCI is −0.64 for the surface, −0.42 for the static deviation test and −0.31 for PCI. The surface can tell apart markets that prices treat as equal. Among learned markets whose PCI differs by at most 0.02, 27% have clearly different responses, with non-overlapping 95% intervals (App. B.8).
W3: What the Full Surface Adds beyond Deviation Tests
Calvano et al. [1] trace price paths after a one-shot deviation with learning frozen, and Harrington [2] proposes testing pricing algorithms for reward-punishment properties. The surface builds on these tests in three ways, and the revision measures each one.
1. A Matched Control
A pre/post contrast mixes the response with shocks and drift. Replacing the matched control with a pre/post contrast lowers AUROC from 0.94 to 0.81, and an independent control from the same snapshot gives 0.91. So the matched control mainly removes bias (App. B.5).
2. The Whole Path as a Feature
Deviation paths are usually inspected by eye or reduced to a scalar such as PD-16. The surface uses the whole controlled path as input. This matters because myopic best response reacts at lag 1 about as strongly as reciprocal punishment ( vs. ), and only the later lags tell them apart. Keeping lag 1 alone gives AUROC 0.74, one depth per regime gives 0.87, and the full surface gives 0.94.
Response to Reviewer Aekw (Part 3/3)
W3: What the Full Surface Adds beyond Deviation Tests (continued)
3. Frozen and Adaptive Evaluation
What a policy encodes differs from what continued learning adds. The CMA notes that one-off audits can quickly become outdated because deployed algorithms keep updating [4], and Adaptive evaluation measures this learning directly. For SAC, the response depth is 0.028 under Frozen and 0.041 under Adaptive evaluation, so a test with learning frozen misses about a third of the response. At the same budget of 15,000 transitions, the surface with both regimes reaches AUROC 0.91, against 0.83 for PD-16 with both regimes and a stabilized denominator. The paired gain is +0.08 [+0.05, +0.11] (App. B.4), and it holds on a pool that adds a rule which learns but never observes the rival (App. B.6).
The surface also predicts responses at intervention settings not used for fitting, with path RMSE 0.031 against 0.061 for PD-16 with history (App. C.9).
W3: Is the Added Value Worth the Effort?
A single reference cell costs 0.75% of the training transitions and keeps most of the value of the full grid. Costs are per policy pair and relative to the steps used to train one learned pair. We measure quality by AUROC on the scripted pool, and for learned sellers by the remedy effect and the Kendall of the pair ranking against the full grid.
| Setting | Transitions | vs. training | Scripted AUROC | Learned ΔPCI / |
|---|---|---|---|---|
| Reference cell, F+A | 0.75% | 0.91 | −0.15 / 0.86 | |
| Grid, 100 pairs per cell | 24% | 0.91 | −0.16 / 0.93 | |
| Grid, 500 pairs per cell | 120% | 0.94 | −0.17 / 1 |
The reference cell uses 250 pairs × 2 rollouts × 15 lags × 2 regimes, the same budget as the equal-budget comparison. It already reaches AUROC 0.91 and 88% of the full-grid remedy effect. The full grid serves prediction at new intervention settings and a complete description across . Frozen rollouts need no gradient updates, so its compute is about 0.7 times that of training (App. D.1).
W3: The Role of the Theory
Thank you for this view. The theory is deliberately simple. It states what observed data cannot identify and what a valid evaluation must copy. These requirements matter in practice. A checkpoint with network weights only, a common shortcut, silently turns Adaptive evaluation into Frozen evaluation and changes the Adaptive response depth by about 30% for SAC and TD3 (App. B.12).
Presentation
We have worked on the parts that took effort to read. Sec. 3.1 now defines the execution state and the learner state, and Sec. 5.1 defines the labels. App. C.7 to C.10 give the market, every scripted rule, the prediction model and the classifier.
References
[1] Calvano, Emilio, et al. "Artificial intelligence, algorithmic pricing, and collusion." American Economic Review 110.10 (2020): 3267-3297.
[2] Harrington, Joseph E. "Developing competition law for collusion by autonomous artificial agents." Journal of Competition Law & Economics 14.3 (2018): 331-363.
[3] U.S. Department of Justice. "United States of America et al. v. RealPage, Inc. et al.; Proposed Final Judgment and Competitive Impact Statement." Federal Register 90.232 (2025): 56286-56335.
[4] Competition and Markets Authority. "Algorithms: How they can reduce competition and harm consumers." Research paper, UK Competition and Markets Authority, 2021.
[5] Xu, Xiangzhe, et al. "Checkpointing and deterministic training for deep learning." Proceedings of the 1st International Conference on AI Engineering (CAIN), 2022: 65-76.
[6] Johnson, Justin P., Andrew Rhodes, and Matthijs Wildenbeest. "Platform design when sellers use pricing algorithms." Econometrica 91.5 (2023): 1841-1879.
Final Thanks
Thank you again for your time and for questions that made the paper more useful. We hope the concrete users, the remedy experiment with welfare outcomes and the cost account answer your concerns about practical value. If they do, we would be grateful if you would consider raising your rating. We are happy to provide more details and look forward to further discussion.
Response to Reviewer Aekw (Part 1/3)
We thank the reviewer for the detailed and constructive comments. Below we address each point in turn.
In short, the access we need is routine for the parties that run pricing algorithms, and regulators and monitors can obtain similar access. On learned sellers, choosing which pairs receive a remedy by the response surface raises consumer surplus by 13.8%, against 7.5% when the same number of pairs is chosen by PCI. One reference cell costs 0.75% of the training transitions and keeps most of this value.
W1 & Q1: Realistic Settings for Snapshots and Shock Replay
The method is built for the parties that run the algorithms or oversee them, and for them this access is routine. With less access, the Frozen surface remains identifiable.
1. Who Already Has This Access
Parties that run the algorithms.
Parties that oversee them.
For all of them, a snapshot is an ordinary checkpoint of weights, optimizer state, buffers and the shock-generator state, and faithful replay from such checkpoints is an established tool in deep learning [5]. Shocks are shared exactly when the simulator draws them. Without shared shocks the estimate stays unbiased and needs more pairs (table below).
2. What Each Level of Access Identifies
Shared shocks cut the variance of each pair by a factor of 4.3, since the two rollouts have a median correlation of 0.77. Natural deviations are rare, since SAC prices about 0.25s below its policy mean in only 0.4% of periods, so intervention is the practical route (App. B.5 and B.11).
3. Query-Only Access
A tester who can query each frozen policy but cannot copy its state runs both rollouts in its own simulator with shared shocks. It rebuilds the execution state by replaying the last 50 observed periods through the policy. This covers the 5-period window of the neural policies exactly and recovers the scripted execution state in 93% of units. With the 15k budget, Frozen AUROC is 0.83, close to the 0.85 with full snapshots (Sec. 6, App. D and D.1).