acceptodds
Withdrawn from ICLR 2027

Beyond High Prices: Auditing Information-Mediated Enforcement in Recurrent Pricing Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

High prices and correlated responses do not prove that pricing agents have learned collusive enforcement. We study this distinction in a continuous-price duopoly with noisy public signals and recurrent agents trained by proximal policy optimization. We use two benchmarks. A two-state trigger model shows when deviations are deterred by future losses. A Gaussian-monitoring model shows how noise weakens this threat. We audit the learned GRU policies using counterfactual price interventions and signal ablation. Across ten seeds and one million interactions, normalized price elevation drops from 0.43 in early training to 0.015 under informative monitoring, and to 0.019 after signal ablation. The paired difference is 0.0037, with a 95 percent bootstrap interval of [−0.0241, 0.0288]. Crucially, losses from forced price cuts hit immediately. They do not stem from adverse future values, and several individual deviations remain profitable. Rival responses are small under informative monitoring and vanish entirely under ablation. These results separate the illusion of coordination from actual dynamic enforcement. We find no evidence of real trigger enforcement in this setting.

Primary area:
reinforcement learning
Keywords:
Algorithmic Pricing, Reinforcement Learning, Imperfect Monitoring, Counterfactual Audit, PPO

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_qamvOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper conducts a simulation of pricing policies represented by GRU networks and trained with PPO. In the simulation they observe that prices converge to Nash and forced price cuts generally lose money. One of their models considers the case where prices are not perfectly monitored.

I was unable to understand from the paper why GRU+PPO should be used for pricing. There is an extensive literature on pricing algorithms in OR and CS and this is not one of the usual algorithms. That literature tends to prove that its algorithms perform well in broad families of environment, such as iid or adversarial demand. The adversarial bounds transfer to equilibrium environments and result in correlated equilibria. I think before we care much about how GRU+PPO works in price competition, we should have justification for why it should be used in pricing. This paper does not give sufficient justification.

It seems like Proposition 2 is false as stated. The mistake in its proof is to drop the αδ\alpha\delta term from the denominator of VC+VPV^C + V^P equation. (I'm not listing this as a critical mistake, because it doesn't change my opinion of the paepr.)

Critical strengths
  • algorithmic collusion is a topic of significant interest.
Critical weaknesses
  • This paper is not very human readable. I got almost nothing from reading the paper's introduction that I didn't already get from the abstract.

  • This paper does not place its results in the now extensive literature on algorithmic collusion. It cites two such papers. For example, even the well known Calvano et al (2021) which studies imperfect monitoring -- which is related to their study -- is not cited.

  • There is insufficient justification given for considering the algorithms that are considered.

  • The paper does not show that the results they obtain in their simulation are likely to be representative of what would happen if their algorithms are run in other settings.

Questions for authors

I have no questions.

2 comments
AuthorsOpenReview ↗

Response to Reviewer: Clarifying the Choice of GRU+PPO, Theoretical Corrections, and Scope of Our Findings

Response to Reviewer

We thank the reviewer for the detailed and helpful comments. We agree with the concerns regarding our choice of GRU+PPO, the need for a clearer connection to prior literature, and the scope of our results. We have revised the main paper and the supplementary material accordingly, and summarize the key changes below.

  1. Why we use GRU+PPO

We agree that GRU+PPO is not a standard pricing algorithm in the operations research literature, and we do not present it as a benchmark or claim any general performance guarantees for it. Instead, we use it as a controlled environment to answer a specific question: do learned pricing policies respond to unilateral deviations through future punishment?

Because our environment features noisy, delayed public reports, agents must condition their pricing decisions on a history of observations rather than the rival's current price. A GRU handles this historical sequence, while PPO accommodates continuous price actions. We have clarified this motivation in Section 2.3 and Supplement C, making it clear that we are not arguing GRU+PPO is superior to established pricing algorithms.

  1. Research question and readability

We agree the original introduction did not frame our research question clearly enough. The core economic distinction we want to highlight is that a high price is not, by itself, evidence of collusion or punishment. A firm can lose money from a price cut even if its rival never responds.

Our study asks whether a deviation is deterred by this immediate profit loss or by lower continuation payoffs, and we explain how our audit separates these two effects. We have rewritten the Introduction around this point and added worked examples in Supplement I so the economic intuition is accessible without needing to parse all the technical details first.

  1. Relation to previous work

We have expanded the literature discussion in the Introduction and Supplement A.5 to better connect with existing work, including Calvano et al. (2020, 2021), Deng et al. (2025), and classical research on imperfect public monitoring.

We explicitly acknowledge that Calvano et al. (2021) already studies learning under imperfect monitoring and uses forced-deviation interventions. We are not claiming to introduce deviation experiments for the first time. Rather, our specific contribution is decomposing a deviation's effect into an immediate profit component and a discounted continuation component using matched counterfactual simulations, alongside an exact negative control for the action-independent reporting channel. This allows us to isolate a specific enforcement mechanism rather than just inferring collusion from observed price levels.

  1. Generality of the results

We agree that our simulations do not prove how GRU+PPO would behave in all pricing environments. Our experiments rely on a specific market specification, a finite horizon, and a particular observation structure. We have made these limitations explicit in Section 6 and Supplement H.

Our findings describe the trained policies and the specific interventions we tested; we do not claim they automatically extend to other algorithms, demand processes, or indefinitely repeated markets. We also agree that comparing our setup with established pricing algorithms (especially those with theoretical guarantees) would help bound the scope of our results. Such comparisons are beyond the scope of our current completed experiments, and we do not present them as such.

  1. Correction of Proposition 2

Thank you for catching the error in Proposition 2. The original derivation failed to properly account for false alarms during cooperation. Writing gg for the immediate deviation gain and h=πC−πPh=\pi^{C}-\pi^{P}, the corrected Bellman equations yield:

VC−VP=h1−δ+δαV^{C}-V^{P}=\frac{h}{1-\delta+\delta\alpha}

with the corrected incentive threshold:

δ∗=gg(1−α)+(β−α)h\delta^{*}=\frac{g}{g(1-\alpha)+(\beta-\alpha)h}

and the feasibility condition:

(β−α)h>αg(\beta-\alpha)h>\alpha g

The feasibility condition now makes it explicit that frequent false alarms can make cooperation unsustainable, even if deviations are more likely to be detected. The full derivation is now provided in Section 5 and Supplements A.1 and I.8.

  1. Clarifying our contribution

Finally, we have refined the statement of our contribution. We are not proposing a new pricing algorithm or a general convergence result. Instead, we provide a method to examine whether future profit losses deter unilateral deviations in learned pricing policies.

In our experiments, almost all the average loss from the tested price cuts occurs immediately, with very small continuation effects. This describes the behavior of the specific policies in our tested environment, not a general claim about the absence of algorithmic collusion.

We are grateful to the reviewer for pointing out the theoretical error and for the feedback that helped us tighten the experimental interpretation and clarify our contribution.

Let me be more specific on why the choice of pricing algorithm (GRU+PPO) makes this paper not relevant. There is a very large space of algorithms that output prices. Obviously most of these algorithms are not worth evaluating. For an algorithm to be worth evaluating we must argue that it is special among algorithms. For example, with provable guarantees or wide use in practice of pricing or other such arguments. A simulation paper especially must be held to a strong standard of justifying its choice of algorithm. What have we learned about algorithmic collusion by the simulation properties of an algorithm that nobody has considered or is likely to use for pricing?

The authors have not seemed interested in understanding the literature on algorithmic collusion, still having only cited three papers in this space. I am quite certain that the people studying algorithmic collusion will to an equal or greater extent uninterested in this paper (due to the flaw above).

In the author response, they say:

The core economic distinction we want to highlight is that a high price is not, by itself, evidence of collusion or punishment.

I don't think this is a correction in understanding to how the literature on price collusion identifies collusive outcomes.

Reviewer_c5yQOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks whether high prices in learned pricing agents mean the agents punish deviations, or whether they merely drift and co-move. Two firms set prices and observe a noisy public report; two recurrent agents each read the last 32 reports, with ten seeds and one million interactions. In one condition the report depends on prices, in the other it is replaced by action-independent noise with delay, horizon and budget unchanged. The frozen policies are probed with forced price cuts, measuring the rival's response and the profit lost, immediately and over the rest of the episode; a two-state trigger model and a Gaussian noise model serve as benchmarks.

Prices settle near the one-period equilibrium in both conditions, differing by about 0.004 with an interval covering zero and both signs. The loss from a cut is almost entirely immediate, the later part is about one hundred-thousandth, and rival responses are exactly zero under the noise condition. The paper concludes there is no evidence of enforcement.

Critical strengths

None

Critical weaknesses
  1. The main comparison is confounded and the audit has no demonstrated power. The noise channel also changes the signal's mean and spread, not only its dependence on actions, and the authors say so; no positive control is run; with ten seeds the interval is about twice the size of the effect it should detect and per-seed differences change sign. The design also predicts the null: episodes last 512 periods with remaining time as an input and no bootstrap, so near-equilibrium pricing is what backward induction gives, and the signal saturates near that level. The result is close to Deng et al. (2025), in an environment arranged to make collusion hard. The abstract's interval [0.0241, 0.0288] also contradicts Table 1 and the point estimate 0.0037. This is decisive for my recommendation: with the sensitivity of the instrument unknown and a design that predicts its own null, a null result cannot be read as evidence that enforcement is absent — which is the paper's central claim — so what remains is a replication in a favourable environment.
  2. The proof's step on the value difference between cooperating and being punished ignores that punishment can also start by mistake. The corrected threshold therefore carries an extra false-alarm term, so the printed one is too low whenever false alarms are possible — exactly the noisy regime studied; the two agree only with perfect monitoring. I can give a small counterexample in which the printed formula says a deviation is deterred although it is profitable. The area chair asked reviewers to check this, and I confirm the objection. It is consequential because this is the paper's only theoretical claim: as printed it cannot be applied or cited, the correction lowers the deterrence threshold and so changes the feasibility condition the paper's reading of the two-state benchmark rests on, and the error is repeated in the appendix rather than being a slip.
Questions for authors

NA

1 comment
AuthorsOpenReview ↗

We corrected Proposition 2, reframed the noise experiment as a comparison of reporting channels rather than an information treatment, acknowledged the missing positive control and other design limits, and narrowed the conclusion to a null finding for the specific policies and price cuts tested.

Response to Reviewer Comments: Corrections to Theory, Experimental Design, and Interpretation

We thank the reviewer for a detailed report. The comments led us to correct the theory, relabel one experiment, and narrow our claims. The revisions to the paper and the supplement are summarized below.

  1. Experimental design and interpretation

Replacing the public signal with independent noise also changes the distribution of reports, so the comparison is not a pure information treatment. Section 2.2 and Supplement B.2 now describe it as a comparison between two reporting channels.

The negative control shows only that future responses are exactly zero when actions cannot affect later observations under our stated architecture. It does not show that the audit would detect actual punishment. Sections 3.2 and 6 draw this distinction, and Supplement H discusses a positive control based on a known enforcement strategy. That experiment has not been run and is not reported as completed.

  1. Finite horizon, saturation, and statistical power

Agents observe normalized remaining time and each episode ends after 512 periods, so our results describe a finite-horizon learning environment rather than a continuing market. The public signal also saturates at the Nash benchmark, which can make some price cuts hard to detect. Both limits are now stated in Sections 2.2 and 6 and Supplement H.

Ten seeds cannot establish that the two channels have equivalent effects. We report the paired difference in normalized price elevation as 0.0037 with a 95% confidence interval of [-0.0241, 0.0288], correct the interval notation, and note that an interval covering zero is not evidence of equivalence.

  1. Correction of Proposition 2

The original proof did not account for false alarms during cooperation. Writing g for the immediate deviation gain and h = π^C - π^P, the corrected Bellman equations give:

VC−VP=h/(1−δ+δα)V^C - V^P = h / (1 - δ + δα)

δ∗=g/[g(1−α)+(β−α)h]δ* = g / [g(1 - α) + (β - α)h]

(β−α)h>αg(β - α)h > αg

The last inequality is the feasibility condition: frequent false alarms can make cooperation unsustainable even when deviations are more likely to be detected. Section 5 and Supplements A.1 and I.8 give the derivation. The result concerns one specified deviation under a fixed trigger strategy; it does not show that PPO agents learn that strategy.

  1. Contribution

The contribution is not that PPO agents converge near Nash prices. A price cut can be unprofitable without any punishment, and our audit separates the immediate loss from the later one. For the deviations tested, almost all average loss is immediate and the continuation effects are very small. This gives a more precise account of why those deviations fail; it does not establish general power to detect collusion. The Introduction, Section 3, and the supplement now position the paper accordingly relative to earlier work, including Deng et al. (2025).

  1. Revised conclusion

The conclusion no longer claims that learning agents cannot punish deviations. It reports that the recurrent PPO policies examined here show no meaningful continuation-based deterrence for the price cuts tested. Positive controls, alternative monitoring channels, and continuing-market experiments are needed before any broader claim.

We thank the reviewer for comments that corrected the theoretical result and tightened the interpretation.

Reviewer_CDEzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the problem of determining whether elevated prices reflect learned enforcement against unilateral deviations. The paper proposes an audit combining matched price interventions, public-signal ablation, and a decomposition of immediate and continuation profits. Experiments with recurrent PPO agents find near-Nash prices and negligible continuation effects, while a trigger-strategy model is intended to provide a theoretical benchmark

Critical strengths
  1. Separating immediate losses from future losses addresses an important ambiguity in deviation tests. A price cut can be unprofitable without any retaliatory response, and the audit makes this distinction explicit

  2. The experiments use ten independently trained seeds and matched intervention rollouts. The paper appropriately distinguishes uncertainty across training runs from uncertainty conditional on a fixed policy

  3. The negative-control argument clearly explains why action-independent observations produce zero subsequent responses under this architecture. The authors also avoid interpreting that mechanically implied result as evidence that collusion has disappeared

Critical weaknesses
  1. To the best of my understanding, Proposition 2 seems to contain an algebraic issue in the continuation-value calculation. Its Bellman equations imply VC−VP=(πC−πP)/(1−δ+δα)V^C-V^P=(\pi^C-\pi^P)/(1-\delta+\delta\alpha), so omitting the false-alarm term changes the incentive threshold and its feasibility condition. Would be good if the authors can verify

  2. The reported welfare values are inconsistent with the stated definition. With v=2v=2, c=1c=1, and market size one, Equation 3 bounds total surplus by one, yet Table 1 reports approximately 1.1596

  3. The audit lacks a functioning enforcement policy as a positive control. Combined with signal saturation near Nash prices and limited deviation coverage, this leaves its sensitivity to actual enforcement unclear

Questions for authors

NA

1 comment
AuthorsOpenReview ↗

we changed the second proposition and reconstructed welfare function

Response to Reviewer

We thank the reviewer for a careful reading. Two of the comments point to errors we should have caught ourselves, and the third led us to state our empirical claim more narrowly than we originally did. The paper and the supplement have both been revised. We describe the changes below.

  1. Proposition 2

The reviewer is right: the false-alarm probability α was missing from the continuation value. Once it is included, the difference between the collusive and punishment values is

VC−VP=(πC−πP)/(1−δ+δα)V^C - V^P = (π^C - π^P) / (1 - δ + δα)

This changes the threshold on δ and also the condition under which punishment is strong enough to deter a deviation. Section 5 now states the corrected proposition. Supplement A.1 gives the derivation line by line, and Section I.8 explains where the original argument went wrong.

  1. Welfare

The reported numbers did not match the definition we had written down. For logit demand, total surplus is consumer surplus plus the profits of both firms, and under that definition there is no reason for the value to be bounded by one. Supplement I.11 now sets out the calculation.

We could not reconstruct what the original simulator actually computed. Rather than guess, we relabel the reported column an "archived welfare index" and state explicitly that it is not verified total surplus. Table 1 and the surrounding text carry that caveat, and no welfare claim in the paper rests on those numbers any longer.

  1. Positive control and audit sensitivity

We do not have a positive control. Our design includes an exact negative control, but no experimental arm with a known enforcement policy, so we cannot demonstrate that the audit would detect enforcement if enforcement were present. Sections 3 and 6 and Supplement H now say this directly, and we sketch how a known trigger strategy could serve as a positive control in follow-up work.

Two related limitations are also stated more explicitly. Near the Nash benchmark, the public signal can stop responding to further price cuts. And we test only two deviation sizes at two intervention dates. Our null results therefore do not rule out enforcement under other policies, other signals, or other deviations, and the conclusion has been rewritten to say so.

  1. The main finding

The claim is now narrower. For the recurrent PPO policies we trained, most of the loss from a unilateral price cut arrives immediately rather than through later punishment. This is a statement about these policies and these deviations. It is not evidence that algorithmic collusion cannot arise; we simply find no sign of continuation-based enforcement in the cases we examined.

We are grateful to the reviewer for comments that made the paper more accurate.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.