acceptodds
Under review as a conference paper at ICLR 2027

RecurBid: A Multi-Scale Recursive Agent with Differentiable Dual Reflection for Real-Time Bidding

Abstract

Real-Time Bidding (RTB) serves as the economic backbone of digital advertising, demanding optimal constrained decision-making across millions of high-frequency second-price and first-price auctions under volatile market dynamics and rigid budget constraints. While recent pioneering attempts to incorporate Large Language Model (LLM) agents into RTB (such as RTBAgent) have demonstrated the promise of cognitive reasoning, existing agent frameworks suffer from four fundamental structural pathologies: (1) , where flat single-pass prompting fails to bridge multi-day macro budget planning, intra-day meso pacing, and sub-millisecond micro-auction execution; (2) , ignoring competitive opponent dynamics and falling prey to the winner's curse; (3) , lacking convergence guarantees for budget pacing; and (4) , rendering hundred-millisecond autoregressive token generation unusable within standard ad exchange Service Level Agreements (). To overcome these foundational limitations, we introduce RecurBid, a principled multi-scale recursive agent framework that integrates cognitive reasoning with differentiable dual control for real-time bidding. RecurBid is founded upon four synergistic pillars: (1) , which formally decomposes the constrained POMDP across nested Macro-Meso-Micro hierarchies through a recursive Bellman formulation; (2) , executing bounded Level- opponent belief filtering to anticipate competitor pacing adaptations and eliminate adverse selection; (3) , a micro-level state-space controller with provable Lyapunov contraction mapping guarantees, ensuring exponential pacing convergence and zero asymptotic budget overspend; and (4) , an asynchronous background reasoning engine that simulates counterfactual pacing trajectories and distills strategic guidance into closed-form, sub-millisecond analytical micro-bidding ( per request). We provide rigorous theoretical proofs for asymptotic budget stability (-contraction), bounded Level- competitive regret , and Pareto-optimality over flat agents. Extensive empirical evaluations across three benchmark datasets (iPinYou multi-campaigns, Criteo attribution, and Taobao ad exchange) against 12 competitive baselines demonstrate that RecurBid achieves +26.4% higher conversions, reduces Effective Cost Per Click (eCPC) by 31.8%, reaches 99.4% budget utilization without morning starvation, and establishes state-of-the-art bidding performance under non-stationary market conditions.

Primary area:
learning on time series and dynamical systems
Keywords:
Real-Time Bidding, Recursive Agent

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_95CzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper identifies four major pathologies of existing bidding agents and proposes a new bidding agent design with theoretical guarantees. Across multiple datasets, the proposed method achieves significant improvements over several baselines.

Critical strengths

The paper designs a bidding agent with theoretical guarantees and achieves substantial improvements over the baselines across multiple datasets.

Critical weaknesses

The paper's writing needs substantial improvement, as the current presentation is rather disorganized and lacks a clear focus. Surprisingly, all the important theoretical results are placed in the appendix and should be moved to the main text. A thorough rewrite of the paper seems necessary before publication.

The paper contains too many typos, for example, "Complete Mathematical Proofs of Theorems A.1, A.1, A.1, and Propositions 1–2."

Questions for authors

For the 1/K1/K upper bound, can we obtain a corresponding lower bound?

Does the 0.42 ms latency account for processing multiple requests in parallel? In bidding scenarios, decisions often need to be made sequentially, such as updating the budget before placing the next bid. What would the latency be in this case?

Assumption (8) seems rather strong. The authors should conduct empirical tests on real-world datasets to examine whether this assumption holds in practice.

Reviewer_xCWFOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

RecurBid proposes a three-level RTB framework: macro budget allocation, meso pacing and reflection, and micro-level bidding with dual budget control. Recursive theory-of-mind models competitors, while asynchronous reasoning keeps auction-time decisions fast. The paper evaluates the system on offline advertising datasets, reports comparisons with bidding baselines, and includes component ablations.

Critical strengths

A valuable direction for holistic autobidding optimization. The paper frames autobidding as a coordinated problem spanning cross-day budget allocation, intraday pacing, and per-auction decisions. This perspective is consequential because optimizing these decisions in isolation may miss interactions across timescales. It gives the work practical relevance, even though the incremental benefit of the proposed integration remains to be established.

Critical weaknesses
  1. Formalize and substantiate the claimed failure modes. Propositions 1 and 2 assert failure modes associated with delayed decisions and static adversary modeling, but the paper does not clearly provide proofs for these propositions or direct experiments isolating the claimed effects. As a result, it remains unclear under what assumptions these failures occur, how they are measured, or how severe they are in practice. The authors should provide rigorous proofs and/or targeted empirical evidence, and clarify how each failure mode motivates a specific component of RecurBid.

  2. Clarify the novelty and value of the cross-layer integration. RecurBid appears to combine cross-day budget allocation, intra-day pacing, and per-auction bidding, each of which has relevant precedents. A unified description alone does not establish what is gained by integrating them. The paper should identify the specific cross-layer mechanisms that distinguish RecurBid from existing hierarchical bidding methods and evaluate their incremental benefit against fair hierarchical baselines or straightforward combinations of existing modules. While the current ablations show that removing components hurts performance, they do not fully isolate the value of the proposed coupling.

  3. Insufficient Experimental Evidence. The baseline comparisons and ablations are informative, but they do not establish whether RecurBid’s gains arise from its proposed cross-level coupling or from combining individually useful components. A controlled comparison against a straightforward combination of cross-day budget allocation, intra-day pacing, and per-auction bidding—or an appropriately matched hierarchical baseline—would better isolate the value of the joint design. This does not require an existing method with an identical three-module structure.

Questions for authors

See "Critical Weaknesses"

Reviewer_TgdhOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a bidding architecture for RTB based on LLM.

  • The architecture relies on a times scale decomposition, where the higher frequency scale relies on dynamic programming.
  • Some regret sub-optimality claims are made about previous similar architecture that the present paper aim at correcting (RTBAgent).
  • Experiments were done on three classic datasets.
Critical strengths

I believe that as it is, the paper needs to me reworked before a fair assessment of its strength can be provided.

Critical weaknesses

Formalism not rigorous enough:

  • some notations never defined (see in particular 3.3 and 3.4), context not fully stated
  • the two propositions statements are vague, and miss proofs, the claim of the abstract on regret does not appear in the main document
  • justification of the structural pathologies claimed by the authors should be done with more details
  • no limitation section
  • there are many papers with theoretical guaranties to deal with budget pacing that the authors did not mention

Experiments

  • I do not understand how the modelling of the competing DSP play a role in the experiment, since the bids are fixed and the auction second price.
  • only one LLM was tested,
  • no baselines from the online learning literature
  • how equations (5), (6) and (7) are solved in the experiments should be more detailed.
Questions for authors
  • please provide a better description of the numerical experiment?
  • can you explain what you gain from a theory of mind in the context of bidding, compared to modelling the CDF of the competition?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.