- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper focuses on the optimization of auto-bidding policies within the context of offline reinforcement learning. The authors argue that optimizing strategies directly within a continuous bidding space is problematic: it is highly susceptible to issues stemming from insufficient coverage in historical data, and it tends to blend distinct decision tendencies, hindering the ability to adapt flexibly to varying market conditions.
To address these challenges, the authors propose PRISM. This framework first learns a set of latent decision modes based on the long-term returns of historical decisions. It then selects an appropriate mode based on the current context and employs a conditional policy to generate specific bidding actions. The methodology centers on mode discovery via "Return-Tilted Information Decomposition," combined with a policy optimization process that incorporates return-weighting and data-support-based filtering.
Experimental evaluations were conducted using the AuctionNet and D4RL datasets. PRISM demonstrated strong performance across various budget settings in AuctionNet, and ablation studies confirmed that both the mode discovery mechanism and the dynamic mode selection process are the core contributions of the approach.
Critical strengths
- Instead of optimizing the bidding strategy directly within a continuous bid space, the authors first learn latent decision modes and then dynamically select a mode based on the current market conditions and budget status. The design of both mode discovery and mode selection is comprehensive.
- The experimental results are robust, particularly the validation of dynamic mode selection, with strong performance demonstrated on AuctionNet, further supported by ablation studies and comparisons against fixed-mode baselines.
Critical weaknesses
- There may be a mismatch in Mode History between the training and inference phases. For instance, during training, the historical modes are derived from the actual actions present in offline data, whereas during inference, the model relies on the modes it has selected itself. Such discrepancies could potentially affect subsequent mode evaluation and selection; however, the paper has not yet explicitly analyzed whether the errors arising from this process accumulate throughout the decision-making sequence.
- The computational overhead of the method lacks thorough analysis. PRISM comprises several modules (Mode Encoder, Mode Evaluator, Support Gate…), with both the Evaluator and the Support Gate utilizing Transformers. Given the requirements for decision-making efficiency in automated bidding, an analysis of inference latency would be helpful.
Questions for authors
- Could the authors empirically evaluate the potential impact of the inconsistency in Mode History between the training and inference phases? Furthermore, as the decision-making process unfolds, does this discrepancy affect the accuracy of subsequent Mode Evaluation and Mode Selection steps?
- Could the authors also provide a comparison of the proposed method against existing baselines in terms of inference latency?
4 comments
Q1: Mode History Across Training and Inference
We agree that the mode history is constructed differently in the two phases: during training, historical modes are derived from the logged actions in the offline data, whereas at inference the history stores the modes selected by the policy itself. We have conducted a dedicated analysis to evaluate whether this discrepancy affects subsequent mode evaluation and mode selection, and whether its effect accumulates over the decision sequence.
Experimental Setup. We replay the same offline trajectories under two mode-history conventions, keeping the logged states and actions fixed. Under , historical modes are assigned to the logged actions by the encoder, following the training convention. Under , the history is updated with the modes selected by the gate and evaluator, following the inference convention. These selections update only the mode history and do not alter the replayed trajectory. This setup isolates the effect of the history convention on subsequent evaluation and selection, without introducing differences in visited states. All model parameters remain frozen throughout the analysis.
Experimental results.
-
Relatively small changes in candidate sets and evaluation rankings. The two history conventions produce different selections at some steps, but their candidate sets and evaluation rankings remain largely aligned: the candidate-set Jaccard is , and the Kendall ranking correlation is , compared with expected values of and for independent random candidate sets and rankings, respectively.
-
Selection divergences are not permanently sustained. After the first divergence, of trajectories select the same mode as the reference again at a later step. This discrepancy does not accumulate into a persistent drift as the decision-making process unfolds.
-
The self-selected-mode history improves deployment performance. In the closed-loop ablation, removing this history channel significantly reduces performance (Score change , 95% CI ). This demonstrates that the history formed from the policy's own selected modes contributes positively to inference-time performance.
Conclusion. The train-inference difference in mode-history construction leads to relatively small changes in candidate sets and evaluation rankings, and selection divergences are not permanently sustained. The self-selected-mode history is beneficial for deployment. We will add this analysis to the appendix.
Q2: Inference Latency and Computational Overhead
Thank you for requesting an inference-latency comparison. We have conducted additional measurements and report the mean and 95th-percentile (P95) latency below:
| Method | Mean (ms) | P95 (ms) |
|---|---|---|
| USCB | 0.26 | 0.31 |
| CQL | 1.57 | 2.61 |
| IQL | 1.59 | 2.64 |
| BCQ | 1.92 | 2.96 |
| CDT | 3.06 | 4.15 |
| DT | 3.62 | 4.68 |
| PRISM | 5.47 | 6.44 |
| DT-score | 6.36 | 8.41 |
| GAVE | 6.95 | 8.06 |
| DiffBid | 57.00 | 62.20 |
| GAS | 84.89 | 94.73 |
PRISM does not use large language models (LLMs) or large-parameter architectures. Compared with lightweight actor-based baselines, PRISM incurs some additional latency, but its mean and P95 latency remain in the single-digit millisecond range. It is faster than DT-score and GAVE on both metrics, and substantially faster than DiffBid and GAS. These measurements show that the additional mode-selection modules introduce only moderate overhead, while avoiding the much higher inference cost associated with iterative generative methods. We will include this comparison in the appendix.
Dear Reviewer MsVm,
Thank you once again for your valuable comments on our submission. As the discussion period draws to a close, we would be grateful to hear whether our responses have adequately addressed your concerns. Should any questions remain, we would be happy to provide further clarification. Your feedback would be especially helpful to us at this stage.
We sincerely look forward to your reply and thank you again for your time and consideration.
Response Overview
We thank the reviewer for the careful assessment and constructive suggestions. We organize our response around the following two questions:
To address these questions, we provide additional mode-history diagnostics, a deployment ablation, and inference-latency measurements against existing baselines.