- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes FastCombo, a recurrent graph policy trained with reinforcement learning for price discovery in single-minded combinatorial auctions. It proves that maximizing immediate clearing probability can require substantially more rounds than informative querying and demonstrates learned informative behavior in a small example. Experiments on CATS show competitive clearing performance and substantially reduced online price-selection computation compared with a Bayesian baseline.
Critical strengths
The theoretical construction clearly explains why informative price queries can improve subsequent decisions, and the controlled example makes the learned behavior interpretable. The reduction in online price-selection cost is supported by multiple training seeds, matched architecture controls, and detailed evaluation protocols.
Critical weaknesses
The evidence for a broader benefit from sequential information acquisition remains limited. The round-count improvement is demonstrated in a small example where another Bayesian precision also achieves the optimum, while Bayesian pricing uses fewer rounds in three of four CATS domains. The larger experiments therefore establish a computational advantage more directly than an advantage in bidder interaction. Since the timing comparison uses a default Bayesian configuration, the tradeoff against cheaper Bayesian settings at comparable clearing performance remains unclear.
The data split limits the generalization conclusions and leaves a comparison detail unresolved. Training, development, and evaluation auctions share the same source-file pool, so generalization to new source files is untested. The Bayesian prior explicitly excludes the evaluated auction’s source bid records, but it is unclear whether those records can appear during policy training. Clarifying this exposure would help assess whether the methods have comparable access to the underlying data.
Questions for authors
How do the round counts and computational advantage change when varying the Bayesian precision and sampling or optimization budget? In particular, can a cheaper Bayesian configuration achieve clearing performance comparable to FastCombo?
Can evaluated bundle–value records occur during FastCombo training, and how does this compare with their exclusion from the Bayesian prior? Are results available with training and evaluation drawn from disjoint source files?
2 comments
The eval on separate source files and comparison to cheaper bayesian settings address my main concern, so I adjusted my score.
Response to Reviewer VNid
Thank you for highlighting the Bayesian-configuration and data-exposure questions. We have completed new precision and source-file separation experiments and examined cheaper Bayesian settings.
Question 1: Performance against tuned or cheaper BayesGEN. We agree that comparison with the default Bayesian configuration alone does not resolve the tradeoff against cheaper settings. We therefore varied both precision and computational budget. We tested λ ∈ {1, 0.1, 0.01}, selecting precision on development data. On independent assessment auctions, Paths BayesGEN changes from 91.65% clearing and 13.55 mean rounds at λ = 1 to 92.35% and 13.49 rounds at λ = 0.1. Arbitrary changes from 69.35% and 20.80 rounds to 69.25% and 20.79 rounds at λ = 0.01. Precision tuning therefore has modest effects in this study.
Cheaper Bayesian settings can achieve similar clearing rates and use fewer rounds, but online pricing remains substantially slower in our tests. On Paths in the separate 100-auction-per-domain timing study, BayesGEN with 128 samples and at most 5 iterations clears 89.0%, versus FastCombo’s 88.9%, and uses 11.24 versus 14.31 mean rounds. However, pricing computation remains 3.04 s per auction versus 20 ms. On Arbitrary, 32 samples and at most 10 iterations give 68.0% clearing and 20.57 rounds, versus FastCombo’s 66.1% and 24.09 rounds, at 4.25 s versus 35 ms. These paired timings use matched hardware. Similar clearing point estimates are not formal evidence of equivalent performance.
A separate matched-hardware Paths study on 100 development markets measures 15 ms per auction for FastCombo, 6.4 s for default BayesGEN, and 13.7 s for development-selected precision. These measurements concern online pricing computation and exclude bidder delays. The timing samples differ from the full clearing benchmark in Table 1. In the revision, we will add these comparisons and clearly distinguish the timing samples and measured costs.
Question 2: Record exposure and separate source files. We agree that separate evaluation auctions do not establish generalization to new source files: evaluated bundle–value records could occur during original FastCombo training. BayesGEN excluded the evaluated auction’s records when fitting its prior, but used other records from the same source file. We acknowledge this difference in access to the underlying data.
To address this concern, the new study separates 60 training, 20 development, and 20 assessment source files per domain, excluding assessment files from both policy and Bayesian-prior fitting. With three training seeds and 2,000 assessment auctions per domain, round-selected FastCombo achieves 93.28% clearing and 12.22 mean rounds on Paths, versus tuned BayesGEN’s 92.35% and 13.49 rounds. On Arbitrary, the results are 65.62% and 23.38 rounds, versus 69.25% and 20.79 rounds. Failures count as 50 rounds. Paths’ clearing and round-difference intervals include zero; Arbitrary still favors BayesGEN.
This tests source-file separation for the new fits. The files were already known to the project, and shared public calibration still uses the full corpus. Bayesian priors are now pooled across training files, and value normalization also differs, so these results complement rather than replace Table 1. We will add the evaluation with separate source files and clarify the data-access and calibration protocols in the revision.
Criticism: Limited evidence for informative pricing and round savings. We agree that the toy experiment does not establish an advantage over every Bayesian precision setting: λ = 0.1 also achieves the optimum. However, its purpose is to demonstrate that RL can learn informative queries that improve on greedy clearing, which λ = 1 reproduces on this instance. A one-step Bayesian objective can also favor informative prices here, even without explicitly optimizing future rounds.
We also agree that the original CATS experiments do not establish general round savings over BayesGEN. However, there is little room to improve its clearing rates: BayesGEN is already within 0.68–2.02 percentage points of the linear-price clearing ceiling. This limits the scope for clearing gains, although it does not explain the round-count differences. To test a setting with more room to improve clearing, we extended Paths to multi-minded bidders, where the linear-price feasibility rate is 97.55%, 33.25 percentage points above BayesGEN’s discovery rate. A FastCombo controller adapted to multi-minded settings discovers supporting prices within 50 queries in 72.65% of 2,000 fresh draws from known source files, versus BayesGEN’s 64.30%. We assess approximate price support using a clearing gap below 0.001, independently of the choice among tied demands at a given price. We will add these experiments and clarify the distinct purposes of the toy and larger-market comparisons in the revision.