- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
D3AO addresses offline assortment optimization by combining a neural choice estimator, a plug-in revenue function, and a binary denoising model trained on historical assortments. During generation, coordinate-wise revenue differences modify the reverse proposal's Bernoulli logits, with stronger guidance toward the final steps. The paper interprets the guidance as a KL-regularized improvement for a local additive surrogate. Synthetic experiments use MNL, MCCM, and MMNL choice models, up to 100 products, and 256 generated candidates per instance. Additional analyses cover exact recovery, diversity, sample size, and changes in the assortment-collection policy.
Critical strengths
-
Choice estimation, a historical assortment prior, and coordinate-wise guidance have clearly separated roles. Theorem 2 gives an exact interpretation of the logit shift for its local surrogate. The unguided comparison also establishes that the revenue signal is important in the tested setting.
-
The method performs strongly on the synthetic MMNL benchmark relative to the misspecified parametric baselines. Reporting means, quantiles, recovery, and diversity is more informative than reporting only the best candidate, and the distribution-shift experiment exposes an important limitation rather than hiding it.
Critical weaknesses
-
Comparisons with MNL-MLE and MCCM-EM change both the choice estimator and the optimizer. Improvements under MMNL misspecification can therefore arise from the neural estimator alone. The unguided ablation establishes the need for guidance, not for a learned diffusion prior. This distinction is especially important because Section 4.1 replaces the standard reverse kernel with direct clean-sample resampling, while the reported maximum guidance strength is 1000. A comparison using the same fitted GAsN/RAsN revenue function with non-diffusion search, under comparable evaluation and time budgets, is needed to substantiate the central contribution.
-
The maximum true-revenue ratio among 256 candidates is a valid oracle statistic, but the learner cannot use true revenue to choose that candidate at deployment. The performance of the candidate selected by estimated revenue is not clearly reported. This omission becomes decisive under collection-policy shift: in Table 9, MMNL with N=80 gives a GAsN candidate mean of 55.18% despite an oracle maximum of 91.34%; the RAsN maximum is 80.36%. The Boltzmann assumption based on true revenue is a controlled benchmark choice, not a consequence of Theorem 1 about actual historical policies. Moreover, the abstract's pessimistic-estimation language lacks a corresponding uncertainty penalty or confidence bound in the method. Together, these issues leave the claimed offline improvement insufficiently supported outside the main data regime.
Questions for authors
- With exactly the same fitted GAsN/RAsN revenue model, how does D3AO compare with multi-start discrete local search or another non-diffusion optimizer at matched evaluation budgets? How sensitive are the results to guidance strength, and does the learned prior improve over reward-guided search from an untrained or simple reference proposal?
- What is the actual final selection rule? Please report the true revenue of the candidate maximizing estimated revenue, alongside the oracle maximum, especially in Table 9. Is any uncertainty-aware pessimistic adjustment implemented?
- How are product revenues generated, Boltzmann assortments sampled for large N, and MMNL reference optima computed or certified? Please also explain why favored-item utility means grow with N, and diagnose Table 8's N=40 GAsN mean decline from 97.77% at 10,000 observations to 73.81% at 50,000. Does this originate in estimation, prior learning, or optimization?
We thank the reviewer for the careful assessment and the detailed numerical checks. New tables R1–R3 are in our reply to Reviewer yEcx; all new results use 10 seeds, the paper's hyperparameters and true optimal ratios (%).
W1. Confounded baselines, learned prior, guidance sensitivity
We agree that MNL-MLE and MCCM-EM change both estimator and optimizer, and that the unguided ablation does not test the prior. With the identical fitted GAsN/RAsN reward:
(a) Direct optimization (Table R2). Multi-start local search (add/drop), simulated annealing and random search, each with D3AO's number of Rθ evaluations, for N∈{40,80,120} and several budgets. Local search and SA match D3AO or are slightly better (up to ~0.5 points at N=120), already at 1/16 of the budget. We no longer claim that diffusion search beats direct optimization of the same reward.
(b) Learned prior (Table R3). Same guidance with the prior versus gϕ=0. The prior does not improve the returned assortment in any setting (n∈{1,000,3,000,10,000}, N∈{40,80,120}, shifted logging); without it, results are equal or slightly better. We state this, and identify reward estimation as the limiting factor on these benchmarks. We did not ablate the reverse kernel of Sec. 4.1; the revision describes it as a design choice and removes the claim that it performs better.
(c) Guidance strength (MMNL, N=80, γ=3). The returned assortment reaches 68.10 ± 5.61% at λmax=10, 79.99 ± 5.00% at 100, 97.42 ± 0.97% at 1000 and 97.60 ± 0.74% at 10000: insensitive once λmax≥1000, much worse when guidance is weak. The effective shift λtΔiRθ depends on the scale of Rθ. All submitted experiments used λmax=1000, γ=3; we did not vary γ.
We also reran both parametric baselines on the new instances (Table R1). They are near-optimal when correctly specified, fall to ~50% under MMNL with Boltzmann data (D3AO 97.13%), and under shifted logging MCCM-EM (97.89%) is slightly better than D3AO (97.20%).
W2. Deployable selection, distribution shift, "pessimistic"
We agree that the oracle maximum cannot be deployed and that the gap matters under shift. The returned decision is now argmaxs∈CRθ(s) over distinct candidates. Under shifted logging (uniform size, then a uniform random subset; MMNL, N=80) it reaches 97.20 ± 0.94% versus the oracle 98.73 ± 0.77%, a gap of ~1.5 points (Table R1). These shifted-policy results were obtained by re-running this setting with the hyperparameters stated in the paper (λmax=1000, γ=3); we will update Table 9 with these reruns in the revision.
The Boltzmann policy is a controlled benchmark choice, not a consequence of Theorem 1; we revise Sec. 2.1. "Pessimistic" was inaccurate (plug-in estimate, no uncertainty penalty); we correct the abstract and list uncertainty-aware selection as future work.
Q1 / Q2
See W1(a)–(c) and W2. Selection uses Rθ only; ground truth is used only for evaluation.
Q3. Experimental details and Table 8
Table 8 decline. We re-ran this setting (MMNL, N=40, β0=1, 10 new instances, statistics over all 256 samples as in the paper). The decline reproduces:
It is not driven by a few runs: the GAsN mean is lower at n=50,000 than at n=10,000 on all 10 instances. Optimizing the same fitted Rθ with D3AO, D3AO without the prior, and local search gives exactly the same assortment on every instance and n. The decline therefore comes from the fitted revenue model (its maximizer has lower true revenue at larger n), not from the prior or the search. We have not identified why (training hyperparameters were fixed across n) and will report this as a limitation rather than presenting the sample-size results as monotone.