- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3(+1)
Read review
Paper summary
The paper introduces a splitting method for estimating SDE terminal distributions under a fixed simulation budget. The authors derive covariance and limit results for the empirical distribution, optimize a variance-based allocation, and construct a mixture of valid trees. A two-phase procedure estimates the required quantities and is evaluated on SDEs and diffusion models.
Overall, the paper addresses an important problem in simulation-based distribution estimation. The motivation is clear, and the theoretical analysis provides useful insights into the dependence created by shared trajectories.
However, the practical advantage of the full allocation procedure and several aspects of its empirical validation need further clarification.
Critical strengths
- Addresses an important problem: how to estimate a terminal distribution more accurately under a fixed simulation budget by sharing partial trajectories.
- The theoretical analysis explicitly accounts for sample dependence and provides covariance formulas, limit theorems, and an approximation guarantee for converting relaxed allocations into valid trees.
- The evaluation covers multiple SDE and diffusion models, different budgets and split schedules, and comparisons with independent sampling, simpler splitting strategies, and an OU oracle.
Critical weaknesses
- The paper emphasizes accurate terminal-distribution estimation. However, the KS allocation minimizes a worst-case variance surrogate rather than the expected KS error itself. Although this distinction is acknowledged, the experiments provide limited explanation of when the surrogate accurately ranks different splitting strategies.
- The practical advantage over simpler splitting strategies is not consistently demonstrated. Learned-c is often competitive on CIFAR-10 and FFHQ; in Table 8, with nine split points and budget 100k, its MMD reduction is 12.7%, compared with 8.8% for the full method.
- The practical procedure relies on estimated variance contributions and, for KS, a finite query set, whose errors remain outside the theoretical guarantees. Although the appendix provides implementation details, sensitivity to the pilot budget and query count is not evaluated, leaving the robustness of the learned allocation unclear.
- The OU oracle comparison is useful, but the claim of close performance needs qualification. At 19 split points, the learned KS reduction is 18.8%, compared with 22.8% for the oracle. The source of this gap is not investigated, and the comparison is limited to the largest budget.
Questions for authors
- Could the authors compare the variance surrogate with measured KS across candidate allocations, to clarify whether the stronger performance of some Uniform-c baselines comes from surrogate mismatch or variance-estimation error?
- Could the authors provide sensitivity results for the pilot budget and query count, and clarify when the full allocation provides a meaningful advantage over Learned-c?
- Could the authors explain the gap in the 19-split OU oracle comparison and extend this comparison to smaller budgets to better evaluate the two-phase procedure?
3 comments
Thank you for the detailed response and the additional experiments. The new results have addressed most of my main concerns. In particular, the agreement between surrogate and empirical KS rankings provides useful support for the proposed allocation objective. The expanded comparisons with Learned-c further clarify the benefits of the full allocation, while the exact-variance experiments offer a convincing explanation of the gap to the oracle. The pilot-budget analysis also helps explain the sensitivity observed in some settings. I am therefore raising my overall rating to 3.
Thank you for your reassessment of our paper. We appreciate your time and consideration and would be happy to discuss any further questions or concerns.
Thank you for the careful review and questions.
W1 / Q1. Does the surrogate rank allocations like KS? On OU process, where the variances are exact, we studied two gaps: between the bounds of Proposition 3, and between KS and the surrogate.
The expected KS error is hard to compute, so we cannot optimize it directly. Proposition 3 bounds it above and below, up to a logarithmic factor, and our allocation minimizes the lower bound. We also minimized the upper bound. After rounding, the dyadic trees produced by the two allocations are identical in all six settings (9, 19 and 39 splits, B=100k and 5M), and hence the results are exactly the same. Here, the log factor does not change the allocation.
For eight splitting strategies, the surrogate and KS orders agree in 26 of 28 pairs (Spearman 0.93) at both budgets. Hence, we believe the two rank allocations in nearly the same order.
W1 / Q1. Why can Uniform-c do better? Since the surrogate ranks allocations well, the main remaining cause is noise in the pilot's variance estimates. To confirm this, we reran Langevin and EDM with a much larger pilot, not counted in the budget. KS reduction over i.i.d. (%), higher is better:
With the larger pilot, the learned methods match or beat the best Uniform-c in every EDM case and at B=1M on Langevin, and come within 3 points at B=100k on Langevin.
W2 / Q2. When does the full allocation beat Learned-c? Note that the full allocation is strictly more expressive than Learned-c, but Learned-c only has to find a single parameter. Hence, when the pilot estimates are noisy, Learned-c can do better. As the budget grows and more effort is spend on learning the allocations, the full allocation becomes the better choice.
We now score MMD on Inception-v3 and DINOv2 features for the image models, as is standard for images (revised Tables 10 and 11). Here, the full allocation beats Learned-c in 43 of 48 settings. In the setting you cite (9 splits, 100k), the two are within 0.3 points (full allocation 21.9% vs Learned-c 21.6% with Inception-v3, and 21.1% vs 21.4% with DINOv2).
The full allocation scheme is complex to understand and can be sensitive to noise at low budgets. Our aim with developing Learned-c was to come up with a simpler and more robust method that requires estimating fewer parameters. Our experiments show that the simpler method can also be highly competitive, and this should be seen as an additional contribution to a practitioner.
W3 / Q2. Pilot budget and query count. The pilot uses B1=CB0.66 of the budget B. As suggested, for each task we tried three values of C, with 9, 19 and 39 split points and 500 runs per setting. The KS reduction of the learned allocation over i.i.d. (%) is:
Every setting beats i.i.d. except EDM with 9 split points and the smallest pilot, at both budgets. There the pilot is too small, so its estimates are noisy and the full allocation does worse than i.i.d. In that setting, Learned-c, with its single parameter, still gives 18–21% (see W2 for why). Otherwise, the results change by only a few points across pilot sizes.
We did not sweep the query count, which was too expensive to run in the rebuttal period.
W4 / Q3. The OU oracle gap. Our two phase algorithm differs from the oracle in two ways: it estimates the variances from a pilot, and it uses 1,024 instead of 10,200 query thresholds. To isolate these errors, we replaced the estimated variances with the exact ones, which closes most of the gap. The KS reduction rises from 15.9% to 20.6% at B=100k (oracle 24.0%) and from 18.3% to 20.8% at 5M (oracle 22.7%). The smaller remaining gap comes from the oracle's denser grid. Hence the algorithm is close to the oracle.
We are grateful for your careful review. We believe the new experiments address the questions you raised, and we hope you will consider raising your score in light of them. We are happy to clarify anything further.