- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes a new method to construct pathwise gradient estimators for discrete random variables, which are exact for order D polynomials. The derive close-form expression for standard discrete distributions and show that their estimator recover other methods already existing in the literature in special cases. They validate their method on standard dataset such as Poisson VAE.
Critical strengths
The paper is clear and the idea is interesting and original. The observation about excess variance in ReinMax is insightful and the fact that this estimator recovers and generalizes some already known one is elegant. The experiments are encouraging and support an advantage of the method over some baselines.
Critical weaknesses
The main issue is the role of D which is not investigated thoroughly and specifically the practical usefulness of using D higher than 2. The bias/variance/efficiency trade-off that it induces is not really analyzed and the experiments show modest gains sometimes from D=2 to D=3 and do not show D above 3, which substantially reduces the practical impact of the method.
Another substantial issue is the application of the method to one-hots. For the one-hot case and D=1 and D=2, the method recovers known estimators, but higher order exactness is in general not possible (consider ). So it is not clear how to extend the method, and assigning a scalar to each one-hot imposing an order that might not be beneficial.
Finally, although the experiments are encouraging, the evidence for the benefits of the higher-order extension over previously known second-order estimators remains limited.
Questions for authors
How would one quantify the bias-variance tradeoff incurred by higher D ? Is there an optimal D or at least a principled heuristic one could use to choose D (probably dependent on f) ? How does your method transfer to the one-hot representation which is wide spread in discrete modeling ? Could you run experiment in a larger scale setting (high dimension) or a setting where other methods struggle more to see a larger more convincing difference ?
4 comments
R3 (2/2): Categorical applicability and larger-scale evidence
Response to Reviewer 3 (Part 2 of 2): Categorical applicability and larger-scale evidence
On higher-order methods for categorical variables
Developing order- categorical estimators is not part of our scope. We show connections to categorical-variable gradients but do not claim a contribution to building higher-order categorical-gradient estimators. We leave building higher-order methods for categorical random variables to future studies.
On larger-scale settings and challenging benchmarks
We respectfully highlight that as shown in the appendix on experiment details, our experiments already include 200-dimensional latent for nonlinear Poisson-VAEs and hierarchical Poisson DEFs trained on RCV1 with 794,414 documents (Sections 6.2–6.3). These settings demonstrate several existing advantages:
- Computational efficiency: both standard Fini orders are faster than EAT-cubic and ReinMax in all six GPU timing configurations (Section 6.4, Table 5).
- Gradient quality: Fini-2 and Fini-3 align closely with the MVD reference while maintaining second lowest gradient variance (second to ST) in the nonlinear diagnostic (Section 6.2, Figure 3).
- Optimization: Fini-2 and 3 achieve the best performance among all competitors on all but one metric in 20 Newsgroups (see existing experiment section 6.1, 6.2, 6.3).
We further provide new equal-time optimization comparisons in Appendix G.9 (Figures 8–9), using measured runtimes to place trajectories on a common time axis. In the displayed nonlinear Poisson-VAE run, Fini-3 reaches a training negative ELBO of 115.47, versus 117.17 for the best non-Fini baseline at the common budget. In the displayed 20 Newsgroups DEF run, Fini-2 reaches 177.50, versus 180.54 for the best non-Fini baseline. Both Fini orders achieve lower negative ELBO than all tested baselines at these budgets. Furthermore, the best losses will not be subverted by running other methods until the end - by the end, other baselines are still worse. In other words, our methods are on a new pareto frontier against other baselines overall.
The statistical tests are hiding the fact that the improvement of Fini-3 over Fini-2 (if it even exist) is marginal (it is usually within the displayed std). The only improvement that is not marginal is when Fini-2 outperforms Fini-3 (in the binomial experiment with the latent of dimension 2). So I don't think it is reasonable to claim a true improvement, which brings us back to the practical usefulness of the method. The fact that order 4 or more are even less useful does not allow any improvement in that direction. Although the motivation and theoretical derivations are nice, if in practice one should basically only use Fini-2, it reduces the impact of the proposed method.
This scope should be acknowledge better in the paper.
Thank you for clarifying my question about categorical variables, and I still think the insight is interesting, as well as the results, I would just like to understand better if Fini-2 is basically the only setting (if other orders could actually improve).
resp to comment OWv2zQICET
We thank the reviewer for their continued engagement.
We agree with the reviewer on two points. First, the ELBO gains of Fini-3 over Fini-2 are modest in absolute terms: where significant, they range from 0.27 to 1.06 nats. Second, the largest single gap favors Fini-2 in the low-dimensional Binomial setting () by 2.4 nats. We do not claim that Fini-3 uniformly improves over Fini-2, and we will state this scope explicitly in the order-selection discussion and limitations.
We do, however, believe the evidence supports Fini-3 as a useful option beyond order two, for three reasons:
-
Gradient quality: Fini-3 reduces relative gradient bias by ~62–63% compared with Fini-2 in all five Poisson-rate buckets, while maintaining comparably low gradient variance (App. G.4, Fig. 6; Sec. 6.2, Fig. 3).
-
Optimization: Fini-3 significantly outperforms Fini-2 in 4/15 settings, versus 1/15 in the opposite direction (two-sided Welch's t-tests, 5% per-comparison level); the remaining 10 do not show statistically significant differences.
- Error-bar clarification: The earlier draft reported standard deviations across five seeds, not standard errors, and all four Fini-3 improvements exceed both methods' previously reported standard deviations. To avoid ambiguity, the tables now report standard errors (highlighted in the captions); the results and statistical conclusions are unchanged.
-
Runtime: With GPU fusion, Fini-3's mean epoch time ranges from 0.54% faster to 1.57% slower than Fini-2 across all six configurations (Sec. 6.4, Table 5), so the lower bias comes at essentially no additional cost.
In summary, the ELBO gains are often modest, but Fini-3 offers lower measured bias and comparable runtime, with significant optimization gains in several settings. We therefore recommend Fini-3 as a starting point, while acknowledging that Fini-2 can be preferable in some cases.
Response to R3 (1/2): Higher-order gains and order selection
We thank reviewer 3 for taking the time to point out the weaknesses and questions.
On the benefit of using Fini 3 over Fini 2:
In our empirical studies, Fini 3 performs significantly better than Fini 2 in more cases than the other way around.
Going from D=2 to 3 shows performance gains. We assess the performance differences between Fini 2 and 3 using two-sided Welch's t-tests, as in Appendix G.1, with five runs per method. At the 5% significance level, Fini-3 significantly improves over Fini-2 on Bernoulli Omniglot (p<10−4), Binomial (d=16) (p≈0.009), Binomial (d=128) (p≈0.044), and Poisson (d=16) (p≈0.015). Conversely, Fini-2 significantly outperforms Fini-3 only on Binomial (d=2) (p≈0.005).
Given this, we discuss more on this in the “How should practitioners choose the estimator order?” section.
On going above order 3, and higher-order tradeoffs between order 3 and 2:
Why we don’t show order >3 in original draft: our work already discusses the pitfall of going order >3: Section 3’s Poisson example and Appendix E.1 show that fourth-order weight variance diverges at small rates. This is precisely the reason we didn’t go beyond order 3 when testing our method.
In new draft, to empirically verify this point, we adapt the gradient-agreement diagnostic of Liu et al. [1, Figure 4] to our nonlinear Poisson VAE (see new draft’s Section 6.2), using a high-precision MVD reference and additionally measuring gradient variance throughout training. We add more baselines including ST (which is order 1) and Fini-4. Cos similarity is computed between the averaged gradient estimate and the reference.
This shows a bias-variance tradeoff with changing the order: Fini-2 and Fini-3 maintain near-1 cos similarity with the second lowest variance (second to ST) and strong, stable optimization. ST has lower variance but poorer alignment and a worse training objective. Fini-4 exhibits substantial variance spikes without a corresponding optimization advantage. These results directly illustrate the practical tradeoff.
How should practitioners choose the estimator order?
We recommend treating the order as a hyperparameter to tune, analogous to temperature in ReinMax and EAT-cubic. Both standard implementations are faster than EAT-cubic and ReinMax in every tested workload–GPU configuration. Furthermore, GPU fusion brings Fini-3’s observed mean epoch time within 1.6% of standard Fini-2 across all six configurations (Table 5). When budget permits, practitioners can compare and validate both.
If only one run is feasible, we recommend starting with Fini-3 for smooth nonlinear objectives. As it shows performance gain, listed in the answer above.
References
[1] Liu, L., Dong, C., Liu, X., Yu, B., and Gao, J. (2023). Bridging Discrete and Backpropagation: Straight-Through and Beyond. NeurIPS.
[7] Ibrahim, M., et al. (2026). A Hitchhiker’s Guide to Poisson Gradient Estimation. ICML, PMLR 306, 49387–49427.