acceptodds
Under review as a conference paper at ICLR 2027

Extending Pathwise Gradients to Discrete Random Variables

Abstract

Pathwise gradients are preferred for continuous random variables because they are unbiased, low variance, and work with a single sample. For discrete variables, however, the pathwise identity cannot generally be exact for every differentiable function. We propose a general framework to construct finite-order exact pathwise gradient estimators for a range of common discrete variables such as Poisson. The estimator is the least-norm solution among all solutions that are unbiased for polynomials of degree at most . The resulting estimators preserve the hard forward sample, require no temperature tuning, and can be implemented in a few lines of codes. Against other admissible solutions, our estimator is unique and minimizes weight variance; in contrast, prior works use categorical variables or augmented representations to approximate non-categorical variables that induces excess variance and computations. To understand approximation bias for functions beyond the prescribed class, we also derive a non-asymptotic bias bound. In experiments our low order methods match or improve tuned baselines across linear, nonlinear and hierarchical latent-variable models, while out-speeding competitors in every runtime benchmark.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
pathwise gradient estimator, Monte Carlo gradient estimation, discrete random variable, Poisson gradient estimator

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vLDjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a new method to construct pathwise gradient estimators for discrete random variables, which are exact for order D polynomials. The derive close-form expression for standard discrete distributions and show that their estimator recover other methods already existing in the literature in special cases. They validate their method on standard dataset such as Poisson VAE.

Critical strengths

The paper is clear and the idea is interesting and original. The observation about excess variance in ReinMax is insightful and the fact that this estimator recovers and generalizes some already known one is elegant. The experiments are encouraging and support an advantage of the method over some baselines.

Critical weaknesses

The main issue is the role of D which is not investigated thoroughly and specifically the practical usefulness of using D higher than 2. The bias/variance/efficiency trade-off that it induces is not really analyzed and the experiments show modest gains sometimes from D=2 to D=3 and do not show D above 3, which substantially reduces the practical impact of the method.
Another substantial issue is the application of the method to one-hots. For the one-hot case and D=1 and D=2, the method recovers known estimators, but higher order exactness is in general not possible (consider f(e)=3e12−2e13f(e) = 3e_1^2 - 2e_1^3). So it is not clear how to extend the method, and assigning a scalar to each one-hot imposing an order that might not be beneficial. Finally, although the experiments are encouraging, the evidence for the benefits of the higher-order extension over previously known second-order estimators remains limited.

Questions for authors

How would one quantify the bias-variance tradeoff incurred by higher D ? Is there an optimal D or at least a principled heuristic one could use to choose D (probably dependent on f) ? How does your method transfer to the one-hot representation which is wide spread in discrete modeling ? Could you run experiment in a larger scale setting (high dimension) or a setting where other methods struggle more to see a larger more convincing difference ?

4 comments
AuthorsOpenReview ↗

Response to R3 (1/2): Higher-order gains and order selection

We thank reviewer 3 for taking the time to point out the weaknesses and questions.

On the benefit of using Fini 3 over Fini 2:

In our empirical studies, Fini 3 performs significantly better than Fini 2 in more cases than the other way around.

Going from D=2D=2 to 3 shows performance gains. We assess the performance differences between Fini 2 and 3 using two-sided Welch's tt-tests, as in Appendix G.1, with five runs per method. At the 5% significance level, Fini-3 significantly improves over Fini-2 on Bernoulli Omniglot (p<10−4p<10^{-4}), Binomial (d=16d=16) (p≈0.009p\approx0.009), Binomial (d=128d=128) (p≈0.044p\approx0.044), and Poisson (d=16d=16) (p≈0.015p\approx0.015). Conversely, Fini-2 significantly outperforms Fini-3 only on Binomial (d=2d=2) (p≈0.005p\approx0.005).

Given this, we discuss more on this in the “How should practitioners choose the estimator order?” section.

On going above order 3, and higher-order tradeoffs between order 3 and 2:

Why we don’t show order >3> 3 in original draft: our work already discusses the pitfall of going order >3>3: Section 3’s Poisson example and Appendix E.1 show that fourth-order weight variance diverges at small rates. This is precisely the reason we didn’t go beyond order 3 when testing our method.

In new draft, to empirically verify this point, we adapt the gradient-agreement diagnostic of Liu et al. [1, Figure 4] to our nonlinear Poisson VAE (see new draft’s Section 6.2), using a high-precision MVD reference and additionally measuring gradient variance throughout training. We add more baselines including ST (which is order 1) and Fini-4. Cos similarity is computed between the averaged gradient estimate and the reference.

This shows a bias-variance tradeoff with changing the order: Fini-2 and Fini-3 maintain near-1 cos similarity with the second lowest variance (second to ST) and strong, stable optimization. ST has lower variance but poorer alignment and a worse training objective. Fini-4 exhibits substantial variance spikes without a corresponding optimization advantage. These results directly illustrate the practical tradeoff.

How should practitioners choose the estimator order?

We recommend treating the order as a hyperparameter to tune, analogous to temperature in ReinMax and EAT-cubic. Both standard implementations are faster than EAT-cubic and ReinMax in every tested workload–GPU configuration. Furthermore, GPU fusion brings Fini-3’s observed mean epoch time within 1.6% of standard Fini-2 across all six configurations (Table 5). When budget permits, practitioners can compare and validate both.

If only one run is feasible, we recommend starting with Fini-3 for smooth nonlinear objectives. As it shows performance gain, listed in the answer above.

References

[1] Liu, L., Dong, C., Liu, X., Yu, B., and Gao, J. (2023). Bridging Discrete and Backpropagation: Straight-Through and Beyond. NeurIPS.

[7] Ibrahim, M., et al. (2026). A Hitchhiker’s Guide to Poisson Gradient Estimation. ICML, PMLR 306, 49387–49427.

AuthorsOpenReview ↗

R3 (2/2): Categorical applicability and larger-scale evidence

Response to Reviewer 3 (Part 2 of 2): Categorical applicability and larger-scale evidence

On higher-order methods for categorical variables

Developing order-D>2D>2 categorical estimators is not part of our scope. We show connections to categorical-variable gradients but do not claim a contribution to building higher-order categorical-gradient estimators. We leave building higher-order methods for categorical random variables to future studies.

On larger-scale settings and challenging benchmarks

We respectfully highlight that as shown in the appendix on experiment details, our experiments already include 200-dimensional latent for nonlinear Poisson-VAEs and hierarchical Poisson DEFs trained on RCV1 with 794,414 documents (Sections 6.2–6.3). These settings demonstrate several existing advantages:

  1. Computational efficiency: both standard Fini orders are faster than EAT-cubic and ReinMax in all six GPU timing configurations (Section 6.4, Table 5).
  2. Gradient quality: Fini-2 and Fini-3 align closely with the MVD reference while maintaining second lowest gradient variance (second to ST) in the nonlinear diagnostic (Section 6.2, Figure 3).
  3. Optimization: Fini-2 and 3 achieve the best performance among all competitors on all but one metric in 20 Newsgroups (see existing experiment section 6.1, 6.2, 6.3).

We further provide new equal-time optimization comparisons in Appendix G.9 (Figures 8–9), using measured runtimes to place trajectories on a common time axis. In the displayed nonlinear Poisson-VAE run, Fini-3 reaches a training negative ELBO of 115.47, versus 117.17 for the best non-Fini baseline at the common budget. In the displayed 20 Newsgroups DEF run, Fini-2 reaches 177.50, versus 180.54 for the best non-Fini baseline. Both Fini orders achieve lower negative ELBO than all tested baselines at these budgets. Furthermore, the best losses will not be subverted by running other methods until the end - by the end, other baselines are still worse. In other words, our methods are on a new pareto frontier against other baselines overall.

The statistical tests are hiding the fact that the improvement of Fini-3 over Fini-2 (if it even exist) is marginal (it is usually within the displayed std). The only improvement that is not marginal is when Fini-2 outperforms Fini-3 (in the binomial experiment with the latent of dimension 2). So I don't think it is reasonable to claim a true improvement, which brings us back to the practical usefulness of the method. The fact that order 4 or more are even less useful does not allow any improvement in that direction. Although the motivation and theoretical derivations are nice, if in practice one should basically only use Fini-2, it reduces the impact of the proposed method.

This scope should be acknowledge better in the paper.

Thank you for clarifying my question about categorical variables, and I still think the insight is interesting, as well as the results, I would just like to understand better if Fini-2 is basically the only setting (if other orders could actually improve).

AuthorsOpenReview ↗

resp to comment OWv2zQICET

We thank the reviewer for their continued engagement.

We agree with the reviewer on two points. First, the ELBO gains of Fini-3 over Fini-2 are modest in absolute terms: where significant, they range from 0.27 to 1.06 nats. Second, the largest single gap favors Fini-2 in the low-dimensional Binomial setting (d=2d=2) by 2.4 nats. We do not claim that Fini-3 uniformly improves over Fini-2, and we will state this scope explicitly in the order-selection discussion and limitations.

We do, however, believe the evidence supports Fini-3 as a useful option beyond order two, for three reasons:

  1. Gradient quality: Fini-3 reduces relative gradient bias by ~62–63% compared with Fini-2 in all five Poisson-rate buckets, while maintaining comparably low gradient variance (App. G.4, Fig. 6; Sec. 6.2, Fig. 3).

  2. Optimization: Fini-3 significantly outperforms Fini-2 in 4/15 settings, versus 1/15 in the opposite direction (two-sided Welch's t-tests, 5% per-comparison level); the remaining 10 do not show statistically significant differences.

    • Error-bar clarification: The earlier draft reported standard deviations across five seeds, not standard errors, and all four Fini-3 improvements exceed both methods' previously reported standard deviations. To avoid ambiguity, the tables now report standard errors (highlighted in the captions); the results and statistical conclusions are unchanged.
  3. Runtime: With GPU fusion, Fini-3's mean epoch time ranges from 0.54% faster to 1.57% slower than Fini-2 across all six configurations (Sec. 6.4, Table 5), so the lower bias comes at essentially no additional cost.

In summary, the ELBO gains are often modest, but Fini-3 offers lower measured bias and comparable runtime, with significant optimization gains in several settings. We therefore recommend Fini-3 as a starting point, while acknowledging that Fini-2 can be preferable in some cases.

Reviewer_RkBNOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose a recipe for constructing pathwise gradient estimators for discrete random variables. Since a pathwise estimator cannot in general be unbiased for every differentiable function of a discrete variable, they relax the requirement to unbiasedness for polynomials up to degree D. Among all the weights that satisfy this property, the authors pick the one with minimal L_2 norm, which is also the weight of minimal variance. The theory section presents propositions characterising the variance: on its optimality within the exactness class, on the excess variance introduced by compound representations, on the bias of the resulting estimator for smooth functions outside the prescribed polynomial class. The authors derive closed-form weights for the Poisson, Binomial and Negative-Binomial distributions, and show that the same recipe recovers classical pathwise gradients for continuous location-scale families and existing second-order estimators for categorical variables. Experiments cover a variety of latent variable models: linear and nonlinear Poisson VAEs, a partially observed GLM, Poisson deep exponential families, and count VAEs.

Critical strengths
  • The derivation is general and clean: the weight is the least-norm vector under the distribution-induced norm, and the recipe needs only the distribution's moments.
  • When no information about f is available, minimising weight variance is a sensible surrogate. No f-independent optimisation of the gradient variance is possible, and minimising the weight norm minimises a bound on gradient variance that holds uniformly over all f with bounded derivative.
  • The proposed approach does not require temperature tuning.
  • The resulting gradient estimate is lightweight making it faster compared to alternatives.
Critical weaknesses
  • The polynomial exactness criterion appears to be a useful trick that drives derivations. However, the paper does not asses whether low-level polynomials can approximate the underlying objectives well for the range of values a Poisson variable has.
  • Half of the experimental settings do not have count structure. Poisson VAEs replace the Bernoulli latents of a standard benchmark with Poisson latents on image data, and the count VAEs swaps latent families on MNIST. Only the spike-train model and the topic models have a genuine count interpretation.
  • Varying D does not lead to new estimators with significant practical gains. The weight variance increases with D as we add more constraints and limit the gradient approximator search space. If the new estimates do not improve results, the contribution reduces to a principled rederivation of a second-order estimator, which the paper itself shows coincides with existing estimators in several cases: classical pathwise for location-scale families, ReinMax at temperature 1 under a symmetry principle, and the one-hot MVE of Hooper and Shekhovtsov.
  • The headline comparisons omit the simplest competitive baselines. Gumbel-Softmax appears only in Table 1. Plain first-order straight-through does not appear anywhere. The score-function estimator appears only in Table 1, where it is the worst entry, and there is no indication that it uses any variance reduction. The natural reference would be a leave-one-out score-function estimator of the kind now standard in policy-gradient practice: with K samples it subtracts, for each sample, the mean of the others as a baseline. It is unbiased, needs no control-variate network, no temperature and no tuning, and costs only additional function evaluations.
Questions for authors
  • Could you add two baselines to Tables 2, 3 and 4: a leave-one-out score-function estimator, and plain first-order straight-through? For the former, a comparison matched on wall-clock rather than on sample count seems fairer in both directions.
  • Given that the order-2 solutions coincide with earlier gradient estimates for scale-location families and one-hot, where would you locate the novelty of the order-2 estimators specifically? In the derivation principle, in the new closed forms for count distributions, or elsewhere?
  • Could you report the bias of Fini against a reference gradient? For instance, you already use MVD as a low-variance reference in the deep-exponential-family diagnostic, and the linear VAE has an analytic gradient. Direct evidence that the bias is small where the objective is not polynomial would support the claim that Proposition 4.5 is informative in these settings, since the bound itself depends on the supremum norm of the (D+1)-th derivative of the decoder, which is never estimated.
3 comments
AuthorsOpenReview ↗

Response to R2 (1/3): Nonlinear accuracy and count-valued models

We thank the reviewer for the detailed assessment and constructive suggestions. We address the concerns and questions below. This is Part 1 of 3.

On approximation accuracy for nonlinear objectives

We have added a nonlinear gradient-bias study following Liu et al. [1, §6.3, Figure 4]. Like their study, we measure cos similarity between the averaged gradient estimate and a reference gradient. This checks whether the approximate gradient points in the correct direction throughout training. We adapt the experiment to our nonlinear Poisson VAE, using a high-precision MVD reference and additionally measuring gradient variance.

The results appear in §6.2, Figure 3. Fini-2 and Fini-3 maintain near-1 cos similarity with the second lowest (second to ST) variance and strong, stable optimization. ST has lower variance but weaker alignment and a worse training objective. Fini-4 retains strong alignment but exhibits substantial variance spikes, accompanied by noisier optimization without a loss advantage. These measurements directly examine the practical gradient-approximation tradeoff outside the polynomial exactness class, supporting Fini-2 and Fini-3 being in the order-wise sweet spot while complementing Proposition 4.5.

Furthermore, we bucketize Poisson rates observed during training and measure relative bias against the reference gradient. The buckets are <0.25<0.25, 0.250.25–0.50.5, 0.50.5–11, 11–22, and 22–44. Appendix G.4, Figure 6 shows that measured relative bias decreases with order across all five buckets: Fini-2's bias is roughly 3–7%, Fini-3's roughly 1–3%, and first-order ST's 30–58%.

On using count-valued latent variables

We respectfully disagree that evaluating count-latent models is a major weakness of a gradient-estimation paper whose scope includes count-variable gradients. Our contribution is improving gradient estimation, rather than establishing that count latents should replace other latent-variable families.

VAEs are established nonlinear benchmarks for evaluating gradient estimators, as in ReinMax [1] and implicit reparameterization gradients [2]. Our Count-VAE experiments extend this evaluation across latent distributions while comparing estimator performance within fixed models, similar to Figurnov et al. [2, Table 4].

The Poisson VAE model we build on was introduced by Vafaii et al. [3], which seminally connects Poisson spike-count representations with perceptual inference, rate coding, and predictive coding. Count-valued latents also have established applications in ecology [4], network tomography [5], and neural sequence modeling [6]. Our spike-train and hierarchical topic-model experiments provide further count-structured applications.

Our evaluation focuses on gradient quality, optimization performance, and computational efficiency. Predictive metrics additionally depend on model assumptions and the training objective, and therefore do not isolate gradient-estimator quality. Nevertheless, Table 3 already reports held-out perplexity alongside ELBO.

References

[1] Liu et al. (2023). Bridging Discrete and Backpropagation: Straight-Through and Beyond. NeurIPS.

[2] Figurnov, Mohamed, and Mnih (2018). Implicit Reparameterization Gradients. NeurIPS.

[3] Vafaii, Galor, and Yates (2024). Poisson Variational Autoencoder. NeurIPS.

[4] Dail and Madsen (2011). Models for Estimating Abundance from Repeated Counts of an Open Metapopulation. Biometrics, 67(2), 577–587.

[5] Vardi (1996). Network Tomography: Estimating Source-Destination Traffic Intensities from Link Data. JASA, 91(433), 365–377.

[6] Williams, Degleris, Wang, and Linderman (2020). Point Process Models for Sequence Detection in High-Dimensional Neural Spike Trains. NeurIPS.

AuthorsOpenReview ↗

Response to R2 (2/3): Order-2 novelty and higher-order gains

Response to Reviewer 2 (Part 2 of 3): Order-2 contribution and higher-order gains

On the order-2 contribution

Firstly, we would like to point out that, to our knowledge, no native second-order Poisson gradient estimator had been derived and evaluated before our study, nor for variables like Negative-Binomial and Beta-Binomial, even though recent work [3, 7] has been dedicated to Poisson gradient estimation. Our native second-order estimators are novel.

Secondly, applying ReinMax (another order-2 method) to Poisson does not yield our Fini-2 estimator, and we extensively study the difference in the original manuscript. Corollary 4.2 characterizes an excess component in categorical ReinMax for Poisson, resulting in additional weight variance. Proposition D.1 further shows for exponential families, adopting Reinmax will also lead to excess weight variance. Empirically, ReinMax shows worse gradient bias, higher variance, and worse optimization in most of our comparisons.

Our contribution also includes direct moment-based constructions and explicit native weights for Poisson, Binomial, Negative-Binomial, and Beta-Binomial distributions (§3; Appendix E), evaluated independently of support size and without categorical truncation. Proposition 4.3 and Corollary 4.4 characterize the weight-variance costs of compound representations and identify when Rao–Blackwellization removes them.

On higher-order gains and choosing the order

We respectfully disagree on the characterization that “Varying DD does not lead to new estimators with significant practical gains”. Increasing DD blindly can lead to variance blow up and performance degradation; however, increasing order from 2 to 3 can lead to practical improvements as demonstrated in our paper.

1. Increasing DD blindly does not lead to monotonic improvement:

The reviewer might have already understood that we cannot blindly increase DD. Our theoretical analysis already characterizes the tradeoff introduced by increasing the exactness order - excessive variance is a concrete failure mode. Higher orders guarantee exactness for a larger polynomial class, while also increasing the minimum achievable weight variance (Proposition 4.1). Proposition 4.5 bounds nonlinear bias under smoothness assumptions. In particular, Section 3 and Appendix E.1 identify a concrete limitation: the variance of fourth-order Poisson weights grows inversely with the rate and diverges as the rate approaches zero.

To investigate this tradeoff empirically, we adapt the gradient-alignment diagnostic of Liu et al. [1, Figure 4] to our nonlinear Poisson VAE, using a high-precision MVD reference and additionally measuring gradient variance throughout training (new Figure 3).

Our empirical results reveal a clear pattern: increasing the order generally improves gradient alignment, but the variance increase can eventually outweigh the improvement in approximation accuracy. ST exhibits the lowest variance but weaker alignment and worse optimization. Fini-2 and Fini-3 maintain near-1 cos similarity with the second lowest variance (second to ST) and achieve the best optimization performance. Fini-4 generally retains near-1 alignment, but exhibits substantially higher variance, including pronounced spikes, resulting in noisier training trajectories and worse final performance.

2. There is a sweet spot for order DD, which is order 2 and order 3. In our empirical studies, Fini 3 performs significantly better than Fini 2 in more cases than the other way around.

Going from D=2D=2 to 3 shows performance gains. We assess the performance differences between Fini 2 and 3 using two-sided Welch's tt-tests, as in Appendix G.1, with five runs per method. At the 5% significance level, Fini-3 significantly improves over Fini-2 on Bernoulli Omniglot (p<10−4p<10^{-4}), Binomial (d=16d=16) (p≈0.009p\approx0.009), Binomial (d=128d=128) (p≈0.044p\approx0.044), and Poisson (d=16d=16) (p≈0.015p\approx0.015). Conversely, Fini-2 significantly outperforms Fini-3 only on Binomial (d=2d=2) (p≈0.005p\approx0.005).

Our practical order-selection guidance and direct baseline comparisons follow in Part 3.

References

[1] Liu et al. (2023). Bridging Discrete and Backpropagation: Straight-Through and Beyond. NeurIPS.

[3] Vafaii, Galor, and Yates (2024). Poisson Variational Autoencoder. NeurIPS.

[7] Ibrahim et al. (2026). A Hitchhiker's Guide to Poisson Gradient Estimation. ICML.

AuthorsOpenReview ↗

Response to R2 (3/3): Order selection and additional baselines

How should practitioners choose the estimator order?

Thus, practitioners should select the order as a hyperparameter, analogous to temperature in ReinMax and EAT-cubic. Both standard implementations are faster than EAT-cubic and ReinMax in every tested workload–GPU configuration. Furthermore, GPU fusion brings Fini-3’s observed mean epoch time within 1.6% of standard Fini-2 across all six configurations (Table 5). When budget permits, practitioners can compare and validate both. If only one run is feasible, we recommend Fini-3 as a starting point for smooth nonlinear objectives, based on its stronger performance across most nonlinear comparisons and essentially comparable GPU runtime.

On more baselines

In related work, and the introduction part of experiments, we have pointed out that Reinmax [1] is empirically the most capable categorical estimator over previous categorical baselines, and EAT-cubic [7] is similarly the most capable specialist estimator for Poisson. Their respective papers [1, 7] compare exhaustively against competitors, where Reinmax beats score function gradients (that have more advanced control variates than RLOO), ST, and GSM across the reported comparisons, and similarly EAT-cubic beats EAT-sigmoid (which beats ST in [3]) and GSM. Thus we compared against the state-of-the-art in the original submission whenever possible.

Nevertheless, we appreciate the reviewer’s suggestion and have added ST, RLOO, and a Gumbel-Softmax baseline to the nonlinear Poisson-VAE and Poisson-DEF experiments underlying Tables 2 and 3 highlighted by red table captions, and added equal-time optimization comparisons in Appendix G.9, Figures 8–9. These direct comparisons preserve our conclusions about Fini’s optimization performance and computational efficiency.

Our profiling shows that Fini requires less update time and peak GPU memory than RLOO even at (K=2K=2). We additionally evaluate (K=4K=4), giving RLOO a larger measured compute allocation under the same training-update budget; this does not overturn the conclusions. The updated Tables 2–3 report the new results, and the appendix provides timing measurements, and configurations.

References

[1] Liu, L., Dong, C., Liu, X., Yu, B., and Gao, J. (2023). Bridging Discrete and Backpropagation: Straight-Through and Beyond. NeurIPS.

[3] Vafaii, H., Galor, D., and Yates, J. L. (2024). Poisson Variational Autoencoder. NeurIPS.

[7] Ibrahim, M., et al. (2026). A Hitchhiker’s Guide to Poisson Gradient Estimation. ICML, PMLR 306, 49387–49427.

Reviewer_GMpgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors propose Fini, a framework for estimating gradients of objectives involving discrete, particularly count-valued, latent variables. Rather than requiring unbiasedness for arbitrary downstream functions, Fini guarantees unbiasedness for polynomials up to a chosen degree. The resulting construction yields computationally inexpensive, closed-form estimators for count distributions such as Poisson, binomial, and negative binomial. The authors provide theoretical analyses of bias and weight variance and report competitive empirical results on count-valued latent-variable models.

Critical strengths

The paper introduces a new framework for propagating gradients through count-valued latent variables, with unbiasedness guaranteed for polynomial downstream functions up to a chosen degree. The resulting estimators are simple to compute and apply directly to count distributions without categorical approximations.

Critical weaknesses

A major weakness is that it remains unclear when the surrogate gradient estimator becomes unreliable due to large bias or variance. Although unbiasedness is guaranteed for polynomials up to a chosen degree, this does not ensure small bias for nonlinear downstream objectives.

It is also unclear in which downstream tasks count-valued latent models offer distinct advantages over alternative latent-variable models. The experiments show that the proposed estimators can train the selected models effectively, but provide limited evidence of when preserving count-valued latent structure is particularly important. The practical contribution would be more compelling with clearer application-specific motivation and evidence that improved gradient estimation translates into meaningful gains on downstream metrics, in addition to improvements in the training objective and computational efficiency.

Questions for authors

Given that increasing the exactness order can also increase variance, how should practitioners choose the estimator order? Could the authors provide practical guidance or diagnostics for identifying when the approximation is reliable and when a higher-order estimator is likely to be beneficial?

2 comments
AuthorsOpenReview ↗

Response to R1 (1/2): Reliability and count-valued models

We thank the reviewer for the thoughtful feedback. We address the concerns and questions below.

On the bias–variance tradeoff and reliability.

Our theoretical analysis already characterizes the tradeoff introduced by increasing the exactness order: higher orders guarantee exactness for a larger polynomial class, while also increasing the minimum achievable weight variance (Proposition 4.1). Proposition 4.5 bounds nonlinear bias under smoothness assumptions. In particular, Section 3 and Appendix E.1 identify a concrete limitation: the variance of fourth-order Poisson weights grows inversely with the rate and diverges as the rate approaches zero.

We further empirically identify excessive variance as a concrete failure mode of increasing the order. To investigate this tradeoff empirically, we adapt the gradient-alignment diagnostic of Liu et al. [1, Figure 4] to our nonlinear Poisson VAE, using a high-precision MVD reference and additionally measuring gradient variance throughout training (new Figure 3).

Our empirical results reveal a clear pattern: increasing the order generally improves gradient alignment, but the variance increase can eventually outweigh the improvement in approximation accuracy. ST exhibits the lowest variance but weaker alignment and worse optimization. Fini-2 and Fini-3 maintain near-1 cos similarity with the second lowest variance (second to ST) and achieve the best optimization performance. Fini-4 generally retains near-1 alignment, but exhibits substantially higher variance, including pronounced spikes, resulting in noisier training trajectories and worse final performance.

These observations are consistent with our theoretical characterization: higher polynomial exactness can improve gradient approximation, but its variance cost can eventually dominate. Although this tradeoff need not follow the same pattern for every downstream function, it appears consistently in our tested setting and helps explain why low orders perform well.

As for how to choose the order in practice, we refer the reviewer to the second comment, "How should practitioners choose the estimator order?"

On using count-valued latent variables.

We respectfully disagree that evaluating on count-latent models is a critical weakness for a gradient-estimation paper whose scope includes count-variable gradients. Our contribution is improving gradient estimation, rather than establishing that count latents should replace other latent-variable families.

VAEs are established nonlinear benchmarks for evaluating gradient estimators, as in ReinMax [1] and implicit reparameterization gradients [2]. Our Count-VAE experiments extend this evaluation across latent distributions while comparing estimator performance within fixed models, similar to Figurnov et al. [2, Table 4].

The Poisson VAE model we test on was introduced by Vafaii et al. [3], which seminally connects Poisson spike-count representations with perceptual inference, rate coding, and predictive coding. Count-valued latents also have established applications in ecology [4], network tomography [5], and neural sequence modeling [6]. Our spike-train and hierarchical topic-model experiments provide further count-structured applications.

Our evaluation focuses on gradient quality, optimization performance, and computational efficiency. Predictive metrics additionally depend on model assumptions and the training objective, and therefore do not isolate gradient-estimator quality. Nevertheless, Table 3 already reports held-out perplexity alongside ELBO.

References

[1] Liu et al. (2023). Bridging Discrete and Backpropagation: Straight-Through and Beyond. NeurIPS.

[2] Figurnov, Mohamed, and Mnih (2018). Implicit Reparameterization Gradients. NeurIPS.

[3] Vafaii, Galor, and Yates (2024). Poisson Variational Autoencoder. NeurIPS.

[4] Dail and Madsen (2011). Models for Estimating Abundance from Repeated Counts of an Open Metapopulation. Biometrics 67(2), 577–587.

[5] Vardi (1996). Network Tomography: Estimating Source-Destination Traffic Intensities from Link Data. JASA 91(433), 365–377.

[6] Williams, Degleris, Wang, and Linderman (2020). Point Process Models for Sequence Detection in High-Dimensional Neural Spike Trains. NeurIPS.

AuthorsOpenReview ↗

Response to R1 (2/2): Order selection and practical diagnostics

How should practitioners choose the estimator order?

We recommend treating the order as a hyperparameter and tuning it, analogous to temperature in ReinMax [1] and EAT-cubic [7]. Both standard implementations are faster than EAT-cubic and ReinMax in every tested workload–GPU configuration. Furthermore, GPU fusion brings Fini-3's observed mean epoch time within 1.6% of standard Fini-2 across all six configurations (Table 5). When budget permits, practitioners can compare both and validate.

If only one run is feasible, we recommend starting with Fini-3 for smooth nonlinear objectives. It achieves higher mean ELBO in 5/6 nonlinear Poisson-VAE settings and 7/9 Count-VAE settings (Tables 2 and 4). Using two-sided Welch's tt-tests with five runs per method, Fini-3 shows statistically significant improvements at the 5% significance level on Bernoulli Omniglot (p<10−4p<10^{-4}), Binomial (d=16d=16, p≈0.009p\approx0.009), Binomial (d=128d=128, p≈0.044p\approx0.044), and Poisson (d=16d=16, p≈0.015p\approx0.015). Fini-2 significantly outperforms Fini-3 only on Binomial (d=2d=2, p≈0.005p\approx0.005). These tests are consistent with the convention in Appendix G.1.

What practical diagnostics indicate unreliability or a benefit from higher order?

Our experiments suggest that large gradient variance can manifest as noisy or spiky training trajectories and reduced optimization performance, as illustrated by Fini-4 in Figure 3. We therefore recommend monitoring optimization stability and loss performance when selecting the order.

If Fini-3 produces persistently unstable updates or pronounced loss spikes, Fini-2 is a natural lower-variance-weight alternative to evaluate. Conversely, when optimization remains stable, Fini-3's additional approximation accuracy may be beneficial. Small Poisson rates also provide a theoretically identified warning against unnecessarily high orders.

These are empirically motivated guidelines rather than universal guarantees. The central contribution of our work remains the general construction of finite-order exact estimators, their minimum-weight-variance characterization, and their efficient application across discrete distributions.

We have incorporated the additional gradient diagnostics into Section 6.2 (Figure 3, p. 9), and the matched-time optimization comparisons into Appendix G.9 (Figures 8–9).

References

[1] Liu et al. (2023). Bridging Discrete and Backpropagation: Straight-Through and Beyond. NeurIPS.

[7] Ibrahim et al. (2026). A Hitchhiker's Guide to Poisson Gradient Estimation. ICML, PMLR 306, 49387–49427.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.