acceptodds
Under review as a conference paper at ICLR 2027

Jumping up and down: Denoiser diffusion models for discrete ordinal data

Abstract

Diffusion models are highly developed in continuous spaces for image and video domains. Recently, major advances have been made for discrete diffusion models for categorical data, specifically in the language domain. In contrast, diffusion models for discrete integer-valued data are less developed, despite the prevalence of this modality, ranging from images and music to gene counts. We introduce Jumping Up and Down (JUD)—a new family of denoiser-based diffusion models for discrete ordinal data. This is the first family of diffusion models for ordinal data which centers around training denoisers, which at the same time allows for bi-directional (up and down) perturbations of the data. The simplicity of the training objective, combined with the flexibility of bi-directional perturbations, leads us to obtain competitive results across different data modalities.

Primary area:
generative models
Keywords:
discrete diffusion models, denoisers, ordinal data

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_rRnSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Jumping Up and Down (JUD), a diffusion model for discrete ordinal data. It combines Binomial-Poisson or Poisson-Poisson corruption with two latent denoisers that estimate the removed and added counts. Experiments include eight synthetic distributions, unconditional CIFAR-10 generation, and hard Sudoku completion. The Binomial-Poisson model achieves a CIFAR-10 FID of 2.07 on 50,000 generated images and a mean Sudoku accuracy of 66.99% with posterior cross-entropy training and digit shuffling.

Critical strengths
  1. Theorem 2.1 gives explicit reverse-time rates in terms of the expected added and removed counts. This provides a concrete link between simple denoising losses and bidirectional sampling.
  2. The authors conduct extensive experiments and the results are promising. Table 2 reports a CIFAR-10 FID of 2.07, the lowest among the discrete models. These results provide useful evidence that the construction can generate competitive images.
Critical weaknesses
  1. The ablation studies are not clear enough. The CIFAR-10 comparison does not isolate the benefit of bidirectional corruption. Appendix G.3 varies the sampler but does not report a Binomial-only model using the same backbone, preconditioning, time-sampling policy, and training budget.
  2. The strongest Sudoku result uses shuffled digits, but Appendix H.1 also changes the updates from 10,000 to 20,000, the learning-rate schedule from cosine decay to constant, and weight decay from 0.05 to zero. The comparison in Table 3 therefore does not isolate digit shuffling, and does not establish a matched training comparison with the published diffusion baselines.
Questions for authors
  1. Is a Binomial-only CIFAR-10 result already available under the same backbone, preconditioning, training-time proposal, and training and sampling budgets as JUD? Please identify any existing matched control and clarify which of these settings are shared by the Binomial Flow result in Table 2.
  2. Does the Sudoku comparison include the update-count, learning-rate, and weight-decay changes described in Appendix H.1?
  3. What noise parameters and initialization were used for the Poisson-Poisson column of Table 1?
3 comments
AuthorsOpenReview ↗

Addressing reviewer's points

All Points

Please see the Addressing major points raised by multiple reviewers thread above where we address major points raised by multiple reviewers. In case anything is unclear or further discussion is warranted please let us know.

Most of my concerns have been addressed in the author response.

We thank the reviewer for the acknoledgment.

Questions For Authors:

  1. Is a Binomial-only CIFAR-10 result already available under the same backbone, preconditioning, training-time proposal, and training and sampling budgets as JUD? Please identify any existing matched control and clarify which of these settings are shared by the Binomial Flow result in Table 2.

This requested controlled experiment has been carried out (see "Up-Down vs Down-Only Process on CIFAR-10" section above) and added to the revised manuscript. We have also commented in detail about the remaining differences between this control and the original Binomial Flow implementation, which explains the FID differences, and clarified the extent to which the Binomial Flow already followed the same approach of JUD with a zero poisson rate.

  1. Does the Sudoku comparison include the update-count, learning-rate, and weight-decay changes described in Appendix H.1?

All the Sudoku results in the revised manuscript and in the section "Controlled Sudoku Shuffling" above now use identical training and inference configurations. We have additionally added the permutation-shift results to all baselines, which allows to isolate the impact of digit shuffling across all methods.

  1. What noise parameters and initialization were used for the Poisson-Poisson column of Table 1?

We have clarified what the experiment in the original and revised manuscripts use in the "Poisson-Poisson Performance and Settings, Other Changes and Results" section above.

We would like to enquire with the reviewer what specific concerns remains unresolved (as most concerns have been addressed in the author response).

Reviewer_xQQaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces JUD, a diffusion framework tailored for discrete ordinal data. The idea is to corrupt integer-valued data in both directions, which means removing counts and adding counts and then train simple conditional-mean "latent denoisers" that predict these perturbations. A key theoretical finding shows that these denoisers determine the reverse-time CTMC rates needed for sampling. The authors demonstrate he framework usings Binomial-Poisson and Poisson-Poisson, which are evaluated on synthetic distributions, CIFAR-10, and Sudoku.

Critical strengths
  • The mathematical derivation between latent conditional-mean denoisers and bidirectional reverse jump rates is elegant and offering a useful counterpart to denoiser-based continuous diffusion for ordinal discrete data.
  • Binomial-Poisson reaches FID 2.07, surpassing other discrete diffusion baselines evaluated in the paper and approaching continuous EDM performance.
  • The methodology is straightforward and easy to reproduce. The relationship to one-directional Binomial diffusion is particularly useful: the appendix shows that Binomial Flow is recovered as a special case.
Critical weaknesses
  • The paper's primary empirical claim is that allowing both upward and downward perturbations improves discrete ordinal diffusion. However, the CIFAR-10 implementation combines JUD with EDM-style preconditioning, a specialized time-sampling strategy, and specific sampler choices. As a result, the comparison to the Binomial Flow baseline is not strictly controlled. This matters, while the FID of 2.07 is impressive, it is difficult to isolate how much of the improvement effectively comes from adding the upward Poisson component vs the surrounding training and sampling design. Hence, a matched Binomial-only ablation study is needed by maintaining the exact same architecture, preconditioning, training budget, and sampling methods.

  • Although the introduction highlights JUD's potential for integer-valued scientific data such as gene-expression counts, MIDI, the real-data experiments are CIFAR-10 and Sudoku, and the paper explicitly notes that Sudoku is not ordinal. This leaves the practical advantage of JUD on genuinely count-valued data relatively untested. Including an experiment on biological counts would significantly strengthen the paper, especially since other recent methods in this space already evaluate on such datasets. Moreover, the strongest Sudoku result is not a clean shuffling ablation: shuffled and unshuffled models use different numbers of updates, learning-rate schedules, and weight decay.

  • The paper omits a highly relevant prior work,CountsDiff [1], which is a closely related diffusion model on the natural numbers that introduces churn/remasking to obtain non-monotone reverse trajectories, allowing earlier sampling mistakes to be corrected. This directly overlaps with JUD's motivation that monotone count trajectories cannot recover from overshooting or undershooting. Still, JUD derives bidirectional reverse dynamics from conditional-mean latent denoisers, whereas CountsDiff uses a different corruption and reverse-dynamics construction. However, the omission makes the novelty appear broader than it is and leaves an important empirical comparison unanswered.

[1] Soatto et al., “CountsDiff: A Diffusion Model on the Natural Numbers for Generation and Imputation of Count-Based Data.”, ICML 2026.

Questions for authors
  1. Can the authors conduct an ablation study on CIFAR-10 that remove the upward Poisson while keeping the architecture, EDM preconditioning, training budget, time sampling, and sampler identical and see how the FID is impacted?

  2. Could the authors evaluate JUD on at least one genuinely count/ordinal real-world domain, such as gene-expression counts or MIDI?

  3. In what ways does JUD differ from CountsDiff, especially its non-monotone reverse sampling mechanism?

  4. Does the improvement from digit shuffling remain when shuffled and unshuffled models use identical optimization settings?

1 comment
AuthorsOpenReview ↗

Addressing reviewer's points

CountsDiff Comparison

The paper omits a highly relevant prior work,CountsDiff [1], which is a closely related diffusion model on the natural numbers that introduces churn/remasking to obtain non-monotone reverse trajectories, allowing earlier sampling mistakes to be corrected. This directly overlaps with JUD's motivation that monotone count trajectories cannot recover from overshooting or undershooting. Still, JUD derives bidirectional reverse dynamics from conditional-mean latent denoisers, whereas CountsDiff uses a different corruption and reverse-dynamics construction. However, the omission makes the novelty appear broader than it is and leaves an important empirical comparison unanswered.

In what ways does JUD differ from CountsDiff, especially its non-monotone reverse sampling mechanism?

[1] Soatto et al., “CountsDiff: A Diffusion Model on the Natural Numbers for Generation and Imputation of Count-Based Data.”, ICML 2026.

We thank the referees for bringing CountsDiff to our attention. We read this work carefully and here we explain how it relates to ours. CountsDiff uses binomial noise, as in Blackout and Binomial Flows. CountsDiff differs from Blackout in two ways. The first difference is that CountsDiff uses non-monotonic noise schedule. This is an implementation aspect of the proposed method and does not address the problem of uni-directional jumps. The second difference is that CountsDiff injects further noise into sampling and allows up and down jumps. This modification indeed aims at addressing the issue of uni-directional jumps, but fundamentally differs from what we do here in JUD. In JUD, the noising process itself allows for up and down jumps which automatically ensures that the sampling process allows for up and down jumps with no need to inject further noise.

We were motivated to develop JUD since in practice, injecting extra noise during sampling using binomial noise yields in fact worse results. Indeed, Blackout, which does not use extra noise in sampling, obtains FID of 4.58 on CIFAR-10, while CountsDiff, which does use extra noise in sampling, obtains a worse FID of 5.20, all the while following the recipe of Blackout. In addition, in the table under Additional CIFAR-10 Experiments (Table 9 in revised paper) we inject further noise (this method is the `` + marginal-preserving rates" options) and find that it yields worse results. To conclude, the injection of extra noise in sampling to a pure death process such as binomial noise does not seem to help, and in fact yields worse results. This necessitates a more principled approach, where the noising process itself allows for up and down jumps (while still learning only denoisers), which is what we achieve in JUD.

We thank the referees for the opportunity to clarify this point. For completeness, we included the FID on CIFAR-10 of CountsDiff in Table 2 and also reference the work and above observations in the Related Works section in the appendix.

Remaining points

For the remaining points, please see the Addressing major points raised by multiple reviewers thread above where we address major points raised by multiple reviewers. In case anything is unclear or further discussion is warranted please let us know.

Reviewer_JduYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a noising process, define a CTMC process with corresponding marginals, and show how to train a denoiser to reverse the dynamics. The core idea of the noising process is to perturb the ordinal data in two directions: up, and down. The main method they focus on uses the Poisson process to model the upward perturbation, and Binomial to model the downward perturbation. They then discuss how to train a diffusion model to reverse these dynamics -- they first show how to train a model by predicted both the expected added / removed noise for each position in the target sequence; and then they show that cross entropy can be used to predict a distribution over the clean value, which can then be used to simulate the correct reverse dynamics. They demonstrate that learning the expected added and expected removed noise is enough to properly simulate the reverse, and that training with cross entropy can recover these expectatios. They show results on synthetic distributions, CIFAR-10, and Sudoku.

Critical strengths
  1. The theory seems very simple (which I consider I big strength) and clean. The core idea -- that we can define two forms of corruption (one increases value, one depletes value), and reverse the joint process by modeling each of these noises separately. I also liked that the method is amenable to cross entropy loss, which falls in line with many of the recent works on diffusion language modeling.

  2. The performance on Cifar 10 is quite strong -- it seems to outperform the reproduction of EDM values.

Critical weaknesses
  1. None of the experiments are on tasks where ordinal representations are important. The motivation gives clear examples -- gene expression, MIDI music, etc -- where ordinal data could potentially be important for learning a good generative model. Yet these are not use as actual experiments.

  2. The images could be polished a bit more. Figure 1 in particular could be improved -- is it really necessary to visualize all the different corruption forms? I would suggest focusing on visualizations of the core method and the variations (Binomial-Poisson, Poisson-Poisson, up-only Poisson, and down-only Binomial). While discussion of previous methods is important, I would put visualizations of them in the appendix to make room for the related works section in the main body.

Questions for authors
  1. Why does Poisson-Poisson do poorly, especially on the Poisson mixture?
  2. The step counts for EDM v.s JUD are not equal -- it would be interesting to see how they perform with matched inference time compute.
  3. What other ordinal data can be used? From the results, sudoku seems better suited towards categorical representations. I think it would be really interesting to focus on experiments where representing the inputs as ordinal provides a clear advantage over categorical representations.
1 comment
AuthorsOpenReview ↗

Addressing reviewer's points

Matched Inference Compute

The step counts for EDM v.s JUD are not equal -- it would be interesting to see how they perform with matched inference time compute.

See the Addressing major points raised by multiple reviewers thread above for the extended ablations for JUD-BP and JUD-PP and comments on this point therein.

Figure 1 and Related-Work Placement

The images could be polished a bit more. Figure 1 in particular could be improved -- is it really necessary to visualize all the different corruption forms? I would suggest focusing on visualizations of the core method and the variations (Binomial-Poisson, Poisson-Poisson, up-only Poisson, and down-only Binomial). While discussion of previous methods is important, I would put visualizations of them in the appendix to make room for the related works section in the main body.

We thank the reviewer for these suggestions which we adopt in the revised manuscripts. We have updated Figure 1 and have included a short Related Works section in the main body of the paper containing the most relevant papers. The extended related works section of the appendix remains as well.

Remaining points

For the remaining points, please see the Addressing major points raised by multiple reviewers thread above where we address major points raised by multiple reviewers.

In addition we note that the CIFAR-10 image data is genuinely ordinal since the values of each pixel (0 to 255) are greyscale values. In case anything is unclear or further discussion is warranted please let us know.

Comments on the paper

5 comments
AuthorsOpenReview ↗

Addressing major points raised by multiple reviewers [part 1]

First of all, we would like to thank the reviewers for the time spent reviewing the manuscript and for their clear, actionable points. We have updated the manuscript addressing all points raised by the reviewers, as well as the anonymous repository. Below we first address the major points raised by multiple reviewers. Reviewer-specific points are addressed in separate threads.

Poisson-Poisson Performance and Settings

rRnS:

What noise parameters and initialization were used for the Poisson-Poisson column of Table 1?

JduY:

Why does Poisson-Poisson do poorly, especially on the Poisson mixture?

CUDA Poisson Sampler Bug

We were also surprised by the large performance gap between the Poisson–Poisson and Binomial–Poisson formulations. After much investigation, we found that the main cause of the poor Poisson–Poisson performance is a bug in PyTorch's CUDA Poisson sampler for Poisson's intensity 16≲λ<6416 \lesssim \lambda < 64 (pytorch#136750).

On CUDA, torch.poisson calls cuRAND's curand_poisson, which for λ<64\lambda<64 uses Knuth's multiplication method ([B]: Sec. X.3.3, Lemma 3.3). This method uses a variable number of uniforms, but PyTorch assumes a fixed number of 20 uniforms per call. The next call uses the uniform samples after these 20, so when a draw uses more, the two calls reuse the same uniforms and their draws are positively correlated (rates λ≥64\lambda \ge 64 use other algorithms and are unaffected, as is the CPU sampler).

The issue is especially pronounced for the difference of two independent Poisson draws, which in JUD-PP (Poisson–Poisson) appears both in training (forward noising) and in sampling (τ-leaping). For independent U,D∼Poisson(λ)U,D\sim\mathrm{Poisson}(\lambda) independent, Var(U−D)=2λ\mathrm{Var}(U-D)=2\lambda. The following script illustrates the issue with the latest PyTorch (2.14.1):

import torch
r = torch.full((10**6,), 48.0, device="cuda", dtype=torch.float64)
def draw(fix):
    g = torch.cuda.default_generators[0]
    x = torch.poisson(r, generator=g)
    if fix:
        g.set_offset(g.get_offset() + 2**32)
    return x
for fix in (False, True):
    torch.manual_seed(0)
    u, d = draw(fix), draw(fix)
    print(f"fix={fix}: Var(U-D) = {(u - d).var():.1f}, expected 96")
# fix=False: Var(U-D) = 40.1, expected 96
# fix=True: Var(U-D) = 95.8, expected 96

The workaround advances the generator by 2322^{32} positions after every CUDA Poisson call.

[B] Devroye, Non-Uniform Random Variate Generation (Springer, 1986), luc.devroye.org/chapter_ten.pdf.

Other Changes and Results

In Table 1 of the submitted manuscript, JUD-PP used linear rates with terminal values matching the data mean and range, initialized sampling from the difference of the two terminal Poisson noises (a Skellam distribution), and used Euler sampling.

Aside from the Pytorch workaround above, we make the following changes:

  • initialize reverse sampling from a Skellam distribution matching the mean and variance of the terminal state X+UT−DTX+U_T-D_T, which accounts for the data distribution;
  • use τ-leaping instead of Euler discretization;
  • fix an issue in the EDM preconditioning (bringing it to parity with the JUD-BF implementation).

Terminal matching and τ-leaping address two sources of JUD-PP error: an inaccurate initial reference and saturation at high jump rates, as the Euler discretization allows at most one jump per step.

The revised results are as follows (the Pytorch CUDA issue described above results in the same underlying uniform variates being reused across sampling steps in general, so we repeat JUD-BP as well):

DatasetBP, Table 1BPPP, Table 1PP
Poisson0.13 ± 0.020.128 ± 0.0420.35 ± 0.020.102 ± 0.023
Poisson Mixture1.39 ± 0.301.665 ± 1.16389.71 ± 0.341.641 ± 0.759
Zero-inflated Poisson0.12 ± 0.010.113 ± 0.0450.32 ± 0.020.163 ± 0.077
Negative Binomial Mixture1.62 ± 0.562.056 ± 0.83717.73 ± 0.412.534 ± 0.281
Beta Negative Binomial0.65 ± 0.060.710 ± 0.2726.14 ± 0.140.796 ± 0.196
Zipf0.39 ± 0.030.324 ± 0.1261.79 ± 0.100.540 ± 0.082
Yule–Simon0.12 ± 0.010.109 ± 0.0300.23 ± 0.020.189 ± 0.068
Correlated Beta Binomial, dimension 10.62 ± 0.100.543 ± 0.05111.39 ± 0.190.517 ± 0.114

In the revised manuscript we have updated Table 1 accordingly and re-ran all methods with the Poisson CUDE fix. JUP methods is competative and perform best in 5/8 distributions.

Additionally, we expanded the Sudoku (see below) and CIFAR10 experiments to include JUD-PP as well.

AuthorsOpenReview ↗

Addressing major points raised by multiple reviewers [part 2]

Additional CIFAR-10 Experiments

For JUD-PP, we use the same exact training setting as JUD-BF. We additionally extend the sampling NFE counts to 128, 64, and 36 (same as EDM baseline with Heun predictor-corrector ODE sampler): The complete CIFAR10 FID results are:

Model and sampler36 NFE64 NFE128 NFE256 NFE512 NFE1024 NFE2048 NFE
JUD-BP Euler471.85470.36327.407.072.222.152.09
JUD-BP τ-leaping52.0622.077.463.272.292.112.07
JUD-BP τ-leaping + marginal-preserving rates69.0034.2513.425.462.952.342.20
JUD-PP$^*$ τ-leaping129.7754.6416.726.193.663.213.09
JUD-PP$^*$ τ-leaping + marginal-preserving rates202.9590.1726.327.943.903.073.09

*: PP starts with very large noise. A uniform-time grid would spend most sampling steps at high noise levels, leaving too few for the rest. We therefore use a geometric grid for JUD-PP sampling, where each step reduces the remaining noise variance by a fixed percentage. Euler sampling (not reported) performed very poorly for JUD-PP on CIFAR-10 because it allows at most one unit jump per coordinate per step. At the high jump rates encountered here, the transition probabilities saturate, preventing the sampler from accurately approximating the reverse dynamics.

The results above are now included in Table 9 in the revised manuscript. The main conclusion is that the JUD-BP performs better than JUD-PP. We conjecture that this due to A) The initial distribution in JUD-PP is only approximately Skellam B) Related, in JUD-PP one is forced to choose high Poisson rates to forget the initial distribution, which in turn requires more sampling steps. In addition, at low NFE the performance of all methods significantly degrades. We note that we did not optimize the sampling grid (we use either linear or geometric) for different NFE levels, and investigating the implementation of JUD at low NFE remains an interesting open question which we leave for future research.

Up-Down vs Down-Only Process on CIFAR-10

rRnS:

The ablation studies are not clear enough. The CIFAR-10 comparison does not isolate the benefit of bidirectional corruption. Appendix G.3 varies the sampler but does not report a Binomial-only model using the same backbone, preconditioning, time-sampling policy, and training budget.

Is a Binomial-only CIFAR-10 result already available under the same backbone, preconditioning, training-time proposal, and training and sampling budgets as JUD? Please identify any existing matched control and clarify which of these settings are shared by the Binomial Flow result in Table 2.

xQQa:

The paper's primary empirical claim is that allowing both upward and downward perturbations improves discrete ordinal diffusion. However, the CIFAR-10 implementation combines JUD with EDM-style preconditioning, a specialized time-sampling strategy, and specific sampler choices. As a result, the comparison to the Binomial Flow baseline is not strictly controlled. This matters, while the FID of 2.07 is impressive, it is difficult to isolate how much of the improvement effectively comes from adding the upward Poisson component vs the surrounding training and sampling design. Hence, a matched Binomial-only ablation study is needed by maintaining the exact same architecture, preconditioning, training budget, and sampling methods.

Can the authors conduct an ablation study on CIFAR-10 that remove the upward Poisson while keeping the architecture, EDM preconditioning, training budget, time sampling, and sampler identical and see how the FID is impacted?

AuthorsOpenReview ↗

Addressing major points raised by multiple reviewers [part 3]

The submitted manuscript does not make this relationship sufficiently explicit: setting the Poisson rate of Binomial–Poisson JUD to zero recovers the method of Binomial Flows [A] which uses binomial corruption process. We call this implementation JUD-BF, and refer to the Binomial Flow model in [A] as BF. Both JUD-BF and BF already share DDPM++ and closely matched training settings ([A]: Sec. 3.2), EDM-derived preconditioning (Sec. 3.2.1, App. J.1, Table 4), training-time sampling motivated by improvement over a baseline (Sec. 3.2.3), and τ-leaping (Sec. 3.2.4). The main implementation differences are:

  • Preconditioning: JUD includes the optimal baseline intercept, whereas BF employs the skip connection c_skip(t)xₜ alone. JUD also uses time-dependent input centering, while BF applies a fixed shift.
  • Training-time sampling: JUD uses a piecewise-constant training-time density, while BF uses a parametric form;
  • Sampling: BF uses a uniform grid in the noise level σ = −log(t + ε) (for small ε), in JUD the grid is uniform in time t;
  • Loss scaling: both use weights 1/(qₜ + ε_loss), but BF ε_loss = 0.1, versus 10⁻⁸ in JUD. Here, qₜ is the expected squared error per coordinate of the corresponding baseline, including any squared mean error. JUD’s affine intercept removes this mean error, whereas BF’s skip-only baseline generally retains it; both use the same baseline-MSE normalization criterion.

The most significant differences are the last two.

[A] motivates the uniform grid on noise σ by the need for small discretization intervals at early times. Intuitively, in an increasing-only process, once a pixel moves away from 0 (black), it gives a strong hint about its final value. This is not the case for JUD-PP and JUD-BP due to the bidirectional ature of the reference and sampling process. The more conservative safety factor in 1/(q_t + ϵ) is required because qₜ vanishes near the clean endpoint in BF (and JUD-BF), and with ε_loss = 10⁻⁸ the loss rescaling over time is very heavy.

For JUD-BF, only the existing removed-count denoiser is required, since the added Poisson counts are identically zero.

For completeness, we repeat the CIFAR10 experiments with JUD-BF, training one denoiser only and setting ε_loss = 0.1 (a first training run failed with a loss spike with ε_loss = 10⁻⁸), keeping everything else the same as the other JUD experiments on CIFAR10. We then tested sampling at 2048 steps both with t-uniform and σ-uniform grids, obtaining FIDs of 21.49 and 4.49, respectively. We note that BF attains FID of 2.94 as opposed to JUD-BF (FID 4.49), which shows that the Binomial-only pertrubation method require more careful tuning. In contrast, JUD-BF required little tuning while at the same time achieving the best FID (2.07) among discrete diffusion models. We made sure to explain more clearly the relation between our methods and BF in section 3.2 in the revised manuscript.

[A] Shenfeld et al, Binomial flows: Denoising and flow matching for discrete ordinal data, ArXiv Preprint.

Additional Experiments on Truly Ordinal Data

xQQa:

Although the introduction highlights JUD's potential for integer-valued scientific data such as gene-expression counts, MIDI, the real-data experiments are CIFAR-10 and Sudoku, and the paper explicitly notes that Sudoku is not ordinal. This leaves the practical advantage of JUD on genuinely count-valued data relatively untested. Including an experiment on biological counts would significantly strengthen the paper, especially since other recent methods in this space already evaluate on such datasets.

Could the authors evaluate JUD on at least one genuinely count/ordinal real-world domain, such as gene-expression counts or MIDI?

JduY:

None of the experiments are on tasks where ordinal representations are important. The motivation gives clear examples -- gene expression, MIDI music, etc -- where ordinal data could potentially be important for learning a good generative model. Yet these are not use as actual experiments.

What other ordinal data can be used? From the results, sudoku seems better suited towards categorical representations. I think it would be really interesting to focus on experiments where representing the inputs as ordinal provides a clear advantage over categorical representations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.