- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper modifies generative diffusion or flow by adding during the generative process some heavy-tailed distributed jumps. The aim is to generate heavy-tailed data in a way that gives a better sampling of the tails. The paper proposes a procedure for adding these random jumps at random times in uniformly random directions; this procedure depends on the power nu defining the heavy tail and two parameters lambda, controlling the rate of jumps, and beta, controlling how this rate evolves with the generative time. The theoretical part proves that this procedure generates the heavy tails, with the exponent which is the one injected in the jumps. It also proves that the jumps do not distort too much the generated distribution. The experimental part first tests the procedure on some toy distribution in one dimension, and then on images. For each image dataset , picking up some appropriate values of the parameters nu, beta and lambda it provides some some evidence of a rather small improvement in IS and FID and ECS, with respect to the standard procedure.
Critical strengths
The procedure of jumps is well defined , it induces heavy tails, and it generates a distribution which does not differ too much from the distribution obtained without jumps
Critical weaknesses
The jumps are not correlated with the data structure. If we focus on images, the paper never measures on the data what is the quantity that ihas a power law distribution and that it aims at reproducing. The procedure uses isotropic jumps which cannot reproduce specific properties of data, unless data would have isotropic tails. The directions of jumps their frequencies and their amplitudes should be learned from data. Here the direction is uniformly random, and the parameters nu, lambda, beta are chosen a priori, or optimized over many trials.
The theoretical part is correct but it basically proves expected results: that a power law distributed jumps create power-law tails, and that the distances between the distributions generated with or without jumps is not too large. It does not address the crucial question: What is the property of data that one wants to reproduce? Does the new method with jumps provide an improvement? The one-dimensional toy example is too simple: with only one variable the isotropy of jumps is no longer a problem.
The experimental part on images finds improvements in FID, IS, ECS which are really small. There is no evidence of improvement on a crucial tail property.
Questions for authors
Could you give an example of a property that you expect has a heavy tail distribution, measure it empirically on the image dataset that you train your model on, and measure the same property for new images generated by the standard procedure on the one hand, and for new images generated by your jump procedure on the other hand.
How do you chose the parameters nu, lambda, beta for each dataset