- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper focuses on flow matching using causal velocities and the error that arises from when denoising tokens in parallel in this setting. They derive explicit form of this error when the target is Gaussian and decompose the KL into a sum of errors that depend on the choice of batches during generation. They use these results to design better sampling schemes that minimize this error.
Critical strengths
The paper has interesting theoretical insights in the gaussian case on the effect of restricting the velocity to be causal. It shows (at least in the gaussian case) exactly why it does not in general generate the correct distribution and they use this to have a criterium to choose the generation order (causally).
Critical weaknesses
The setting is not very well explained (in which case we would want to have these kind of causal denoisers) and more importantly, the few cases mentioned in the introduction are not seriously considered in the experiments. The gaussian results are interesting but almost all the theoretical results are limited to this very toy-ish distribution, giving little insights as to what happens in the general case, and the empirical evidence does not complement this enough.
Questions for authors
Could you test your method in actual large-scale setting where training a standard (non-causal) denoiser is not possible, or not preferable for some reason ? How much of the theory transfers from the gaussian case to other more complex distribution like the ones encountered in real settings ?