- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes CONDOR, a method for single-step discrete text diffusion. The method proposed first samples a noise to seed the generation process, and second outputs a coherent block based on this noise seed. To encourage the model to output coherent text blocks instead of uniform random sampling as would naively happen in a block-diffusion method, the authors propose two stages of training. In the first stage, the diffusion language model trains solely on the minimum loss token. In the second stage, the minimum loss tokens are sampled repeatedly using a confidence score and fed back into the model to create coherent target data under the models distribution for consistency. The authors evaluate the method at a small scale on several tasks, most importantly text continuation and label-to-MNIST generation.
Critical strengths
The task being addressed is important, and single-step diffusion language models have a clear motivation and the area is interesting to prior work.
The proposed method is moderately novel, although confidence-based diffusion language model sampling for few step generation has been done before.
Critical weaknesses
First, I will start with soundness concerns. There is no connection between the noise seed and the generation outputs. Although the noise seed is added to the masked inputs, the mask tokens provide no mutual information to the model about the output, whereas noise in traditional diffusion masks visible data. Because the visible tokens receive no noise there is no justification for this point. Because the network is small scale, large noise that are out of distribution will of course harm the generation quality as seen in Table 3. This doesn't indicate a purpose for the noise seed itself.
Second, the minimum loss measure for generation has a strong soundness concern. Unlike single step generation methods based on confidence, which is a deterministic function of the data, the minimum loss is stochastic. A model does not know which token will have the minimum loss. Therefore, the minimum token being sampled is stochastic, invalidating a key setting for most self-distillation work, even though the decoding is greedy. Furthermore, backpropagating only on the minimum loss token violates standard langugage modeling. The minimizer for this loss is not necessarily a distribution. Take the distribution on two characters either a,a or b,b. Then the optimal loss objective predicts a for one token with all probability and b for a different token.
Third, the results are difficult to interperet and generally low quality. I focus here on the MNIST experiment, where the outputs are excessively blurry. MNIST is a toy dataset and outputs should be nearly perfect, highlighting the soundness concerns.
Questions for authors
What is the purpose of the noise seeding?
Why is minimum token loss an appropriate measure here, when the model itself cannot know what token will have the minimum loss?
Why aren't the MNIST outputs sharp?
Thank you for the review. Several soundness concerns appear to stem from reading the WTA winner in Eq. (4) as a single token. Prior work already explains how WTA trains a complete candidate and demonstrates how independent noise draws can yield different outputs for the same context.
1. How does the seed affect the output? Winner-take-all (WTA) trains the closest of k predictions toward the target. Guzmán-Rivera et al. (2012) formalized this objective in Multiple Choice Learning: Learning to Produce Multiple Structured Outputs. In Stochastic Multiple Choice Learning for Training Diverse Deep Ensembles, Lee et al. (2016) backpropagated each example’s loss through its winning predictor. For each of k noise draws, Eq. (4) sums cross-entropies over all masked positions. Every masked position in the lowest-loss draw contributes to the update.
In Self-organized formation of topologically correct feature maps, Kohonen (1982) updated the winning unit and its neighbors. Theorem 1 of Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses (Rupprecht et al., 2017) shows that under squared error, optimal predictors represent the means of their assigned targets. In Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks, Gupta et al. (2018, Eq. 5) trained on the closest of k future trajectories generated from independent noise for the same motion history.
Suppose the only valid blocks are
aaandbb. On a training example labeledaa, four noise draws might decode toab,bb,ba, andbb. Suppose the draw that producedabhas the lowest total cross-entropy againstaathis time. It wins under Eq. (4), and both positions are trained ona, pushing the second position towardaas well.Even a winning draw that already decodes to
aacan improve. If it assignsaprobabilities 0.8 and 0.9, its loss is still positive. Cross-entropy pushes both probabilities toward 1 for that draw. Onbbexamples, other draws can specialize inbb. Repeating these updates can make mixed blocks such asabandbarare while preserving both valid outputs. In our earlier four-token experiment,aaaa,bbbb,cccc, andddddeach accounted for about a quarter of the outputs. We observed no other sequence.On TinyStories, we held each prefix fixed and greedily decoded one block for each of 16 noise draws. CONDOR produced an average of 11.31 distinct valid continuations per prefix, compared with 0.25 for plain cross-entropy (Table 2). Even without distillation, WTA alone (k=4) raised this count to 1.19.
2. “The minimum loss measure for generation.” We cannot tell which step this phrase refers to. Could you clarify whether your soundness concern applies to the WTA training loss, the construction of self-distillation targets, or inference with the trained model? WTA selects the noise draw with the lowest total loss over the masked positions (Eq. 4). To construct a distillation target, the student takes the vocabulary argmax at each position and the teacher greedily refills selected positions (§3.2). At inference, we draw noise once per block and take the vocabulary argmax at each position in a single forward pass (§3.3).
MNIST sharpness. Some strokes in Figure 3 are blurry. We included MNIST as a simple example of generating an image in one forward pass given a digit word. Each image token represents a 7×7 patch and predicts all 49 pixel values together (§3.3). We did not tune the hyperparameters or refine the image decoder specifically for sharpness.