- Rating
- 3
- Confidence
- 1
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors prevent a theoretically-supported method for choosing the number of parallel tokens to decode at once in diffusion language models. They prove bounds and show that their results hold empirically on benchmark performance.
Critical strengths
- The method is theoretically sound.
- The method has been validated empirically on actual useful benchmark tasks.
Critical weaknesses
I am not an expert in information theory or randomized algorithms, so I don't have substantial methodological feedback. I am experienced in language models, and I do think that this provides a principled solution to a real problem.
Questions for authors
- I think you should use a different word besides "batch" to describe multiple tokens being decoded at once. This could be confused with batching of unrelated sentences. Perhaps you could use "chunk" or "burst". This is not a critical suggestion, though, and just one sentence clarifying the distinction would be enough.
- From the Figure 1, it's not clear that we can do arbitrary non-left-to-right orders for the parallel decoded batches, although your method makes it seem that in fact, the tokens in the batch don't need to be contiguous. Please clarify the diagram accordingly.
Thank you again for your encouraging response and for taking the time to consider our rebuttal.
Just a small administrative note: in your latest response, you mentioned that you would raise your score, but the rating in your formal review is still displayed as 2. Could you please check whether the intended update has been saved? If you are already planning to update it, please excuse the reminder.
Regarding your note to the AC, confidence 2 does not require expertise in this specific topic; it allows for little or no direct research experience, provided the reviewer has a general familiarity with the field. If your original confidence of 2 still reflects your familiarity with the field, would you consider retaining it? We would be happy to clarify any remaining technical questions.
Your further support would mean a great deal to us. Thank you again for your time, and we hope you have a wonderful day.