- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
TEMPO reduces the cost of diffusion language model inference by varying numerical precision across decoding steps and token positions. Predictions from the previous step determine whether the next step runs entirely at low precision. Otherwise, full precision is retained for the prompt and estimated output region. The paper evaluates task accuracy, inference speed, and agreement with the original model. Agreement measures how often a method reproduces the same tool call on examples where the model at full precision produces a valid call.
Critical strengths
- The paper shows that positions beyond the emitted tool call can still affect decoding. On LLaDA evaluated on BFCL, removing these positions reduces accuracy on BFCL from 89% to 73%, while updating them at low precision retrains 86% accuracy. This is a useful finding that supports retaining these positions during inference.
- The method uses predictions already available from the previous step and requires no additional forward pass. The reported speedup of to when added to parallel decoding and boundary fill demonstrates that it can provide further acceleration after these optimizations.
Critical weaknesses
- The reported accuracy and speed do not come from the same experimental settings. Table 1 evaluates accuracy and output agreement at a generation length of 128. Table 3 reports latency at 1024 without reporting either metric for those configurations. In addition, Table 1 uses "computation saved" to describe both positions evaluated at low precision by TEMPO and positions removed by Learn2PD. These operations have different costs, so the percentages cannot establish which method is more efficient. The main results leave unclear how much accuracy and agreement are retained at the reported speeds.
- The main ablations do not sufficiently isolate the benefit of adapting precision to the current predictions. Figure 2 shows that content is usually committed in later steps. A fixed schedule that uses low precision early and full precision later already follows this observation. The comparisons with random allocation and reversed gate decisions in Table 2 show that the choice of steps matters, but do not establish how much the proposed gate improves over this simple schedule. That improvement is important for assessing the contribution of the method.
Questions for authors
- For the configurations in Table 3, what are the task accuracy and agreement with the model at full precision? How does TEMPO compare with uniform quantization when both use the same generation length and decoding optimizations? Reporting these results together would show whether TEMPO provides additional speed while preserving comparable outputs.
- Does adaptive gating improve accuracy or speed over a fixed schedule? The fixed schedule uses low precision in early steps. In later steps, it uses full precision for the prompt and predicted content, while other positions remain at low precision. Choose the fraction of early steps using the same calibration data, and keep this fraction fixed across tasks and step counts.
2 comments
response to authors
The rebuttal is difficult to follow, and the newly added experiments do not adequately address my concerns. I also have concerns about the authors' attitude toward the discussion. Therefore, I will maintain my original score and continue to recommend rejection.
We thank the reviewer for recognising the reserved-region finding and that TEMPO needs no additional forward pass.
W1. In one configuration, TEMPO adds 1.20–1.76× over Par+Fill with outputs consistent with Par+Fill; we report the two kinds of "saved" separately and use measured latency as the common cost unit.
W2. Against the fixed schedule the reviewer describes, the gate achieves higher fidelity on hard tasks and saves more computation on easy ones. We built the schedule exactly as described; see Q2.
Q1. With the same generation length and decoding optimisations, TEMPO adds speed over Par+Fill with outputs consistent with it, and its fidelity is up to 16.0 points above uniform quantisation. Full results, including FP, Par, latency and memory, are in our response to Reviewer qKBW (W2, Q1).
Q2. Yes. Against the fixed schedule the reviewer describes, the gate achieves higher fidelity and accuracy on hard tasks and saves more computation at equal fidelity on easy ones. A fixed schedule applies one fraction to every task and sample, whereas the gate allocates precision per sample and step. The full table and analysis are in our response to Reviewer Cm1g (W2).
We built the fixed schedule exactly as described: early steps fully at low precision; in later steps the prompt and predicted content at full precision and other positions at low precision (TEMPO's routing); the early fraction chosen from the same calibration data and fixed across tasks and step counts. LLaDA W3A16, Dream W4A16; BFCL, BFCL-multiple, MetaTool, API-Bank; T=16/32; n=100.