acceptodds
Under review as a conference paper at ICLR 2027

TEMPO: Temporal and Positional Precision for Diffusion Language Model Agents

Abstract

Diffusion large language model (dLLM) agents recompute a fixed-length generation region at each denoising step. Reserved positions beyond the emitted call account for most of this computation. Removing them reduces accuracy because they provide context for uncommitted predictions, requiring a reduction in computation per position. Quantisation reduces this cost, but existing quantisers use a fixed configuration across steps and positions, irrespective of the decoding state defined by committed tokens and current predictions. We identify three phenomena that enable precision to adapt to this state. Filler-first ordering places end-of-text and padding commitments before content commitments, allowing early steps to run at low precision. Margin persistence correlates logit gaps across steps, indicating the risk of quantisation-induced prediction changes before each pass. Slack tolerance permits low-precision updates of the reserved region with limited changes in accuracy. These phenomena motivate TEMPO (Temporal and Positional Orchestration of Precision), a training-free hierarchical precision schedule. A step-wise gate selects low-precision steps using cached gaps. On other steps, position-wise routing retains full precision only for the prompt and estimated content region. Across four agent benchmarks and three backbones, TEMPO achieves 84% mean call fidelity, compared with 76% for full quantisation and 66% for Learn2PD. Adding TEMPO to parallel decoding with boundary fill yields a further to speedup on Spider and API-Bank. The full composition achieves up to at 128 denoising steps. Our code is openly accessible to support future research.

Primary area:
foundation or frontier models, including LLMs
Keywords:
diffusion language models, efficient inference, quantization, dynamic mixed precision, parallel decoding

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_nC4HOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TEMPO reduces the cost of diffusion language model inference by varying numerical precision across decoding steps and token positions. Predictions from the previous step determine whether the next step runs entirely at low precision. Otherwise, full precision is retained for the prompt and estimated output region. The paper evaluates task accuracy, inference speed, and agreement with the original model. Agreement measures how often a method reproduces the same tool call on examples where the model at full precision produces a valid call.

Critical strengths
  1. The paper shows that positions beyond the emitted tool call can still affect decoding. On LLaDA evaluated on BFCL, removing these positions reduces accuracy on BFCL from 89% to 73%, while updating them at low precision retrains 86% accuracy. This is a useful finding that supports retaining these positions during inference.
  2. The method uses predictions already available from the previous step and requires no additional forward pass. The reported speedup of 1.20×1.20\times to 1.45×1.45\times when added to parallel decoding and boundary fill demonstrates that it can provide further acceleration after these optimizations.
Critical weaknesses
  1. The reported accuracy and speed do not come from the same experimental settings. Table 1 evaluates accuracy and output agreement at a generation length of 128. Table 3 reports latency at 1024 without reporting either metric for those configurations. In addition, Table 1 uses "computation saved" to describe both positions evaluated at low precision by TEMPO and positions removed by Learn2PD. These operations have different costs, so the percentages cannot establish which method is more efficient. The main results leave unclear how much accuracy and agreement are retained at the reported speeds.
  2. The main ablations do not sufficiently isolate the benefit of adapting precision to the current predictions. Figure 2 shows that content is usually committed in later steps. A fixed schedule that uses low precision early and full precision later already follows this observation. The comparisons with random allocation and reversed gate decisions in Table 2 show that the choice of steps matters, but do not establish how much the proposed gate improves over this simple schedule. That improvement is important for assessing the contribution of the method.
Questions for authors
  1. For the configurations in Table 3, what are the task accuracy and agreement with the model at full precision? How does TEMPO compare with uniform quantization when both use the same generation length and decoding optimizations? Reporting these results together would show whether TEMPO provides additional speed while preserving comparable outputs.
  2. Does adaptive gating improve accuracy or speed over a fixed schedule? The fixed schedule uses low precision in early steps. In later steps, it uses full precision for the prompt and predicted content, while other positions remain at low precision. Choose the fraction of early steps using the same calibration data, and keep this fraction fixed across tasks and step counts.
2 comments

We thank the reviewer for recognising the reserved-region finding and that TEMPO needs no additional forward pass.

W1. In one configuration, TEMPO adds 1.20–1.76× over Par+Fill with outputs consistent with Par+Fill; we report the two kinds of "saved" separately and use measured latency as the common cost unit.

  1. Same-configuration quality comes from the timed kernels (RTX A6000, L_g=1024, T=128, n=100); the full table is in our response to Reviewer qKBW (W2).
  2. Positions removed by Learn2PD are no longer computed, whereas TEMPO's low-precision rows are still computed at lower precision. As their costs differ, we report them as separate quantities and compare efficiency across methods by measured latency.

W2. Against the fixed schedule the reviewer describes, the gate achieves higher fidelity on hard tasks and saves more computation on easy ones. We built the schedule exactly as described; see Q2.

Q1. With the same generation length and decoding optimisations, TEMPO adds speed over Par+Fill with outputs consistent with it, and its fidelity is up to 16.0 points above uniform quantisation. Full results, including FP, Par, latency and memory, are in our response to Reviewer qKBW (W2, Q1).

  1. TEMPO adds 1.20–1.76× over Par+Fill with outputs consistent with Par+Fill.
  2. On Dream BFCL, at similar speed to uniform W8A8 (1.34× vs. 1.37×), TEMPO is 16.0 points higher in fidelity and 10 in accuracy.
  3. On LLaDA BFCL, API-Bank and Spider, TEMPO's fidelity exceeds uniform quantisation (88.7 vs. 84.5, 69.9 vs. 67.1, 25.3 vs. 23.2).

Q2. Yes. Against the fixed schedule the reviewer describes, the gate achieves higher fidelity and accuracy on hard tasks and saves more computation at equal fidelity on easy ones. A fixed schedule applies one fraction to every task and sample, whereas the gate allocates precision per sample and step. The full table and analysis are in our response to Reviewer Cm1g (W2).

We built the fixed schedule exactly as described: early steps fully at low precision; in later steps the prompt and predicted content at full precision and other positions at low precision (TEMPO's routing); the early fraction chosen from the same calibration data and fixed across tasks and step counts. LLaDA W3A16, Dream W4A16; BFCL, BFCL-multiple, MetaTool, API-Bank; T=16/32; n=100.

  1. Accuracy: on API-Bank the gate's fidelity is 3.4–8.7 points higher, and at T=32 its accuracy is also higher: 64 vs. 57 (LLaDA) and 24 vs. 17 (Dream).
  2. Speed (low-precision share): on MetaTool, at equal fidelity, the gate runs 13–33 points more at low precision.
  3. On average the gate saves more with higher fidelity and accuracy: LLaDA fidelity 87.9 vs. 86.9, accuracy 79.4 vs. 78.5, low-precision share .69 vs. .58; Dream 81.3 vs. 80.7, 65.5 vs. 65.4, .87 vs. .81. The average gap is small because the gain appears as quality on hard tasks and as savings on easy ones, and a single-metric average dilutes each.
Reviewer_nC4HOpenReview ↗

response to authors

The rebuttal is difficult to follow, and the newly added experiments do not adequately address my concerns. I also have concerns about the authors' attitude toward the discussion. Therefore, I will maintain my original score and continue to recommend rejection.

Reviewer_Cm1gOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TEMPO changes precision across denoising steps and positions when diffusion language models generate tool calls. The idea is that filler tokens are often committed early, cached logit margins can flag sensitive predictions, and the reserved tail still matters even though it is not emitted. A gate decides whether a whole step can run at low precision; otherwise, the prompt and estimated content region stay at full precision while the tail uses low precision. Across the tested backbones and tool-use tasks, the method preserves more full-precision outputs than uniform quantization and gives additional speedups when combined with parallel decoding and boundary fill.

Critical strengths

The reserved-region experiments are the most interesting part for me. In table 7, removing, freezing, and updating this region at lower precision have quite different effects. On LLaDA-BFCL, oracle-length removal loses 16 accuracy points while low-precision updates lose three. That gives a good reason to keep computing at positions that look dispensable from the final output. The cases where buffered removal improves accuracy are useful too: they show why matching a decoding trajectory and getting the task right are separate questions.

The schedule is simple, and the ablations check several things that could go wrong. I liked the comparison between cached and full-precision gate signals in app. F, since calibration and deployment do not read the same signals. There is also a fixed-tail baseline in app. J, alongside the random and inverted controls. The paper acknowledges where that simpler schedule is competitive. These checks help explain what the gate is doing.

Critical weaknesses

My main concern is that the quality and speed results use different configurations. table 1 uses W3A16/W4A16/W8A16 fake quantization at generation length 128. The kernels in app. A use INT8/FP8 or fused W8A16, and table 3 times length 1024 with additional decoding optimizations. There is an accuracy check for the full composition in table 4, but it covers BFCL and MetaTool, while table 3 times Spider and API-Bank; app. H also says all accuracy/fidelity runs use fake quantization. I still can't tell what quality the actual timed kernels deliver, or how they compare with uniform W8A16 using the same decoding optimizations. That comparison matters more to my assessment than the large speedup of the full composition over fixed-step decoding.

I'm also not yet convinced that the online gate gives a better quality–latency tradeoff than a fixed temporal schedule. The fixed-tail results in table 14 are competitive, and the text says fidelity is similar at matched budgets. Avoiding a separate choice of how many final steps to protect is useful, but I'd like to see whether it buys a better operating point across tasks.

Questions for authors
  1. For the actual configurations in table 3, can you report quality alongside latency for full precision, parallel decoding, parallel decoding + fill, uniform quantization + parallel decoding + fill, and TEMPO? Please include the precision formats and peak memory, and specify whether quality was checked through the timed kernels.

  2. Does the gate beat a fixed-tail schedule at matched measured latency on held-out tasks, with the same positional routing and kernels? Also, what does “Saved (%)” count in table 14? The fixed-tail rows equal the fraction of low-precision steps even though app. J says all methods use position-wise routing.

  3. Are the BFCL trajectories used for calibration disjoint from the reported BFCL evaluation items? Were the thresholds recalibrated for the INT8/FP8 timing paths? The split and the calibration used for each precision would help assess how well the gate transfers.

2 comments

We thank the reviewer for recognising the reserved-region experiments, the cached-gap control and the fixed-tail baseline.

W1. In one configuration, TEMPO adds 1.20–1.76× over Par+Fill with outputs consistent with Par+Fill; against uniform quantisation with the same decoding optimisations, its fidelity is up to 16.0 points higher. All quality numbers come from the timed kernels; the full table is in our response to Reviewer qKBW (W2).

  1. On Dream BFCL, at similar speed (1.34× vs. 1.37×), TEMPO exceeds uniform W8A8 by 16.0 fidelity points (79.0 vs. 63.0) and 10 accuracy points (81 vs. 71). On LLaDA BFCL, API-Bank and Spider its fidelity is also higher (88.7 vs. 84.5, 69.9 vs. 67.1, 25.3 vs. 23.2).
  2. Fidelity is against FP at the same T; on Spider and API-Bank parallel decoding itself changes outputs, so Par+Fill is the better reference.

W2. Yes. A fixed schedule is fixed: one fraction for every task and sample. Tasks need different protection, so one fraction under-protects hard tasks and over-protects easy ones. TEMPO's gate allocates precision per sample and step from one calibrated threshold and does better at both ends. We built the fixed schedule as Reviewer nC4H describes: fraction from the same calibration data, fixed across tasks and step counts, same routing. LLaDA W3A16, Dream W4A16, T=16/32, n=100, disjoint calibration. Cells: fidelity / low-precision share; Δ = gate − fixed.

FixedGateΔ fid.Δ share
API-Bank (hard)LLaDA T=1664.7/.5671.8/.46+7.1−.10
LLaDA T=3265.2/.5973.9/.49+8.7−.10
Dream T=1633.3/.7836.7/.67+3.4−.11
Dream T=3247.6/.8154.8/.79+7.2−.02
MetaTool (easy)LLaDA T=1698.9/.5698.9/.880.0+.32
LLaDA T=32100/.59100/.920.0+.33
Dream T=1698.9/.8298.9/.950.0+.13
Dream T=3298.9/.8497.8/.97−1.1+.13
4-task meanLLaDA86.9/.5887.9/.69+1.0+.11
Dream80.7/.8181.3/.87+0.6+.06

(Share: measured row share for Dream, step share over all samples for LLaDA.)

  1. Hard tasks: the fixed fraction under-protects; the gate protects more and gains 3.4–8.7 fidelity points. On LLaDA API-Bank the gate beats the fixed tail at every k (T=16: 71.8 vs. best 68.2; T=32: 73.9 vs. best 72.8).
  2. Easy tasks: the fixed fraction over-protects; at equal fidelity the gate runs 13–33 points more at low precision.
  3. On average the gate saves more with higher fidelity. The average gap looks small because the gains lie on different axes: on hard tasks the gate spends compute for fidelity, on easy tasks it keeps fidelity and saves compute, so any single-metric average dilutes one of them. A fixed fraction cannot do this: it stays at one budget for every task.
  4. The budget adapts: with one threshold, LLaDA's low-precision share ranges from .46 (API-Bank) to .92 (MetaTool); the fixed schedule stays at .56–.59.

Q1. In the configurations of Table 3, TEMPO adds 1.20–1.76× over Par+Fill with outputs consistent with Par+Fill; all quality numbers come from the timed kernels.

  1. Quality and latency for the five configurations: see our response to Reviewer qKBW (W2).
  2. Formats: FP uses the original 16-bit weights; uniform quantisation runs all rows in W8A8; TEMPO runs low-precision rows in W8A8 and protected rows as a two-plane form of the same int8 weights in one GEMM.
  3. Peak memory: LLaDA 24.7–26.5 GiB, Dream 22.3–22.9 GiB (resident memory: response to qKBW, Q1).

Q2. On held-out tasks the gate reaches a better operating point than the calibrated fixed schedule: higher fidelity on hard tasks, more savings on easy ones, with no per-task choice of k.

  1. Calibration uses only BFCL-live, so API-Bank and MetaTool are held out. On API-Bank the gate gains 3.4–8.7 fidelity points; on MetaTool, at equal fidelity, it runs 13–33 points more at low precision (W2).
  2. "Saved (%)": in Table 14 the fixed-tail rows reported the nominal low-precision step fraction k/T, and TEMPO's row the measured row share. W2 uses one metric for both: measured row share (Dream) and step share over all samples (LLaDA).

Q3. We recalibrated each precision separately on data disjoint from evaluation; all new results use the matching thresholds, and results agree with the original thresholds.

  1. The original trajectories were the first 15–20 BFCL-simple items, which overlap with evaluation; the submitted timing path reused the weight-only thresholds.
  2. Split: BFCL-live (n=100), disjoint from the BFCL-simple and BFCL-multiple evaluation sets, with no API-Bank, MetaTool or Spider items.
  3. Thresholds: deployed weight-only precision (W3A16/W4A16/W8A16) LLaDA 5.54, Dream 3.42, CoDA 0.125; timed W8A8 kernel LLaDA 1.46, Dream 1.56.
  4. Replacing the original thresholds changes fidelity by −2.0 to 0 (LLaDA) and 0 to +2.1 (Dream) points across four tasks × T=16/32.

Thanks for the additional results. The timed-kernel evaluation and disjoint calibration address much of my first concern. The comparison between the gate and a fixed schedule at the same measured latency is still missing, so I will keep my score at 2.

Reviewer_qKBWOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

Standard fixed-length dLLMs recompute the entire generation region at every denoising step. For short structured outputs like tool calls, most of that region is padding at the end of the sequence. The authors show this padding still matters, since truncating or freezing it hurts accuracy as dLLMs are trained with bidirectional non-causal attention. Their proposal is to compute it more cheaply rather than remove it.

The paper introduces TEMPO, which does this with two mechanisms. A step-wise gate reads logit gaps cached from the previous pass. If every likely commit is filler or confidently predicted, the whole step is run with quantization. Otherwise, rows past an estimated content boundary run with quantization and the prompt and content stay at full precision. The method is training-free and needs one threshold per backbone.

On LLaDA-8B, Dream-7B and CoDA-1.7B across BFCL, API-Bank and MetaTool, TEMPO reports 84% mean agreement with full-precision outputs. Full quantisation reaches 76% and Learn2PD 66%. Combined with parallel decoding and early termination, TEMPO claims to add a further 1.20–1.45× speedup.

Critical strengths

The central observation in the paper is its best contribution. The reserved region can account for 57-84% of positions for a median tool call. On LLaDA-BFCL, removing it drops accuracy 16 points and freezing it drops 11 points. The authors also find that confidence-ranked samplers commit filler first, and show that via a random-order control that the effect comes from ranking.

The empirical work is carefully done. The paper reports both call fidelity as well as accuracy, and the authors report McNemar tests. They compare candidate gate signals with CIs, and check that cached gaps behave like full-precision ones, and enforce a same-GPU protocol after finding nondeterminism across hardware. The limitations section is also candid.

Critical weaknesses

(W1): My main concern is the scope of the paper. TEMPO targets an inefficiency of fixed-length dLLMs, when these models are given more generation length than needed. This can cover short structured outputs like tool-calls, which is the only evaluation setting covered in the paper. The authors do not explore the impact on more free-form text, or where e.g. reasoning might be interleaved with tool-calls as in modern LMs. Moreover, block dLLMs grow the sequence as needed and remove the core problem at its source. Semi AR block decoding enables KV caching, which is also LLaDA's usual inference mode, and this likely suppresses both the filler-first pattern and positional savings. For long-form generation that fills Lg, there is little filler and the boundary sits near the end, so TEMPO should reduce to near full precision. None of these settings is tested: there is no block decoding, no long-form task, no chat, and every run is greedy.

Even within this scope, the gate doesn't seem to benefit much over heuristics that are more obvious. In table 14, simply keeping the last k steps at full precision roughly matches it. On LLaDA at T=16, the fixed tail reaches 62.5% saved and 89.6% fidelity, against TEMPO’s 66.2% and 90.1%. On CoDA, random and inverted gates land within two points of TEMPO. In task accuracy, TEMPO averages 63 against 62 for full quantization, while spending more compute.

(W2): Some experimental settings seem conflated. The paper doesn't seem to show quality and speed for the same configuration. Fidelity is measured with fake-quantization with weight-only settings: W3A16, W4A16, W8A16. This is a compute-bound regime, and all positions are processed each step. Weight-only quant saves memory traffic but does not save FLOPs. Row routing also requires both weight copies in memory, so the fake-quant path peaks near 30 GB vs 17.5 GB. The speedups instead come from INT8/FP8 and fused W8A16 kernels and the fidelity is never reported here

Also, the headline 99x speedup comes from older measurements at Lg=1024, T=128, and seems to be due to step reduction and early termination. TEMPO contributes 1.2-1.45x. For that system, only task accuracy is reported. Dream-BFCL jumps from 0.75 to 0.86, which might be due to behavioral changes that the paper's own fidelity argument says should be measured. The Learn2PD comparison is also hard to interpret - it saves 0% compute on Dream but changes about a third of calls, presumably because its learned parallel decoding alters the commit schedule rather than because of truncation.

Questions for authors

(Q1): Can the authors accurately report the memory overhead?

(Q2): What is the call fidelity under the exact kernels and precisions used for timing, and what is the deployment memory footprint with both weight copies resident?

(Q3): Are the BFCL calibration trajectories disjoint from the BFCL evaluation items?

1 comment

We thank the reviewer for recognising the reserved-region finding and our evaluation.

W1 (scope). In fixed-length agent calls the reserved region is the main source of computation, and TEMPO still accelerates tasks with longer outputs. While block-decoding and long-form runs are not feasible in the rebuttal period:

  1. The reserved region covers 57–84% of positions for the median tool call.
  2. Routing keeps the estimated content at full precision. On Spider, with longer SQL outputs (26–35 steps vs. 4–5 on BFCL), TEMPO still adds 1.39× (LLaDA) and 1.26× (Dream) over Par+Fill.

W1 (gate vs. heuristics). The gate is adaptive, while a fixed tail is fixed: with one calibrated threshold the gate protects more on hard tasks and saves more on easy ones (table in our response to Reviewer Cm1g, W2).

  1. On API-Bank the gate's fidelity exceeds a fixed schedule calibrated on the same data by 3.4–8.7 points; on MetaTool, at equal fidelity, it runs 13–33 points more at low precision.
  2. TEMPO's mean accuracy (63) matches full precision (64), with mean fidelity 82.6 vs. 72.3 for full quantisation.

W2 (configurations). In one configuration, TEMPO adds 1.20–1.76× over Par+Fill with outputs consistent with Par+Fill; all quality numbers come from the timed kernels. We add W8A8, which the timed kernel supports. At more aggressive precision (W3A16/W4A16, Table 1) TEMPO's fidelity exceeds full quantisation by 9–25 points, so the gain depends on the configuration. TEMPO is bounded below in every setting:

  1. Speed: faster than Par+Fill in all 6 settings (1.20–1.76×).
  2. Quality: fidelity on par with uniform quantisation (within 1.0 point) or higher (up to +16.0).

A6000, L_g=1024, T=128, n=100; low-precision rows W8A8 (so compute drops), protected rows a two-plane form of the same int8 weights; W8A8 thresholds from disjoint data. Cells: fidelity/accuracy/latency (s), speedup over Par+Fill in brackets.

FP acc/sParPar+Fill+Uniform W8A8+TEMPO
LLaDA API-Bank46/24.4167.1/54/2.8767.1/54/2.6767.1/53/1.71 (1.56×)69.9/53/1.90 (1.41×)
LLaDA BFCL85/23.8585.6/84/0.9985.6/84/0.9784.5/85/0.70 (1.39×)88.7/84/0.81 (1.20×)
LLaDA Spider28/23.7725.3/32/5.6625.3/32/4.8723.2/32/3.32 (1.47×)25.3/33/3.51 (1.39×)
Dream API-Bank12/35.5133.3/35/5.4937.5/36/4.7433.3/33/2.60 (1.82×)33.3/34/2.69 (1.76×)
Dream BFCL75/28.0580.2/83/1.1480.2/83/1.1463.0/71/0.83 (1.37×)79.0/81/0.85 (1.34×)
Dream Spider11/25.587.0/30/9.387.0/30/5.857.0/26/4.75 (1.23×)6.0/26/4.64 (1.26×)
  1. On Dream BFCL, at similar speed to uniform W8A8 (1.34× vs. 1.37×), TEMPO is +16.0 in fidelity and +10 in accuracy.
  2. Fidelity is against FP at the same T; on Spider and API-Bank parallel decoding itself changes outputs, so Par+Fill is the better reference.

W2 (99×). TEMPO's speedup compounds with the step reduction of parallel decoding, adding 1.20–1.76× over Par+Fill. LLaDA BFCL: Par 24.1×, Fill 1.02×, TEMPO 1.20× (total 29.5×). Dream BFCL: 24.6×, 1.00×, 1.34× (33.0×). Dream API-Bank: 6.5×, 1.16×, 1.76× (13.2×).

W2 (Dream-BFCL accuracy). We agree that accuracy changes do not reflect the method's effect here, so fidelity is our primary metric.

  1. On Dream BFCL, parallel decoding alone raises accuracy from 75 to 83 while ~20% of calls differ from FP (fidelity 80.2).
  2. On LLaDA BFCL, uniform W8A8 matches FP accuracy (85) yet changes 15.5% of calls (fidelity 84.5).

W2 (Learn2PD). That removing the reserved region hurts quality rests on controlled interventions, not on the Learn2PD comparison. We agree that on Dream Learn2PD's changes come from its learned decoding order; it serves as an end-to-end comparison with a published method.

  1. On LLaDA-BFCL, oracle-length removal lowers accuracy by 16 points, freezing by 11, low-precision updates by 3.
  2. On Dream, Learn2PD removes 0–5% of positions, so its results reflect its full method.

Q1. With both copies resident, deployment memory is 20.2–21.4 GiB, 6.0–6.5 GiB above FP-only, fitting one 48 GB GPU (real-kernel path; peak below the ~30 GiB of fake-quant).

GiBFP onlyBoth copiesDecoding peak
LLaDA14.9321.4424.7–26.5
Dream14.1920.1722.3–22.9

Q2. Under the timed kernels and precisions, TEMPO keeps call fidelity on par with full-precision weights: 88.7 (LLaDA BFCL), 98.9 (LLaDA MetaTool), 79.0 (Dream BFCL), 98.8 (Dream MetaTool), vs. 85.6/98.9/80.2/98.8 for full-precision weights with the same decoding. Memory: see Q1.

Q3. After recalibrating on data disjoint from evaluation, results are unchanged.

  1. The original trajectories were the first 15–20 BFCL-simple items, which overlap with evaluation. Recalibrating on the disjoint BFCL-live split (n=100) changes fidelity by −2.0 to +2.1 points.
  2. All new results use disjoint calibration; split and per-precision thresholds are in our response to Reviewer Cm1g (Q3).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.