acceptodds
Under review as a conference paper at ICLR 2027

LOGIT-PATH TRACING: EXACT GREEDY DECODING PARTITIONS OF LANGUAGE MODELS UNDER AFFINE LOGIT CONTROLS

Abstract

Many language-model decoding methods expose continuous parameters, such as guidance scales and objective weights. We study settings in which token logits are affine in these parameters and ask which greedy outputs occur anywhere in the control domain. Finite sampling can detect observed outputs but cannot certify the absence of unobserved output cells. Under greedy decoding, affine token logits form an upper envelope whose intersections partition the parameter domain into regions with a constant next-token prediction. We introduce Logit-Path Tracing (LPT), which recursively constructs these regions and verifies their winners against the full vocabulary. Because pairwise logit gaps are affine, vertex verification certifies each polyhedral cell, yielding the complete greedy decoding partition under exact arithmetic. For finite-precision logits, we extend LPT with ε-robust certification, using a pairwise-gap error tolerance to separate certified and unresolved regions. We evaluate three base models on 200 test prompts over one- and two-dimensional control domains, with the pairwise-gap tolerance calibrated on a separate set of 100 prompts. LPT constructs the complete greedy decoding partition, with a mean of 5.23–10.36 output cells per prompt in one dimension and 18.82–68.30 in two dimensions. Using these complete partitions as reference, finite sampling observes only 45.0–87.9% of the output cells under matched execution time and 33.9–63.1% under matched model-evaluation budgets.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Controllable Language Generation, Decoding-Time Control, Greedy Decoding, Logit Interpolation, Parametric Verification

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_yLqGOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the partition of a continuous control domain induced by greedy language-model decoding when the token logits are affine in the control parameters. It proposes Logit-Path Tracing (LPT), which constructs candidate upper-envelope cells, verifies their winners against the full vocabulary at the cell vertices, and recursively propagates all verified token branches. Under exact arithmetic, the paper claims recovery of the complete greedy-decoding partition. It further introduces an ε-robust variant based on an assumed uniform bound on pairwise-logit-gap errors. Experiments cover three 1.5B–3B models, 200 test prompts, and one- and two-dimensional control domains.

Critical strengths
  • Finite sampling can discover outputs but cannot certify the absence of an output between sampled parameter values. This distinction is relevant to controllable decoding and model auditing.

  • For a fixed prefix, affine token logits induce polyhedral winner regions, and full-cell verification reduces to evaluating affine gaps at vertices.

Critical weaknesses
  • Theorem 1 assumes exact arithmetic, whereas the experiments use fp32 model logits, float64 geometry, and fixed numerical tolerances. There is no interval arithmetic, exact rational geometry, directed rounding, or final certified verification showing that narrow cells cannot be lost. Dense-grid agreement is not an exactness proof. The paper should sharply distinguish the exact-arithmetic theorem, the partition of stored numerical affine functions, and the behavior of the executed model.

  • The theorem (Eq. 10) requires a uniform pairwise-gap error bound over every relevant parameter value, prefix, and token pair. The maximum discrepancy observed on 100 calibration prompts under bf16 versus fp32 does not provide such a bound for unseen prompts, prefixes, kernels, or hardware. The theoretical certificate is valid conditionally, but the experimental ε\varepsilon should be described as an empirical stress-test tolerance rather than a formal finite-precision certificate.

  • The theory retains lower-dimensional tie cells and propagates every maximizing branch. The reported one-dimensional implementation breaks ties deterministically, while the two-dimensional implementation does not export zero-area tie atoms. Zero-measure sets are irrelevant to area coverage but not to the motivating question of whether any control value produces a target output. The paper needs one consistent tie semantics and experimental validation of the corresponding implementation.

  • Cell recall and coverage are measured relative to the ε=0\varepsilon = 0 LPT partition. Dense grids provide finite-point consistency checks, while candidate-set invariance only tests stability within the same implementation. Independent exhaustive or symbolic validation on tractable autoregressive instances is needed.

  • The paper acknowledges that the point-query samplers do not use a prefix trie, whereas prefix caching is central to LPT’s efficiency. A prefix-cached uniform or adaptive sampler would provide a substantially stronger comparison. Runtime should also be decomposed into model inference, caching, and geometric processing, with peak memory reported.

  • Upper envelopes of affine functions and vertex verification of affine inequalities are standard tools in parametric optimization and computational geometry. The main contribution is their recursive application to autoregressive decoding. The paper should more precisely isolate the novel algorithmic component relative to MERT-style envelope methods, parametric analysis, and existing continuous LM interpolation work.

Questions for authors

refer to weaknesses

6 comments
AuthorsOpenReview ↗

Response to Reviewer yLqG (1/4): Exact Arithmetic and Robustness Certification

Thank you for the review and for the detailed comments on numerical certification, tie handling, completeness verification, and sampling baselines.

(1) Exact arithmetic and numerical implementation

To verify the numerical correctness and completeness of LPT, we developed an independent verifier based on exact rational arithmetic, using stored full-vocabulary affine token logits obtained from Qwen2.5-3B.

The verification was conducted as follows:

  1. Exact coefficient representation: We converted the stored binary floating-point coefficients into mathematically equivalent rational numbers using their mantissas and powers-of-two exponents. This preserves the stored coefficient values exactly, without introducing additional rounding errors.

  2. Region construction: We computed token-winner regions using exact rational halfspace intersections, eliminating floating-point errors in the geometric calculations.

  3. Full-vocabulary verification: We verified candidate winners against the entire vocabulary at each region's vertices using exact rational comparisons.

  4. Recursive decoding: We recursively expanded all valid token branches to independently reconstruct the complete set of possible greedy outputs.

  5. Result comparison: We compared the reconstructed greedy output sequences, region dimensions, and vertices against the original LPT implementation.

We evaluated three prompts in each of the one- and two-dimensional control settings (T = 64). The results are summarized below.

Control DimensionPromptsMatched Greedy OutputsDisagreements
d = 13360
d = 234000
Total64360

Disagreements indicate missing or additional greedy outputs, or differences in their region dimensions, between LPT and the independent verification.

The independent verifier recovered all 436 greedy outputs identified by LPT, with no missing or additional outputs or region-dimension differences. The maximum discrepancy between the computed region vertices was 7.08 × 10⁻¹⁶.

By preserving the original stored affine coefficients exactly and performing the geometric verification with rational arithmetic, this experiment directly tests whether LPT's floating-point geometry misses or introduces greedy output regions. The complete agreement provides independent evidence of numerical correctness and completeness for the tested fixed affine decoding systems.

Following the reviewer's suggestion, the revised PDF distinguishes three levels explicitly (Section 5): (i) Theorem 1 guarantees completeness for exact affine logit functions under exact arithmetic; (ii) the stored numerical affine system: LPT's floating-point geometry recovers exactly the same greedy outputs as exact rational computation on the tested systems, as shown above; and (iii) the executed model: a fresh FP32 execution reproduced all 1,540 representative outputs, whereas BF16 reproduced 31.10%, so agreement under a different numerical precision is implied by neither (i) nor (ii) (see our response to Reviewer hn5J).

(2) Robustness certification

We agree with the reviewer that the calibrated tolerance is empirical rather than a formal finite-precision bound. Eq. (10) requires a uniform bound on pairwise logit-gap errors across the relevant controls, prefixes, and token pairs. The BF16–FP32 discrepancy measured on calibration prompts provides an empirical tolerance, rather than such a uniform bound.

At this tolerance, only 0–0.5% of the control domain satisfies the robustness margin criterion. In the revised PDF, we clarify the assumption underlying Eq. (10) in Section 5 and present the results in Figure 4 and Table 13 (Table 6 in the original submission) as empirical robustness analyses (Section 6.5; Appendix C). As reported in our response to Reviewer hn5J, the stability radii of the recovered outputs (medians of 0.0108 and 0.0463 logits for outputs found only by LPT and for those also found by sampling) are far below this tolerance, which explains the low certified coverage; LPT reports these margins for every output.

AuthorsOpenReview ↗

Response to Reviewer yLqG (2/4): Tie Handling and Consistency

(3) Tie handling and consistency

To validate the handling of lower-dimensional tie cells and every maximizing branch, we conducted a separate verification using exact rational arithmetic and common refinement on small autoregressive test instances.

We constructed eight test instances with V = 3, T = 2, and d ∈ {1, 2}, varying random seeds and whether exact ties were imposed between root-token logit functions.

The verification was conducted as follows:

  1. Tie construction: Construct affine token logits with and without imposed ties.

  2. Greedy output enumeration: Enumerate all 3² = 9 possible token sequences and construct their greedy-winner inequalities, retaining every maximizing branch.

  3. Common refinement: Subdivide the control domain into disjoint partition regions and boundaries, including lower-dimensional tie cells and zero-area tie atoms, using exact rational arithmetic.

  4. Independent verification: Compute all possible greedy outputs at a representative point within each partition region or boundary using direct all-maximizer greedy decoding, and compare the resulting output sets.

The results are summarized below.

dSeedImposed TieRegions & BoundariesDisagreements
10No70
10Yes70
11No50
11Yes50
20No230
20Yes230
21No270
21Yes210
Total—8 instances1180

Imposed Tie indicates whether two tokens at the initial decoding step are assigned identical affine logit coefficients. Disagreements count differences in the possible greedy output sets between common refinement and direct all-maximizer greedy decoding.

Across the eight test instances, all 118 partition regions and boundaries were correctly labeled, comprising 46 zero-dimensional points, 55 one-dimensional intervals or edges, and 17 two-dimensional regions. Among these, 62 admitted multiple greedy outputs, and all output sets matched direct all-maximizer greedy decoding with zero disagreements.

These results validate the handling of every maximizing branch, including lower-dimensional tie cells and zero-area tie atoms, in the tested exact-arithmetic instances. The revised PDF states one tie semantics explicitly (Section 3; Appendix E.1). The theory and LPT's traversal follow all-maximizer semantics, and the implementation already traces every maximizing branch. Comparisons with sampling additionally apply a deterministic lowest-token-ID rule, since a sampler decodes a single output per control value. Because persistent ties can occur on positive-measure regions, the two counts can differ (e.g., 51.5 vs. 51.3 supports for Qwen3.5-9B at T = 64); we report both (Appendix E.1, Table 7) and have corrected the appendix statement that they differ only on lower-dimensional ties. We also distinguish output supports, which may overlap, from disjoint partition cells (Section 3), and describe the common-refinement construction of such cells validated above on small instances (Appendices D and E.1).

AuthorsOpenReview ↗

Response to Reviewer yLqG (3/4): Independent Verification of Completeness

(4) Independent verification of completeness

To independently validate LPT's completeness, we conducted two verification experiments using exact rational arithmetic: (a) exhaustive verification on small autoregressive models and (b) independent verification using affine token logits obtained from a real language model.

(a) Exhaustive verification. We constructed 80 autoregressive test instances with V = 3, T = 3, and d ∈ {1, 2}, including instances with exact ties.

The verification was conducted as follows:

  1. Sequence enumeration: Enumerate all 3³ = 27 possible token sequences for each test instance.

  2. Greedy-winner conditions: Construct the affine inequalities requiring each token in a sequence to maximize the logits at its corresponding prefix.

  3. Exact feasibility verification: Use exact rational arithmetic to determine whether each sequence is feasible for any control parameter value, including lower-dimensional tie cells.

  4. Independent comparison: Compare the feasible greedy outputs and their region dimensions against LPT under two candidate-set configurations.

The results are summarized below.

SettingInstances × ConfigurationsComparisonsDisagreements
V = 3, T = 3, d ∈ {1, 2}80 × 21600

Disagreements indicate missing or additional greedy outputs, or differences in their region dimensions, between LPT and the independent verification.

Across 80 test instances and two candidate-set configurations, all 160 comparisons matched, confirming that LPT recovered every feasible greedy output among all 27 possible sequences, including outputs restricted to lower-dimensional tie cells.

(b) Verification using real-LM logits. We additionally verified LPT using the stored full-vocabulary affine token logits obtained from Qwen2.5-3B, with the independent exact-rational verifier described in (1).

As reported in the table in (1), the verifier independently reproduced all 436 greedy outputs over positive-measure control regions across three prompts in each of the one- and two-dimensional control settings (T = 64), with zero disagreements.

These results provide independent evidence of complete greedy output recovery for the tested affine decoding systems. The revised PDF reports this verification in Appendix D (Tables 4–5).

Reviewer_hn5JOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The authors propose Logit Path Tracing (LPT), a method that finds every greedy output a model can produce when its logits depend linearly on a continuous control, such as interpolating between two LoRA adapters. Since the greedy token can only change where two logit lines cross, the control space splits into cells, and LPT builds and checks these cells step by step until EOS or 64 tokens. On three small models with one or two control dimensions, sampling baselines find only 45.0–87.9% of LPT's cells at matched time and 33.9–63.1% at matched model evaluations.

Critical strengths
  • The problem is well motivated. Sampling can show that an output exists but can never show that it doesn't, and for auditing this difference matters.
  • The method is simple, and I went through the proof in Appendix B and believe it is correct under the stated assumptions.
  • The authors are open about the limits of their own setup, for example that the sampling baselines do not share prefixes. Table 12, which lists every cell with its margin and text, makes the results easy to inspect.
Critical weaknesses
  • The cells that only LPT finds seem numerically fragile. In Table 12 their winning margins are 0.0009–0.016 logits, while the calibrated bf16–fp32 error ε̂ is 2.7–16.8 in Table 2. At ε = ε̂ the certified coverage falls to 0–0.5% in Table 6, so in practice the guarantee holds for one particular fp32 run. Table 9 then reports that bf16 reproduces the fp32 partition with 100% recall, which I find hard to square with the size of ε̂.
  • The comparison with sampling is not quite fair. The baselines decode every sample from scratch, and the adaptive baseline keeps splitting the largest cell even when neighbouring samples agree (l.788). A prefix-cached sampler, or one that splits where neighbours disagree, would be a stronger baseline. The scope is also narrow: T = 64, at most two control dimensions and models up to 3B.
Questions for authors

see weaknesses

6 comments
AuthorsOpenReview ↗

Response to Reviewer hn5J (1/2): Numerical Stability and Precision Consistency

Thank you for the review and for the constructive suggestions on numerical stability and sampling baselines.

(1) Numerical stability and precision consistency

We conducted additional experiments to examine numerical consistency across precisions. We identified an error in Table 9 of the original submission, which incorrectly reported 100% agreement between FP32 and BF16, and we thank the reviewer for pointing this out. We independently evaluated greedy decoding in both precisions at 1,540 representative control parameter values from Qwen2.5-3B (d = 2, T = 64).

Let L denote the greedy outputs identified by LPT and S = U ∪ A denote those identified by either uniform (U) or disagreement-adaptive (A) sampling in the time-matched comparison.

Output SetOutputsFP32 MatchBF16 Match
L ∩ S969100%39.42%
L − S571100%16.99%
L (All)1,540100%31.10%

Match indicates identical complete greedy output sequences at the same representative control parameter values. We evaluate one maximum-margin representative control from each of the 1,540 positive-measure FP32 reference supports. Both FP32 and BF16 point decoders use lowest-token-ID tie-breaking.

FP32 reproduced all 1,540 original greedy output sequences, while BF16 reproduced 479 (31.10%). Precision sensitivity is not specific to the outputs found only by LPT: even among outputs also found by sampling, only 39.42% remained identical under BF16 (16.99% for outputs found only by LPT). As the margin analyses below show, agreement is governed by the winning logit margin rather than by the method that discovered an output. These results correct the previously reported precision agreement in Table 9 and quantify the numerical sensitivity of the recovered outputs.

We further examined the relationship between logit margins and cross-precision agreement. The following table reports FP32–BF16 agreement across four margin quartiles.

Logit MarginFP32–BF16 Agreement
Q1 (Smallest 25%)11.2%
Q2 (25–50%)19.2%
Q3 (50–75%)30.1%
Q4 (Largest 25%)63.9%

Results are computed from the same 1,540 representative control parameter values, grouped by winning logit margin.

Agreement increases with the logit margin, from 11.2% in the smallest-margin quartile to 63.9% in the largest. This supports the reviewer's observation that outputs with small winning margins are particularly sensitive to precision changes. At the empirically calibrated BF16–FP32 error tolerance, only 0–0.5% of the control domain satisfies the robustness margin criterion.

Stability of outputs found only by LPT. To quantify this sensitivity beyond representative points, we computed for each of the 1,540 outputs whether an interior region remains in which the winning margin exceeds a pairwise-gap perturbation bound ε at every decoding step.

ε (logits)Found only by LPT (571)Also found by sampling (969)
0.00192.82%100.00%
0.0151.84%94.01%
0.10.70%14.86%

Fraction of outputs that retain a nonempty interior region whose winning margin exceeds ε at every decoding step. This does not imply that the original region or its area is retained.

Most outputs found only by LPT are not degenerate: 92.82% retain such a region at ε = 0.001 logits, and their median stability radius is 0.0108, compared with 0.0463 for outputs also found by sampling. However, both radii are far below the calibrated BF16–FP32 tolerance (2.7–16.8 logits), so neither group is robust at that scale, consistent with the BF16 results above. Outputs missed by sampling are thus the narrow, low-margin ones, and LPT reports this margin for every output, allowing an auditor to filter outputs by the tolerance relevant to a given deployment.

These results distinguish two properties: LPT recovers greedy decoding regions for a fixed affine logit system, while cross-precision consistency depends on the numerical stability of the underlying logits. In the revised PDF, we have replaced Table 9 with this corrected analysis (Section 6.5; Appendix F.6, Tables 16–18) and clarified the uniform error-bound assumption in Eq. (10), together with the interpretation of Figure 4 and Table 6 (now Table 13), in Section 5 and Appendix C.

AuthorsOpenReview ↗

Response to Reviewer hn5J (2/2): Stronger Sampling Baselines and Experimental Scope

(2) Stronger sampling baselines and experimental scope

Following the reviewer's suggestions, we implemented (1) prefix-cached uniform sampling and (2) disagreement-adaptive sampling. The former reuses affine token logits for previously evaluated prefixes, while the latter prioritizes control regions where neighboring samples produce different outputs. Both baselines were evaluated under the same model-evaluation budget as LPT for each prompt, with no limit on the number of sampled control values; prefix caching allows many control values to share model evaluations.

The following table compares LPT with the strengthened sampling baselines on the same 20 prompts for each model (d = 2, T = 64).

ModelLPT OutputsUniform RecallAdaptive RecallLPT Time (s)Uniform Time (s)Adaptive Time (s)
Qwen2.5-3B77.057.63%60.43%127.6120.5118.5
SmolLM2-1.7B28.899.07%99.07%32.31,183.263.3

Results are averaged over 20 prompts per model. LPT Outputs denotes the average number of distinct greedy outputs over positive-measure control regions. Recall is averaged per prompt. Times include the measured sampling and processing costs. Counts report positive-measure output supports returned by all-maximizer LPT, not disjoint partition atoms.

The strengthened baselines achieve 57.63–60.43% recall on Qwen2.5-3B, while both reach 99.07% on SmolLM2-1.7B. This difference is accompanied by substantially fewer distinct greedy outputs for SmolLM2 (28.8 vs. 77.0 per prompt). Near-complete empirical recall, however, is distinct from establishing that no output was missed, which is the question LPT targets. The timing results also reveal different trade-offs: sampling is slightly faster on Qwen2.5-3B but recovers fewer outputs, whereas on SmolLM2-1.7B LPT has the lowest median time (29.8 s vs. 59.7 s for adaptive and 241.5 s for uniform sampling); the uniform mean is dominated by the long-tail case discussed below.

Discovery of narrow output regions. To measure the computational effort required for sampling to discover all distinct greedy outputs, we conducted an additional experiment using prefix-cached uniform sampling, recording when each new output was first discovered. For a SmolLM2-1.7B prompt (d = 2, T = 64), LPT identified 24 distinct greedy outputs.

Outputs FoundSampled Values
11
22
33
44
55
88
……
19425
201,241
212,226
227,049
239,730
242,101,710

Sampled Values denotes the cumulative number of evaluated control parameter values, not model forward passes.

Uniform sampling discovered 23 of the 24 outputs within 9,730 sampled control values, but identified the final output only at sample 2,101,710. The complete sampling run took 13,970 seconds, compared with 29.4 seconds for LPT to recover all 24 outputs. This demonstrates that even when sampling achieves high output recall, discovering the remaining outputs can require substantially greater computational effort.

Experimental scope. We additionally evaluated Qwen3.5-9B and extended generation lengths up to T = 256. Detailed scalability results and sampling comparisons are provided in our response to Reviewer WeGb. These results, together with the strengthened baselines and the narrow-region trace above, are included in the revised PDF (Section 6.3, Table 2; Appendix F.2, Tables 10–12).

We hope these results address your concerns, and we would be happy to clarify any remaining questions.

Thank you for the corrected Table 9 and the stronger baselines. I have raised my Soundness score. Two remaining points:

  1. With the stronger baselines, the gap to sampling becomes much smaller. Also, by your own numbers, only 17% of the outputs found only by LPT stay the same under bf16.

  2. Almost all outputs have margins far below ε̂, yet 31% of them stay exactly the same under bf16, and 63.9% in the top margin quartile. This suggests the actual bf16 error on the top competing tokens is much smaller than ε̂, which is probably driven by tail tokens that never win. Would calibrating ε only on the top competing tokens give a certified coverage that is not close to zero?

Reviewer_WeGbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focused on the fundamental bottleneck of Finite Sampling which cannot certify the absence of unobserved output cells. To dress this problem, the authors proposed Logit-Path Tracing approach which recursively constructs these regions and verifies their winners against the full vocabulary. Because pairwise logit gaps are affine, vertex verification certifies each polyhedral cell, yielding the complete greedy decoding partition under exact arithmetic. For finite-precision logits, they extend LPT with ε-robust certification, using a pairwise-gap error tolerance to separate certified and unresolved regions. They evaluated three base models on 200 test prompts over one- and two-dimensional control domains, with the pairwise-gap tolerance calibrated on a separate set of 100 prompts and show consistency improvement.

Critical strengths
  1. The formulation is clear and the proposed approach is simple and effective.
  2. The experiments show consistent performance improvements.
Critical weaknesses
  1. The selected foundation models are too weak and old. For example, the maximum model size is 3B. Therefore, it is unclear whether the proposed approach can work well on stronger and latest foundation models.

  2. The proposed approach seems to be difficult to scale up to long-form generation setting. I noticed the proposed approach will slow down the evaluation process as indicated by the Table 1. Meanwhile, currently the experimental settings only considers 64 tokens. So it is unclear whether the proposed approach can work on more long-form generation settings. Whether the evaluation cost will be unaffordable.

Questions for authors

See above

1 comment
AuthorsOpenReview ↗

Response to Reviewer WeGb: Larger Models, Longer Generations, and Computational Cost

Thank you for the review and for recognizing the clarity and simplicity of our formulation. We agree that model scale and generation length are important for assessing the practical applicability of LPT, and we address both points below.

Larger models. As the reviewer pointed out, our original experiments used models of at most 3B parameters. We therefore extended the evaluation to Qwen3.5-9B with d = 2 and T ∈ {64, 128}, evaluating 10 prompts at each generation length. All 20 runs completed. The table compares the number of distinct greedy outputs identified by LPT, uniform sampling, and disagreement-adaptive sampling under matched execution-time budgets.

TLPT OutputsLPT Time (min)Uniform OutputsUniform Time (min)Adaptive OutputsAdaptive Time (min)
6451.34.9223.14.9224.24.92
128110.824.6442.224.6446.924.64

Results are averaged over 10 prompts per setting. Output counts and sampling recall use positive-measure LPT output supports reachable under lowest-token-ID tie-breaking (51.5 and 112.4 supports at T = 64 and T = 128 when all maximizing branches are included). LPT runtime includes tracing all maximizing branches.

At T = 128, LPT identified 2.63× and 2.36× more distinct greedy outputs than uniform and adaptive sampling, respectively, within the same execution-time budget. Averaged per prompt, uniform and adaptive sampling recovered 42.55% and 46.57% of the greedy outputs identified by LPT, respectively, and adaptive sampling covered 83.80% of the control domain. Peak CUDA allocation for LPT was 16.86 GiB.

Longer generations and computational cost. As the reviewer noted, our original experiments were limited to 64 tokens. We therefore extended the evaluation to longer generation horizons (T = 128 and T = 256) using Qwen2.5-3B and SmolLM2-1.7B. The table below uses d = 2, FP32, and the same 10 prompts per model and horizon. Uniform and adaptive sampling were given per-prompt wall-clock time budgets matched to LPT, with no limit on the number of sampled control values.

ModelTLPT OutputsMatched Time Budget (min)Uniform RecallAdaptive Recall
Qwen2.5-3B128158.610.8637.88%42.83%
Qwen2.5-3B256242.937.8635.82%41.13%
SmolLM2-1.7B12854.32.3776.54%78.85%
SmolLM2-1.7B25686.29.5361.21%66.77%

LPT Outputs denotes the mean number of distinct greedy outputs over positive-measure control regions. Recall is averaged per prompt relative to these outputs. Counts and recall use positive-measure supports reachable under lowest-token-ID tie-breaking; these counts coincide with the all-maximizer support counts in the evaluated long-horizon settings. The matched time budget is the mean LPT execution time, with the same per-prompt budget assigned to both sampling baselines.

At T = 256, LPT identified an average of 242.9 distinct greedy outputs on Qwen2.5-3B, while uniform and adaptive sampling achieved 35.82% and 41.13% recall, respectively. On SmolLM2-1.7B, LPT identified 86.2 outputs, compared with 61.21% and 66.77% recall. From T = 128 to T = 256, the recall of both baselines decreases for both models, so the gap between LPT and time-matched sampling widens as the generation horizon grows.

We agree that LPT adds computational cost, as reflected in Table 1, and that this cost grows with the generation horizon. Regarding cost, the matched budgets above are the mean LPT runtimes: at T = 256, 37.86 minutes per prompt for Qwen2.5-3B and 9.53 minutes for SmolLM2-1.7B, with a longest run of 119.51 minutes (Qwen2.5-3B). LPT is an offline analysis rather than a change to inference-time decoding: for a fixed prompt and decoding configuration, its stored output supports can be reused for subsequent control-space queries. In total, the full scalability evaluation comprises 140 runs across one- and two-dimensional control settings. Here, T denotes the maximum generation length, with EOS stopping. These results are now included in the revised PDF (Section 6.3, Table 2; Appendix F.2, Tables 10–12).

We hope these results address your concerns, and we would be happy to clarify any remaining questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Scalability, Stronger Baselines, Precision, and Independent Verification

We thank all reviewers for their careful and constructive feedback. We conducted additional experiments and revisited the numerical evaluation.

Scalability. We completed 140 runs, extending the evaluation to (1) a larger model, Qwen3.5-9B with d = 2 and T ∈ {64, 128}, and (2) longer generations, up to T = 256 with Qwen2.5-3B and SmolLM2-1.7B. Under per-prompt time budgets matched to LPT at T = 256, uniform and adaptive sampling recovered only 35.82–66.77% of LPT's outputs.

ModelTLPT OutputsMatched Time Budget (min)Uniform RecallAdaptive Recall
Qwen2.5-3B128158.610.8637.88%42.83%
Qwen2.5-3B256242.937.8635.82%41.13%
SmolLM2-1.7B12854.32.3776.54%78.85%
SmolLM2-1.7B25686.29.5361.21%66.77%

LPT Outputs denotes the mean number of distinct greedy outputs over positive-measure control regions. Recall is averaged per prompt relative to these outputs. Counts and recall use positive-measure supports reachable under lowest-token-ID tie-breaking; these counts coincide with the all-maximizer support counts in the evaluated long-horizon settings. The matched time budget is the mean LPT execution time, with the same per-prompt budget assigned to both sampling baselines.

Sampling. We strengthened the baselines with (1) prefix-cached uniform sampling and (2) disagreement-adaptive sampling, under matched model-evaluation budgets. On the same 20 prompts (d = 2, T = 64), adaptive sampling recovered 60.43% of distinct greedy outputs for Qwen2.5-3B and 99.07% for SmolLM2-1.7B. Qwen2.5-3B produced an average of 77.0 distinct greedy outputs over positive-measure control regions, compared with 28.8 for SmolLM2-1.7B.

ModelLPT OutputsUniform RecallAdaptive RecallLPT Time (s)Uniform Time (s)Adaptive Time (s)
Qwen2.5-3B77.057.63%60.43%127.6120.5118.5
SmolLM2-1.7B28.899.07%99.07%32.31,183.263.3

Results are averaged over 20 prompts per model. LPT Outputs denotes the average number of distinct greedy outputs over positive-measure control regions. Recall is averaged per prompt. Times include the measured sampling and processing costs. Counts report positive-measure output supports returned by all-maximizer LPT, not disjoint partition atoms.

Precision and verification. We addressed (1) numerical consistency across precisions and (2) independent verification of LPT's completeness. We identified an error in Table 9 of the original submission, which incorrectly reported 100% agreement between FP32 and BF16. At 1,540 representative control parameter values, FP32 reproduced all 1,540 greedy output sequences, while BF16 reproduced 479 (31.10%). Here, L denotes the outputs identified by LPT and S = U ∪ A those identified by uniform (U) or disagreement-adaptive (A) sampling.

Output SetOutputsFP32 MatchBF16 Match
L ∩ S969100%39.42%
L − S571100%16.99%
L (All)1,540100%31.10%

Match indicates identical complete greedy output sequences at the same representative control parameter values. We evaluate one maximum-margin representative control from each of the 1,540 positive-measure FP32 reference supports. Both FP32 and BF16 point decoders use lowest-token-ID tie-breaking.

For independent verification, we implemented a separate verifier that constructs token-winner regions using exact rational halfspace intersections and full-vocabulary checks at region vertices. Using affine token logits obtained from Qwen2.5-3B, we evaluated three prompts in each of the one- and two-dimensional control settings. The verifier recovered exactly the same 436 greedy outputs over positive-measure control regions as LPT, with no missing or additional outputs.

Control DimensionPromptsMatched Greedy OutputsDisagreements
d = 13360
d = 234000
Total64360

Disagreements indicate missing or additional greedy outputs, or differences in their region dimensions, between LPT and the independent verification.

We have uploaded a revised PDF incorporating these results: prefix-cached baselines and scaling (Section 6.3; Appendix F.2), exact-arithmetic verification (Appendix D), tie semantics and counting rules (Section 3; Appendices E.1 and F.2), and the corrected cross-precision analysis replacing the original Table 9 (Section 6.5; Appendix F.6). Section 5 and Appendix C clarify the scope of numerical certification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.