- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the partition of a continuous control domain induced by greedy language-model decoding when the token logits are affine in the control parameters. It proposes Logit-Path Tracing (LPT), which constructs candidate upper-envelope cells, verifies their winners against the full vocabulary at the cell vertices, and recursively propagates all verified token branches. Under exact arithmetic, the paper claims recovery of the complete greedy-decoding partition. It further introduces an ε-robust variant based on an assumed uniform bound on pairwise-logit-gap errors. Experiments cover three 1.5B–3B models, 200 test prompts, and one- and two-dimensional control domains.
Critical strengths
-
Finite sampling can discover outputs but cannot certify the absence of an output between sampled parameter values. This distinction is relevant to controllable decoding and model auditing.
-
For a fixed prefix, affine token logits induce polyhedral winner regions, and full-cell verification reduces to evaluating affine gaps at vertices.
Critical weaknesses
-
Theorem 1 assumes exact arithmetic, whereas the experiments use fp32 model logits, float64 geometry, and fixed numerical tolerances. There is no interval arithmetic, exact rational geometry, directed rounding, or final certified verification showing that narrow cells cannot be lost. Dense-grid agreement is not an exactness proof. The paper should sharply distinguish the exact-arithmetic theorem, the partition of stored numerical affine functions, and the behavior of the executed model.
-
The theorem (Eq. 10) requires a uniform pairwise-gap error bound over every relevant parameter value, prefix, and token pair. The maximum discrepancy observed on 100 calibration prompts under bf16 versus fp32 does not provide such a bound for unseen prompts, prefixes, kernels, or hardware. The theoretical certificate is valid conditionally, but the experimental should be described as an empirical stress-test tolerance rather than a formal finite-precision certificate.
-
The theory retains lower-dimensional tie cells and propagates every maximizing branch. The reported one-dimensional implementation breaks ties deterministically, while the two-dimensional implementation does not export zero-area tie atoms. Zero-measure sets are irrelevant to area coverage but not to the motivating question of whether any control value produces a target output. The paper needs one consistent tie semantics and experimental validation of the corresponding implementation.
-
Cell recall and coverage are measured relative to the LPT partition. Dense grids provide finite-point consistency checks, while candidate-set invariance only tests stability within the same implementation. Independent exhaustive or symbolic validation on tractable autoregressive instances is needed.
-
The paper acknowledges that the point-query samplers do not use a prefix trie, whereas prefix caching is central to LPT’s efficiency. A prefix-cached uniform or adaptive sampler would provide a substantially stronger comparison. Runtime should also be decomposed into model inference, caching, and geometric processing, with peak memory reported.
-
Upper envelopes of affine functions and vertex verification of affine inequalities are standard tools in parametric optimization and computational geometry. The main contribution is their recursive application to autoregressive decoding. The paper should more precisely isolate the novel algorithmic component relative to MERT-style envelope methods, parametric analysis, and existing continuous LM interpolation work.
Questions for authors
refer to weaknesses
6 comments
Response to Reviewer yLqG (2/4): Tie Handling and Consistency
(3) Tie handling and consistency
To validate the handling of lower-dimensional tie cells and every maximizing branch, we conducted a separate verification using exact rational arithmetic and common refinement on small autoregressive test instances.
We constructed eight test instances with V = 3, T = 2, and d ∈ {1, 2}, varying random seeds and whether exact ties were imposed between root-token logit functions.
The verification was conducted as follows:
-
Tie construction: Construct affine token logits with and without imposed ties.
-
Greedy output enumeration: Enumerate all 3² = 9 possible token sequences and construct their greedy-winner inequalities, retaining every maximizing branch.
-
Common refinement: Subdivide the control domain into disjoint partition regions and boundaries, including lower-dimensional tie cells and zero-area tie atoms, using exact rational arithmetic.
-
Independent verification: Compute all possible greedy outputs at a representative point within each partition region or boundary using direct all-maximizer greedy decoding, and compare the resulting output sets.
The results are summarized below.
| d | Seed | Imposed Tie | Regions & Boundaries | Disagreements |
|---|---|---|---|---|
| 1 | 0 | No | 7 | 0 |
| 1 | 0 | Yes | 7 | 0 |
| 1 | 1 | No | 5 | 0 |
| 1 | 1 | Yes | 5 | 0 |
| 2 | 0 | No | 23 | 0 |
| 2 | 0 | Yes | 23 | 0 |
| 2 | 1 | No | 27 | 0 |
| 2 | 1 | Yes | 21 | 0 |
| Total | — | 8 instances | 118 | 0 |
Imposed Tie indicates whether two tokens at the initial decoding step are assigned identical affine logit coefficients. Disagreements count differences in the possible greedy output sets between common refinement and direct all-maximizer greedy decoding.
Across the eight test instances, all 118 partition regions and boundaries were correctly labeled, comprising 46 zero-dimensional points, 55 one-dimensional intervals or edges, and 17 two-dimensional regions. Among these, 62 admitted multiple greedy outputs, and all output sets matched direct all-maximizer greedy decoding with zero disagreements.
These results validate the handling of every maximizing branch, including lower-dimensional tie cells and zero-area tie atoms, in the tested exact-arithmetic instances. The revised PDF states one tie semantics explicitly (Section 3; Appendix E.1). The theory and LPT's traversal follow all-maximizer semantics, and the implementation already traces every maximizing branch. Comparisons with sampling additionally apply a deterministic lowest-token-ID rule, since a sampler decodes a single output per control value. Because persistent ties can occur on positive-measure regions, the two counts can differ (e.g., 51.5 vs. 51.3 supports for Qwen3.5-9B at T = 64); we report both (Appendix E.1, Table 7) and have corrected the appendix statement that they differ only on lower-dimensional ties. We also distinguish output supports, which may overlap, from disjoint partition cells (Section 3), and describe the common-refinement construction of such cells validated above on small instances (Appendices D and E.1).
Response to Reviewer yLqG (3/4): Independent Verification of Completeness
(4) Independent verification of completeness
To independently validate LPT's completeness, we conducted two verification experiments using exact rational arithmetic: (a) exhaustive verification on small autoregressive models and (b) independent verification using affine token logits obtained from a real language model.
(a) Exhaustive verification. We constructed 80 autoregressive test instances with V = 3, T = 3, and d ∈ {1, 2}, including instances with exact ties.
The verification was conducted as follows:
-
Sequence enumeration: Enumerate all 3³ = 27 possible token sequences for each test instance.
-
Greedy-winner conditions: Construct the affine inequalities requiring each token in a sequence to maximize the logits at its corresponding prefix.
-
Exact feasibility verification: Use exact rational arithmetic to determine whether each sequence is feasible for any control parameter value, including lower-dimensional tie cells.
-
Independent comparison: Compare the feasible greedy outputs and their region dimensions against LPT under two candidate-set configurations.
The results are summarized below.
| Setting | Instances × Configurations | Comparisons | Disagreements |
|---|---|---|---|
| V = 3, T = 3, d ∈ {1, 2} | 80 × 2 | 160 | 0 |
Disagreements indicate missing or additional greedy outputs, or differences in their region dimensions, between LPT and the independent verification.
Across 80 test instances and two candidate-set configurations, all 160 comparisons matched, confirming that LPT recovered every feasible greedy output among all 27 possible sequences, including outputs restricted to lower-dimensional tie cells.
(b) Verification using real-LM logits. We additionally verified LPT using the stored full-vocabulary affine token logits obtained from Qwen2.5-3B, with the independent exact-rational verifier described in (1).
As reported in the table in (1), the verifier independently reproduced all 436 greedy outputs over positive-measure control regions across three prompts in each of the one- and two-dimensional control settings (T = 64), with zero disagreements.
These results provide independent evidence of complete greedy output recovery for the tested affine decoding systems. The revised PDF reports this verification in Appendix D (Tables 4–5).
Response to Reviewer yLqG (1/4): Exact Arithmetic and Robustness Certification
Thank you for the review and for the detailed comments on numerical certification, tie handling, completeness verification, and sampling baselines.
(1) Exact arithmetic and numerical implementation
To verify the numerical correctness and completeness of LPT, we developed an independent verifier based on exact rational arithmetic, using stored full-vocabulary affine token logits obtained from Qwen2.5-3B.
The verification was conducted as follows:
Exact coefficient representation: We converted the stored binary floating-point coefficients into mathematically equivalent rational numbers using their mantissas and powers-of-two exponents. This preserves the stored coefficient values exactly, without introducing additional rounding errors.
Region construction: We computed token-winner regions using exact rational halfspace intersections, eliminating floating-point errors in the geometric calculations.
Full-vocabulary verification: We verified candidate winners against the entire vocabulary at each region's vertices using exact rational comparisons.
Recursive decoding: We recursively expanded all valid token branches to independently reconstruct the complete set of possible greedy outputs.
Result comparison: We compared the reconstructed greedy output sequences, region dimensions, and vertices against the original LPT implementation.
We evaluated three prompts in each of the one- and two-dimensional control settings (T = 64). The results are summarized below.
Disagreements indicate missing or additional greedy outputs, or differences in their region dimensions, between LPT and the independent verification.
The independent verifier recovered all 436 greedy outputs identified by LPT, with no missing or additional outputs or region-dimension differences. The maximum discrepancy between the computed region vertices was 7.08 × 10⁻¹⁶.
By preserving the original stored affine coefficients exactly and performing the geometric verification with rational arithmetic, this experiment directly tests whether LPT's floating-point geometry misses or introduces greedy output regions. The complete agreement provides independent evidence of numerical correctness and completeness for the tested fixed affine decoding systems.
Following the reviewer's suggestion, the revised PDF distinguishes three levels explicitly (Section 5): (i) Theorem 1 guarantees completeness for exact affine logit functions under exact arithmetic; (ii) the stored numerical affine system: LPT's floating-point geometry recovers exactly the same greedy outputs as exact rational computation on the tested systems, as shown above; and (iii) the executed model: a fresh FP32 execution reproduced all 1,540 representative outputs, whereas BF16 reproduced 31.10%, so agreement under a different numerical precision is implied by neither (i) nor (ii) (see our response to Reviewer hn5J).
(2) Robustness certification
We agree with the reviewer that the calibrated tolerance is empirical rather than a formal finite-precision bound. Eq. (10) requires a uniform bound on pairwise logit-gap errors across the relevant controls, prefixes, and token pairs. The BF16–FP32 discrepancy measured on calibration prompts provides an empirical tolerance, rather than such a uniform bound.
At this tolerance, only 0–0.5% of the control domain satisfies the robustness margin criterion. In the revised PDF, we clarify the assumption underlying Eq. (10) in Section 5 and present the results in Figure 4 and Table 13 (Table 6 in the original submission) as empirical robustness analyses (Section 6.5; Appendix C). As reported in our response to Reviewer hn5J, the stability radii of the recovered outputs (medians of 0.0108 and 0.0463 logits for outputs found only by LPT and for those also found by sampling) are far below this tolerance, which explains the low certified coverage; LPT reports these margins for every output.