- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper aims to address the limitation that the fixed schedule in line-coupled language models (LCLM) copy a confidently-wrong value in cross-line dependencies. To this end, the authors propose the Lattice Language Model (LATLM), which represents a document as a 2D lattice of token cells (rows = variable-length "units" aligned with sentences / reasoning steps; columns = token positions).
A single 482M checkpoint supports three decoding strategies: AR-ORDER (one line at a time), LCLM-ORDER (first K frontier cells per round), and the new DEFER policy, which commits only frontier cells whose predictive entropy falls below a calibrated threshold and otherwise lets lines wait. Training uses sampled commit trajectories under a 2D causal mask, so every cell is supervised in one forward pass. An append-only cache with query-state reuse keeps "score then commit" at the cost of one re-score.
On cross-line dependence, where fixed schedules are supposed to fail, DEFER reaches 96.5% full-trace accuracy on mixed chains in 28.9 rounds, versus Native LCLM at 78.5% / 25.6 rounds and AR at 100% / 64.8, and critically, LatLM's own LCLM-ORDER on the same weights collapses to 29.0%, isolating the gain to the schedule rather than to a stronger base model.
Critical strengths
1 The lattice is well-defined object. This gives the paper a precise geometric baseline against which the DEFER gate's "selectively hold rows back" behaviour can be interpreted.
2 The ablation is carefully designed and convincing. AR-ORDER, LCLM-ORDER, and DEFER share one set of weights. That makes every ablation a comparison within a single trained model. The "LatLM, LCLM-ORDER = 29.0%" row of Table 3 rules out "the gate wins because the model is better" and isolates the schedule.
3 The method's four design decisions compose into a working system. Representation, residual-stream wiring, training objective, and decoder fallback are co-designed. it is this co-design, not any single component, that lets one checkpoint support three different decoders at near-exact caching cost.
Critical weaknesses
1 Every reported experiment uses one 482M model which means the paper's main claims may not survive scaling. Modern parallel-decoding papers (speculative decoding, diffusion LLMs) typically evaluate at 1B–8B at minimum. The authors acknowledge this, but the acknowledgement does not substitute for the scaling issue.
2 Tasks where DEFER achieves large gains are highly templated / synthetic. Mixed chains, state tracking, branching traces, Records, List batch are all templated row formats. There is no evaluation on natural reasoning (e.g. GSM8K / MATH / HumanEval-style unit segmentation) or on free-form generation with naturally variable sentence lengths.
Questions for authors
Please refer to the weakness section.
Response to Reviewer qA6k
We thank the reviewer for recognizing the lattice formulation and the comparisons between decoding policies on the same checkpoint.
W1. Evaluation beyond 482M parameters
We evaluated one parameter scale, 482M. At this scale, comparing policies on the same checkpoint isolates the effect of changing the commitment schedule. A larger model might infer a missing intermediate value more reliably, which could reduce the benefit of waiting. The relationship between predictive entropy and errors could also change with scale. Our empirical claims are therefore limited to 482M. The size of the scheduling gain at larger scales remains open.
W2. Natural reasoning and free-form generation
Natural reasoning. We chose controlled tasks so that we could identify the exact result a later step needs and observe what happens when the model commits that step before the result is available. The same-checkpoint comparisons show when waiting helps on these tasks. Whether DEFER yields a similar accuracy–round trade-off on natural reasoning benchmarks such as GSM8K remains open.
Free-form generation. We added an evaluation on 200 held-out documents, with 64 separate documents for calibration (Appendix E, Table 7). During sampled generation, LatLM predicts each end-of-line token, so individual row lengths vary within a supplied six-row budget. For each policy, we measured conditional content bits per byte (BPB) on the reference continuations under teacher forcing and timed the generation of sampled continuations, including prompt prefill and scheduling.
All seven configurations were evaluated at batch size one in BF16 with FlashAttention. Throughput accounts for the amount of content each policy actually generated. We retained all requests, including the 11 that reached the cap under sequential LatLM and the 11 under DEFER at q = 0.50.
Sequential LatLM and AR had nearly identical conditional BPB on these documents (0.8734 and 0.8735). DEFER at q = 0.95 delivered 1.23× AR’s content throughput, with BPB rising to 0.8852. The line-start delay policy achieved both lower BPB (0.8823) and higher throughput (1.62× AR) than DEFER at that setting.