acceptodds
Under review as a conference paper at ICLR 2027

TEMPO: Temporal Error-Modulated Precision Overlays for Recursive Transformer Quantization

11 views

Abstract

Recursive Transformers repeatedly reuse the same parameters across computational loops, yet the resulting quantization errors are neither uniform nor equally consequential across recurrent roles. We introduce TEMPO, a post-training quantization framework that models this asymmetry through Temporal Error-Modulated Precision Overlays. TEMPO retains a shared low-bit weight backbone and augments selected weight tiles with sparse INT2 residual planes whose contributions are controlled by compact temporal codes. Instead of ranking corrections by local reconstruction error, TEMPO estimates how loop-specific perturbations propagate through the recurrent trajectory and influence the final prediction. This terminal-sensitivity signal drives temporal-code assignment and residual selection, while a global budget allocator concentrates additional bits on corrections with the largest expected distortion reduction. The resulting representation separates globally shared quantized weights from lightweight loop-dependent overlays, preserving the parameter efficiency of recursive architectures without duplicating per-loop weights. TEMPO provides a compact, trajectory-aware mechanism for correcting recurrent quantization error and exposes temporal residual structure as a first-class compression primitive.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Recursive Transformers, Post-Training Quantization, Temporal Coding, Low-Bit Quantization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4DW2OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose TEMPO to solve the compounding quantization error in recursive Transformers. They do this by combining a shared INT4 weight backbone with sparse INT2 residual overlays and loop-conditioned scales. They reformulate recursive quantization as a terminal-risk-driven capacity allocation problem and show that precision overhead is concentrated within a sparse subset of coordinates. They empirically evaluate their method on Ouro recursive models under W4A4 and W4A8 settings and show that TEMPO significantly reduces perplexity and recovers zero-shot accuracy compared to baselines.

Critical strengths
  1. The paper is well written and easy to follow.
  2. They demonstrate that local layer-wise MSE is decoupled from recurrent performance and introduce an analytical terminal-sensitivity metric to guide error allocation across loops.
  3. By augmenting a globally shared INT4 backbone with loop-conditioned scales and sparse INT2 residual planes, they avoid the storage cost of duplicating full per-loop weights.
Critical weaknesses
  1. No real wall-clock speedups. Their code materializes effective weights back into bf16 before computation. So, the real world implications are unknown
  2. The calibration process requires multiple forward and backward passes, but no total wall clock calibration time and peak memory usage is reported.
  3. Experiments are conducted on a single model family (Ouro-1.4B and 2.6B). I'm not sure how their method generalizes across model families and benchmarks.
Questions for authors
  1. What is the total wall-clock time and peak GPU memory footprint required for the calibration?
  2. How does TEMPO perform on standard dense LLMs like Llama-3 or Qwen used in recursive loops for test-time compute scaling?
  3. Can you do some comparisons on recursive VLMs and state-space models?
Reviewer_oYazOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TEMPO, a post-training quantization method for recursive Transformers. Because shared weights are applied repeatedly across recurrent loops, TEMPO estimates cross-loop gradient coupling using terminal scalar probes, compresses the resulting directional sensitivity into a tractable tilewise surrogate, and uses this surrogate to guide representation search, shared INT4 code optimization, loop-conditioned scales, and sparse shared INT2 residual overlays. Experiments on Ouro-1.4B and Ouro-2.6B under W4A4 and W4A8 report notable gains over the cited LoopQ results (improving the five-task W4A4 average by roughly 3.21 and 8.18 percentage points, respectively), with residual-free variants also retaining substantial improvements. At the same time, the appendices note that baseline numbers are imported directly from LoopQ and that the current evaluation dequantizes weights into BF16 linear layers rather than executing native low-bit kernels.

Critical strengths
  1. TEMPO directly targets a core challenge of quantizing shared recursive weights: because the same weight perturbation is reused across recurrent steps, equal local quantization errors at different tiles or loops do not have equal impact on the final output. Rather than relying on an ad hoc post-quantization saliency heuristic, the method connects a single tractable surrogate to gauge/clipping search, code/scale fitting, and residual allocation. In addition, Appendix A clearly states the isotropic projection and omitted cross-block terms, while Appendix B properly scopes the top-K optimality guarantee to fixed, equal-cost candidates under the additive surrogate.

  2. The reported accuracy improvements are sizable, particularly on Ouro-2.6B under W4A4. Importantly, the residual-free control still improves the five-task average from 58.132% (as cited from LoopQ) to 65.718%, showing that the gains do not come solely from extra INT2 residual capacity. The ablations over calibration sequence count, random seeds, and residual density—together with the candid distinction in the appendix between packed code size, full model-state size, and BF16-cached runtime memory—are also helpful.

Critical weaknesses
  1. The main baseline comparisons are not conducted under a matched evaluation protocol and storage budget. Table 1 imports baseline numbers (including LoopQ) directly from prior reports rather than re-evaluating them within the TEMPO setup. Notably, LoopQ [1] uses Pile calibration data and group size 32, whereas TEMPO uses C4 and an input-group size of 128 (Appendix D.1–D.2). Moreover, Appendix D.5 notes that full evaluation configurations for LAMBADA accuracy, MMLU, and the perplexity benchmarks are missing from the bundled evaluation scripts, even though these tasks drive a large portion of the reported gains. On the storage side, the default overlay adds 2 * rho = 0.5 bits per weight on top of INT4 codes (4.5 code bits before scales, indices, transforms, and unquantized layers), and the footprint comparison with LoopQ relies on assumptions about how LoopQ reports model size in GB. Finally, because the implementation materializes and caches BF16 weights at runtime (Appendix D.7), the results validate simulated quantization accuracy and serialized size rather than actual low-bit inference speed or memory savings. A controlled comparison under an identical calibration/evaluation pipeline and explicit byte-level storage accounting is needed to put the gains on firm footing.

  2. The ablation studies do not cleanly isolate the paper's two main technical components: off-diagonal cross-loop coupling and risk-ranked INT2 overlay placement. First, replacing the coupling matrix C with the identity matrix removes both step-wise diagonal sensitivity and off-diagonal cross-loop interactions at once, so it does not test whether off-diagonal coupling improves over the diagonal approximation diag(C). Given that Figure 2 already shows very high rank correlation between diagonal and full sensitivity, comparing full C directly against diag(C) is essential. Furthermore, Appendix C notes that the full model does not achieve the highest five-task average across all ablations, yet only highlights selected per-task wins rather than reporting the full ablation table across all metrics. Second, removing the INT2 overlay simultaneously reduces bit capacity, and there is no equal-budget baseline (such as random, uniform, or local-error-based tile allocation) to show that terminal-risk ranking itself drives the overlay's benefit. Finally, describing LoopQ [1] as purely local or uniformly adaptive understates prior work, as LoopQ (Section 4.4) already incorporates trajectory-aware calibration with recurrent hidden-state and final-output supervision. The paper should frame its novelty more precisely around the terminal-probe compression and sparse overlay parameterization, backed by ablations that isolate those choices.

Questions for authors
  1. Can you provide baseline comparisons (especially against LoopQ and residual-free TEMPO) evaluated under the exact same calibration data, group size, and task evaluation harness, alongside a full byte-level breakdown of model storage (codes, scales, overlay indices, transforms, and unquantized parameters)?

  2. Holding calibration data, storage budget, and search budget fixed, how do identity C, diag(C), and full C compare across all five downstream tasks and both perplexity benchmarks? Similarly, under an equal byte budget, how does risk-ranked INT2 overlay placement compare against uniform, random, or local-reconstruction-error placement? Please provide the complete ablation table across all evaluated metrics.

  3. In Figure 2, how is the ground-truth "terminal risk" ordering defined and computed (e.g., via the compressed surrogate, the uncompressed linearized probe, or actual end-to-end output perturbation on held-out data)? In addition, please clarify whether the implementation normalizes the coupling matrix by its maximum entry (as written in the main text) or by its trace (as described in Appendix D.3).

Reviewer_4EevOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed a quantization method called TEMPO for recurrent or recursively applied transformers. It combines a shared W4 weight base with sparse INT2 residual tiles, using loop-conditioned scaling and temporal codes informed by calibration trajectories. The allocation objective is based on a terminal-sensitivity surrogate. Experiments on Ouro 1.4B and 2.6B models report W4A4 and W4A8 results across reasoning and perplexity benchmarks, alongside ablations and sensitivity analyses.

Critical strengths
  1. The motivaiton of this paper is clear
  2. The paper is generally well organized
Critical weaknesses
  1. My first concern is that although the empirical observation motivating the paper is interesting, but the contribution is narrowly scoped: recursive LMs remain a niche architecture, and the evaluation is limited to the Ouro family. Much of TEMPO’s machinery—GPTQ-style initialization, gauge transforms, group-wise quantization, and scale optimization—is inherited from existing LLM PTQ work, with the main novelty being terminal-sensitivity weighting and sparse INT2 allocation. The paper should clarify why recursive LMs require dedicated quantization methods and what the broader takeaway is for the quantization community beyond this architecture family.

  2. My second concern of this paper is that the paper’s central claim—that INT2 residual capacity should be allocated to weight tiles selected by terminal-risk surrogate reduction—is not yet sufficiently validated as a general selection rule. The main evidence is limited to two models from the same Ouro recursive family, with fine-grained ablations mostly on Ouro-1.4B, and the paper lacks a matched-budget, strategy-level comparison of terminal-gain top-K against local-error top-K, random top-K, and uniform allocation. Moreover, since the selected mask is calibration-dependent, the stability of the chosen tiles across calibration seeds/sample counts is not reported. These gaps make it unclear whether the gains come from the terminal-risk selection rule itself or merely from adding sparse INT2 residuals to some subset of tiles; at minimum, a matched-budget strategy comparison and mask-stability analysis should be added.

I am willing to raise the rate if the concern is addressed

Questions for authors

na

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.