- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose TEMPO to solve the compounding quantization error in recursive Transformers. They do this by combining a shared INT4 weight backbone with sparse INT2 residual overlays and loop-conditioned scales. They reformulate recursive quantization as a terminal-risk-driven capacity allocation problem and show that precision overhead is concentrated within a sparse subset of coordinates. They empirically evaluate their method on Ouro recursive models under W4A4 and W4A8 settings and show that TEMPO significantly reduces perplexity and recovers zero-shot accuracy compared to baselines.
Critical strengths
- The paper is well written and easy to follow.
- They demonstrate that local layer-wise MSE is decoupled from recurrent performance and introduce an analytical terminal-sensitivity metric to guide error allocation across loops.
- By augmenting a globally shared INT4 backbone with loop-conditioned scales and sparse INT2 residual planes, they avoid the storage cost of duplicating full per-loop weights.
Critical weaknesses
- No real wall-clock speedups. Their code materializes effective weights back into bf16 before computation. So, the real world implications are unknown
- The calibration process requires multiple forward and backward passes, but no total wall clock calibration time and peak memory usage is reported.
- Experiments are conducted on a single model family (Ouro-1.4B and 2.6B). I'm not sure how their method generalizes across model families and benchmarks.
Questions for authors
- What is the total wall-clock time and peak GPU memory footprint required for the calibration?
- How does TEMPO perform on standard dense LLMs like Llama-3 or Qwen used in recursive loops for test-time compute scaling?
- Can you do some comparisons on recursive VLMs and state-space models?