- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
LatentLM adapts a pretrained LLM with LoRA into a compressor, decompressor, and latent-space inference model. It autoregressively represents input text using a variable number of discrete “Z-tokens,” with regularizers intended to control length, maintain contextual consistency, and encourage reuse of latent codes. Experiments cover reconstruction, continuation, RULER, summarization, and long-context QA. The paper also analyzes whether the number of allocated Z-tokens correlates with information-density proxies and studies reuse of compressed contexts across multiple queries.
Critical strengths
-
Learning discrete, variable-length latent codes inside an existing generative model is an interesting interface between context compression and language modeling. Unlike fixed soft-token compressors, the representation can in principle be stored, inspected, and reused.
-
The paper evaluates several aspects beyond reconstruction, including continuation, long-context retrieval, QA, summarization, latent-space inference, allocation behavior, and multi-query amortization. The appendix explicitly reports compression overhead and break-even points, which is valuable.
Critical weaknesses
-
The main efficiency presentation is misleading for ordinary inference. The headline latency and TTFT tables exclude the compression pass while describing the results as end-to-end improvements. The appendix shows that compression takes 25.1–74.6 seconds, compared with savings of only 0.27–1.14 seconds per query, requiring approximately 65–92 queries over the same document to break even. Thus, the proposed method substantially increases latency for single-query and moderately reused contexts, and its practical applicability is much narrower than the main presentation suggests.
-
Reconstruction quality is difficult to interpret because teacher-forced and free-decoding BLEU are not consistently foregrounded. The ablation reports 99.41 teacher-forced BLEU but only 72.39 free-decoding BLEU, raising the possibility that the near-perfect reconstruction numbers in the principal tables mainly reflect teacher forcing. Faithful decompression should be evaluated primarily under autonomous decoding, with clear generation settings and uncertainty estimates.
Questions for authors
-
Are the reconstruction results in Tables 1 and 3 teacher-forced or freely decoded? Please report both throughout, including exact-match or edit-distance statistics and failure rates under autonomous decompression.
-
Can the main efficiency tables include compression, decompression, and downstream generation in one end-to-end measurement? What workloads realistically reuse exactly the same compressed representation for 65–92 or more queries?
-
Does the compressed representation itself require first materializing the full-context KV cache, and what is the peak memory and latency of the compression phase?
-
How sensitive are the downstream results to the autoregressive compression decisions? In particular, what is the variance across compression samples or decoding seeds?