acceptodds
Under review as a conference paper at ICLR 2027

Time Series Compression with Foundation Models

Abstract

Time-series volumes continue to grow across domains, making efficient lossless compression essential for storage and transmission. Classical codecs and specialized numerical compressors often miss nonlinear or long-range temporal structure, while neural compressors that learn such structure typically demand per-dataset training and extra model storage. Large pre-trained foundation models offer strong sequence modeling with zero-shot transfer, suggesting a route to compression without task-specific fine-tuning. This paper connects next-token and next-value prediction to encoding-based compression and presents a unified zero-shot framework for numerical time series: LLMs drive arithmetic coding from predictive token distributions, and LTSMs drive residual coding from point forecasts. Lightweight numeric tokenization schemes are further studied without modifying frozen tokenizers. Extensive experiments show that publicly available foundation models can be applied directly; in particular, pre-trained LLMs attain higher compression ratios than the compared general-purpose, numerical, and LTSM-based compressors on most datasets, under a shared public-checkpoint accounting.

Primary area:
learning on time series and dynamical systems
Keywords:
time series, data compression, foundation model

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_i2ymOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This work investigates the lossless compression of numerical time series data based on large pre-trained foundational models. It aims to leverage the strength of such large pre-trained foundational models in realizing the zero-shot transfer of such compression capacity.

The main contribution is the framework which has bi-lateral branches for LLM probability coding and TSFM residual coding. The effectiveness of this framework is justified via integer and floating-point time series without dataset specific fine-tuning. The key idea is applying the next-token prediction into helping the time series forecasting. The validation of this presented framework is done based on 6 time series datasets.

Critical strengths

To me, I think the clear problem description and the motivation of studying this problem is one of the main strength. The bi-lateral framework for leveraging the large pre-trained foundational model is also clearly described.

Critical weaknesses

The most critical weakness is that the key idea connecting LLMs to data compression is no longer new. The previous study, e.g., Language Modeling is compression, the LLMZip etc., already present similar ideas. The difference in terms of applying tokenization into time series numerical data is thus incremental compared with existing ones.

Another critical weakness is that the theoretical part is a largely restatement of those already presented in previous studies.

The methodological development is majorly a simple bringing of existing concepts to a specific targeted problem without sufficient original development done by authors.

The validation is also insufficient due to the dataset coverage and baseline coverage.

Questions for authors

Q1. In Table 1, the time series foundational model based compressor seems quite ineffective, with compression ratios slightly higher than 1 in many cases. Does it indicate that the time series foundational model based compressor will undermine the framework presented?

Q2. The numerical results seem not consistently support the application effectiveness claim. It does not show sufficient convinces that the presented "outperform on most datasets"? Meanwhile, the dataset coverage is also limited.

Q3. Why related works is put after the numerical studies? This content organization is so weird.

2 comments

W

We agree that the link between LLMs and data compression is established in LLMZip [m1] and Language Modeling Is Compression [m2]. Section 2.2 uses the known relation between negative log-likelihood and code length as background. LLMTime [m3] likewise studies numeric tokenization for zero-shot forecasting.

Our empirical question is how well two types of predictions from frozen models translate into compression gains on the same numerical time series without dataset-specific fine-tuning. The LLM path uses next-token probabilities directly in arithmetic coding. The TSFM path uses point forecasts and applies static arithmetic coding to their residuals.

We also compare numeric tokenizations while holding the GPT-2 checkpoint and series fixed. In Table 1, Text-Only gives a higher ratio than Number in all 18 pairwise comparisons across three GPT-2 sizes and six series. For example, on TH-GPS with GPT-2-large, the ratios are 4.013 for Number, 8.875 for Text-Only, and 9.236 for Space-Number.

Q1

We agree that several TSFM compression ratios are only slightly higher than 1. For example, Table 1 gives a best TSFM ratio of 1.111 on ZH-Root. The best TSFM ratio is below LZMA on five of six series. On TH-Climate, the best TSFM achieves 6.052, compared with 4.714 for value-level static arithmetic coding and 10.614 for LZMA. On USGS-Earthquakes, the best TSFM reaches 4.161, compared with 3.924 for LZMA and 4.753 for value-level static arithmetic coding.

We will present the TSFM branch as a comparison within the framework and distinguish its results from the compression gains observed with LLM probability coding. One possible explanation for the ratios near 1 is that small forecast errors can still produce many distinct residual symbols. If their frequencies are spread across those symbols, a static arithmetic coder has fewer opportunities to assign short codes. A follow-up could compare the TSFM against a simple delta predictor using the same residual representation and coder to assess how much the forecasts contribute.

Q2

We will qualify “outperform on most datasets” by specifying both the dataset and baseline coverage. Under shared-checkpoint accounting, the best LLaMA 2 configuration has the highest reported ratio among the methods in Table 1 on five of six series (City-Temp, TH-GPS, ZH-Root, TH-Climate, and Cyber-Vehicle). On City-Temp, one of the two public datasets, the best LLaMA 2 ratio is 7.081, compared with 4.312 for LZMA. On USGS-Earthquakes, the other public dataset, value-level static arithmetic coding achieves 4.753, compared with 4.682 for the best LLaMA 2 entry. This series has a small discrete domain and skewed value frequencies, which can favor static value coding. With four private series in the current evaluation, broader comparisons on public data against predictive codecs would help establish how often the gains carry over.

Q3

Thanks for the suggestion. We will move Related Work before the numerical studies and discuss LLM-based compression, numeric forecasting, and predictive residual coding there. This will make clear which ideas the framework draws on and what the comparisons in Table 1 are intended to assess.

References

[m1] Valmeekam et al. LLMZip: Lossless Text Compression using Large Language Models. arXiv:2306.04050, 2023.

[m2] Delétang et al. Language Modeling Is Compression. ICLR, 2024.

[m3] Gruver et al. Large Language Models Are Zero-Shot Time Series Forecasters. NeurIPS, 2023.

Reviewer_i2ymOpenReview ↗

Response to Authors' Response

Thank you very much for your efforts in responding my pretty harsh comments. Indeed, time-series partnering with LLMs is flooding with too many staying at the level of simply taking other concepts and built it with another. Meanwhile, in the era of agentic research, the empirical results are no longer strong enough and we need to see the theory and science behind it. Sorry that I talked too much and hope you don't feel that my previous comments are too offensive.

Honestly, I do have some expectations in terms of technical in-depth and hope my feedback can help you move towards a higher quality work in the future.

Reviewer_5tUxOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper explores time series compression using pretrained foundation models without dataset-specific fine-tuning. It presents two approaches: serializing numerical values into text and using an LLM’s next-token probabilities for arithmetic coding, and using a time series foundation model to predict values and then encoding the residuals. The evaluation covers six integer and floating-point datasets, comparing against Lzma, Gorilla, and static arithmetic coding. The authors also examine numerical representations, context length, prediction length, and batch size. Results show that appropriately configured LLMs achieve higher compression ratios on most evaluated datasets.

Critical strengths

S1: The paper brings LLM probability coding and TSFM residual coding into a common zero-shot framework without dataset-specific fine-tuning. This provides a useful perspective for comparing the compression capabilities of different foundation models, extending the study’s relevance beyond a single architecture.

S2: Appropriately configured LLMs achieve higher compression ratios than the evaluated baselines on most tested datasets. The analysis of numerical tokenization and runtime settings also helps explain the factors affecting performance. Despite limitations in baseline coverage and throughput, these results provide preliminary empirical support for using foundation models in time series compression.

Critical weaknesses

W1: Important compression baselines are missing from the experimental comparison, including ALP, Chimp, Elf, DeXOR, and Sprintz. Comparisons of compression ratios and compression/decompression speeds are needed to assess whether foundation models’ benefits justify their computational cost.

W2: Important dataset details are missing, such as decimal precision and numerical ranges. Evaluation on public datasets with higher precision, greater fluctuations, and larger sizes would strengthen the evidence for robustness and scalability.

Questions for authors

Q1: If residual computation and reconstruction use standard floating-point arithmetic throughout, exact recovery is not guaranteed in some cases. For example,setting x = 0.123456789, y = 123456789, and z = x - y yield y + z ≠ x. Please explain how the method handles such cases and clarify the numerical representations used and the conditions guaranteeing exact recovery.

Q2: Several discrepancies and implementation details require clarification: (i) Please define“MB/D”in Figure 5 and explain how this throughput measure relates to the points/s reported in Table 1. (ii) Please specify the criteria for selecting“good”tokens in Section 3.1.1, whether the restriction is applied during input tokenization or output probability processing, and whether the resulting probabilities are renormalized.

2 comments

W1

Thanks for suggesting these compression baselines. We will discuss ALP [m1], Chimp [m2], Elf [m3], and DeXOR [m4] for the floating-point series, and Sprintz [m5] for integer series at supported widths. Table 1 gives comparisons with LZMA, Gorilla, and character- and value-level static arithmetic coding. A follow-up could compare compression ratios and compression and decompression speeds on the same observations and with the same reconstruction target. This would help assess whether the storage savings justify the computational cost.

W2

We use one column per dataset. As reported in Table 2, City-Temp has 100,000 floating-point values and USGS-Earthquakes has 75,809 integer values. We will make this scope explicit when discussing robustness and scalability. A follow-up could report decimal precision, numerical ranges, and the spread of consecutive differences, and use these statistics to select larger public datasets with higher precision and greater fluctuations.

Q1

Thanks for the example. Section 3.2.3 forms the residual z = x - y. For x = 0.123456789 and y = 123456789, standard floating-point arithmetic can discard low-order bits in z, so y + z can differ from x. We will clarify the conditions for exact recovery. For values on a known decimal grid, one could use scaled integers, derive the same integer forecast on both sides, and use an integer type wide enough to avoid overflow. For binary floats, one could encode the XOR of the value and forecast bit patterns, then recover the value using the same forecast bits. These are possible exact representations for the residual path.

Q2(i)

MB/D means megabytes per day. Figure 5 uses the original, uncompressed data size. For each run, we divide the original size in MB by the elapsed time in seconds and multiply by 86,400. Equivalently, for the same run, we can multiply points/s by the original size in MB per point and by 86,400. Figure 5 varies the settings on TH-Climate, while Table 1 reports average points/s. We will state the conversion in the caption.

Q2(ii)

In Number, the “good” tokens are existing GPT-2 tokens for single digits and punctuation needed to write and separate numbers. The restriction applies during both input tokenization and output probability processing. For instance, the input 364 is split into 3, 6, and 4 (Figure 3(b)). For arithmetic coding, the distribution over the same set is renormalized by dividing each retained probability by their sum. We will clarify the selection criteria and both uses in Section 3.1.1.

References

[m1] Afroozeh et al. ALP: Adaptive Lossless Floating-Point Compression. Proc. ACM Manag. Data, 2023.

[m2] Liakos et al. Chimp: Efficient Lossless Floating Point Compression for Time Series Databases. PVLDB, 2022.

[m3] Li et al. Elf: Erasing-based Lossless Floating-Point Compression. PVLDB, 2023.

[m4] Lv et al. DeXOR: Enabling XOR in Decimal Space for Streaming Lossless Compression of Floating-point Data. PVLDB, 2026.

[m5] Blalock et al. Sprintz: Time Series Compression for the Internet of Things. Proc. ACM IMWUT, 2018.

Reviewer_5tUxOpenReview ↗

Response to Author's Rebuttal

The response clarifies several implementation details, particularly the throughput definition and token selection strategy. However, my major concerns remain only partially addressed, as the authors' responses largely consist of promises to clarify, report, or discuss these issues (W1, W2, and Q1) in the revised manuscript, without providing additional experimental results, concrete data, or sufficient technical justification to substantiate their claims. Overall, I hope these comments will strengthen this manuscript in the future.

Reviewer_A7rLOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies zero-shot lossless compression of univariate numerical time series with frozen foundation models. In the LLM path, each series is serialized as decimal text and GPT-2 or LLaMA 2 next-token probabilities drive arithmetic coding; the authors compare Text-Only, Number, and Space-Number numeric representations without modifying tokenizers. In the TSFM path, TimesFM, Timer-XL, and Chronos2 produce point/patch forecasts whose residuals are encoded using static arithmetic coding. The theoretical discussion recalls the established relationship between predictive negative log-likelihood and arithmetic-coding length. The evaluation covers six integer or floating-point series (four private) and compares with LZMA, Gorilla, and static arithmetic coding. The authors report that LLaMA 2 variants attain the best compression ratios on five of six datasets (Table 1), although throughput is orders of magnitude below conventional codecs. They propose an archival setting in which the checkpoint is shared and its size is excluded from per-file storage.

Critical strengths
  1. Promising zero-shot empirical result under an explicit deployment assumption. The work applies a shared, frozen LLM as a probability model for numerical coding without per-dataset retraining. The best LLaMA 2 results exceed the reported conventional baselines on five of the six datasets (Table 1). This is a practically relevant proof of concept for an archival setting in which model weights can be amortized across many files.
  2. Useful study of numeric serialization. The comparison of Text-Only, Number, and Space-Number preprocessing across GPT-2 scales shows that token boundaries materially affect coding efficiency, with Text-Only generally outperforming forced digit-level splitting (Table 1). This is an actionable observation for reusing frozen language models on numeric sequences.
Critical weaknesses
  1. Bit-exact losslessness and decoder reproducibility are not established. Sections 3.1.3 and 3.2.3 provide a conceptual encoding description, but not an end-to-end, decodable specification of the file format, finite-precision probability/CDF representation, decoder termination, and inference synchronization across model versions or hardware. For floating-point values, encoding "r = x - " "x" ̂ and recovering "x" ̂" + r" in ordinary finite-precision arithmetic does not universally ensure bit-for-bit reconstruction; an exact residual representation or correction mechanism must be specified. Moreover, static arithmetic coding estimates full-file symbol frequencies; the decoder needs those frequencies or an explicitly shared model, and any per-file side information should count toward compressed size. The absence of reported byte-/bit-exact round-trip tests directly affects the headline lossless claim.
  2. Limited baselines and cost-aware validation weaken the significance claim. Although Section 5 discusses Sprintz and TS2DIFF, Table 1 contains only LZMA, Gorilla, and simple static arithmetic coding as conventional baselines; stronger predictive/delta-based time-series codecs are missing. Four of six datasets are private and two have fewer than 6,000 points (Appendix C, Table 2). More importantly, LLaMA-2-7B processes only 6.767 points/s versus "3.314 × " 〖"10" 〗^"5" points/s for LZMA (about "49,000×" slower, Table 1); Appendix C.4 assumes decoding throughput equals encoding throughput instead of measuring it. Excluding a 13–25 GB LLaMA checkpoint is reasonable only under the stated shared-checkpoint scenario, for which the computation, memory, service, and storage-amortization costs are not quantified. Since LM-based entropy coding and forecast-residual coding are established ideas, stronger end-to-end evidence is needed to establish the significance of this particular application.
Questions for authors
  1. Exact reconstruction and reproducibility (clarification essential to soundness). What guarantees bit-exact decoding for both the text/LLM and floating-point TSFM paths? Please specify numeric serialization, residual arithmetic or correction, probability quantization, checkpoint/backend consistency, the final bitstream format, and termination. Please report exact encode–decode equality checks (including float bit patterns) on the evaluated datasets.
  2. Full compressed-size accounting (clarification essential to the reported ratios). Do all reported ratios include file headers, initial uncompressed prefix, sequence length, static-AC frequency tables/residual alphabets, and other decoder-required side information? Are original and compressed sizes measured from the same source representation (textual decimal values versus binary numerical arrays)?
  3. Strength of comparison and realistic deployment cost (evidence that could change the recommendation). Can you compare on the two public datasets against at least one strong predictive time-series codec (e.g., Sprintz, TS2DIFF, or a delta-plus-entropy pipeline), and provide measured end-to-end compression/decompression time and memory use? Under what dataset volume or reuse level do the compression savings amortize a shared LLaMA checkpoint and its inference cost? Please also clarify the reproducibility status of the four private datasets.
1 comment

W1 & Q1

Thanks for raising the question of bit-exact decoding. In the LLM path, numeric serialization produces comma-separated decimal text (Section 3.1.1). A decoder would start from the initial uncompressed prefix (Section 3.1.3). It could recover the same tokens if both sides use the same symbol order and finite-precision cumulative distribution function (CDF) at each step. A reversible token mapping then recovers the text. Recovering the original float bit patterns also requires a bit-preserving formatter and parser.

We will outline a bitstream format with the prefix, a token count for decoder termination, and the coded payload. To reproduce the integer CDFs, both ends could pin the checkpoint revision, tokenizer, backend, and precision and use the same probability quantization rule. Bit-exact round-trip checks would compare source and decoded bytes or float bit patterns on each evaluated series.

For the floating-point TSFM path, Section 3.2.3 forms a residual by subtraction, so exact reconstruction also depends on the residual representation. With fixed-scale decimals, one could use scaled integers and exact addition and subtraction, given the same integer forecast and sufficient integer width. For binary floats, one could code the XOR of the value and forecast bit patterns and invert it using the same forecast bits. We will describe these as possible extensions for bit-exact reconstruction.

Q2

Appendix C.4 specifies that shared model weights are excluded. We will distinguish the reported ratios from full compressed-size accounting, which would count decoder-required side information such as file headers, the initial uncompressed prefix, sequence length or a stopping marker, coder termination, and per-file static-AC frequency tables or residual alphabets. For example, if frequencies are estimated from the full file and sent to the decoder, the table counts toward the compressed size. A direct comparison would use one source representation across codecs, either textual decimal values or binary numerical arrays, and divide its size by the complete archive size.

W2 & Q3

Thanks for suggesting predictive time-series codecs. Sprintz [m1] and TS2DIFF [m2] would be useful comparisons on the two public datasets, City-Temp and USGS-Earthquakes. For reproducibility, we will clarify in Table 2 that reproducing the other four results requires access to the private series (TH-GPS, ZH-Root, TH-Climate, and Cyber-Vehicle). TH-GPS and Cyber-Vehicle contain 3,441 and 5,751 points, respectively.

Table 1 reports 6.767 points/s for LLaMA 2-7B and 331,400 points/s for LZMA, a gap of about 49,000 times. Appendix C.4 assumes equal compression and decompression throughput. A deployment comparison could measure end-to-end compression and decompression time separately and report peak memory use.

For instance, on City-Temp, the reported ratios of 7.064 for LLaMA 2-7B and 4.312 for LZMA imply a saving of about 9.0% (1/4.312 - 1/7.064) if both use the same source bytes. At that rate, roughly 144 GB of similar input would offset one additional 13 GB checkpoint (Table 3), considering storage alone.

As described in Section 1, one frozen model serves many files and adapts its predictions to each file using the preceding tokens. Both ends can download the same public checkpoint revision when needed for encoding or decoding. This lets us exclude model weights from per-archive storage in Table 1. Download and inference still incur costs.

References

[m1] Blalock et al. Sprintz: Time Series Compression for the Internet of Things. Proc. ACM IMWUT, 2018.

[m2] Xiao et al. Time Series Data Encoding for Efficient Storage: A Comparative Analysis in Apache IoTDB. PVLDB, 2022.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.