- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Byte-Pair Patching (BPP), a controllable dynamic tokenizer for time-series foundation models that combines fixed-size patching with Byte-Pair Encoding (BPE). In the discovery stage, normalized time series are divided into non-overlapping patches, assigned discrete identifiers using K-means, and merged into variable-length motifs through BPE. In the representation stage, these identifiers determine only motif boundaries, while fixed-dimensional token embeddings are constructed from the corresponding continuous observations using patch projection, intra-motif self-attention, pooling, and cross-attention. An occurrence-count threshold controls which merge rules are applied at inference, allowing a single trained checkpoint to operate at different compression levels without retraining. Experiments on GIFT-Eval and fixed-window forecasting across 20 datasets show competitive forecasting accuracy relative to BPE and fixed-patching baselines, with compression reaching 18.54× relative to point-wise tokenization on GIFT-Eval and model-side inference latency reduced by up to 25% compared with fixed patching of size 16, excluding tokenization time.
Critical strengths
- Addressing the token count in time series holds clear practical significance. Since this length directly impacts computational costs, applying necessary compression is highly justified.
- By supporting inference-time compression control without retraining, this method significantly reduces the training cost for practical deployment.
Critical weaknesses
- The absence of comparisons with stronger TSFMs, such as Sundial [1] and Chronos-2 [2], restricts a comprehensive assessment of the method's competitive performance.
- The existing ablations examine representation stage design choices, continuous versus quantized inputs, and compression thresholds, but do not isolate the contribution of BPE-derived motif boundaries under an otherwise identical representation stage.
[1] Sundial: A Family of Highly Capable Time Series Foundation Models
[2] Chronos-2: From Univariate to Universal Forecasting
Questions for authors
- Would it be possible to compare the proposed method with stronger TSFMs like Sundial and Chronos-2, or alternatively report the MASE metric to facilitate a unified evaluation on GIFT-Eval?
- Could the authors compare BPE-derived motif boundaries with fixed-length grouping of the same base patches while keeping the representation stage identical?
- Is it feasible to transfer BPP to other TSFM architectures and evaluate the performance by either full retraining on the identical pre-training corpus or applying a direct fine-tuning approach?
- Could the authors report the inference time comparison with the tokenization process included?
5 comments
Thank you for the thoughtful responses and the effort put into the additional experiments. These help clarify the method's benefits. I have three follow-up questions:
- How is MASE computed and aggregated in Table 3? Does it follow the official GIFT-Eval leaderboard protocol, using the geometric mean of MASE normalized by the seasonal-naive baseline?
- What is the end-to-end inference speedup on Sundial-base with BPP, including tokenization?
- On Sundial, the performance degradation from adding BPP seems noticeably larger than the gains reported in the BPE ablations. Although these are different settings, could you clarify whether the actual inference savings justify this trade-off?
Responding to reviewer 147Q follow-up questions
We thank the reviewer for their response and are pleased that our clarifications and experiments helped clarify the benefits of our method. Below, we answer the reviewer’s follow-up questions.
-
Yes, we follow exactly the official GIFT-Eval leaderboard protocol. For all configurations, we compute MASE using the official splits, prediction horizons, and seasonalities, then divide it by the corresponding MASE of the Seasonal Naive baseline from the GIFT-Eval results repository. We report the geometric mean of these ratios.
-
We measured Sundial’s end-to-end inference latency with a batch size of 1024, following the Sundial evaluation notebook in the GIFT-Eval repository. Compared with Sundial’s original P16 tokenizer, BPP with base patch size 16 reduces total latency by 11.5% and 17.0% at values of and , respectively. Sundial’s flow-matching head accounts for a substantial portion of inference time and takes similar time across all three models. If we exclude this head’s timing, the latency gains are approximately 21% and 31%, respectively.
-
Thank you for raising this point. In the BPE ablation, both models are trained under identical conditions, with only the BPE module removed. BPP-n achieves superior accuracy while only using about half as many tokens. Although the accuracy improvement is modest, it comes alongside substantially greater compression and test-time compression control. These findings support our claim that BPE is necessary for the combination of accuracy, adaptive compression, and test-time flexibility provided by our design.
In the Sundial experiment, BPP is at a disadvantage. Specifically, the pretrained Sundial backbone uses the TimeBench dataset for training with a 16-point fixed-size patching tokenizer at the input layer. We replace this patcher with our BPP tokenizer and fine-tune using the Chronos corpus to align the tokenizer and the backbone. The “Original” baseline in Table 3 of the most recent draft simply tunes the existing backbone and static patching tokenizer on the Chronos corpus. We hypothesize that this mismatch in training corpora and the limited fine-tuning budget result in the accuracy gap.
We resort to this setup only because the Sundial training corpus is not publicly available: Indeed, training a BPP-fused Sundial backbone from scratch with this dataset would be the fairest baseline to compare against the numbers reported by Sundial. Despite this constraint, the adapted BPP-Sundial variant with uses 21% fewer tokens with a modest relative MASE increase of about 0.9%, from 0.854 to 0.862. We will sharpen the text around Table 3 to clarify this.
Addressing reviewer 147Q concerns
We thank the reviewer for reviewing our manuscript. We are grateful that the reviewer finds our main task practically significant and that BPP significantly reduces the training cost of practical deployment. Here are our responses to the reviewer’s concerns:
Weaknesses
Our original experiments use a shared backbone and Chronos-based training pipeline to isolate the effect of tokenization. Comparisons across different architectures can provide useful context, but they are very difficult to use to draw meaningful conclusions about the tokenizer’s contribution, since they use different pretraining corpora, different data preprocessing/normalization pipelines, and none release the exact pretraining code and data behind their public checkpoints.
Nevertheless, we do acknowledge that evaluation with another TSFM would strengthen our results; therefore, we integrated BPP into Sundial-base and fine-tuned both the modified model and the original model under the same protocol. BPP reduces the average input token count from 28.21 to 20.97, a reduction of more than 25%, while MASE changes from 0.8537 to 0.8649 and MSE from 0.6508 to 0.6675. We kindly refer the reviewer to Section 2 of our main comment for further discussion and subsection 4.6 of the revised manuscript for the complete results. We hope this additional evaluation addresses the reviewer’s concern.
To emphasize BPE’s contribution, we added BPP-nobpe, an ablation that removes BPE while keeping the representation module and training setup unchanged. BPP-n achieves lower relative MSE than BPP-nobpe at both scales, improving from 0.7070 to 0.6953 at the tiny scale and from 0.6783 to 0.6725 at the small scale, while increasing compression from 8× to 15.10×. These results show that BPE merging improves the accuracy and compression trade-off. Beyond these gains, BPE also enables test-time compression control in combination with our representation module, a capability lost when BPE is removed. The results, added to Table 2 under BPP-noBPE, are highlighted in blue and summarized in the table below:
Questions
We hope this discussion and the additional experiments address the reviewer’s concerns about our work.