- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces LEXIS, an interface that represents time series through patch statistics and compositional shape tokens. It adds 50 shape symbols to Qwen3-4B, initializes their embeddings using axis/level geometry, and trains the model through series–text alignment. The model outperforms models trained with decimal inputs on three time-series understanding benchmarks. Component interventions are used to examine which parts of the representation the model relies on.
Critical strengths
- The representation is compact and decodable: each shape token has a known contribution to the curve. It uses roughly one quarter of the tokens required by decimal serialization.
- The experiments go beyond aggregate accuracy. Masking and shuffling show that the model uses shape information and temporal order. Against the trained decimal baseline, LEXIS gains 3.36, 2.41, and 2.91 percentage points on the three benchmarks.
Critical weaknesses
- The initialization ablation changes both geometry and scale. New embeddings are placed at the typical vocabulary radius, while the baseline starts near the mean. As Appendix A.4 explains, this also changes their initial accessibility to the output head. Table 3 therefore leaves open whether the gain comes from meaningful symbol geometry or simply better-scaled initialization.
- The training comparison needs clearer accounting. Appendix B describes the same examples, one epoch, and a common packed-token schedule, despite large differences in sequence length. Actual steps and example exposure for each variant would resolve this ambiguity. The soft-token baseline also omits series-writing examples, so its comparison includes a difference in supervision.
Questions for authors
- Does geometric initialization still help against a radius-matched control with shuffled symbol-to-position assignments?
- Please report actual steps, tokens, and example exposure for each representation, including how long examples are handled.
- How are the coordinate error and free-share metrics in Table 6 defined? The reported 0.6% coordinate/radius deviations and 14.7% orthogonal squared-norm share appear difficult to reconcile under the centered definitions in Appendix A.4.
2 comments
Dear Reviewer ueDQ,
Thank you again for your careful review. We appreciate that you found our representation compact and decodable, and found that our experiments go beyond aggregate accuracy to show how the model uses shape information and temporal order.
In our response, we added a radius-matched control that separates the geometry of the initialization from its scale (R1), reported the samples, steps and input tokens of every trained variant together with a supervision-matched control for the soft-token baseline (R2), and gave the definitions behind Table 6 with a correction of its free share label (R3).
Your comments have made this work stronger, and we would be delighted to discuss any remaining questions. Please do not hesitate to contact us for further discussion.
Authors
We thank the reviewer for their time and thoughtful feedback and address the concerns below.
R1: Geometry or scale of the initialization
We run the radius-matched control, which places the symbols on the same shell (same norm as the pretrained vocabulary) with random coordinates, following the same training recipe.
The control separates the two effects (TSAQA / TSE / TB):
Both therefore contribute: scale gives about 1 point on every benchmark, and geometry gives most of the gain on TSAQA and TSE, while the gain on TB comes from scale. This is consistent with Section 4.4 of the paper, and we will add the control to the revision.
R2: Training accounting
The corpus has 857,912 stage 1 (s1) and 478,233 stage 2 (s2) samples. Samples are packed into sequences of 4,800 tokens, each stage is one epoch, and samples longer than the sequence length are dropped at export (evaluation has no cutoff).
Soft-token supervision. The soft-token baseline cannot emit a series and is trained only on the samples whose target is text. As a direct control we trained LEXIS-4B on exactly those samples (reading rows only): 76.18 / 59.92 / 35.79 against 71.73 / 55.90 / 37.65 for soft tokens, so the lead on TSAQA and TSE holds at equal supervision (+4.45, p=0.0003; +4.02, p=0.039).
R3: Definitions in Table 6
Each symbol embedding E is centered on the pretrained mean m. U holds the 26 placement directions, κ is the scale, ϕ the target coordinates and R the pretrained radius.
Free share in Table 6 is a ratio of norms, which the caption and the text mislabel as a share of squared norm. As a share of squared norm it is 2.2% for LEXIS (0.1472≈0.022), which is consistent with the 0.6% coordinate and radius deviations. We will correct the caption and the text.
We thank the reviewer again for their feedback that strengthens this work.