- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes GLEAN, a lightweight framework for time-series question answering that replaces the LLM reasoning backbone with a recurrent latent reasoning module. It combines frozen time-series and text encoders, iterative evidence retrieval, and structured prediction heads. Experiments on TSAQA demonstrate competitive performance against time-series LLMs with substantially lower inference computation.
Critical strengths
The paper explores a practically meaningful direction by decoupling time-series perception from reasoning. The extensive ablation studies and computational efficiency analysis provide useful empirical insights into the effectiveness of lightweight alternatives to LLM-based approaches.
Critical weaknesses
- Limited methodological novelty. The core architecture largely combines established techniques, including frozen pretrained encoders, Perceiver-style iterative cross-attention, recurrent parameter sharing, and Sinkhorn-based structured prediction. The paper lacks a sufficiently novel mechanism or deeper insight beyond integrating these existing components.
- Insufficient evidence for the central claim. All experiments are conducted on TSAQA, which consists of closed-ended questions with predefined answer spaces. This setting naturally favors specialized discriminative models, and the findings cannot readily generalize to open-ended time-series reasoning. Moreover, several fine-tuned general-purpose LLMs still outperform GLEAN, weakening the broader claim that LLM-based reasoning is unnecessary.
Questions for authors
- What is the essential methodological difference between GLEAN's recurrent reasoning module and existing Perceiver-style iterative latent processing architectures? Could the authors provide a direct comparison?
- Can the authors validate GLEAN on additional benchmarks, particularly open-ended time-series QA tasks, to support the generality of their central claim?
6 comments
Response to Reviewer oDwU (Part 2 of 4)
W1 / Q1 (continued): the direct comparison with Perceiver-style processing
Direct comparison (Q1). We replaced the core while holding the frozen encoders, the bank, the K = 12 slots and the readouts fixed: an unshared Perceiver-IO-style stack at matched depth and wiring, a plain transformer over the bank and the slots swept over depth, and a small language model in the reasoner's place.
| Reasoner over the same frozen evidence bank | Trainable params | Reasoner GFLOPs | TSAQA overall |
|---|---|---|---|
| none, pooled readout (Table 2) | n/a | n/a | 70.79 |
| Qwen3-0.6B + LoRA | LoRA on 0.6B | n/a | 74.65 |
| plain transformer, L = 2 | 7.32M | 3.85 (0.50×) | 75.30 |
| plain transformer, L = 4 | 10.87M | 7.17 (0.93×) | 76.90 |
| plain transformer, L = 8 | 17.97M | 13.82 (1.80×) | 77.03 |
| plain transformer, L = 12 | 25.07M | 20.46 (2.67×) | 76.79 |
| unshared core, Perceiver-IO style | 41.07M (5.0×) | 7.67 (1.00×) | 76.60 |
| GLEAN shared recurrent core | 8.21M | 7.67 (1.00×) | 77.33 |
TSAQA test split, 42,000 questions; reasoner GFLOPs are the reasoning module's own forward cost under the paper's FLOP counter.
Analysis. Untying the weights is the comparison asked for most directly: identical FLOPs, 5.0× the trainable parameters, 0.73 points lower accuracy, so sharing is the better configuration at this budget rather than a compression compromise. Dropping our wiring does not recover the result either: the plain transformer reaches 75.30 at matched parameters and 77.03 at its best depth, needing 2.2× the parameters and 1.80× the reasoner FLOPs to land 0.30 short, and four further layers reduce accuracy. A language model in the same position reaches 74.65. Neither depth, nor parameters, nor a language model reproduces what the shared recurrent wiring does.
Read with the pooled-readout row, the same table also locates where reasoning capacity pays, which is the deeper insight we would claim: perception in the frozen encoders is already saturated (the pooled readout scores 90.83 there, above every time-series LLM, against 58.03 on relational questions), the reasoner is worth +11.13 on relational questions and only +0.24 on classification, and what still fails sits in the representation rather than in the reasoner, which Appendix J measures and the second corpus reproduces.
Modification. §4 Related Work states that we do not claim iterative latent cross-attention or cross-step weight sharing, and names the three differences above. New Appendix M ("Reasoner Comparisons at Fixed Evidence") with Table 10 and per-cell results in Table 11. §3.3 names the evidence-source interventions as such, so the architecture controls are unambiguous.
(continued in Part 3 of 4)
Response to Reviewer oDwU (Part 3 of 4)
W2 / Q2: Evidence for the central claim, and additional benchmarks
W2. Insufficient evidence for the central claim. All experiments are conducted on TSAQA, which consists of closed-ended questions with predefined answer spaces. This setting naturally favors specialized discriminative models, and the findings cannot readily generalize to open-ended time-series reasoning. Moreover, several fine-tuned general-purpose LLMs still outperform GLEAN, weakening the broader claim that LLM-based reasoning is unnecessary.
Q2. Can the authors validate GLEAN on additional benchmarks, particularly open-ended time-series QA tasks, to support the generality of their central claim?
Response. The paper does not make the broader claim. §1 states the opposite fact: "General-purpose LLMs fine-tuned on TSAQA remain more accurate overall, at 51-733× the computation, and we report where and why they lead (§3.2)", and the conclusion adds that "the answer spaces we study are closed. These findings therefore do not establish sufficiency for unrestricted language-based reasoning." The abstract scopes the claim the same way, to whether a large language model is needed as the reasoning engine. Concretely, three of the six fine-tuned general-purpose LLMs score above GLEAN, Qwen3-0.6B (78.32), Qwen3-8B (84.29) and LLaMA3.1-8B (85.26), at 51×, 733× and 576× our FLOPs per question, and three score below (Ministral-8B 74.74, LLaMA3.2-1B 73.48, Gemma3-1B 69.70); Table 5 and Figure 1c report this.
On the setting favouring discriminative models: it is the same setting for every system in the comparison, all fine-tuned on the same training split under the same protocol and scored by the same parser. The control is the pooled readout, which keeps our frozen encoders and structured readouts but has no reasoner: it reaches 70.79, 4.27 points below TimeOmni-1-7B. A discriminative readout over a predefined answer space does not by itself produce the result.
Further benchmark, retrained under the identical protocol (Q2). We retrained every system on Time-MQA, converted into the two answer formats of our readouts, 90,278 questions split 72,222 / 4,514 / 13,542, with three seeds for GLEAN. We report by subset, because the test mix is dominated by two perception-style subsets.
| Time-MQA | anomaly det. (5,974) | classification (5,550) | characterization (1,917) | Overall (13,542) |
|---|---|---|---|---|
| GLEAN (ours), 3 seeds | 71.15 ± 0.62 | 83.20 ± 0.17 | 62.60 ± 0.60 | 74.71 ± 0.37 |
| ChatTime-1-7B | 73.82 | 76.99 | 52.11 | 71.76 |
| ChatTS-8B | 62.71 | 69.42 | 65.36 | 65.76 |
| TimeOmni-1-7B | 64.31 | 66.52 | 66.25 | 65.34 |
| PATRA-7B ¹ | 32.11 | 18.40 | 20.14 | 24.74 |
| TimeMQA-Qwen2.5-7B ² | 69.44 | 70.72 | 64.95 | 69.19 |
¹ PATRA-7B produced no parseable answer in 43.4% of its responses, so its overall figure is largely a compliance rate. ² in-domain: pre-trained on this corpus, so not a like-for-like row. Three further subsets (101 questions, 0.75% of the split) are reported as counts.
Analysis. GLEAN leads overall by +2.95 over the strongest comparable baseline, from classification (+6.21), and is behind on the other two large subsets, by 2.67 on anomaly detection and 3.65 on characterization. What bears on generality is not the margin but that the same profile reappears: strong on perception-style subsets, behind on language-heavy characterization, which is the split Table 1 reports on TSAQA. That supports reading the limitation as a property of the frozen representations rather than of one benchmark's construction.
(continued in Part 4 of 4)
Response to Reviewer oDwU (Part 1 of 4)
We thank the reviewer for recognising that decoupling time-series perception from reasoning is a practically meaningful direction, and that the ablations and the efficiency analysis give useful empirical insight into lightweight alternatives to LLM-based approaches. We address the two concerns point by point below.
W1 / Q1: Methodological novelty, and the difference from Perceiver-style architectures
Response. Although pretrained encoders, cross-attention, and sinkhorn-based module are established, and the paper credits them as such rather than claiming them, the contribution is not their assembly.
(1) First, a measurement that published work cannot provide. Existing time-series QA systems place perception and reasoning inside one LLM, and each receives the signal differently, so no published comparison can attribute accuracy to either stage. To our knowledge this is the first work in time-series question answering to hold the perception stage fixed and compare reasoners over identical evidence, a language model included, under one FLOP convention, which is what makes the question in the title answerable at all.
(2) Second, three specific differences in how the latent state is used, each isolated by its own ablation rather than asserted. State-conditioned reads are not among them, since Perceiver's queries also come from the current latents, and Related Work states that content matching and Sinkhorn assignment "are established operators, not new contributions".
(3) Third, the accuracy this combination reaches per unit of computation: 77.33 overall with 8.2M trainable parameters, at 87 to 693 times fewer FLOPs per question than the five fine-tuned time-series LLMs.
(continued in Part 2 of 4)