acceptodds
Under review as a conference paper at ICLR 2027

LineupRL: Verifiable Reinforcement Learning for Time Series Captioning via Caption-to-Series Identification

Abstract

Time series captioning is a fundamental step in time series understanding and can also serve as the bridge between signal and natural language. Supervised fine-tuning (SFT) relies on a larger model's captions and cannot exceed their quality. Reinforcement learning (RL) can, but its rewards were designed for other modalities and other tasks, and they transfer poorly to open-ended generation in the time series domain. We address this by proposing LineupRL, a reinforcement learning with verifiable rewards (RLVR) pipeline whose reward is caption-to-series identification. The reward model is a frozen large language model (LLM) verifier that reads the generated caption and the candidate time series as raw values, never the chart, and must pick the described time series from multiple distractors. Matching is a far lighter demand on the verifier than writing questions or judging a caption, so an off-the-shelf LLM can supply the reward. Across two captioning benchmarks, and on forecasting and reconstruction where the predictor sees only the caption, LineupRL outperforms SFT and RL baselines on every metric. The 3B vision language model (VLM) trained by LineupRL also outperforms, at 1/24 of the parameters, the 72B VLM whose captions the SFT baseline is distilled from. Our case study shows that LineupRL resists reward hacking, and that the captioner it trains both traces the trend and names the values at key points.

Primary area:
learning on time series and dynamical systems
Keywords:
time series understanding, reinforcement learning with verifiable rewards, vision language models, multimodal learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SuTtOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This work proposes LineupRL, an RLVR framework for time series captioning. The issue is that existing training frameworks such as SFT and standard RL don’t suit this task. LineupRL uses Qwen2.5-VL-3B as the base model and trains it to generate a caption given a line plot. The verifier is a frozen Qwen2.5-14B that provides the reward supervision. Among multiple numeric time series, one of which is the true time series of the input, the verifier has to pick the one being described by the caption generated by the base model. This is an easy task for the verifier because it is just a multiple-choice question. The authors use three criteria for evaluation, including entailment with reference captions, identification between caption-series and series-caption (scored by the verifier), and caption-based forecasting and reconstruction. Results show that LineupRL outperforms SFT, answerability RL, and LLM-as-judge in every setup they test.

Critical strengths
  • The paper is clearly written and written with care, unlike most papers that exhibit obvious AI-written traces. Figures, diagrams, and tables are clear, as are their captions.
  • The reward is simple and elegant. Identifying the right numeric time series among close alternatives has a deterministic ground truth, so it can't be hacked. It also asks the verifier only to match a caption to numbers, which is a simple task.
  • Consistent and significant gains. LineupRL beats SFT and both RL baselines on every metric in table 1. The 3B model also beats or matches the 72B teacher on most metrics.
  • Strong controls. The authors tried rotating the true series through every position, a no-caption check, empty- and shuffled-caption controls for forecasting, a temporal split, and an audit of the overlap between training and evaluation data.
  • Baselines are analyzed clearly. The authors quantified the judge hack, where 99.9% of captions end with text addressed to the grader, which makes up 43.8% of their characters. The answerability failure is traced, where only 28% of the generated questions survive verification.
Critical weaknesses
  • Only Qwen2.5-VL-3B-Instruct is used as the base model for LineupRL. It’s possible that LineupRL works exceptionally well with this model but fails on some models.
  • No limitations section, not even in the appendix.
  • Limited novelty. The reward is essentially the listener/discriminability reward from image captioning [1, 2], reframed to time series with an LLM as the listener.
  • The reward favors transcription over description. The verifier can barely use qualitative captions. The trained captions read more like verbalized tables of values rather than true numeric descriptions.
  • Flawed metric. Identification and caption-only forecasting both reward reporting numbers, but entailment is more fragile. It also never penalizes unsupported or hallucinated content, since it only checks whether the caption implies the reference.
  • Missing key baseline. There's no best-of-N or rejection-sampling fine-tuning with the same verifier. Without it, "RL beats SFT" might just mean "a good filter beats a verbose and templated teacher."
  • Single training seed for every fine-tuned model and every ablation setting.
  • The captioner sees only a line plot. Text LLMs given the raw values outperform every chart-reading captioner on identification (by ~0.2 on BEDTime, less on CaTS-Bench), presumably because they read the values directly rather than off ticks. Yet LineupRL is applied only to a single 3B VLM.
  • Impact beyond captioning is untested. It is unclear whether training with LineupRL improves or deteriorates the captioner’s performance in general, beyond captioning.

References

[1] Luo et al., Discriminability Objective for Training Descriptive Captions. CVPR 2018.

[2] Dessì et al., Cross-Domain Image Captioning with Discriminative Finetuning. CVPR 2023.

Questions for authors
  • The captioner reads numbers by looking at the tick labels from the chart. How sensitive are the results to rendering choices such as tick density, aspect ratio, or line thickness?
  • Does training with LineupRL improve the VLM's general ability to process charts and time series, or does it only improve TS captioning?
Reviewer_RcwWOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LineupRL for time series captioning. Its reward is built upon caption-to-series identification: a frozen LLM verifier takes the generated caption and candidate raw time series values, and selects the matched series. The paper is well written, while its novelty should be further clarified.

Critical strengths
  1. The paper is well written, and its intended contribution is clearly presented.
  2. The ablation experiments are comprehensive.
Critical weaknesses
  1. The novelty is limited. The core proposal is essentially a new reward function under the reinforcement learning framework for time series captioning, and the fundamental advantages over existing reward designs are not well established.
  2. The work lacks adequate comparisons against existing time-series captioning approaches, from both mechanism and experimental perspectives, making it hard to validate its superiority.
Questions for authors
  1. The paper recognizes two mainstream reward paradigms for time series captioning and discusses their drawbacks. However, it does not clearly explain why the proposed reward can mitigate these issues.
  2. The paper claims to tackle open-ended time series captioning, but training relies on a fixed dataset. All candidate time series segments are sampled within this limited data pool. How can the generalization to out-of-distribution unseen time series be guaranteed?
  3. The reward design is narrow. It only measures whether captions can stably match the target time series. This objective may fail to develop strong captioning ability. The LLM may only receive reward for partial descriptions that are just distinctive enough for identification.
  4. Most experimental results come from ablation studies. Many advanced time series captioning methods have been published, yet direct comparisons with these approaches are missing. This prevents a solid assessment of the proposed method.
  5. The evaluation metrics should be expanded. Time series captioning requires evaluation along both correctness and completeness. In particular, metrics checking whether critical temporal characteristics are retained are currently absent.
Reviewer_utmjOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper is interested in time-series (TS) captioning, by fine-tuning a vision-language model (Qwen2.5-VL-3B) with Reinforcement Learning with Verifiable Reward (RLVR). The VLM is given the TS rendered as a chart, and generates a caption for it. During RLVR, the reward is based on a retrieval task: the correct TS must be identified from its caption among a candidate set of distractors. The distractors are chosen to match statistics of the original TS among the training set, to make the retrieval harder. The method is evaluated on two TS-captioning benchmarks, and several forecasting benchmarks.

Critical strengths
  • The paper is clearly written, and the proposed approach of fine-tuning with RLVR is a sound, well-presented approach.
  • The comparison with other RL baselines clearly shows that the retrieval-based verifiable reward performs better than the alternative RL setups. This is the paper's main methodological contribution.
Critical weaknesses

Major:

  • The method only considers VLMs that receive the TS as a chart. It does not consider fine-tuning LLMs directly on the raw TS values. The paper does not explain what the chart gives over raw values, so this design choice looks arbitrary. This matters because Table 1 suggests LLMs are already strong at the retrieval tasks. That raises doubts about whether the chart-based pipeline is needed at all.
  • Table 2 leaves out several baselines from Table 1. This makes it hard to interpret the forecasting results and compare them across tables. Table 1 suggests LLMs do well on retrieval but worse on entailment. However, entailment looks like a metric crafted for this setup, while retrieval and forecasting are concrete, practical evaluation scores. Strong performance on retrieval and forecasting should count for more. Without full baselines on those tasks, the evidence that the method is better is weaker.

Minor:

  • the way the results are bolded leaves out all cases where the baselines are actually better than the proposed approach, which is misleading at first read.
Questions for authors
  1. What specific advantage does rendering the TS as a chart give over feeding raw values to an LLM? Is there an empirical comparison on raw values? Furthermore, if the goal is to specifically consider VLMs, have you considered other ways to render a TS as an image (e.g. Gramian Angular Field, or heatmaps)?
  2. Why does Table 2 leave out some baselines from Table 1, especially the LLM baselines that are strong at retrieval? Could the forecasting comparison be extended to all of them?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.