acceptodds
Withdrawn from ICLR 2027

Image Synthesis as an Intermediate for Controllable Time Series Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Semantic-driven time-series generation offers a promising way to improve downstream learning in few-shot forecasting, but directly generating numerical sequences from language often fails to preserve the intended temporal structure. We propose VisualBridge, which uses time-series plots as a visual intermediate to bridge high-level temporal semantics and numerical sequences. An MLLM first converts plotted series into structured semantic representations, enabling explicit control over temporal properties such as trend, seasonality, and volatility. We then learn a semantic editing policy with downstream forecasting rewards, allowing the generation process to favor temporal patterns that are beneficial for the target task. The resulting sequences are further modeled by a temporal VAE to produce consistent multivariate augmentations. Experiments on standard public forecasting benchmarks demonstrate that VisualBridge improves few-shot forecasting over conventional augmentation methods, with ablations validating the roles of visual semantic grounding, learned semantic control, and VAE-based generation.

Primary area:
learning on time series and dynamical systems
Keywords:
Time-Series Data Augmentation, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_oiBhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces VisualBridge, a framework for controllable time-series augmentation in few-shot forecasting. The method first converts real time-series windows into plots and uses a multimodal language model to extract structured semantic descriptions covering properties such as trend, seasonality, volatility, change points, and anomalies. These semantic records are then converted into generated plots, from which numerical sequences are recovered through curve extraction and source-scale calibration.

The main methodological contribution is a semantic editing policy trained using downstream forecasting feedback. The policy performs a single bounded edit to trend strength, seasonality strength, or volatility level and is optimized using a GRPO-style single-step contextual-bandit objective. Edited generations are evaluated relative to matched no-edit image generations using forecasting rewards computed on chronological folds within the few-shot training split. Accepted candidates are blended with their original windows, with 90% of each anchor coming from the original sequence and 10% from the image-recovered generation.

A temporal VAE subsequently models complete multivariate input-target windows using reconstruction, first-difference, KL, and forecasting-consistency objectives. Posterior draws around accepted augmentation anchors are filtered and ranked using a frozen forecaster, and the retained samples are combined with real data to train a final iTransformer from scratch. The method improves MAE and MSE over the reported augmentation baselines on all eight evaluated datasets.

Critical strengths

1. Interesting use of visual representations for semantic control.

Using time-series plots as an intermediate representation is an intuitive and potentially useful way to connect qualitative temporal concepts with numerical sequences. The paper provides both cross-model and same-model comparisons suggesting that plot-mediated generation preserves certain recurring structures better than direct numerical generation, particularly periodicity and cycle-count characteristics.

2. Forecasting-guided semantic editing is more compelling than unconditional augmentation. The policy does not merely generate more samples; it learns which bounded semantic modifications appear useful for a downstream forecasting task. The matched no-edit generation provides a more meaningful control than comparing only against original-data training, and Table 7 indicates that this matched identity reference improves the resulting augmentation policy across the evaluated datasets.

3. The method contains several thoughtful controls.

The reward computation uses chronological folds, purge gaps, matched proxy initialization, and no access to the official validation or test sets. The paper also evaluates transfer of fixed augmentations to PatchTST and DLinear, where improvements are reported on the four tested datasets. These experiments provide some evidence that the resulting synthetic data are not useful only to the iTransformer used for policy learning.

4. Consistent headline forecasting results. The full method reports lower MAE and MSE than the real-data-only model and all listed augmentation baselines across all eight datasets. The improvements are substantial on some datasets, particularly ETTh1, ETTh2, and Traffic.

Critical weaknesses

1. The experiments do not establish that image synthesis is necessary in the final augmentation pipeline. This is the central weakness. The complete method contains an image generator, curve extraction, source-range calibration, a 90/10 source-generation blend, candidate filtering, reward-based semantic selection, VAE reconstruction, decoder forecasting-consistency refinement, posterior sampling, and frozen-forecaster selection. As a result, the final forecasting gain cannot be cleanly attributed to the visual intermediate.

The existing Direct versus Visual-only comparison shows that images can improve selected semantic-fidelity metrics when generating from a semantic description, but it does not answer the more important downstream question: whether image synthesis remains necessary once the same semantic edits are implemented directly in numerical space and passed through the identical blending, VAE, filtering, and selection pipeline.

A critical missing ablation is therefore: semantic policy -> direct numerical edit -> ame 90/10 blend ->same VAE and candidate selection.

Without this control, the work demonstrates that the full compound pipeline is effective, but not that visual generation is the essential mechanism.

2. Some reported gains are small relative to the complexity of the method.

Although the method wins on all eight datasets, several improvements over real-data-only training are modest. Weather MSE improves from 0.176 to 0.173, Electricity from 0.173 to 0.171, Exchange from 0.091 to 0.085, and ETTm2 from 0.190 to 0.186. These gains may be useful, but the paper does not provide confidence intervals, statistical tests, or sufficient seed variation to demonstrate that the improvements are robust relative to the considerable construction cost.

3. Reported variance understates overall experimental uncertainty.

Table 2 averages over three generated datasets while fixing the final forecasting seed at 2023. Therefore, the reported standard deviations capture augmentation-generation variation but not variation from forecasting-model initialization and optimization. This makes values such as 0.0000.0000.000 or 0.0010.0010.001 difficult to interpret as evidence of general robustness. More complete experiments should independently vary the policy-training seed, image-generation samples, VAE seed, and final forecaster seed.

4. The comparison set is weak for a modern generative time-series paper.

The main forecasting comparisons use Gaussian noise, convolution, TimeGAN, and ADA. The related-work section itself discusses flows and diffusion-based generation, including Diffusion-TS, but no modern diffusion-based generator is included in the main experiments. Consequently, the paper establishes superiority over relatively conventional augmentation baselines, but not over current learned generative approaches.

5. The cost-versus-benefit analysis is inadequate. The appendix provides an asymptotic statement that visual generation scales linearly with the number of window-channel pairs, but it does not report actual API calls, wall-clock time, monetary cost, accelerator usage, or total compute by dataset. This omission is material because semantic extraction and image generation operate per source-window/channel pair, and policy learning evaluates multiple actions with matched identity generations.

Questions for authors
  1. Could the authors implement a reasonable numerical-space counterpart for each controlled attribute and pass those candidates through the same downstream pipeline? Even one transparent implementation, accompanied by sensitivity analysis, would help determine whether the image path adds value beyond stochastic numerical editing.

  2. Why are all final forecasting runs conducted with a fixed seed? Please report results over independently varied final-forecaster seeds in addition to independently generated augmentation sets. Ideally, the policy, image sampling, VAE, and final forecaster should all be rerun under several independent seeds, with confidence intervals or paired statistical tests.

  3. What is the total computational and monetary cost of producing the augmentation set for each dataset? Please report the number of semantic-model calls, image-generation calls, proxy-forecaster training runs, GPU hours, wall-clock time, and estimated API cost. A comparison with the runtime and cost of each baseline would help assess whether the reported gains justify the method’s complexity.

  4. How does the method compare with modern diffusion-based time-series generators? The absence of a recent learned generative baseline, ideally a diffusion-based method adapted to the same few-shot augmentation protocol, limits the strength of the empirical comparison.

Reviewer_xn8dOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper "IMAGE SYNTHESIS AS AN INTERMEDIATE FOR CONTROLLABLE TIME SERIES GENERATION" proposes a novel paradigm of using semantic-driven image synthesis as an intermediate step to generate controllable time-series data.

Critical strengths
  1. Innovative Visual-Intermediate Paradigm for Continuous Time Series: By framing 1D continuous sequence generation as 2D plot image synthesis (s→Is→as→I~s→s~s \to I_s \to a_s \to \tilde{I}_s \to \tilde{s}), the authors ingeniously bypass the autoregressive tokenization and scale-calibration pitfalls of LLMs on raw numbers. Table 1 demonstrates that this visual pathway drastically reduces periodicity error (Per-F1 up to 84.3% vs. 64.5% on ETTh1) and turning-point errors compared to direct numerical generation.
  2. Task-Directed Closed-Loop Optimization via Downstream Rewards: Rather than generating random augmentations that may distort task distributions, the framework introduces a contextual bandit policy trained via GRPO that selects ordinal semantic edits specifically rewarded by downstream forecasting performance gains on held-out probes. Table 3 and Figure 4 rigorously show that this reward-guided selection is critical to avoiding the severe performance degradation suffered by conventional augmentations (such as TimeGAN and Gaussian noise).
Critical weaknesses
  1. Severe Practical Dependency on Proprietary Closed APIs and Unaddressed Computational Overhead: The core workflow requires per-channel MLLM structured extraction and per-channel image generation using closed commercial APIs (gpt-5.6-luna and gpt-image-2). For high-dimensional datasets such as Traffic (862 channels) or Electricity (321 channels), generating hundreds of windows requires tens of thousands of separate image generation API calls. The paper does not provide an explicit dollar-cost, latency, or carbon footprint audit, nor does it evaluate whether open-source alternatives can replace these proprietary backbones without severe degradation in curve recovery fidelity.
  2. Confounded Pipeline Attribution and Unverified Controllability of Final Samples: The visual contribution is not isolated in the full pipeline: because accepted visual curves only act as anchor seeds for a temporal convolutional VAE with consistency filtering, the relative performance contribution of visual image synthesis versus the temporal VAE backbone and reward-filtering gate remains confounded in end-to-end forecasting. Crucially, final-sample controllability is unverified: while the authors validate semantic fidelity on intermediate curves prior to VAE sampling, they do not verify that posterior draws in Daug\mathcal{D}_{aug} actually preserve the requested semantic modifications. Furthermore, generating channels independently ignores cross-channel co-integration and spatiotemporal relationships during the primary visual synthesis stage.
Questions for authors
  1. Isolation of Visual Contribution vs. VAE Modeling: In Figure 4, the ablation includes w/o VAE (using visual anchors directly) and Visual only (visual synthesis without learned policy or VAE), but how much of the final forecasting gain in Table 2 is attributable to the visual diffusion intermediate versus the temporal VAE posterior smoothing and forecaster consistency filtering? Can the authors isolate the visual contribution in the full pipeline with a controlled baseline that applies the identical VAE and reward selection to standard non-visual augmentations?
  2. Verification of Controllability on Final Augmented Samples: Has the team evaluated whether final-sample controllability is preserved after VAE posterior sampling (si,kaug=Dη(zi,k)s_{i,k}^{aug} = \mathcal{D}_\eta(z_{i,k}))? Specifically, if the semantic policy requested an increased trend or suppressed volatility, do the final samples in Daug\mathcal{D}_{aug} actually exhibit those target attributes under the MLLM evaluation metric, or does the VAE reconstruction pull the samples back toward the unconditional data prior?
  3. Reproducibility with Open-Source Backbones: Have the authors evaluated whether open-source vision-language models (e.g., Qwen2-VL-7B/72B) and open image generators (e.g., Stable Diffusion 3 / FLUX) can substitute for gpt-5.6-luna and gpt-image-2 in the QA extraction and curve synthesis stages? What is the impact on curve extraction accuracy and downstream forecasting MSE?
  4. Computational and API Cost Breakdown: What was the total API monetary cost and wall-clock time required to generate the augmented datasets across the 8 benchmarks, particularly for high-dimensional datasets like Traffic (C=862C=862) and Electricity (C=321C=321)?
Reviewer_zub7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

An interesting idea of using generated plots as intermidiate to control how time series are generated. However, the gains are relatively small with quite complex pipeline and no clear main improvement.

Critical strengths
  1. Using plots as a bridge between text and numbers is an intuitive and fun idea, and the paper gives some support for it.
  2. The matched no-edit control - it compares each edit against the same visual pipeline without the edit, so the reward measures the effect of the edit itself rather than the effect of adding generated data in general
Critical weaknesses
  1. The gains are small. Most improvements are in the second or third decimal, all final models use a single forecasting seed. Please do more testing as it might be that improvement gets lost actually.
  2. Closed models. Please evaluate your results on several different open-weight models so the results could be easily reproduced.
  3. Complex and not very clear pipeline. Lack of ablation studies for each improvement.
Questions for authors
  1. Do all of the parts you described are essential for improvements?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.