acceptodds
Under review as a conference paper at ICLR 2027

Joint Visual-Temporal Diffusion Model for Probabilistic Time Series Imputation

Abstract

Score-based diffusion models have emerged as powerful probabilistic tools for time series imputation, but their denoising process is still primarily conditioned on observed temporal values. When missing values are dense or form long contiguous gaps, such temporal context becomes sparse and fragmented, weakening the structural guidance available for generation. We revisit this problem from a complementary perspective: even when numerical observations are incomplete, the macroscopic shape of a time series often remains visually discernible in its line-graph rendering. Rather than treating imputation as an image-generation problem, we use this visual representation as a structural prior for temporal diffusion. We propose VT-Diff, a Joint Visual-Temporal Diffusion framework for probabilistic time series imputation. VT-Diff renders multivariate time series as variable-aware line graphs and encodes them with a Vision Transformer to obtain global and local structural tokens. The central component is a Cross-Attention Bottleneck with learnable temporal anchors, which translates spatial visual tokens into time-step-wise conditions for the temporal denoising network. This design allows visual cues to guide the global shape and local variations of the imputed trajectory, while the diffusion model performs generation in the original numerical time-series domain. Extensive experiments on diverse missing patterns show that VT-Diff achieves competitive or superior performance against strong baselines and produces more structurally coherent imputations, especially under extreme missingness.

Primary area:
generative models
Keywords:
Time Series Imputation, Diffusion Models, Visual Structural Prior

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_2GdKOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes VT-Diff, which renders a multivariate series with missing values as line charts, extracts global and local visual features with a pretrained ViT, converts them via a cross-attention bottleneck with learnable temporal anchors into per-timestep conditions injected into a CSDI-style denoising network, thereby "perceiving structure in the visual domain while generating values in the temporal domain," and compares against 17 baselines on 7 datasets under 3 missingness patterns.

Critical strengths
  1. A novel cross-modal idea that avoids the trap of "generating in pixel space." The authors clearly distinguish what visual representations are good at (geometric shape, overall trend) from what numerical imputation requires (precise values), and choose to let vision serve only as a condition while keeping generation in the temporal domain. This is a sensible design choice. Using learnable temporal anchors as queries to cross-attend to visual patches, aligning 2-D spatial tokens into 1-D per-step conditions, is a clean interface that could transfer to other conditional diffusion frameworks.
  2. The motivation targets a real pain point. Context degradation of conditional diffusion models under high missing rates and long contiguous gaps is a recognized challenge. Approaching it from the angle of "supplementing structural priors" complements frequency-domain priors (FGTI, LSCD) and graph priors (PriSTI) naturally.
Critical weaknesses
  1. The core motivation does not hold at the information level, or at least is not argued: the rendered image contains no new information, and the "visually discernible shape" comes precisely from the interpolation it is supposed to overcome.
    • The rendered image is a deterministic function of (X~,M)(\tilde X, M) plus linear interpolation. Inside a long gap, the "macroscopic shape" shown in the image is simply a straight line; under high missing rates, the so-called "still discernible shape" also comes from linearly connecting sparse observations. Therefore, the core claim that "the shape remains visually discernible even when numerical observations are incomplete" amounts to "the contour after linear interpolation," which provides no information in the missing regions that a temporal encoder cannot obtain.
    • If the gains are real, the only remaining possible sources are: inductive biases or prior knowledge from the pretrained ViT (why ImageNet natural-image priors apply to line charts needs justification); a larger receptive field and a different encoding scheme; or additional parameter capacity. The paper does not disentangle these explanations, nor does it provide any theoretical or mechanistic analysis, e.g., in what sense is visual encoding better than directly feeding the interpolated series and mask into a temporal encoder?
    • Rendering itself also introduces loss and bias, which the paper does not discuss:
      • Values are quantized to pixel resolution, and the more variables there are, the lower the per-cell resolution;
      • Global coordinate bounds are taken from the entire training set, making them sensitive to outliers and distribution shift;
      • The grid layout introduces artificial spatial adjacency: variables in neighboring cells have no semantic relation, yet ViT's spatial attention treats them as neighbors.
  2. The technical contribution is mainly a combination of mature components, without matching depth of analysis.
    • The denoising network follows the CSDI-style temporal/feature Transformer; "learnable queries cross-attending to external tokens" is standard practice in Perceiver, DETR, and Q-Former; the auxiliary reconstruction loss on the global token is merely a conventional regularizer, and its output x^\hat x is not used at inference.
    • The methodological novelty lies in "plugging ViT features into CSDI," yet the integration is not carefully examined: why does fine-tuning only the last two layers avoid "attention collapse"? How do the temporal anchor queries correspond to actual time positions? Is this purely learned, or is there a positional prior? What justifies the division of labor between the global token and the patch tokens?
    • The positioning relative to related work is also insufficient. The paper does not discuss works that convert time series into images and use visual diffusion models for generation and imputation (e.g., ImagenTime, Naiman et al., NeurIPS 2024), nor works that use pretrained vision models for time series tasks (e.g., VisionTS). The feature comparison in Table 1 is overly simplified and does not establish the distinctiveness of this work.
Questions for authors

Clarifications (answers may affect my rating):

  1. Since the rendered image is a deterministic function of the input, please argue from an information or representation perspective: what structure can visual encoding provide that a temporal encoder would in principle find hard to learn? Why do ImageNet-pretrained priors apply to line charts?
  2. Inside long gaps, the shape in the image is just linear interpolation. How exactly does the visual condition help recover non-linear variations inside the gap? Please provide a mechanistic explanation or targeted analysis.
  3. The grid layout places different variables in adjacent cells. Does ViT's spatial attention introduce meaningless cross-variable associations? How does this compare with "rendering each variable separately with a shared encoder"?
  4. Please clarify the inference procedure: is rendering done with mask MM or MoriM^{ori}? Does the merge in line 8 of Algorithm 2 overwrite evaluation positions? How should the inconsistency between line 11 of Algorithm 1 and Eq. 7 be understood?
  5. A brief experimental question: could you add controls with "a non-visual encoder of equal capacity" or "a randomly initialized ViT" to separate the effect of the visual modality from model capacity? How should some results (e.g., a CRPS of 0.003 on Stock, and errors that do not vary monotonically with gap length) be explained?

Suggestions:

  • Compare with other imaging schemes (GAF, RP, delay embedding) to justify the choice of line charts.
  • Reduce rhetorical expressions and add quantitative analysis of how the conditioning signal works, e.g., the mutual information between the condition cc and the true missing values, or probing analyses.
7 comments
AuthorsOpenReview ↗

Responses to Q1, Q2 and Q3

We sincerely appreciate the reviewer's careful evaluation and constructive feedback. Thank you for highlighting these important issues. We address each comment below.

Q1: Visual Encoding Advantages and Pretraining Transferability.

A1: We sincerely thank the reviewer for this insightful question. We agree that visual rendering introduces no additional information beyond the masked observations and interpolated values, but provides a different structural inductive bias. By arranging variable-wise trajectories into a 2D grid, VT-Diff enables the ViT to capture geometric patterns, including local fluctuations, turning points, and global trends. Although temporal encoders can theoretically learn similar patterns, it is difficult for them to infer these geometric relationships, particularly when missing observations disrupt temporal continuity. To validate its effectiveness, we compare VT-Diff with a capacity-matched numerical encoder under 50% point missingness. The results (RMSE/MAE) are presented as follows:

Conditioning VariantETTh1ILIPEMS-Traffic
Numerical Conditioning0.354/0.2560.600/0.3240.354/0.136
Visual Conditioning (Ours)0.302/0.1740.397/0.1960.337/0.126

As shown in the table, visual conditioning consistently outperforms numerical conditioning, supporting the effectiveness of the proposed visual representation. Regarding ImageNet pretraining, we leverage transferable low-level visual features, such as edges, orientations, and contours, rather than natural-image semantics. The attention maps in Fig. 6 provide qualitative evidence that the ViT attends to observation-dependent visual regions in time-series line charts, supporting its ability to capture relevant visual structures.

We have further clarified the motivation for visual encoding and the role of ImageNet pretraining in the revised manuscript.

Q2: Nonlinear Reconstruction in Long Missing Gaps.

A2: We sincerely thank the reviewer for this important question. We acknowledge that linear interpolation does not provide the true nonlinear structure within missing intervals. In VT-Diff, the interpolated segments serve as a visual representation rather than the final imputation results.

Specifically, the ViT extracts structural features from available observations, while the Cross-Attention Bottleneck aligns local visual features with temporal positions. These features condition the diffusion model, which learns temporal dependencies from training data and generates plausible nonlinear variations within missing regions. Thus, nonlinear reconstruction arises from the learned conditional generative process rather than the interpolated lines themselves. We have further clarified this mechanism in Section 3.2 (Visual Rendering of Time Series), with the revision highlighted in blue.

Q3: Potential Spurious Cross-Variable Associations in Grid Layouts and the Choice of Independent Variable Encoding.

A3: We sincerely thank the reviewer for this thoughtful question. The grid layout provides a consistent spatial organization rather than assuming that adjacent variables are correlated. Each variable is rendered in a separate cell with a distinct color, while ViT self-attention enables interactions across all visual patches, allowing the model to learn potentially useful cross-variable relationships.

Compared with encoding each variable separately using a shared encoder, our approach enables joint visual modeling of both individual trajectories and cross-variable context. Separate encoding can avoid direct spatial interactions between unrelated variables, but also limits cross-variable feature learning within the visual encoder.

Nevertheless, the grid layout has a limitation: spatial adjacency may introduce spurious cross-variable associations that do not reflect actual variable relationships. We have added a discussion of this limitation to appendix, with the revision highlighted in blue.

AuthorsOpenReview ↗

Responses to Q4, Q5 and Suggestions

Q4: Mask Usage and Inference Procedure.

A4: We sincerely thank the reviewer for identifying these inconsistencies.

In our formulation, MoriM^{ori} denotes the original observation mask, whereas MM denotes the conditional mask obtained by artificially hiding a subset of originally observed values. During training and masked evaluation, rendering is based on the conditional observations X⊙MX\odot M, rather than the complete observations indicated by MoriM^{ori}. We acknowledge that Algorithm 2 does not explicitly distinguish these masks. In particular, using MoriM^{ori} in Line 8 would overwrite artificially masked evaluation positions. The merging operation should instead use the conditional mask MM:

X^=x0ta⊙(1−M)+Xob⊙M.\hat X=x_0^{ta}\odot(1-M)+X^{ob}\odot M.

We have corrected Algorithm 2 accordingly. Regarding Algorithm 1, Line 11 omits the mask weighting and normalization specified in Eq. (7). We have revised the pseudocode to match the original observation-weighted reconstruction objective.

Q5: Visual Contribution and Performance Analysis.

A5: We sincerely thank the reviewer for this constructive comment. To disentangle visual representation, model capacity, and pretraining effects, we compared a numerical encoder, a randomly initialized ViT, and a pretrained ViT under identical experimental settings. The results under 50% point missingness (RMSE/MAE) are presented as follows:

Conditioning VariantETTh1ILIPEMS-Traffic
Numerical Conditioning0.354/0.2560.600/0.3240.354/0.136
Randomly Initialized Visual Conditioning0.362/0.2570.441/0.2230.423/0.221
Pretrained Visual Conditioning (Ours)0.302/0.1740.397/0.1960.337/0.126

The results show that pretrained visual conditioning consistently achieves lower RMSE and MAE across all three datasets, demonstrating the importance of visual pretraining and its effectiveness beyond model capacity and auxiliary supervision.

Regarding the performance on Stock, the low CRPS may be partly attributed to the complementary information among its six financial variables. Open, High, Low, Close, and Adjusted Close capture closely related aspects of stock price movements, while Volume provides additional information about trading activity. These inter-variable relationships may provide useful contextual information for reconstructing missing values, contributing to the model's strong imputation performance. We have added the controlled comparison to Section 4.2 and the discussion of Stock performance to Appendix E.1, with the revisions highlighted in blue.

Q6: Comparison of Line Charts with Alternative Time-Series Representations.

A6: We sincerely thank the reviewer for this valuable suggestion. Following this suggestion, we compared line charts with three alternative imaging schemes, namely GAF, RP, and Delay Embedding, under identical experimental settings. The results on ETTh1, ILI, and PEMS-Traffic under 50% point missingness (RMSE/MAE) are presented as follows:

Visual RepresentationETTh1ILIPEMS-Traffic
GAF0.346/0.2160.459/0.2360.586/0.291
RP0.343/0.2050.807/0.4910.496/0.219
Delay Embedding0.319/0.1790.409/0.2470.393/0.149
Visual Conditioning (Ours)0.302/0.1740.397/0.1960.337/0.126

As shown in the table, line-chart-based visual conditioning achieves the lowest RMSE and MAE across all three datasets. This advantage may be attributed to its direct preservation of temporal order, value trajectories, and local geometric patterns. These results support the use of line charts as an effective representation for our visual conditioning framework. We have added this comparison and the corresponding discussion to the Appendix, with the revisions highlighted in blue.

Q7: Quantitative Analysis of the Information Encoded in the Conditioning Signal.

A7: We sincerely thank the reviewer for this valuable suggestion. In VT-Diff, the conditioning signal c is generated through cross-attention between learnable temporal queries and visual patch tokens. Each query corresponds to a specific time position and retrieves relevant visual features, providing temporally aligned structural information to guide the diffusion denoising process.

Our existing comparison between visual and numerical conditioning provides indirect quantitative evidence of the effectiveness of this mechanism. As shown in the controlled comparison reported in Q5, visual conditioning achieves lower RMSE and MAE across ETTh1, ILI, and PEMS-Traffic under identical experimental settings. However, we acknowledge that these results do not directly quantify the information encoded in c, as suggested by the reviewer. We have acknowledge the lack of direct information-level analysis as a potential limitation.

AuthorsOpenReview ↗

Responses to W1, W2

Q8: Rendering-Induced Information Loss and Bias.

A8: We sincerely thank the reviewer for highlighting these important limitations. We acknowledge that pixel quantization may lose fine-grained numerical details, training-set coordinate bounds may be sensitive to outliers and distribution shifts, and grid layouts may introduce artificial spatial proximity between unrelated variables. In our framework, visual rendering provides auxiliary structural guidance, while numerical reconstruction remains in the original time-series domain. Nevertheless, these rendering-related effects may influence visual feature extraction. We have clarified these trade-offs and discussed them as potential limitations in the Appendix, with the revisions highlighted in blue.

Q9: Technical Novelty and Component Contributions.

A9: We sincerely thank the reviewer for highlighting these important limitations. We acknowledge that pixel quantization may lose fine-grained numerical details, training-set coordinate bounds may be sensitive to outliers and distribution shifts, and grid layouts may introduce artificial spatial proximity between unrelated variables. Although visual rendering serves only as an auxiliary source of structural guidance and numerical reconstruction remains in the original time-series domain, these rendering-related effects may still influence visual feature extraction. We have therefore acknowledged these issues as potential limitations and discussed them in the Appendix, with the revisions highlighted in blue.

Q10: Rationale for Visual-Temporal Integration.

A10: We sincerely thank the reviewer for this insightful comment. Regarding the ViT fine-tuning strategy, we freeze most pretrained parameters and fine-tune the patch embeddings and the final two Transformer layers with a smaller learning rate. This strategy aims to preserve pretrained visual representations while allowing task-specific adaptation. It is intended to reduce the risk of disrupting pretrained features during cross-modal alignment, rather than guarantee the avoidance of attention collapse.

For temporal alignment, the L learnable queries are assigned to the L temporal positions in sequential order. Their representations are learned during training, without an additional handcrafted positional prior. Each query generates a corresponding time-step-wise condition through cross-attention with visual patch tokens.

The global token and local patch tokens serve complementary roles. The global token captures macroscopic trajectory information and is used for auxiliary trend reconstruction during training, whereas local patch tokens preserve fine-grained variations and generate the conditioning signal for diffusion denoising. Although the global token is not directly used during inference, its reconstruction objective regularizes the shared visual encoder.

We have revised the manuscript to provide a clearer explanation of the ViT fine-tuning strategy, the temporal alignment mechanism, and the complementary roles of global and local visual representations.

Q11: Comparison with Related Visual Time-Series Methods.

A11: We sincerely thank the reviewer for pointing out these relevant studies.

ImagenTime (Naiman et al., NeurIPS 2024) employs invertible transformations to convert time series into image representations and utilizes visual diffusion models for time-series generation and imputation. VisionTS explores the use of pretrained vision models for time-series forecasting. These approaches differ from VT-Diff in their task objectives and the role of visual representations. Specifically, VT-Diff focuses on probabilistic time-series imputation. It uses line-graph representations of incomplete observations to extract visual structural features, which are transformed into time-aligned conditioning signals for numerical-domain diffusion. Unlike approaches that perform generation or reconstruction primarily in the image domain, VT-Diff retains numerical-domain diffusion to ensure precise value reconstruction.

We have expanded the Related Work section to discuss ImagenTime and VisionTS, and revised Table 1 to provide more detailed comparisons of their tasks, visual representations, and roles of visual models.

Reviewer_EZngOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies probabilistic time series imputation with diffusion models. The authors argue that when missing rates are high or gaps are long, diffusion imputers that condition only on observed values (CSDI-style) have little context, while the overall shape of the series remains visible in a line plot. VT-Diff therefore renders the multivariate series (gaps bridged by straight lines) as per-variable line plots tiled into a grid, encodes the image with a pretrained ViT, and uses cross-attention with learnable "temporal anchors" to turn the patch tokens into per-time-step conditions for a numerical-domain diffusion denoiser; the global ViT token is used only for an auxiliary reconstruction loss. On 4 datasets × 6 missingness settings, VT-Diff reports the lowest CRPS in 23 of 24 cases against 5 probabilistic baselines, with the largest gains under contiguous gaps.

Critical strengths
  1. Clear motivation and a sensible design. Structure is taken from a visual view while values are generated in the numerical domain, which keeps numerical fidelity and avoids the precision problem of pixel-space reconstruction.
  2. Extensive empirical evidence. The evaluation spans 7 datasets, 17 deterministic and probabilistic baselines, point/subsequence/block missingness at several rates, three metrics, and mean±std over three runs. It is complemented by an ablation, a qualitative comparison, attention analyses and an efficiency study. VT-Diff achieves the best CRPS in nearly all settings of Table 2, including against recent strong baselines such as FSDI, with especially large gains under contiguous gaps.
  3. Well written and well presented. The paper is clearly organized and easy to follow; Figures 1–3 explain the pipeline well, and the tables are readable (best/second-best marked). The authors are also candid about efficiency.
Critical weaknesses
  1. Attribution of the gain to the visual representation could be sharper. The rendered image is built from the same (linearly interpolated) observations (Sec. 3.2), and the model is trained jointly with a regression-style auxiliary loss L_recon (Eq. 7), so the improvement may also stem from the interpolation prior (a similar idea underlies PriSTI [1], a baseline here), from added encoder capacity, or from the auxiliary loss. In Table 3, all four ablation variants on ETTh1 land at RMSE ≈ 0.38 versus 0.302 for the full model, which makes it hard to tell which component matters. One control, replacing "rendering + ViT" with a capacity-matched non-visual encoder using the same input and loss, would make the central claim much more convincing.

  2. Related work and novelty positioning. The paper sets aside image-domain imputation because of a "modality gap" (Sec. 2), but image-based generative approaches with invertible transforms, such as ImagenTime [2], already handle interpolation without a precision gap and are not discussed. The Cross-Attention Bottleneck (learnable queries attending to patch tokens) is also closely related to the output-query decoding of Perceiver IO [3]. A short discussion of these would clarify what is new.

  3. Some claims go beyond the evidence. The text states consistent, significant gains "even under extreme conditions like 90%", whereas in Table 2 FGTI is clearly better on Weather POINT-90% (0.062 vs. 0.104), and in several settings the differences to the best baseline are within the reported standard deviations (e.g., ETTh1 POINT-50: 0.167±0.000 vs. 0.166±0.005). Likewise, "structurally coherent" imputations and "concentrated" uncertainty intervals are central claims but are supported only by pointwise/marginal metrics (RMSE/MAE/CRPS) and one qualitative figure; CRPS cannot see joint structure (structure-sensitive scores such as the variogram score [4] exist), and no interval coverage is reported. Toning the claims down, or adding a structure-sensitive metric and coverage, would address this.

Overall, I lean towards acceptance: the idea is simple and well motivated, the empirical evidence is extensive, and the paper is well written. The points above mainly concern how the gain is attributed and how strongly the claims are worded, and they seem addressable in revision.

References

Questions for authors
  1. Could the authors run the control described in Weakness 1 (same interpolated input and L_recon, capacity-matched non-visual encoder)? Relatedly, is the image rendered from the conditional input M (as Fig. 2 suggests) or from M^ori during training?
  2. How is c injected into the denoiser, and, since c has no variable axis, how is variable-specific shape information retained? How many variables and what image resolution are used for high-dimensional datasets such as PEMS-Traffic?
3 comments

We sincerely thank the reviewer for the positive feedback and support. Your encouragement is greatly appreciated. Below, we provide a point-by-point response to your questions.

Q1: Attribution of the gain to the visual representation could be sharper.

A1: We sincerely thank the reviewer for this constructive comment. To better isolate the contribution of visual representation, we conducted a controlled comparison between VT-Diff and a capacity-matched numerical conditioning baseline. Both variants share identical settings but differ in conditioning inputs: visual trajectories versus interpolated values with an observation mask. We evaluate both variants on ETTh1, ILI, and PEMS-Traffic under 50% point missingness, reporting RMSE/MAE as follows:

Conditioning VariantETTh1ILIPEMS-Traffic
Numerical Conditioning0.354/0.2560.600/0.3240.354/0.136
Visual Conditioning (Ours)0.302/0.1740.397/0.1960.337/0.126

The results show that visual conditioning consistently achieves lower RMSE and MAE across all three datasets. These improvements provide direct evidence of the additional benefits of visual representation for time series imputation. We have added this controlled comparison and the corresponding discussion to Section 4.2 (Ablation Study) of the revised manuscript.

Q2: Related work and novelty positioning.

A2: We sincerely thank the reviewer for highlighting these relevant works and the need to clarify our novelty.

Regarding ImagenTime, we acknowledge that invertible time-series-to-image transformations enable image-domain generation and interpolation without an inherent information loss from the transformation. We have refined our discussion of the modality gap, which primarily concerns non-invertible line-graph rendering. Unlike ImagenTime, VT-Diff uses visual representations as structural conditioning signals while performing probabilistic imputation directly in the numerical domain.

Regarding Perceiver IO, we acknowledge the similarity in using learnable queries and cross-attention. Our Cross-Attention Bottleneck builds on this established mechanism, specifically aligning ViT patch features with temporal positions to generate conditioning signals for diffusion-based imputation, rather than directly decoding output values.

We have incorporated both references and expanded the related-work discussion in Appendix A, with the revisions highlighted in blue.

Q3: Some claims go beyond the evidence.

A3: We sincerely thank the reviewer for this careful and constructive assessment. We acknowledge that some claims in the original manuscript were overstated. While VT-Diff achieves competitive performance across diverse missingness settings, it does not outperform all baselines in every case, and some improvements fall within the reported standard deviations. Regarding structural coherence and uncertainty estimation, we acknowledge that CRPS evaluates marginal predictive distributions and does not directly capture joint temporal structure. Moreover, narrower prediction intervals do not necessarily indicate better calibration.

We have revised the manuscript accordingly to ensure that our conclusions are fully supported by the available evidence, with all revisions highlighted in blue.

Q4: Clarification of image rendering mask, visual-condition integration, variable-specific information, and input configurations for high-dimensional datasets.

A4: We sincerely thank the reviewer for this important question. During training, images are rendered from the conditional observations X0ob=X⊙MX_0^{ob}=X\odot M, rather than MoriM^{ori}, ensuring that artificially masked targets are excluded from visual conditioning. As illustrated in Fig. 9, the visual condition c is first projected through an MLP and then injected into the denoiser through cross-attention in both Temporal Learning and Feature Learning modules. Although c has no explicit variable axis, variable-specific patterns are encoded through the spatial arrangement and distinct colors of individual line graphs, and further exploited by Feature Learning. For PEMS-Traffic, we use the first 32 variables and a uniform image resolution of 384 ×\times 384. We have clarified the conditioning mechanism in Section 3.3 and Appendix D.5, and provided the dataset configuration in Appendix D.1.

Thanks for the authors' reply. All of my concerns are addressed and I would like to keep my initial rating.

Thank you for your careful review and continued support. Your valuable comments and suggestions have greatly helped us improve and strengthen our work. We sincerely appreciate your time and effort.

Reviewer_uDG9OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a visual-temporal diffusion framework for probabilistic time series imputation, where visual representations of incomplete sequences serve as conditioning signals for the diffusion denoising process. Experiments across seven datasets and three missingness patterns against 17 baselines demonstrate the effectiveness of the method.

Critical strengths
  1. Overall, the paper is technically sound. Using visual representations to guide numerical generation is a sensible design that combines a structural inductive bias with
  2. The experiments cover diverse application domains, representative baselines, and missing patterns, demonstrating substantial improvements over SOTA methods in several metrics.
  3. Ablation studies, hyperparameter sensitivity and in-depth analysis further support the effectiveness of each module design.
Critical weaknesses
  1. The performance improvements may arise from visual pretraining, increased model capacity, or auxiliary supervision. The current ablation studies do not adequately isolate the contributions of these factors.
  2. The main experiments use sequences of length 48 and retain only the first 32 variables in high-dimensional datasets. Although Appendix E.3 examines sensitivity to sequence length, it remains unclear whether the reported gains over the baselines persist for longer sequences and when all variables are included.
  3. Sensitivity to the interpolation choice is also unexplored.
Questions for authors
  1. How much improvement remains when the same interpolated observations are supplied through a numerical conditioning encoder with comparable capacity and same auxiliary supervision?
  2. How important is visual pretraining?
  3. How does the method perform on longer sequences and the full set of variables in high-dimensional datasets?
1 comment

We sincerely thank the reviewer for the positive assessment and valuable feedback. Our responses are provided below.

Q1: How much improvement remains when the same interpolated observations are supplied through a numerical conditioning encoder with comparable capacity and same auxiliary supervision?

A1: We thank the reviewer for raising this important question. To answer it, we compare VT-Diff with a capacity-matched numerical conditioning baseline. Both variants share the same model and training settings, differing only in their conditioning inputs: rendered trajectories versus interpolated values with the corresponding conditional mask. The evaluation is conducted on ETTh1, ILI, and PEMS-Traffic under 50% point missingness. The results are presented as follows (reported as RMSE / MAE):

Conditioning VariantETTh1ILIPEMS-Traffic
Numerical Conditioning0.354/0.2560.600/0.3240.354/0.136
Visual Conditioning (Ours)0.302/0.1740.397/0.1960.337/0.126

As shown in the table, visual conditioning consistently outperforms numerical conditioning across all three datasets. These results demonstrate that the improvements cannot be attributed solely to increased model capacity or auxiliary supervision, highlighting the additional contribution of visual conditioning to time series imputation. We have added this controlled comparison and the corresponding discussion to Section 4.2 (Ablation Study) of the revised manuscript.

Q2: How important is visual pretraining?

A2: We sincerely thank the reviewer for raising this important question. To evaluate the contribution of visual pretraining, we conducted an additional ablation study comparing VT-Diff with a randomly initialized ViT under identical experimental settings. The results under 50% point missingness (RMSE/MAE) are presented as follows:

VariantETTh1ILIPEMS-Traffic
w/o Visual Pretraining0.362/0.2570.441/0.2230.423/0.221
VT-Diff (Ours)0.302/0.1740.397/0.1960.337/0.126

The results show that visual pretraining consistently leads to lower RMSE and MAE across all three datasets, demonstrating its important role in learning effective visual representations. We have added these additional ablation results and the corresponding discussion to Section 4.2 (Ablation Study).

Q3: How does the method perform on longer sequences and the full set of variables in high-dimensional datasets?

A3: We sincerely thank the reviewer for this valuable suggestion. To evaluate performance on longer sequences, we conducted additional experiments on ETTh1 and PEMS-Traffic with sequence lengths of 48, 96, and 128 under 50% point missingness. The RMSE results are presented below:

DatasetSequence LengthM.TCNGlocal.FSDIOurs
ETTh1480.3160.4150.3010.298
ETTh1960.3150.4180.4960.300
ETTh11280.3240.4070.4630.321
PEMS-Traffic480.4400.4830.4030.336
PEMS-Traffic960.4180.4070.4740.376
PEMS-Traffic1280.4170.4150.4810.414

As shown in the table, VT-Diff achieves the lowest error across all tested sequence lengths on both datasets. Although the error increases with sequence length, the model maintains competitive performance compared with the baselines.

Regarding the full set of variables in high-dimensional datasets, due to limited computational resources, our experiments retain only 32 variables. We have clarified the 32-variable setting in Appendix D.1 and added a discussion of this potential limitation in the appendix, with the revisions highlighted in blue.

Q4: Sensitivity to the interpolation choice is also unexplored.

A4: We thank the reviewer for pointing out this aspect. To investigate the sensitivity to interpolation choices, we evaluate three interpolation methods (Linear, Cubic Spline, and PCHIP) on ETTh1, ILI, and PEMS-Traffic under 50% point missingness, with RMSE/MAE as evaluation metrics. The results are presented as follows:

Interpolation MethodETTh1ILIPEMS-Traffic
Cubic Spline0.308/0.1720.392/0.1930.334/0.122
PCHIP0.305/0.1700.401/0.1980.338/0.124
Linear (Ours)0.302/0.1740.397/0.1960.337/0.126

As shown in the table, the three interpolation methods yield comparable imputation performance. Linear interpolation remains competitive while offering a simple and efficient rendering strategy. These results suggest that VT-Diff is not overly sensitive to the choice of interpolation method. We have added this sensitivity analysis and the corresponding discussion to Appendix E.3.

Reviewer_dVbROpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VT-Diff for probabilistic time series imputation. It renders incomplete observations as line graphs and uses a pretrained Vision Transformer to extract features that condition a numerical diffusion model. An attention module aligns visual features with temporal positions, while an auxiliary reconstruction objective supports representation learning.

Critical strengths
  1. Generation remains in numerical space, avoiding the need to recover numerical values from generated images.
  2. The evaluation covers seven datasets, multiple missing patterns, component ablations, downstream forecasting, and computational cost.
Critical weaknesses
  1. The ablations show that the proposed visual branch is useful, but do not clarify whether its benefits are specific to the image representation. A comparable numerical conditioning branch using the same interpolated observations and reconstruction supervision would help distinguish the value of rendering from the benefit of adding an auxiliary encoder.
  2. The main comparisons include severe missingness and long gaps, but the component study is restricted to moderate random masking. It therefore does not directly establish whether the visual branch accounts for the gains in the difficult settings used to motivate the method. A targeted analysis of these settings, including sensitivity to the interpolation used for rendering, would make the explanation more convincing.
  3. CDTI [1], which the paper discusses but does not evaluate, is a particularly relevant comparison because it also conditions diffusion on features extracted from a transformed representation of the observations. Comparing with CDTI, where feasible, would help assess the value of visual features relative to a closely related spectral conditioning approach. A recent method such as T1 [2] would also provide a useful comparison for reconstruction accuracy under sparse observations.
  4. CRPS provides meaningful evidence about marginal predictive distributions, but does not directly establish structural coherence within generated trajectories. A targeted assessment of temporal or variable dependence would strengthen this claim. The interpretation of narrower prediction intervals also requires coverage results or a more restrained discussion.

My main concern is that the current evidence does not establish a sufficiently distinct and significant contribution from the visual representation. The reported improvements leave unresolved whether the advantage depends on representing the observations visually, and this uncertainty concerns the central contribution of the paper. This is the main basis for my rejection recommendation.


Reference

[1] Liu et al., “Time-Frequency Conditioned Diffusion for Multivariate Time Series Imputation,” ICDE, 2026.

[2] Park et al., “T1: One-to-One Channel-Head Binding for Multivariate Time-Series Imputation,” ICLR, 2026.

Questions for authors

I have no additional questions beyond the points raised in the weaknesses above.

4 comments
AuthorsOpenReview ↗

Responses to W1, W2

We sincerely thank the reviewer for the valuable comments and suggestions. We have carefully considered each concern and provide our point-by-point responses below.

Q1: Evaluating the Unique Benefits of Visual Representation.

A1: We sincerely thank the reviewer for this valuable suggestion. To isolate the contribution of visual representation, we compare VT-Diff with a capacity-matched numerical conditioning baseline. Both variants share the same model and training settings, differing only in their conditioning inputs: rendered trajectories for the visual variant and linearly interpolated values with the corresponding conditional mask for the numerical variant. We evaluate both variants on ETTh1, ILI, and PEMS-Traffic under 50% point missingness, with results reported in terms of RMSE/MAE:

Conditioning VariantETTh1ILIPEMS-Traffic
Numerical Conditioning0.354/0.2560.600/0.3240.354/0.136
Visual Conditioning (Ours)0.302/0.1740.397/0.1960.337/0.126

As shown in the table, visual conditioning consistently outperforms numerical conditioning across all three datasets. These results suggest that the performance improvements are not merely attributable to the introduction of an auxiliary encoder or reconstruction supervision. Instead, rendering time series as images provides additional structural information that benefits the imputation process. We have added this controlled comparison and the corresponding discussion to Section 4.2 (Ablation Study) of the revised manuscript.

Q2: Evaluating Visual Branch Contributions under Challenging Missingness and Sensitivity to Interpolation Methods.

A2: We sincerely thank the reviewer for this valuable and constructive suggestion. To further assess the visual branch under severe missingness, we conduct an additional ablation study at 90% point missingness on ETTh1, ILI, and PEMS-Traffic. The results are presented as follows (reported as RMSE / MAE):

VariantETTh1ILIPEMS-Traffic
w/o Visual Condition0.568/0.3521.197/0.5760.588/0.278
VT-Diff (Ours)0.535/0.3341.019/0.5550.519/0.238

As shown in the table, VT-Diff consistently achieves lower RMSE and MAE across all three datasets. These results demonstrate the effectiveness of visual conditioning even under extremely high missingness.

We further investigate the sensitivity of VT-Diff to different interpolation methods, including Cubic Spline, PCHIP, and Linear interpolation, under 50% point missingness. The results are presented as follows(reported as RMSE / MAE):

Interpolation MethodETTh1ILIPEMS-Traffic
Cubic Spline0.308/0.1720.392/0.1930.334/0.122
PCHIP0.305/0.1700.401/0.1980.338/0.124
Linear (Ours)0.302/0.1740.397/0.1960.337/0.126

The results show that the three interpolation methods yield comparable performance, with relatively small variations across datasets, demonstrating the robustness of VT-Diff to interpolation choices. We adopt linear interpolation for its simplicity and computational efficiency while maintaining competitive imputation performance. We have added this sensitivity analysis and the corresponding discussion to Appendix E.3.

AuthorsOpenReview ↗

Responses to W3, W4

Q3: Comparison with Recent and Closely Related Methods (CDTI and T1).

A3: We sincerely thank the reviewer for suggesting these relevant comparisons. Following this suggestion, we added CDTI and T1 as additional baselines and evaluated them on ETTh1, ILI, and PEMS-Traffic under point, sequential, and block missing patterns. Specifically, we adopted a 50% missing rate for point and block missing, and a subsequence length of 20 time steps for sequential missing. The results (RMSE/MAE) are presented as follows:

DatasetSettingCDTIT1VT-Diff (Ours)
ETTh1POINT0.606/0.3280.317/0.1760.302/0.174
ETTh1SEQ0.325/0.1910.296/0.1870.248/0.149
ETTh1BLOCK0.774/0.4980.478/0.2990.477/0.283
ILIPOINT1.454/1.1830.577/0.3630.397/0.196
ILISEQ1.034/0.7341.138/0.7360.432/0.236
ILIBLOCK1.806/1.1951.047/0.6580.995/0.539
PEMS-TrafficPOINT0.427/0.1710.409/0.2060.337/0.126
PEMS-TrafficSEQ0.447/0.1820.417/0.1970.311/0.121
PEMS-TrafficBLOCK0.457/0.1950.433/0.2090.337/0.131

As shown in the table, VT-Diff consistently achieves lower RMSE and MAE than both CDTI and T1 across all three datasets and missing patterns. These results demonstrate the effectiveness of VT-Diff across different missing patterns and its competitive reconstruction performance. We will incorporate these comparisons into the revised manuscript and expand the discussion of the differences between visual conditioning and existing spectral-conditioning approaches.

Q4: Clarifying Structural Coherence and Uncertainty Evaluation.

A4: We thank the reviewer for this insightful comment. Regarding CRPS, we note that it has been adopted in previous probabilistic generative imputation studies, including FGTI [1], which uses CRPS to evaluate the discrepancy between learned and ground-truth predictive distributions. Following this established practice, we use CRPS to assess the quality of marginal predictive distributions. Nevertheless, we agree that CRPS alone does not directly quantify temporal or cross-variable dependencies within generated trajectories. We have clarified the scope and limitations of CRPS in the Metrics paragraph of Appendix D.4 (Evaluation Metrics), with the revisions highlighted in blue.

Regarding prediction intervals, we agree that narrower intervals alone do not necessarily indicate better uncertainty calibration. We have revised the discussion of prediction intervals in Appendix E.2 (Qualitative Visualizations), with the revisions highlighted in blue.

[1] Yang X, Sun Y, Yuan X, et al. Frequency-aware generative models for multivariate time series imputation[J]. Advances in Neural Information Processing Systems, 2024.

Thank you for addressing my comments; I have raised my score from 1 to 2, though the overall contribution and significance remain somewhat limited for acceptance.

Thank you very much for considering our responses and updating your assessment. We sincerely appreciate your acknowledgment of our clarifications and understand that you still have concerns about the overall contribution and significance of our work.

1. Methodological contribution

To clarify the contribution of our VT-Diff, we highlight its key differences from related approaches. Unlike ImagenTime[1], which performs diffusion in the image domain, and CDTI[2], which uses spectral features to condition diffusion, our VT-Diff uses visual features as structural conditions for numerical-domain probabilistic imputation, thereby combining global visual structure with precise numerical reconstruction. Furthermore, it is precisely this combination that enables VT-Diff to avoid pixel-space generation and “the need to recover numerical values from generated images”. This advantage was highlighted in your comment and similarly recognized by Reviewer 2GdK as a novel cross-modal design.

2. Supporting evidence

To further substantiate these contributions, our experimental results provide comprehensive evidence. Specifically, the ablation studies in Section 4.2, as shown in Table 3, verify the benefits of visual conditioning beyond model capacity and auxiliary supervision, as well as the contribution of visual pretraining. The sensitivity analysis in Appendix E.3, as shown in Table 9, evaluates the impact of line-graph representations. Experiments under 90% point missingness further demonstrate the effectiveness of visual guidance under severe missingness. These findings highlight the potential of VT-Diff for probabilistic imputation with severely incomplete observations and provide a basis for further exploring visual structural priors in time-series reconstruction.

We hope that these distinctions and supporting results help clarify the contribution and significance of our work, and would sincerely appreciate to receive your favorable consideration and support for our work. Thank you again for your time and thoughtful evaluation.

[1] Naiman I, Berman N, Pemper I, et al. Utilizing image transforms and diffusion models for generative modeling of short and long time series[J]. Advances in Neural Information Processing Systems, 2024.

[2]Liu Y, Wang Z, Liu J, et al. Time-Frequency Conditioned Diffusion for Multivariate Time Series Imputation[C]//2026 IEEE 42nd International Conference on Data Engineering (ICDE). IEEE, 2026.

Comments on the paper

1 comment

We sincerely thank all reviewers for their insightful reviews and positive perspectives, which are a great encouragement to us. Specifically, the reviewers commented:

  • The proposed approach presents a novel cross-modal idea (Reviewer 2GdK), with clear motivation and a sensible design (Reviewer EZng). Furthermore, the motivation targets a real pain point (Reviewer 2GdK).
  • The framework avoids the trap of “generating in pixel space” (Reviewer 2GdK), with generation remaining in numerical space (Reviewer dVbR). Moreover, using visual representations to guide numerical generation is a sensible design (Reviewer uDG9). The use of learnable temporal anchors also provide a clean interface that could transfer to other conditional diffusion frameworks (Reviewer 2GdK).
  • The paper provides extensive empirical evidence (Reviewer EZng). The evaluation covers seven datasets, multiple missing patterns, component ablations, downstream forecasting, and computational cost (Reviewer dVbR), demonstrating substantial improvements over SOTA methods in several metrics (Reviewer uDG9). Ablation studies, hyperparameter sensitivity and in-depth analysis further support the effectiveness of each module design (Reviewer uDG9).
  • The paper is well written and well presented (Reviewer EZng), being clearly organized and easy to follow (Reviewer EZng).

The reviewers also raised insightful and constructive concerns. We made every effort to address all the concerns by providing sufficient evidence and requested results.

Here is the summary of the major revisions:

1. More Ablation Studies [Reviewers: dVbR, uDG9, EZng, 2GdK]

We have added capacity-matched numerical conditioning and visual pretraining comparisons in Section 4.2 (Table 3), as well as an additional ablation study under 90% point missingness. The results demonstrate the contributions of visual representations and visual pretraining, and further confirm the effectiveness of visual conditioning under severe missingness.

2. Additional Baseline Comparisons [Reviewers: dVbR, EZng, 2GdK]

Following the reviewers' suggestions, we have added CDTI and T1 as additional baselines and evaluated them across three datasets and three missing patterns. We have also expanded the related-work discussion to clarify the differences between VT-Diff and existing spectral-conditioning approaches, as well as image-based time series generation and imputation methods.

3. Sensitivity Analysis and Visual Representation Comparisons [Reviewers: dVbR, uDG9, 2GdK]

We have conducted additional experiments comparing Linear, Cubic Spline, and PCHIP interpolation, as well as different visual representations, including GAF, RP, and Delay Embedding. The results demonstrate the robustness of VT-Diff to interpolation choices and the effectiveness of line-chart representations. We also evaluated the method on longer sequences, confirming its competitive performance as sequence length increases.

4. Additional Theoretical and Mechanistic Explanations [Reviewers: EZng, 2GdK]

We have further clarified the theoretical motivation and underlying mechanisms of visual conditioning, including the roles of visual representations and pretraining, the extraction of geometric patterns, and the generation of nonlinear variations within missing intervals. These explanations provide further insights into how visual structural information guides the diffusion model for time series imputation.

Regarding the review status, Reviewer uDG9 has not yet responded, and Reviewer EZng has maintained a Weak Accept rating. Reviewer dVbR raised their rating from 1→21 \rightarrow 2, while Reviewer 2GdK raised their rating from 2→32 \rightarrow 3 and increased their confidence rating to 4.

All updates are highlighted in blue in the revised paper. The valuable suggestions from reviewers are very helpful for us to revise the paper to a better shape. We hope you will appreciate our revised version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.