- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper provides benchmarking across diverse model families (Diffusion, Flow Matching, Autoregressive, GANs), offering useful empirical insights regarding model convergence, sampling speed, and structural signal fidelity under non-stationary noise.
Critical strengths
The paper offers valuable practical design principles for sensor time-series generation, such as demonstrating that flow-matching models (SiT) serve as strong baselines, fixed-range min-max normalization outperforms z-score normalization for generative targets, and spectrogram time-frequency conditioning enhances high-frequency signal generation (e.g., EEG).
Critical weaknesses
As shown in Figure 3, the dataset is heavily skewed towards one specific task category. This disparity creates potential bias when reporting aggregated metrics across categories.
Paired metrics are computed "in the normalized signal space used during training". Finding 2 compares z-score and min-max by MSE/MAE in each scheme's own space. Those scales differ, so the numbers are not comparable. Per-window min-max also discards absolute amplitude that the metrics then cannot see, and it is itself fragile to artifact spikes.
Questions for authors
MAR and autoregressive baselines often depend heavily on the continuous-to-discrete tokenization or latent patchification strategy. How discrete/patch tokens were constructed for MAR on continuous multi-channel signals, and whether continuous AR formulations were considered?
4 comments
Response to Reviewer HWGB (2/2)
Q3: "MAR and autoregressive baselines often depend heavily on the continuous-to-discrete tokenization or latent patchification strategy. Could the authors clarify how discrete/patch tokens were constructed for MAR on continuous multi-channel signals, and whether continuous AR formulations were considered?"
We appreciate this question and would like to clarify how MAR handles continuous sensor signals in our implementation. Our MAR baseline already operates in a continuous-valued autoregressive formulation. Following Li et al. (2024), Autoregressive Image Generation without Vector Quantization, we model the conditional distribution of each token using a lightweight diffusion head. As described in Appendix G.3 (lines 2062–2070), signal values are not converted into discrete codebook indices or amplitude categories. Continuous autoregressive modeling is therefore already represented in our comparison. To adapt this formulation to sensor data, we operate directly on normalized raw signals, treating sensor channels analogously to RGB channels in images. We use a one-dimensional convolution to construct continuous embeddings from non-overlapping temporal patches, with its kernel size and stride both equal to the patch length. MAR performs masked autoregressive modeling over these patches, while its diffusion head directly generates the continuous waveform values across all channels within each patch. Our implementation therefore does not require a VAE latent representation or a VAE decoder after generation. This is analogous to the pixel-space MAR formulation demonstrated in Appendix D.1 of Li et al. (2024) [1], where the model directly generates RGB pixel patches without an image tokenizer. We have expanded the MAR description to make this continuous, tokenizer-free formulation explicit (Appendix G.3, lines 2070–2073).
References:
- Autoregressive Image Generation without Vector Quantization. NeurIPS, 2024.
We sincerely thank the reviewer once again for their time and constructive feedback. To support reproducibility and future research on sensor-signal generation, we are committed to releasing the following resources:
- Benchmark. The SensorGen benchmark construction pipeline, including preprocessing, task configurations, and evaluation protocols, subject to the data use agreements of the source datasets.
- Pretrained model. Checkpoints of the generative models trained for SensorGen’s task settings, where permitted by the applicable data use agreements, together with example code for inference.
- Code. We will open-source the complete code to facilitate reuse by the broader research community.
We hope our detailed responses and the additional experiments have addressed all of your concerns, and we would be happy to answer any remaining questions during the discussion period. We would be very grateful if you might consider raising your score in light of these updates.
Thanks a lot for your comment, the authors answered all my concerns. I ll keep my score.
Thank you for your encouraging remarks!
Thank you for your encouraging feedback and confirmation that our rebuttal has fully addressed your concerns! Your constructive comments, especially regarding the presentation of methodology and study design, was highly valuable and will be incorporated into the revised manuscript. Please feel free to let us know if there are other clarifications or experiments we can offer.
Response to Reviewer HWGB (1/2)
We thank the reviewer for the thoughtful feedback and for recognizing the practical value of our findings and design principles for sensor signals generation. We hope the clarifications and additional results below further address your concerns and reinforce your positive assessment of the paper.
We thank the reviewer for raising this concern. We agree that the distinction between dataset composition and evaluation weighting could be made clearer, and clarify our aggregation procedure below.
First, we would like to clarify that the data proportions shown in Figure 3 do not represent the weights used to aggregate task performance. Figure 3 provides a comprehensive breakdown of the collected data based on the number of windows available for each generation setting. These window counts do not determine how much each setting contributes to the reported performance averages.
In fact, as stated in Section 4.1 (lines 301–302), we aggregate results within each task category (e.g., Interpolation & Extrapolation) and report category-level averages. The results for different categories are reported separately in Tables 2–6. Therefore, the larger volume of sleep-imputation data does not influence the reported Semantic-to-Signal, Translation, or Editing results.
Second, we further prevent larger datasets from dominating the category-level averages by assigning equal weight to each task setting within the category. Each setting is evaluated on its corresponding dataset, and its contribution to the category-level average does not depend on the number of data windows in that dataset. This allows the results to better reflect model performance across different dataset properties and signal configurations.
Finally, we report the individual results for all 14 settings in Appendix C.1. For the reviewer’s convenience, we map the category-level averages to the corresponding setting-level results below:
Thus, the dataset-volume imbalance shown in Figure 3 does not give larger datasets disproportionate weight in the reported averages.
We have revised the evaluation description to state the equal-weight aggregation protocol explicitly in the revised paper(Sec. 4.1, lines 301–303).
We thank the reviewer for the insightful observations. We agree that comparing normalization strategies requires a common evaluation scale, and would like to clarify a possible misunderstanding with our evaluation procedures.
The normalization comparisons reported in our manuscript already use a common evaluation scale. Predicted time series from both normalization methods are mapped to the same [-1,1] scale defined by the corresponding ground-truth signal before computing MSE, MAE, and SMSE. The sentence cited by the reviewer was intended to describe the default evaluation setting for our main comparisons, which consistently use min–max normalization. While in Appendix F.2, Evaluation settings for analysis studies seeks to discuss analysis study-specific evaluation settings, it did not explicitly describe the shared-scale procedure for the normalization experiment. We apologize for this ambiguity and have fully clarified the procedure in the revised manuscript (Section F.2).
Furthermore, we extend the normalization comparison to glucose signal translation, using the same shared-scale evaluation protocol, to further substantiate Finding 2.
Min–max normalization achieves lower errors across all three metrics for both models, reducing MSE by 12.3% for DiT and 22.9% for SiT relative to z-score normalization. These results demonstrate that the advantage observed in our original analysis extends to an additional task, providing further support for Finding 2. In summary, the additional experiment strengthens Finding 2 by demonstrating the benefit of min–max normalization on another task under the common evaluation scale.
We have made the shared-scale evaluation procedure explicit (Appendix F.2, line 1959), and incorporated the additional normalization comparison on glucose signal translation (Sec. 5, lines 467–472; Appendix D.2, line 1324; Table 24).