acceptodds
Under review as a conference paper at ICLR 2027

UGapSyn: Utility-Gap Synthesis for Adapting Financial Time-Series Foundation Models

Abstract

Synthetic price paths can supplement the limited market history used to adapt time-series foundation models to financial forecasting, and generators are usually judged by how realistic these paths are. Under a fixed adaptation budget, however, each synthetic window replaces a real one, so even a realistic path can hurt the adapted model if the displaced real window was more useful. To capture this trade-off, we define the utility gap as the reduction in forecasting risk relative to the same adaptation without replacement. Because this gap depends on the model being adapted and on the real windows being replaced, it must be measured by adapting the same model with and without the replacement. Utility-Gap Synthesis (UGapSyn) makes this comparison by continuing adaptation from a shared state with synthetic or real windows at the replaced positions, and selects the configuration of a controllable GARCH-jump generator by the gap measured on validation data. During adaptation, it adjusts the amount and kind of synthetic data only when repeated measurements on historical data agree. With three foundation models on two US equity panels and one Chinese panel, UGapSyn achieves lower mean test MSE than real-data adaptation in all nine settings and than all five synthesis and condensation baselines on both US panels for every model. On Chronos-Bolt-Base, it lowers MSE relative to real-data adaptation by 2.4% on US_71 and 2.7% on US_14L.

Primary area:
learning on time series and dynamical systems
Keywords:
time-series foundation models, financial time series, synthetic data, model adaptation, time-series forecasting

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_kb3SOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper tries to addressing the efficient usage of synthetic price path data on training time-series foundation models. The authors define the utility gap as the reduction in forecasting risk relative to the same adaptation without replacement. UGapSyn makes the comparison by continuing adaptation from a shared state with synthetic or real windows at the replaced positions, and selects the configuration of a controllable GARCH-jump generator using the gap measured on validation data. Experiments on US and China stock markets further show the effectiveness of UGapSyn.

Critical strengths
  • High-quality financial data is usually limited. Effective usage of synthetic data is important to train financial time series foundation model.
  • A controllable GARCH-jump generator is reasonable design with finance insights.
  • The flow of this paper is easy to follow.
Critical weaknesses
  • As real history market data is always considered very valuable, I do not think budget is the bottleneck for training financial foundation model. As a result, I think adding synthetic data as extra information is better than replacing them with real historical data.
  • The setting of this work has the potential to be more general. Besides financial settings, it may be extend to the training of different time-series foundation models on different domains.
  • Even though there are lots of generation models working on creating synthetic financial market data, there is no gold standard to verify the how realistic they are. Unlike CV or NLP, it is hard to distinguish the quality of synthetic financial time-series data. Addressing the quality of synthetic financial data is the preliminary for the success of this work.
  • UGapSyn use the gain of data samples to decide the usefulness of different data samples. However, explanability is quite important in finance, using one performance on some specific testing periods is not promising enough. It is more likely to be overfitting than real improvement.
  • Very large scale experiments are required as this is about training foundation models. The current datasets are at most medium-level for me.
  • Using time-series data sample with part of overlap can enlarge the size of available data. Maybe this can be used in the design of this framework.
  • Although I can see the potential of this work, the current proposed method is a bit incremental for me.
Questions for authors

See weakness above.

6 comments

Thank you for your review and comments! We have considered each of them carefully, and the paper is clearer and better supported as a result. We respond to each point below. All changes in the revised paper are in blue, and the margin tags that refer to your comments are in purple (kb3S·W1, for example). The tags for the other two reviewers are in red and green.

W1: Replacing or adding synthetic data

Thank you for this comment on the problem setting, which led to one of the most useful additions in this revision. The setting does not assume that the computing budget is a bottleneck. It fixes the budget so that synthetic data are compared with the real data they replace under the same conditions (points (1) and (2)). We also ran the experiment you propose, in which synthetic data are added to the real data (point (3)).

(1) Why the budget is fixed even though it is not the bottleneck. We fix the budget so that a change in forecasting error can be attributed to the data, which Section 2 now states. Spending more updates on the real data did not help in our controls either: extending real-data adaptation to 1,092 updates raises test MSE from 0.1877 to 0.2433 on US_71, from 1.1229 to 1.4204 on US_14L and from 1.9575 to 2.0425 on CSI300 (Table 5). The question is therefore which windows should fill a short schedule.

(2) How the method treats the value of real historical data. Real data are the reference in our formulation. A synthetic window counts as useful only if the adapted model forecasts better with it than with the real window it replaces (Eq. 3), and the synthetic configuration is selected by this comparison on validation, where it improves on real-data adaptation (Appendix B.8).

(3) Adding synthetic data as extra information versus replacing real data. Synthetic data help when they are added as well. We kept all 128 real windows in each block and added 64 windows, either synthetic or further real ones, so that both cases use 76 updates.

ConfigurationUpdatesUS_71US_14L
Real windows only520.18761.1171
Replace 50% (fixed zero drift)520.18291.0859
Add 64 real windows per block760.19001.1197
Add 64 synthetic windows per block760.18421.1023
Add synthetic minus add real [95% CI]-0.0058 [-0.0077, -0.0041]*-0.0174 [-0.0258, -0.0090]*

Adding synthetic windows gives lower MSE than adding real windows by 0.0058 and 0.0174, with intervals excluding zero, and a lower mean than real-data adaptation with 52 updates. The benefit of synthetic data thus does not depend on the replacement setting. Adding more real windows does not improve on real-data adaptation (0.1900 against 0.1876, and 1.1197 against 1.1171), which is consistent with point (1), and replacement with 52 updates still has the lowest mean on both panels. The references in this table are those of Appendix B.8 (Table 10).

Revision: (1) Section 2 now explains that the budget is fixed so that a change in forecasting risk can be attributed to the training data. (2) Section 4.2 summarizes the experiment in which windows are added, with full results in Appendix B.8 (Table 13).

W2: Generality beyond finance

We focused the experiments on finance because market history is limited and non-stationary, so the question of whether synthetic data are worth using is most pressing there. The method itself is not tied to finance, for two reasons.

(1) The definition and the measurement are not specific to finance. Replacement utility only requires a pretrained model, a real training sequence and held-out feedback data (Section 2), and the three backbones we adapt are general time-series foundation models.

(2) The generator is the only finance-specific part. In another domain GARCH-jump would be replaced by a generator suited to the data there. Replacing it with fractional Brownian motion or a block bootstrap leaves the rest of the method unchanged, and the measurement proceeds in the same way (Appendix B.8, Table 14).

Revision: Section 5 now notes that the definitions make no financial assumption and that the approach can be applied to other domains with a suitable generator.


W3: Realism of synthetic financial data

The quality of generated financial series is indeed hard to judge directly, and there is no accepted standard of realism. This is the situation the paper addresses. It measures the effect of synthetic data on forecasting directly, so a realism standard is not a prerequisite for it, and the introduction now says so. The three points below describe this measurement, why fidelity metrics cannot take its place, and the basic checks that generated paths still pass.

(1) Judging synthetic financial data without a gold standard for realism. Without such a standard, we judge synthetic data by replacement utility. From the same starting state and under the same budget, adaptation continues with either the synthetic windows or the real windows they replace, and the two forecasting errors are compared (Section 2). Synthetic data count as good only if they help forecasting more than the real data they replace.

(2) Whether fidelity metrics could serve as the quality criterion. Fidelity metrics do not rank sources consistently in our experiments. On the US panels smaller fidelity distances generally go with larger utility, but on CSI300 their rank correlations with utility lie between -0.35 and +0.17 (Section 4.3, Tables 6 and 8). Fidelity can also miss a harmful source: on US_71 the energy distance of SDForger is in the same range as that of our parametric sources, yet its MSE is 0.2192 against 0.1877 for real-data adaptation (Table 1).

(3) Basic quality checks on the generated paths. Generated prices must be finite and positive, and a path must not duplicate a real window or another generated path (Section 3.1). Table 6 reports fidelity scores of all sources together with their utility, and Table 8 relates all six scores to utility.

Revision: (1) The introduction now states that financial series have no gold standard of realism against which fidelity scores can be calibrated. (2) It also states that evaluation against the displaced real data requires no separate judgment of realism.

W4: Explainability and overfitting

Thank you for raising these two concerns. Addressing them made the paper more explicit about how each decision is made and why the selected data help. Usefulness is decided on validation data, and test data are not used in selecting the configuration or in adjusting the policy (point (1)). The gain in the main results rests on this single choice, and points (2) and (3) explain why it is unlikely to overfit and what makes the selected data useful.

(1) Whether usefulness is decided by performance on the test periods. Usefulness is decided on the validation periods, and the test periods serve only to evaluate the frozen configuration. Each panel is split in time order into training, calibration, validation and test periods (Table 17). The generator configuration is selected on validation and then frozen, and runtime adjustments use only the calibration panels, which come before the validation period.

(2) Whether the improvement is overfitting to specific test periods. The selection leaves little room for overfitting. It chooses among a few candidates registered in advance, and the single selected configuration is applied to the three panels and all three backbones without reselection (Appendix B.3). With this one configuration, UGapSyn has lower mean test MSE than real-data adaptation in all nine combinations of backbone and panel (Table 1), and the ten-seed paired intervals on Chronos-Bolt-Base exclude zero on both US panels (Table 2). A separate ten-seed comparison on Chronos-Bolt-Small gives the fixed configuration a lower mean on all three panels as well (Table 4). On a later US period, from January 2024 to September 2026, which played no part in any selection, UGapSyn still has the lower mean (Table 3(b)). The gain is also not confined to one stretch of time: split into half-year blocks, the validated configuration is better than real-data adaptation in 14 of the 20 blocks (see our reply to Reviewer taeP, W3 & Q3).

(3) Explaining why the selected data are useful. The generator is a parametric financial model, so the selected data can be described and their effect traced to specific properties. The selected configuration has zero simulated drift, equal weights on four regions of variance scale and persistence, and 50% replacement. Removing the simulated drift changes test MSE from 0.1855 to 0.1832 on US_71 and from 1.9911 to 1.9531 on CSI300, while on US_14L the two drift settings are nearly equal (1.0874 and 1.0885, Table 3(a)). Where the drift setting matters, paths without a drift estimated from the training period help more, which agrees with a common observation in finance that the average return of one period is a poor guide to the next. The region weights we compared change test MSE by at most 0.0004 in the ten-seed paired comparison (Table 12) and by a few thousandths in the three-seed diagnostic (Table 6). The amount of synthetic data matters as well: lowering the replacement level from 50% to 30% raises test MSE from 0.1829 to 0.1854 on US_71 and from 1.0859 to 1.0990 on US_14L (Table 15).

Revision: (1) Sections 2 and 3 now state that all decisions use only training and feedback data and that test data are used in neither selection nor revision, and Section 4.1 describes the test and later evaluations as held-out. (2) Section 5 notes that, on US_71 and CSI300, the drift setting changes test MSE far more than the region weights compared in Appendix B.8. (3) Appendix B.3 describes the selected configuration in financial terms and states that it is used for all three backbones without reselection.

Reviewer_ptDZOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes UGapSyn for pretrained model adaptation on synthetic data with a fixed budget. The authors consider the task as a replacement problem and identify the gap between fidelity and usefulness of synthetic data. Specifically, UGapSyn contains a replacement utility that estimates the scores of candidate synthetic windows through paired continuations from the same adaptation state, and replacement policies that either keeps a real training window or replaces the window with a synthetic sequence generated by a controllable GARCH-jump process. The configuration the largest utility gap is selected and frozen before adaptation, while during adaptation, the replacement policies change only when repeated screening and a later confirmation agree. Experimental results show that the proposed UGapSyn generally achieves lower MSE than real data adaptation and other generative baselines across several foundation models on distinct market datasets.

Critical strengths
  1. This paper studies an advanced problem of fixed-budget adaptation of pretrained foundation models on synthetic data and correctly identifies the gap in latest ML time series research that fidelity and diversity are insufficient objectives, usefulness in downstream applications are more important.

  2. The designed replacement protocol is easy to follow and seems flexible to incorporate diverse time series generative models.

  3. UGapSyn achieves better results across several downstream forecasting models and datasets in the experiment.

Critical weaknesses
  1. The replacement is conducted for each asset independently. Therefore, the dataset after replacement may not preserve original multivariate structures, such as cross-asset dependence.

  2. This paper compares UGapSyn with several generative baselines, but does not clearly specify how the baselines are employed to generate synthetic data, some of which are originally designed for multivariate time series generation.

  3. Fixing the replacement level at 50% for the generative baselines is a little unfair, since UGapSyn naturally has a more flexible replacement level during adaptation by design.

Questions for authors
  1. Why is the GARCH-jump process selected for synthetic data generation? Deep generative models may lack comparable interpretable control, but many other statistical methods can be considered generative, such as fractional Brownian motion. Why not try these alternatives?

  2. How would UGapSyn + the bootstrap method perform? Would its performance be different from real data adaptation?

  3. Some generative baselines (e.g., Diffusion-TS, L2D-Diff) are proposed for multivariate time series generation, but are used here to generate univariate sequences. Is this a fair comparison? Dose the per-asset replacement policy constrain their full capability? How would their performance change if replacing multivariate windows?

  4. How would the performance of generative baselines change when using different replacement levels (e.g., 30% and 70%)?

5 comments

Thank you for your careful review and positive assessment! Your questions have been very helpful to us. We respond to each point below. All changes in the revised paper are in blue, and the margin tags that refer to your comments are in green (ptDZ·W1, for example). The tags for the other two reviewers are in red and purple.

W1: Cross-asset dependence

We replace single-series windows because the forecasters we adapt read one series at a time. The role of cross-asset dependence differs between these forecasters and a multivariate one, so we take the two cases in turn.

(1) Cross-asset dependence with the present univariate forecasters. Chronos-Bolt and TimesFM-2.5 process each window independently of the other series, and this is also the case when they are adapted on real data. The alignment between assets is therefore not used in training, and replacing a window does not remove cross-asset information that the model would otherwise use.

(2) Preserving cross-asset dependence with a multivariate forecaster. A forecaster that reads several assets together would need this dependence to be preserved. The unit of replacement should then be an aligned multi-asset window from a joint generator, and the utility gap and the paired measurement carry over unchanged.

Revision: (1) Section 5 now states that the evaluated backbones process each window independently, so replacement removes no cross-asset information that they use. (2) Section 5 also describes the multivariate case, in which the replaced unit is an aligned multi-asset window from a joint generator.


W2 & Q3: How the external methods are used and whether the comparison is fair

Thank you for your careful reading of the baseline setup! We have added Table 19 to Appendix C.3, which states how each baseline is run. Point (1) summarizes it, and points (2) to (4) answer the three questions in Q3.

(1) How each baseline is used to generate synthetic data. Each method is fitted on the training period of each panel and generates univariate windows of length 148 (128 context days plus the 20-day horizon), saved as a fixed pool. During adaptation these windows fill the 64 replaceable positions in each block of 128, the same positions that UGapSyn may replace. Diffusion-TS and L2D-Diff run with one channel, FTS-Diffusion is itself a univariate method, SDForger uses a GPT-2 backbone, and MTT+CondTSF uses DLinear experts as in the official code. Table 19 also marks which methods use a shortened schedule or stop early on a held-out part of the training period.

(2) Whether it is fair to run multivariate generators on univariate series. We consider the comparison fair for the task studied here, the adaptation of a univariate forecaster. All methods share the pretrained weights, real-data plan, replaceable positions, 52 main updates and evaluation panels (Section 4.1), so the runs differ only in the windows placed at the replaced positions. The result also does not depend on runtime feedback: with the validated configuration held fixed at the same 50% level, our mean MSE on US_71 and US_14L is 0.1832 and 1.0885, below the best external means of 0.1865 and 1.0970 (Tables 1 and 3(a)).

(3) Whether per-asset replacement constrains their full capability. What limits the multivariate generators here is the forecaster. A multivariate window has to be split into single series before it is fed to Chronos-Bolt or TimesFM-2.5, so the forecaster cannot use its cross-asset relations. This holds for any generator.

(4) How their performance would change if multivariate windows were replaced. With the present forecasters, the cross-asset structure of multivariate windows cannot contribute, for the reason in point (3). Diffusion-TS and L2D-Diff can benefit from modelling assets jointly only when the forecaster also reads several assets together. That is a different forecasting task, to which the utility gap applies directly (see our reply to W1).

Revision: (1) Appendix C.3 has a new Table 19 that states how each baseline is run, and its text is aligned with the table. (2) Section 5 discusses the multivariate setting (see our reply to W1).

W3 & Q4: Replacement level of the external methods

The baselines are set to 50% because this is the initial level of UGapSyn, so that the methods differ only in the content of the data. Point (1) shows that the adjustable level is not what separates UGapSyn from the baselines, and point (2) reports the comparison at 30% and 70% that you suggest.

(1) Whether the flexible replacement level gives UGapSyn an advantage over baselines fixed at 50%. The difference between our method and the baselines is not due to adjusting the level. With the level fixed at 50% throughout and no adjustment, our mean MSE on US_71 and US_14L is 0.1832 and 1.0885, below the best external means of 0.1865 and 1.0970 and no higher than UGapSyn with runtime adjustment (0.1832 and 1.0930, Tables 1 and 3(a)).

(2) How the baselines perform at other replacement levels (30% and 70%). The comparison is unchanged at 30% and 70% for Diffusion-TS, which we evaluated at these levels together with our fixed configuration. Diffusion-TS is the external method closest to ours on US_71 and the only one below real-data adaptation on both US panels with this backbone, whereas FTS-Diffusion, which has a lower mean on US_14L, is above real-data adaptation on US_71 (Table 1).

Replacement levelDiffusion-TS, US_71Diffusion-TS, US_14LOurs, US_71Ours, US_14L
30%0.18681.10840.18541.0990
50%0.18611.10320.18291.0859
70%0.18601.10870.18421.0881
Real-data adaptation0.18761.11710.18761.1171

Diffusion-TS changes by less than 0.001 across the three levels on US_71, and 50% is its best level on US_14L, so the 50% level did not put it at a disadvantage. Our fixed configuration has a lower mean at all three levels on both panels. The other four external methods are above real-data adaptation at 50% on US_71 (0.1916 to 0.2192 against 0.1877, Table 1), so replacing fewer windows would be expected to move them toward real-data adaptation, which UGapSyn improves on with an interval excluding zero (Table 2). The references in this table are those of Appendix B.8 (Table 10).

Revision: (1) The new paragraph in Section 4.2 reports the comparison with Diffusion-TS at 30%, 50% and 70% replacement. (2) Appendix B.8 gives the full results (Table 15).


Q1: Choice of the GARCH-jump process

Thanks for the question, and for suggesting fractional Brownian motion. We chose GARCH-jump mainly for the control it gives over the generated data (point (1)), and we ran the experiment you suggest (point (2)).

(1) Why the GARCH-jump process was selected for generation. We chose GARCH-jump mainly because its properties can be set one at a time. Drift, variance scale, persistence, tails and jumps each have their own parameters, so a change in utility can be traced to a specific property of the synthetic data. With this control the paper shows that the drift setting matters much more than the region weights we compared. Removing the simulated drift changes test MSE by up to 0.038 across panels (Table 3(a)), whereas the region weights change it by at most 0.0004 in the ten-seed paired comparison (Table 12) and by a few thousandths in the three-seed diagnostic (Table 6). The process also reproduces the volatility clustering, heavy tails and occasional large moves of daily returns with few parameters (Section 3.1), which keeps the candidate paths within plausible market behavior. Fractional Brownian motion has Gaussian increments with constant volatility and a single shape parameter, the Hurst exponent, so it offers less room for this kind of analysis.

(2) Fractional Brownian motion as an alternative statistical generator. With fractional Brownian motion in place of GARCH-jump, the utility gap relative to real-data adaptation is +0.0046 and +0.0296 on US_71 and US_14L, with intervals excluding zero, and -0.0021 on CSI300. GARCH-jump gives +0.0047, +0.0311 and +0.0024 (the full table is in our reply to Q2). What the two sources share is that their volatility is set from the training returns while their paths carry no average return from the training period, which fits the role of the drift setting described in point (1). With sampled drift, the same generator is less useful on US_71 and CSI300, and on CSI300 it is worse than real-data adaptation (1.9911 against 1.9575, Table 3(a)). On US_14L the two drift settings are nearly equal. The utility-gap measurement therefore applies to other statistical generators as well, and we keep GARCH-jump for the control it gives.

Revision: (1) Section 3.1 now explains that GARCH-jump is used for the control it gives over drift and variance dynamics. (2) Section 3.1 also notes that paths from other sources can be evaluated by the same utility gap and points to the comparison in Appendix B.8 (Table 14).

Q2: UGapSyn with a bootstrap source

To see whether the benefit of synthetic data merely comes from reusing real data, we followed your suggestion and used a block bootstrap of training returns as the source, alone and mixed with GARCH-jump with equal probability. The table gives the utility gap of each source, the MSE of real-data adaptation minus the MSE with that source, so a positive value means the source is better.

Utility gap at 50% [95% CI]US_71US_14LCSI300
GARCH-jump (ours)+0.0047 [+0.0026, +0.0068]*+0.0311 [+0.0265, +0.0360]*+0.0024 [-0.0103, +0.0136]
Fractional Brownian motion+0.0046 [+0.0021, +0.0070]*+0.0296 [+0.0234, +0.0361]*-0.0021 [-0.0092, +0.0077]
Stationary block bootstrap+0.0009 [-0.0015, +0.0029]+0.0323 [+0.0265, +0.0388]*+0.0004 [-0.0170, +0.0131]
GARCH-jump + bootstrap (equal mixture, fixed 50%)+0.0025 [+0.0002, +0.0047]*+0.0340 [+0.0253, +0.0427]*-0.0009 [-0.0145, +0.0104]

(1) Whether UGapSyn with the bootstrap differs from real-data adaptation. With the bootstrap as its source, UGapSyn differs from real-data adaptation on US_14L and cannot be distinguished from it on the other two panels. On US_14L the bootstrap is better by 0.0323, with an interval excluding zero, about the same gain as GARCH-jump (+0.0311). On US_71 and CSI300 its gaps are +0.0009 and +0.0004, with intervals including zero, and on US_71 GARCH-jump is better than the bootstrap by 0.0038, with an interval excluding zero. Reused real returns therefore account for the gain on US_14L but not on US_71.

(2) Mixing the bootstrap with the GARCH-jump generator. The equal mixture gives +0.0025 and +0.0340 on the two US panels, both with intervals excluding zero, and the highest value on US_14L among all sources. On US_71, GARCH-jump alone is better than the mixture by 0.0022, with an interval excluding zero.

(3) Why the paper keeps GARCH-jump as its source. The paper selects the configuration by validation performance averaged over the three panels (Appendix B.3). On this criterion GARCH-jump has a mean validation ratio of 0.9775 against 0.9866 for the mixture, so the reported configuration continues to use GARCH-jump.

Revision: (1) Appendix B.8 reports the bootstrap source and its equal mixture with GARCH-jump (Table 14). (2) Section 3.1 points to this comparison.

We appreciate your close reading of how the baselines are used and your suggestion of other statistical sources. The comparisons at other replacement levels and with fractional Brownian motion and the block bootstrap are now in the paper. If anything is still unclear, we would be glad to explain it further.

Reviewer_taePOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

UGapSyn evaluates generated training windows against the real windows they replace under a fixed main adaptation budget. Paired continuations guide generator selection and possible policy updates, with lower mean test MSE than real-data adaptation in all nine primary settings, but little additional benefit from runtime revision.

Critical strengths

S1: The right reference for synthetic data. A synthetic window should be compared with the real window it displaces when the training budget is fixed. The utility-gap formulation makes this opportunity cost explicit.

S2: Controlled measurement of the replacement effect. The paired continuations share the starting model, optimiser state, and real-data schedule. This is a direct way to reduce unrelated training variation when comparing synthesis choices.

S3: Evidence beyond one configuration. The study covers three backbones and three equity panels, with paired U.S. comparisons and a later-period evaluation. These results support the value of the selected synthetic data.

Critical weaknesses

W1: Runtime revision adds little. The fixed zero-drift configuration matches or improves the main UGapSyn results. Runtime revision therefore adds 1,039 auxiliary updates without a clear benefit in the primary settings.

W2. Historical feedback transfers poorly. The measured historical utility often has the wrong sign for later utility, with weak rank correlations across periods. This limits the evidence that the controller can identify useful policy changes under market drift.

W3. Uncertainty is narrower than the general claim. The strongest intervals describe training-seed variation conditional on the selected configuration and frozen generator pools. They do not capture uncertainty from generator fitting, configuration selection, or a new market period.

Questions for authors
  1. Comparison with a simpler selection procedure. Can the authors compare against a simple fixed-generator validation search under the same total selection and adaptation budget? This would clarify whether repeated policy revision is worth its additional cost.

  2. When historical utility is informative. Under what observed conditions does historical utility predict later utility reliably? This is central to judging the controller, given the weak transfer reported in the audit.

  3. Broader uncertainty accounting. How sensitive are the gains to refitting the competing generators and evaluating additional time blocks? This would show whether the advantage extends beyond the particular frozen pools and training seeds.

3 comments

Thank you for your careful review and valuable comments! We respond to each point below. All changes in the revised paper are in blue, and the margin tags that refer to your comments are in red (taeP·W1, for example). The tags for the other two reviewers are in green and purple.

W1 & Q1: Cost of runtime revision and a simpler selection procedure

Validation selection and runtime revision do two different jobs in UGapSyn. Validation selects the configuration, and this is what produces the gain in the primary settings (point (1)). The search you suggest shows the same at a matched budget (point (2)). Runtime revision can protect a run whose starting configuration has not been validated for the panel at hand, which is where its cost can pay off (point (3)).

(1) Role of runtime revision in the primary settings. In the primary settings the starting configuration has already been selected on validation, so little is left for runtime revision to correct. Holding this configuration fixed and using only the 52 main updates gives an equal or slightly better result: UGapSyn minus the fixed configuration is 0.0000, +0.0045 and +0.0005 on US_71, US_14L and CSI300 (Table 2). The gain over real data in these settings therefore comes from the selected synthetic data and does not depend on the 1,039 auxiliary updates.

(2) Comparison with a fixed-configuration validation search under the same budget. We ran a validation search over 20 fixed configurations and compared its choice with real-data adaptation and with our fixed configuration. The search uses about 1,040 updates per panel, close to the auxiliary updates of one UGapSyn run, and ranks the configurations by the utility gap on validation.

Chronos-Bolt-Base, ten seeds, test MSEUS_71US_14LCSI300
Real-data adaptation0.18761.11711.9549
Fixed zero drift (control in Section 4.3)0.18291.08591.9525
Selected by the search (zero drift, region 2, 50%)0.18281.08561.9523
Search minus real data [95% CI]-0.0048 [-0.0069, -0.0027]*-0.0315 [-0.0362, -0.0271]*-0.0026 [-0.0129, +0.0096]

The search selects zero drift at 50% replacement, as the paper does. On test its choice is within 0.0004 of our fixed configuration on every panel and improves on real-data adaptation on both US panels with intervals excluding zero. Since the fixed configuration in turn matches or improves on UGapSyn (point (1)), the utility-gap criterion does at least as well when it is applied by a simple search as when it is applied within UGapSyn. Full results are in Appendix B.8 (Tables 10 to 12).

(3) When repeated revision is worth its 1,039 auxiliary updates. Runtime revision is worth its cost when a selected configuration is applied without repeating the selection, because it can then limit the loss if that configuration turns out to be unsuitable. Section 4.5 examines this case with retreat to real data enabled: mean error falls in two of the three harmful starts, and on CSI300 revision recovers 0.0246 of a 0.0336 loss (Table 4). When the configuration has been validated for the panel, as in the primary settings, a fixed configuration does as well with the main budget alone. The revised paper therefore presents runtime revision as an optional safeguard and reports its cost separately from the main budget.

Revision: (1) The abstract, contribution (1) and Section 3 now present runtime revision as an optional safeguard. (2) Contribution (3) states that a fixed configuration retains the gain with the 52 main updates alone. (3) Section 4.3 summarizes the budget-matched search, with full results in Appendix B.8 (Tables 11 and 12).

W2 & Q2: Transfer of historical utility

Thank you for this thoughtful comment. Runtime revision acts on utility measured on historical panels, so the paper audits whether such utility carries over to a later period (Section 4.4). The audit tells us where the controller can be relied on, and the gains in the main results are obtained without relying on it (see our reply to W1 & Q1). Points (1) and (2) explain the two audit results you cite, and point (3) gives the condition you ask for, which Section 4.4 now states.

(1) Why historical utility often has the wrong sign for later utility. The screening panel directly follows the training period, and utility measured there understates the later value of synthetic data. All 132 audited candidates improve on real replay in the test period, yet screening judges 46%, 81% and 67% of them worse on the three panels, which leaves the sign agreement of 54%, 19% and 33% reported in the paper (Tables 7(c) and 9). Acting on such feedback would mainly remove useful synthetic data. This agrees with the validation result, which disables retreat in the main configuration.

(2) Why the rank correlations across periods are weak. The candidates compared at runtime in the primary settings are all useful and differ very little. Their differences are about as large as the differences between two repeated measurements of the same candidate (separation ratio 1.03, Table 7(a)), so their order cannot be expected to hold in a later period.

(3) Conditions under which historical utility predicts later utility. Our results support a necessary condition: the difference between candidates has to exceed both the underestimation in point (1) and the disagreement between repeated measurements in point (2). The second part can be checked during adaptation, because this disagreement is measured at every refresh and scales the retreat threshold when retreat is enabled (Eqs. 8 and 10). The condition is not sufficient, since agreement across repetitions does not ensure agreement across periods. The candidates in the primary settings do not meet it. Runtime feedback did lead to a useful change from a harmful start, where revision recovers 0.0246 of a 0.0336 loss on CSI300 (Section 4.5).

Revision: (1) Section 4.4 now states the condition under which historical feedback can inform a policy change. (2) Section 4.4, Appendix B.6 and the captions of Tables 7 and 9 now say that the regrets in Table 7(b) are group medians (the text previously said means), and that the group ending on real data is one retreat and the following refresh of the same run. The numbers are unchanged.

W3 & Q3: Scope of the uncertainty

We appreciate this careful comment on what the intervals cover. The significance statements in the paper concern the ten-seed paired comparisons on Chronos-Bolt-Base, and Appendix B.1 states what these intervals are conditional on. We take the three sources you mention in turn.

(1) Sensitivity to refitting the competing generators. The gain over Diffusion-TS holds across three refits of that generator. Diffusion-TS is the external method closest to UGapSyn on US_71 and the only one below real-data adaptation on both US panels with this backbone (Table 1).

Diffusion-TS pool at 50%, ten seedsUS_71US_14L
Generator seed 10.18611.1032
Generator seed 20.18631.1029
Generator seed 30.18601.1111
Fixed zero drift (ours)0.18291.0859
Ours minus best refit [95% CI]-0.0032 [-0.0045, -0.0017]*-0.0170 [-0.0226, -0.0110]*

Our fixed configuration is below all three pools on both panels. Against the best pool on each panel the difference is 0.0032 and 0.0170, with paired intervals excluding zero, so the comparison with Diffusion-TS does not depend on one particular fit. Our own generator draws new paths in every seed, so the ten-seed intervals already include its randomness.

(2) Sensitivity to a new market period and additional time blocks. On a held-out later US period, from January 2024 to September 2026 with 534 forecast origins per panel, UGapSyn still has lower mean MSE than real-data adaptation and all five external methods on both panels (Table 3(b)). The configuration was frozen before this evaluation. We also split the test and later periods of the two US panels into half-year blocks and compared our fixed configuration with real-data adaptation in each. It is better in 14 of the 20 blocks, and in 8 of the 10 blocks whose paired intervals exclude zero. The two exceptions among these ten are the first half of 2025 on US_71 and the second half of 2023 on US_14L.

(3) Uncertainty from configuration selection. The search in our reply to W1 & Q1 repeats the selection over 20 candidates. It again selects zero drift at 50% replacement, and its choice is within 0.0004 of our fixed configuration on test.

Revision: (1) Section 4.2 has a new paragraph on the three refits of Diffusion-TS and on the 30% and 70% replacement levels, with full results in Appendix B.8 (Tables 15 and 16). (2) Appendix B.1 points to the refits and the search and states which intervals the significance statements use. (3) Section 4.2 and the conclusion describe the later US evaluation as held-out.

The matched-budget search you suggested, together with your questions on transfer and uncertainty, helped us state more clearly where the gain of UGapSyn comes from and what runtime revision is for. We are glad to discuss any of these points further.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.