acceptodds
Under review as a conference paper at ICLR 2027

Synthetic Control with Foundation Models: Nonlinear Gains and Context-Axis Sensitivity

Abstract

Synthetic control (SC) predicts a target’s untreated path using comparable unaffected units as donors. We show that classical SC with convex weights admits a restricted single-query attention form, motivating frozen foundation models (FMs) as in-context predictors. We formalize the finite-context tradeoff: nonlinear approximation gains must exceed the recovery error. We also identify context-axis sensitivity: transposing the same panel can change a checkpoint’s predictions. Our two-way centering procedure reduces the mean absolute log-MSE gap between orientations by 61–85%. Our simulations show tabular FMs recover nonlinear structure better than tuned CatBoost and kernel ridge when sufficient context exists. In a real-panel study spanning 673 prediction tasks from 12 datasets, we find that centered FMs are competitive when pre-treatment history is long relative to donor-pool size, while synthetic difference-in-differences (SDID) performs best when it is short. In a semi-synthetic experiment using real U.S. traffic-fatality data, we show that tabular FMs with a suitable context-axis interface can reduce the time to reach 80% test power for a 0.7-standard-deviation effect from 12 to 9 months versus the best baseline SDID, a 25% reduction.

Primary area:
foundation or frontier models, including LLMs
Keywords:
tabular foundation models, time-series foundation models, in-context learning, synthetic control, counterfactual prediction, transfer learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_JBhDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Synthetic control (SC) reformulates the evaluation of policy interventions as a prediction problem, which depends on the donor-target relationship before treatment and further utilize it to predict the target's unobservable path. However, the SC estimators are essentially constrained regressions because the features size are much larger than that of the pre-treatment window size, making it challenging for existing tabular learners. This work is therefore motivated to show the SC admits a single-query attention-form representation, which lays the foundation of using Foundation Model (FM) for it. Specifically, empirical explorations are established to further understand the recoverable nonlinear headroom and context-axis sensitivity.

Critical strengths
  1. The problem setting itself is meaningful in practice, learning the donor-target relationship from limited pre-treatment history and then apply the rule for post-treatment period reliably will help to estimate the impact of a policy with low time cost.
  2. The proposed two-way centering algorithm helps to reduce orientation disagreement dramatically, which reduces the context-axis sensitivity.
Critical weaknesses
  1. The writing in this paper requires significant improvement. Generally, a well-written paper should allow a reader to grasp the research idea or at least the specific problem being addressed simply by reading the abstract. However, I found it difficult to extract these essential information from the abstract. More concerning is the inclusion of numerous terms that are left explanation within the given context, such as convex weight, finite-context tradeoff, and context-axis sensitivity. Almost half of the abstract contents are just enumerating the experiments conducted rather than explaining the research gap of existing work and the specific challenges.
  2. In line 53, the author claimed that "Classical SC represents the prediction as an affine rule of donor labels, forgoing nonlinearity entirely. FMs can bring in the nonlinearity." However, a single activation function is sufficient to introduce non-linearity, is such a heavy FM really necessary? I'd like to see some ablation studies or more discussion that demonstrate the necessity of the FM.
Questions for authors
  1. The basic principle of the SC is to "learn the donor-target relationship before treatment and uses the donors’ post-treatment outcomes and pre-treatment histories to predict the target’s path that would have occurred without the treatment." as claimed in line 33. However, does the donor-target relationship keep stable across timesteps? More evidence are preferred, such as theoretical analysis or experiment on controlled synthetic data.
  2. What is the difference between "treats FMs as off-the-shelf regressors(line 50)" and "use frozen FMs for prediction task (line 44)"?
  3. What is the meaning of convex weights in the abstract part?
1 comment
AuthorsOpenReview ↗

rewritten abstract, MLP ablation, new Theorem 1, Corollary 1, relationship stability

Thank you for recognizing the practical value of the problem and that two-way centering sharply reduces orientation disagreement.

PDF changes

  • Abstract and Introduction, pp. 1–2: state the problem, gap, and obstacles; define terms.
  • Section 3, p. 5: add Theorem 1 on orientation and regularization.
  • Section 4.1, Table 1, p. 7: add MLP and polynomial ridge at 20 and 64 donors.
  • Appendix B.6, p. 22: prove the new theorem.

1. Abstract and terms (W1, Q3). The revised abstract states the problem, the gap (when a pretrained FM beats methods fitted to the same data), and two obstacles: the error a nonlinear predictor removes must exceed the error of learning it from limited history (the "finite-context tradeoff"), and the same FM predicts differently depending on whether periods or units form its input rows ("context-axis sensitivity," p. 4). It explains each idea in plain words instead of just naming it. The Introduction defines unit, target, and donor with the 1987 U.S. speed-limit law (p. 1).

Convex weights (Q3) are nonnegative donor weights summing to one, so each prediction is a weighted average of donor outcomes; the abstract now says "weighted average."

Theorem 1 (p. 5) gives one mechanism for context-axis sensitivity. A tabular FM trained as a prior-fitted network approximates the Bayesian posterior mean under its training prior (Müller et al., 2022). Under an idealized linear-Gaussian prior with coefficient variance σw2/p\sigma_w^2/p, that mean is ridge with penalty proportional to the feature count pp, so transposing changes the penalty from γJ\gamma J (time rows) to γT0\gamma T_0 (unit rows).

2. Is a heavy FM necessary? (W2) A single activation makes a predictor nonlinear. The question is whether it can recover the nonlinear donor–target relationship from a short history accurately enough that the gain exceeds the error of learning it (Proposition 1, p. 4). We tested the single-activation case directly: a one-hidden-layer GELU MLP, tuned on pre-treatment validation for width, L2L_2 penalty, and stopping, averaging three refits. FM/MLP recovery-error ratios (MSE against the conditional mean; below one favors the FM) on 4,320 EX1(a) tasks per donor count, pooled over T0=32,128,512T_0=32,128,512 (geometric means, paired 95% tt-intervals over 30 seeds; Table 1):

FMJ=20J=20J=64J=64
TabFM0.62 [0.57, 0.68]0.36 [0.34, 0.39]
TabPFN0.98 [0.86, 1.13]0.44 [0.39, 0.50]
TabDPT0.80 [0.74, 0.87]0.82 [0.75, 0.89]

Against all four fitted nonlinear learners in Table 1 (CatBoost, kernel ridge, MLP, polynomial ridge), TabFM and TabDPT have lower recovery error at both donor counts, every interval below one. So does TabPFN, except with 20 donors against kernel ridge and the MLP, where the intervals include one.

On history needed: extending EX1(a) to T0=32,64,128,256,512T_0=32,64,128,256,512 (new simulations, not in the PDF), TabFM and TabPFN break even earlier than CatBoost in 37 and 34 of 90 nonlinear settings, and later in none; TabDPT is earlier in 20, all with 20 donors, and later in 10, all with 64. The FMs' weights are pretrained once and used frozen; the MLP is tuned and refit on every task.

Whether an FM is needed depends on the panel: with two-way centering, FMs are competitive with SDID when history is long relative to the number of donors, and SDID is more accurate otherwise (Section 4.3).

3. Stability of the donor–target relationship (Q1). Like every SC method, we assume the relationship learned before treatment continues to hold (p. 1); it cannot be verified for the treated unit after treatment (p. 10).

  • Controlled simulations. EX1(a) and EX1(b) hold the donor–target map fixed across the cutoff, so their recovery errors measure how well each method learns that stable relationship from limited history (Section 4.1).
  • Theory. Under two-way centering, adding arbitrary unit levels and a common time path, including shocks after the cutoff, to every unit changes neither the prediction errors nor the orientation disagreement of any fixed base learner (Proposition 3, p. 21). Only the relationship net of these components must be stable.
  • Real panels. On 673 real-panel tasks, post-cutoff target outcomes are withheld for scoring while donor outcomes remain available, so relationship changes after the cutoff enter the reported errors (Section 4.3).

4. "Off-the-shelf" versus "frozen" (Q2). Line 50 described prior work (Richards et al., 2026); line 44 described ours. The model use is the same by design: a pretrained model applied without weight updates. The difference is what is tested. SCBench compares pretrained models with three linear baselines and no nonlinear ones (p. 3). We test frozen FMs against four fitted nonlinear learners with a break-even criterion (Proposition 1), model their dependence on orientation (Theorem 1) and reduce it with two-way centering, and evaluate complete procedures on 673 real-panel tasks.

Reviewer_w3yUOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This work compares tabular foundational models (FMs) with statistical baselines such as ridge regression and gradient boosting methods such as CatBoost. The authors formalize the improvement gained by the FMs in terms of recoverable nonlinear headroom and show that under sufficient context, the FMs perform better than the statistical methods. Moreover, the authors test the context-axis sensitivity of the FMs and show that passing the context in a specific form improves the model predictions.

Critical strengths
  1. The authors have performed comprehensive experiments and compared many baselines on a large number of tabular datasets.

  2. The appendices are thorough including versions, tuning grids, DGP details, which helps reproducibility.

Critical weaknesses
  1. The paper is quite hard to follow. Several terms are used without ever explaining their meaning such as “nonlinear-headroom”, “centering”, “orientation disagreement”. The authors write in the initial part of the intro: “Many policy interventions cannot be evaluated by randomized trials because only one state, country, or cohort is treated. Synthetic control (SC) reframes this as a prediction problem. Similar untreated units, called donors, often share trends and shocks with the treated target, making their outcomes useful predictors.” But the authors never define what a donor, unit or the target is. What are “trends and shocks”.

  2. What are the contributions of this work? Showing that tabular FMs are better than ridge regression and time series based learners under sufficient context seems like an obvious conclusion to make. Moreover, showing that transposing the data boosts the performance of tabular FMs does not carry any meaning unless the authors can provide sound justification for it.

In Figure 2, authors showed how to transpose the table. It is too trivial, no?

  1. The paper includes fitted nonlinear baselines (CatBoost, KRR) in the primary comparison, so gains can’t be attributed to “nonlinearity” alone.

  2. The bound in Proposition 1 seems like a definition rather than a proposition that requires a proof. Similarly Proposition 2 has a one line proof and follows directly from the assumptions made. What is the insight from these propositions?

Questions for authors
  1. Can you add stronger nonlinear baselines, such as polynomial ridge regression, an MLP?

  2. Did the authors ensure any kind of equivalence between the methods before comparison in terms of computation or running time?

However, they are unlikely to change my rating.

5 comments
AuthorsOpenReview ↗

new Theorem 1, MLP and polynomial-ridge baselines, run time

Thank you for recognizing the breadth of our experiments and the reproducibility details in the appendices. We address your concerns below.

PDF changes; new experiments below

  • Abstract and Introduction, pp. 1–2: clarify terms and contributions.
  • Section 3, p. 5: add Theorem 1 on orientation and regularization.
  • Section 4.1, Table 1, p. 7: add MLP and polynomial-ridge baselines.
  • Appendix B.6, p. 22: prove the new theorem.

1. Clarity (W1). The Introduction now uses the 1987 U.S. speed-limit law: each state is a unit, pooled adopting states form the target, and states that kept the old limit are donors. Safer vehicles illustrate gradual trends. Severe winters illustrate shared shocks. Two-way centering subtracts unit averages and shared time movements before prediction, then restores them.

2. Orientation changes the conclusion (W2). Transposing a panel is trivial (Figure 2); its effect is not. Orientation disagreement is the gap between one model's predictions with time rows and with unit rows. Ridge with a common penalty, no intercept and no orientation-specific scaling has none (Shen et al., 2023). Tabular FMs do: across 673 real tasks, the three FMs with raw inputs have geometric-mean MSE ratios to classical SC of 1.68–2.60 with time rows but 0.97–1.09 with unit rows (weighting datasets, targets within datasets, and tasks within targets equally). Two-way centering lowers aggregate MSE in all six checkpoint–orientation cells and cuts the mean absolute log-MSE gap by 61–85% (EX3).

3. Why orientation matters (W2, W4). Regularization (new Theorem 1). A tabular FM trained as a prior-fitted network approximates the Bayesian posterior mean under its training prior (Müller et al., 2022). Under a linear-Gaussian prior with coefficient variance σw2/p\sigma_w^2/p, that mean is ridge with penalty proportional to the feature count pp: transposing changes it from γJ\gamma J (time as rows) to γT0\gamma T_0 (unit as rows). Unless T0=JT_0=J, the predictions differ and the orientation with more labeled rows is shrunk less. The gap survives centering, consistent with EX3, where the dependence on T0/JT_0/J weakens but remains (slope −0.57-0.57 to −0.19-0.19).

Preprocessing. Post-period donor outcomes are labels with unit rows but query features with time rows, where feature transforms can suppress changes beyond the pre-treatment range. On the same tasks, with fixed checkpoints and one ensemble member, removing TabPFN's quantile transform lowers time-row MSE by 53% and eliminates 64 constant prediction paths; adding query clipping to TabFM raises MSE by 54% and creates constant paths on the same 64 tasks.

4. Stronger nonlinear baselines (W3, W4, Q1). Proposition 1 separates the gain beyond the best linear rule from learning error; their difference is recoverable nonlinear headroom. With the task fixed, pretraining can change only learning error. This motivates comparisons with fitted nonlinear learners.

Recovery MSE scores error against the known conditional mean on 4,320 matched EX1(a) tasks (20 donors; T0=32,128,512T_0=32,128,512). Ratios are FM/baseline geometric means (paired 95% tt-intervals, 30 panel seeds):

FMvs. MLPvs. polynomial ridge
TabFM0.62 [0.57, 0.68]0.19 [0.16, 0.22]
TabPFN0.98 [0.86, 1.13]0.30 [0.27, 0.34]
TabDPT0.80 [0.74, 0.87]0.24 [0.22, 0.27]

Polynomial ridge uses a full quadratic basis and tunes 19 penalties. The one-hidden-layer MLP tunes width, regularization, and stopping, then averages predictions from three refits. Both use only pre-treatment validation.

The MLP improves on CatBoost (4) and KRR (15), with ratios 0.53 and 0.89. TabFM and TabDPT improve further. TabPFN and MLP show no clear pooled difference. At T0=512T_0=512, MLP/TabPFN is 0.64 [0.52, 0.79]. Polynomial ridge wins in some long-history quadratic settings.

5. Compute time (Q2). Median seconds per 24-period prediction in EX1(a), including fitting and tuning:

MethodHardwareT0=32T_0=32T0=128T_0=128T0=512T_0=512
TabFMA10G GPU0.1560.1600.259
TabPFNA10G GPU0.2450.2470.274
TabDPTA10G GPU0.5710.5720.577
Chronos-2A10G GPU0.2170.2160.237
TimesFM-2.5 (target-only)A10G GPU1.3391.3411.359
CatBoost (4)CPU, 8 threads0.4584.4289.512
CatBoost (32)CPU, 8 threads11.091513.5111528.447
KRR (15)CPU0.0070.2385.888
RidgeCPU0.0020.0110.019
SimplexCPU0.1330.0890.067
SDIDCPU0.7651.3304.536

Parenthesized numbers give tuning candidates. tabular FMs use one ensemble member.

I attempted to read the rebuttal. I could not even understand some statements in the rebuttal itself.

"Severe winters illustrate shared shocks"

"Ridge with a common penalty, no intercept and no orientation-specific scaling has none (Shen et al., 2023)."

I think writing of the paper requires special attention. As of now, it suffers many of the pitfalls that a AI-generated writing typically has. Here, I am not saying that the text was necessarily generated by AI.

I will keep my score.

AuthorsOpenReview ↗

Clarifying two quoted writing from Reviewer w3yU

Thank you for identifying these passages. We clarify their meaning below.

“Severe winters illustrate shared shocks.” Unusually harsh weather can affect traffic-fatality rates in several states during the same months, including states that raised their speed limits and states that did not. We used “shared shock” to describe this common influence on their outcomes.

“Ridge … has none.” “Has none” refers to no difference between the predictions obtained from the two input table orientations. For ridge regression, treating time periods as rows or treating units as rows gives the same prediction when neither table is rescaled, both fits use the same ridge penalty, and neither fits an intercept (Shen et al., 2023, Theorem 1). This equivalence shows that changing the table orientation need not change the prediction. Our paper studies why the tabular FMs we evaluated respond differently and how centering reduces that sensitivity.

The revised paper adds the MLP and polynomial-ridge comparisons as requested in the original review (Table 1, p. 7) and Theorem 1, which explains how orientation changes regularization in a Bayesian-ridge model (p. 5). Our first reply reports computational settings and runtimes. Could you please indicate whether these additions address your original technical concerns, and identify any specific concerns that remain?

Reviewer_3h4yOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies whether pretrained foundation models can improve synthetic control. It explains that learning nonlinear relationships helps only when the gains outweigh the errors caused by limited data. It also shows that predictions can change substantially depending on whether time periods or units form the rows of the input table. The authors propose a simple centering procedure that removes unit averages and shared movements over time to reduce this sensitivity. The authors validate their approach through simulations, 673 prediction tasks from 12 real datasets, and an experiment with artificial treatment effects added to traffic-fatality data.

Critical strengths
  1. The paper shows that input arrangement can change the apparent success of a foundation model, and provides a useful correction. Figure 6 shows that centering reduces time-row TabPFN’s mean squared error ratio relative to classical synthetic control from 2.60 to 0.94, and TabFM’s from 1.68 to 0.76. Thus, preprocessing changes these models from performing worse than the baseline to performing better. Section 4.2 also reports a 61–85% reduction in sensitivity to input orientation across three models. This is important because researchers could otherwise reach different conclusions about the same model simply by arranging the same observations differently.

  2. The simulations provide direct evidence that the models can recover nonlinear relationships better than fitted nonlinear alternatives. The authors vary history length, donor count, noise, and nonlinear strength, using 30 paired random seeds per setting. In Table 1, with 20 donors, TabFM’s recovery error is 0.33 times CatBoost’s and 0.55 times kernel ridge’s. These comparisons strengthen the contribution: the gains are demonstrated against flexible alternatives, rather than only against linear synthetic control.

Critical weaknesses
  1. The real-data results do not establish that a practical selection procedure reliably improves on SDID. In Figure 6, SDID has the lowest overall error ratio, 0.71, compared with 0.76 for the best fixed foundation-model configuration. For histories at least twice the donor-pool size, centered TabFM has a better point estimate than SDID, 0.70 versus 0.85, but the figure does not provide a direct paired uncertainty estimate for that difference. More importantly, Appendix E tests the proposed model-selection procedure on simulations, rather than across the 673 real-data tasks. Since the best model and input arrangement vary, this leaves uncertainty about whether users can obtain the reported gains using only information available before prediction.

  2. The headline 25% reduction in detection time is less robust than the broader prediction results. In the fixed-cohort FARS experiment, 80% power at nine months means detection at 20 of 25 evaluation dates. These dates have overlapping windows, and Appendix D.4 reports only three nonoverlapping windows at the twelve-month horizon. No uncertainty interval is given for the difference in detection time. Furthermore, Table 10 shows that, with randomly formed cohorts, no method reaches 80% power for the same 0.7-standard-deviation effect even after twelve months. The experiment supports a benefit in the fixed cohort, but provides limited evidence about how reliably that particular timing improvement would recur.

Questions for authors
  1. Were the model and input arrangement highlighted in the nine-month FARS result fixed before examining the power results, or identified after comparing the configurations in Table 9?

  2. When the Appendix E selection procedure uses only data available before each prediction cutoff, how does the resulting complete procedure compare with SDID across the 673 tasks?

  3. How uncertain is the nine-versus-twelve-month detection-time difference after accounting for overlapping evaluation windows? Does the advantage remain when using non-overlapping windows?

1 comment
AuthorsOpenReview ↗

Method selection, paired comparisons with SDID, and FARS uncertainty

Thank you for recognizing the importance of context-axis sensitivity and the evidence for nonlinear gains.

** pdf changes**

  • Abstract and Introduction, pp. 1-2: report random-cohort power in the abstract.
  • Section 3, p. 5: add Theorem 1 and Corollary 2 on orientation, regularization, and asymptotic risk.
  • Section 4.1, Table 1, p. 7: add MLP and polynomial-ridge baselines.

1. Selection before prediction (W1, Q2). On the 673 real-panel tasks, the Appendix E selector used only information available before each prediction cutoff. Of these, 148 had enough history for backtesting. On that subset, the weighted geometric mean of selector/SDID MSE ratios was 0.509, using Figure 6 weights. The remaining 525 tasks used SDID as the fallback, so their predictions match SDID by construction.

Backtests moved the origin to T0−jHT_0-jH, for up to three non-overlapping horizon-length holdouts, keeping context length, donors, and horizon fixed. Donor placebos used the origin T0−HT_0-H. We fixed the Appendix E defaults before scoring: 10% stability tolerance, three holdouts, and SDID fallback. A backtest needs additional history beyond the original context; most EX4 tasks already use all available history. The target outcomes after T0T_0 were reserved for final scoring.

For tasks whose pre-treatment period is at least twice the donor count, the table gives paired MSE ratios of the Table 2 FM cells to SDID (geometric means, Figure 6 weights; 95% bootstrap over targets, datasets held fixed). Below 1 favors the FM:

FM (Table 2, p. 9)Ratio to SDID [95% CI]
TabFM, centered time rows0.82 [0.69, 0.98]
TabPFN, centered time rows0.89 [0.75, 1.06]
TabDPT, centered time rows0.95 [0.77, 1.18]
Chronos-20.89 [0.71, 1.13]
TimesFM-XReg0.89 [0.70, 1.11]

2. FARS design choices (Q1). We fixed the checkpoints, input orientations, and evaluation schedule before examining power. Every arm was evaluated at months 3, 6, 9, and 12, four equally spaced looks over the study period. Table 10-11 in the updated draft (Table 9-10 in the original submission) reports the full grid. The nine-month result is from raw unit-row TabPFN and TabDPT.

3. FARS timing and uncertainty (W2, Q3). The 25% reduction in detection time in Section 4.4 applies to the fixed historical cohort of the 10 states that adopted in April 1987 only. We retain that cohort to anchor the evaluation in an existing policy study. At a 0.7-SD effect, raw unit-row TabPFN and TabDPT detect the effect at 20 of 25 origins by month 9; SDID detects it at 14 of 25 by month 9 and 22 of 25 by month 12.

The data from January 1982 through March 1987 provide 63 months before the actual speed-limit change. To construct the test set, we use 27 months of context and 12 months of follow-up, totaling 39 months per window. This gives 25 possible monthly start dates. Among these, the maximum number of disjoint 12-month follow-up periods is three. We evaluate all 25 start dates and report their overlap in Appendix D.4.

Table 11 in the updated draft pdf (Table 10 in the original submission) examines 1,000 randomly selected cohorts using 10 out of 38 candidate states in each cohort. This gives 40 random cohorts at each origin. At a 0.7-SD effect and month 12, CatBoost has 58.5% power and SDID 48.8%. The FM results below are the time-row entries marked "2W" in Table 11. We add paired 95% intervals for power gains, in percentage points:

FMPowerGain over SDID (95% CI)Gain over CatBoost (95% CI)
TabDPT68.1%19.3 [12.7, 25.9]9.6 [7.0, 12.2]
TabPFN65.1%16.3 [9.9, 22.7]6.6 [4.5, 8.7]
TabFM62.6%13.8 [5.7, 21.9]4.1 [1.0, 7.2]

Since the testing windows overlap, we also construct block-bootstrap CIs using blocks of 3, 6, and 12 consecutive start dates. The 95% CIs for TabDPT's and TabPFN's power gains over CatBoost remain above zero at all three block lengths. For TabFM, the CIs include zero at some block lengths with one member and at all three with 32 ensemble members.

At 0.7 SD under random cohorts, the gain is higher month-12 power, as reported in the revised abstract. At 1.0 SD, observed power reaches 80% at month 9 for centered unit-row TabPFN and TabDPT, and Chronos-2. The first non-FM baselines reach it at month 12 (Table 11).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.