acceptodds
Under review as a conference paper at ICLR 2027

Learning to Fluctuate: Statistical Foundations for Causal Tabular Pretraining

Abstract

Causal tabular foundation models amortize effect estimation across synthetic mechanisms, but supervision by the mechanism-level effect , which we call latent-effect supervision, rewards posterior shrinkage instead of directly encoding the repeated-sample response needed in a fixed deployment population. We introduce fluctuation-supervised pretraining (FSP): each synthetic table is labeled by its average treatment effect plus its efficient influence-function fluctuation, while deployment remains a single frozen forward pass. Along the path , we prove an endpoint transition: every fixed retains label ambiguity of order , whereas full fluctuation makes the Gaussian label observable and reduces optimal finite-stratum causal label-prediction risk to order . One finite-pretraining bound combines label, network, episode-sampling, and optimization errors; its resulting sampling defect controls fixed-mechanism bias, mean squared error, variance, Gaussian approximation, and, with variance-head accuracy, studentized coverage. Complementary lower bounds separate the local ATE risk that deployment observations cannot erase from the excess risk of a generic finite-dictionary episode-learning problem. Experiments trace the learned sampling response. Across 24 nonlinear continuous-covariate cells at trained context lengths, continuous-row FSP lowers checkpoint-mean macro RMSE by 7.0% versus S-learner and wins all 12 weak-overlap cells; validation-selected Summary FSP deploys faster per table than S-learner and faster than the released CausalPFN-S pipeline in our warm one-thread CPU benchmark. Under effect shift, matched Raw FSP lowers mean-checkpoint RMSE by 54.2% and teacher defect by 99.0% versus latent-effect supervision, and RMSE by 10.2% versus the released CausalPFN-S checkpoint. Known-effect semisynthesis tests coverage; two randomized-study evaluations show that lower RMSE can coexist with residual attenuation.

Primary area:
causal reasoning
Keywords:
Causality

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_v4eHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes fluctuation-supervised pretraining (FSP): instead of labeling synthetic tables with the true effect θ(P), it uses the oracle influence-function target θ(P) + P_n ψ_P. The goal is to teach a frozen model efficient repeated-sampling behavior. Theory characterizes label learnability, connects pretraining error to inferential guarantees, and establishes lower bounds. Experiments include a four-stratum simulator, low-dimensional continuous covariates, semisynthesis, and two coarsened randomized studies.

Critical strengths
  1. The matched-backbone ablation clearly illustrates how supervision changes the learned estimator’s behavior. Under effect shift, Raw FSP improves RMSE from .1364 to .0624 relative to latent-effect supervision.
  2. The theory is careful within its stated setting and distinguishes label-prediction risk from causal-effect estimation risk.
  3. The reported deployment speedups provide a concrete computational motivation for amortizing estimation.
Critical weaknesses
  1. The motivation for the theoretical objective is insufficiently established.
    The tension between prior-averaged prediction accuracy and fixed-population frequentist behavior is well known. The contribution therefore needs to go beyond identifying shrinkage and showing how to reproduce an efficient estimator’s sampling behavior. The paper does not sufficiently explain why this is the most useful objective for causal pretraining, or which unmet statistical need it addresses relative to existing estimation and correction procedures. Shrinkage can be appropriate under a well-specified prior and a prediction-risk objective; its presence alone does not establish a deficiency that requires removing it. My concern is primarily the significance of the chosen theoretical problem, rather than the correctness of the analysis.

  2. The theory does not sufficiently explain the statistical benefit of pretraining.
    A central motivation for causal pretraining is that structure learned across tasks can improve estimation with scarce deployment data. FSP’s theory characterizes how well a pretrained model can approximate an oracle influence-function target, but does not identify when this yields a finite-sample advantage over per-table AIPW/DML or what transferable structure produces that advantage. In particular, the n−2n^{-2} label-prediction result concerns the constructed training target, while deployment ATE risk remains of order n−1n^{-1}. This distinction is acknowledged, but leaves the practical significance of the faster label-learning rate unclear. Computational amortization is valuable; it should be distinguished more clearly from a statistical benefit due to transfer.

  3. The empirical evidence is insufficient to resolve these concerns.
    The paper includes useful effect-shift, overlap, and smoothness experiments, but evaluation remains limited to low-dimensional settings (d ≤ 4) and coarsened real data. This provides limited evidence of transfer across substantially different confounding structures or realistic high-dimensional and mixed-covariate tables. The results are also mixed: S-learner has lower RMSE than both FSP variants in all three stratified mechanisms in Table 3, although FSP improves on the reported AIPW/DML baseline. The continuous experiments show gains at trained lengths, but S-learner overtakes FSP in aggregate at n = 1024. Raw FSP’s native coverage of .706 in the weak-overlap diagnostic further shows that learning the mean response does not ensure reliable uncertainty estimates. These findings support a narrower contribution than a broadly established advance in transferable causal estimation.

Questions for authors
  1. What specific unmet need motivates learning the efficient sampling response through pretraining, rather than using per-table AIPW/DML or correcting a pretrained estimator at deployment? Is the intended primary contribution computational amortization, improved finite-sample estimation, or inferential calibration?
  2. What structure is transferred under FSP, and can you identify a regime where that transfer yields a statistical advantage over per-table estimation? A quantitative comparison involving pretraining size, deployment sample size, complexity, and mismatch between training and deployment would help establish the theoretical contribution’s significance.
  3. How robust are the benefits under substantially different confounding mechanisms, higher-dimensional or mixed covariates, and broader benchmarks such as IHDP or ACIC?
  4. Beyond the existing null and coverage diagnostics, how do the methods compare on small-sample null rejection rates, interval width, and power using their native uncertainty estimates? This would clarify the inferential tradeoff rather than treating reduced attenuation as sufficient evidence of improvement.

Answers to Questions 1–3 could change my assessment.

11 comments
AuthorsOpenReview ↗

Thanks for the review and here is the kind response

W1. Why is the efficient sampling response a meaningful objective for causal pretraining?

“The motivation for the theoretical objective is insufficiently established.”

“My concern is primarily the significance of the chosen theoretical problem, rather than the correctness of the analysis.”

Response to W1. Shrinkage can improve prior-averaged prediction under a well-specified prior. Our objective is reliable repeated-sample estimation in a fixed deployment population. The unmet need arises when a causal foundation model is pretrained across synthetic populations but deployed to analyze repeated samples from one fixed, unknown population. For that population, the questions are whether estimates have the right repeated-sampling center, whether their fluctuations reflect the available information, and whether reported uncertainty has its intended coverage. A hospital analyzing successive treatment cohorts, for example, needs its estimates and intervals to behave correctly for its own patient population, not merely to score well across simulated hospitals. Low risk averaged over training mechanisms does not, by itself, answer any of these fixed-population questions—particularly when the deployment effect differs from the training-prior center. Our simpler scientific question is: Can we train a causal foundation model once to act like an efficient statistical estimator on new tables, rather than only to predict their latent effects?

Why the EIF, rather than an arbitrary anti-shrinkage penalty? For a regular, asymptotically efficient ATE estimator under the specified causal model, the canonical first-order expansion at a fixed mechanism PP is

θ^(Dn)=θ(P)+PnψP+oP(n−1/2),\widehat\theta(D_n)=\theta(P)+P_n\psi_P+o_P(n^{-1/2}),

where ψP\psi_P is the efficient influence function. This identifies the sampling response we want a frozen model to reproduce; the matched-label comparisons evaluate this response separately from point RMSE. In particular, a model can have low point error by attenuating sample-to-sample movement, yet fail to reproduce the first-order behavior required for efficient frequentist inference. The objective is therefore tied to a specific statistical use case, not chosen merely because the EIF is familiar.

Why does this remain an open problem when classical correction already exists? AIPW, cross-fitted DML, and related one-step or targeted corrections can construct efficient estimators by estimating nuisances and correcting the estimate on each new table. More directly related, the PFN one-step posterior correction of Melnychuk et al. (2026) recovers nuisance posteriors and corrects the PFN-based procedure at deployment. These are important, often stronger alternatives, not deficiencies we claim to have removed. They address a different computational arrangement: construct or correct the estimator at test time. We ask whether synthetic pretraining can amortize that first-order correction into the native frozen map, so that new tables require neither nuisance refitting nor an additional EIF correction. Simulator-known mechanisms are used to construct supervision only during pretraining; they are not available to, or assumed known by, the deployed model. This distinction matters when many cohorts or experimental tables must be processed repeatedly, while also creating a new statistical question: when does the learned correction survive finite pretraining and a change of population?

What is mathematically new and difficult? We supervise each synthetic table with TP(Dn)=θ(P)+PnψPT_P(D_n)=\theta(P)+P_n\psi_P, rather than the mechanism-level constant θ(P)\theta(P). Surprisingly, the fluctuating target can be easier to learn from the table: Theorem 4.2 proves sharp n−2n^{-2} minimax label-prediction risk for full fluctuation in our finite-stratum class, while every fixed partial-fluctuation level below one retains an n−1n^{-1} term. The gain comes from cancellation of first-order label ambiguity, not from assuming the unknown effect becomes easier to estimate. The next obstacle is to pass from an oracle synthetic label to an actual trained network. Theorem 4.3 bounds the sampling defect of the uncorrected frozen output using label learnability, network approximation, finite training episodes, optimization, and explicit mechanism-transfer conditions; it then relates that defect to fixed-population bias, variance, Gaussian approximation, and coverage conditional on variance-head accuracy. Complementary lower bounds (Theorems 4.4–4.6) distinguish the information supplied by pretraining episodes from the irreducible n−1n^{-1} risk of estimating the deployment effect. This is not a standard one-step analysis with consistent test-time nuisance fits: the object whose sampling law must be controlled is a pretrained, frozen table-to-estimate rule.

AuthorsOpenReview ↗

Thanks for the review and here is the kind response(cont'd)

Do these distinctions change observable performance? The matched Raw experiment holds architecture, simulator stream, optimizer, and training budget fixed, changing only the label. Near the prior center, latent-effect supervision has lower point RMSE (0.0176 versus 0.0621), exactly where shrinkage can be beneficial. When the effect shifts, the ordering reverses: FSP lowers RMSE from 0.1364 to 0.0624 (54.2%), reduces sampling defect by 99.0%, and moves the sampling-response slope from −0.058 to 0.972 (Figure 3, Table 3). Thus the experiment does not merely show that shrinkage exists; it isolates a change in which statistical procedure the same network has learned. The seven-level fluctuation ablation tests the predicted supervision mechanism, and the larger weak-overlap experiments identify settings in which the learned rule retains point-accuracy gains. At deployment, the computational distinction is also measurable: on the tested Criteo-visits workload, design FSP takes 5.57–5.61 ms per table on one CPU thread, versus 8.66 ms for optimized AIPW, without per-table nuisance refitting (Appendix P.3).

The significance of the problem is instead the missing bridge between amortized causal prediction and fixed-population efficient inference. FSP supplies a task-aligned supervision target, a counterintuitive label-learnability result, a finite-pretraining-to-inference analysis, and controlled evidence that the learned sampling response—not simply the amount of shrinkage—determines when a frozen model can deliver accurate, reusable causal estimation.

W2. When does EIF-supervised pretraining yield a statistical transfer benefit?

“The theory does not sufficiently explain the statistical benefit of pretraining.”

Response to W2. Pretraining can reduce the cost of learning an adjustment from a small deployment table when tasks share prognostic structure. Our shared-prognostic model gives an explicit threshold: the frozen estimator has lower risk than deployment-only least squares when its offline adjustment error is smaller than the nuisance-learning penalty paid on that table. Appendix Q.5 (pp. 61–62) gives the complete derivation.

What transfers, and when does it help? Consider independent randomized tasks with Z=2A−1∈{−1,1}Z=2A-1\in\{-1,1\}, Pr⁡(Z=1)=1/2\Pr(Z=1)=1/2, X∼N(0,Id)X\sim N(0,I_d), and X,Z,εX,Z,\varepsilon mutually independent, with known σ2>0\sigma^2>0:

\qquad \varepsilon\sim N(0,\sigma^2).$$ The prognostic coefficient $\beta$ is shared, while the ATE $\theta$ may change arbitrarily between tasks. Pretraining yields $\widehat\beta_M$, independent of the deployment table. Within the frozen residual-score family $f_b(D_n)=2n^{-1}\sum_i Z_i(Y_i-X_i^\top b)$, let $\mathcal E_M=\mathbb E_{\rm pre}\|\widehat\beta_M-\beta\|^2$. Direct calculation gives $$\begin{aligned} R_{\rm frozen}(M,n) &=\frac{4\sigma^2}{n}+\frac{4\mathcal E_M}{n},\\ R_{\rm table\text{-}LS}(n) &=\frac{4\sigma^2}{n} \left(1+\frac{d}{n-d-2}\right), \qquad n>d+2. \end{aligned}$$ Here table-only least squares jointly estimates $\theta$ and the $d$ prognostic coefficients; in this Gaussian model it attains the minimax risk over unrestricted $(\theta,\beta)$ for estimators using only the deployment table. Consequently, $$\boxed{ R_{\rm frozen}(M,n)<R_{\rm table\text{-}LS}(n) \quad\Longleftrightarrow\quad \mathcal E_M<\frac{\sigma^2d}{n-d-2}. }$$ The same sufficient condition beats the explicitly specified two-fold, known-propensity AIPW comparator with foldwise pooled linear nuisance fits; its risk is at least $\frac{4\sigma^2}{n}\bigl(1+\frac{d}{n/2-d-1}\bigr)$ for even $n$ and $n/2>d+1$. Appendix Q.5 specifies the foldwise least-squares nuisance fits used in this comparison. **Why can EIF labels recover the shared adjustment?** For a synthetic training table of size $m$, write $u_j=2m^{-1}\sum_i Z_{ji}Y_{ji}$ and $v_j=2m^{-1}\sum_i Z_{ji}X_{ji}$. Its exact oracle EIF teacher is $T_j=\theta_j+2m^{-1}\sum_i Z_{ji}\varepsilon_{ji}$, so $$u_j-T_j=v_j^\top\beta.$$
AuthorsOpenReview ↗

Thanks for the review and here is the kind response(cont'd)

The task-specific effect and outcome noise cancel in the supervision equation. This identifies how EIF labels recover the shared adjustment within the stated score family. Within this specified score family, empirical minimization of exact EIF-label loss therefore recovers β\beta almost surely with M≥dM\ge d independent Gaussian-design training tables. With independent mean-zero label noise of variance ω2/m\omega^2/m, it instead yields EM=ω2d/[4(M−d−1)]\mathcal E_M=\omega^2d/[4(M-d-1)] for M>d+1M>d+1. Thus, when d=o(n)d=o(n) and MM is sufficiently larger than nn, pretraining replaces the table-only adjustment cost of order d/n2d/n^2 by an offline-learning cost of order d/(nM)d/(nM). The gain can be even more pronounced when d/n→κ∈(0,1)d/n\to\kappa\in(0,1): with negligible offline error, Rfrozen/Rtable-LS→1−κR_{\rm frozen}/R_{\rm table\text{-}LS}\to1-\kappa. The oracle variance remains 4σ2/n4\sigma^2/n; the gain comes from reducing the additional nuisance-learning cost. Because the risk is independent of θ\theta, the gain persists under arbitrary treatment-effect shifts. Under prognostic shift, the same threshold applies with EM\mathcal E_M measured against the deployment coefficient.

How does this relate to n−2n^{-2} teacher learnability? The two results answer complementary questions. Theorem 4.2 shows, in a separate finite-stratum model, that a full-EIF label can be predicted from the table with An=O(n−2)A_n=O(n^{-2}); the observable stratum estimator already witnesses this rate without pretraining. Theorem 4.3 explains why the rate matters for a frozen model: in the exact decomposition

n(θ^−θ(P))=1n∑i=1nψP(Oi)+nθ^−TP(Dn),\sqrt n(\widehat\theta-\theta(P)) = \frac{1}{\sqrt n}\sum_{i=1}^n\psi_P(O_i) +\sqrt n\\{\widehat\theta-T_P(D_n)\\},

its label-learning contribution to the scaled remainder is nAn=O(n−1)nA_n=O(n^{-1}), while approximation, optimization, finite-episode cost (proportional to nlog⁡(2Nξ/δ)/Mn\log(2N_\xi/\delta)/M), and mechanism transfer must also be controlled. The shared-prognostic specialization goes beyond this recovery guarantee: it identifies a concrete transferable structure and a threshold for strict finite-nn improvement over fitting it from scratch.

Does the effect appear beyond the specialization? In Figure 19, holding the shifted deployment mechanism and n=512n=512 fixed while raising MM from 2,048 to 32,768 reduces sampling defect from 2.163 to 0.050, moves n MSE/Vn\,\mathrm{MSE}/V from 1.616 to 0.998, and improves native nominal-95% coverage from 91.1% to 94.5%. In the separate matched transfer experiment, FSP has lower point RMSE in all 12 tested cells against each of the logistic and saturated AIPW specifications, with favorable pointwise paired intervals. Appendix Q.1 reports the full grid and the separate sampling-response diagnostics. The statistical benefit of pretraining is therefore precise: shared information learned offline can remove a quantifiable nuisance-learning penalty at deployment, while EIF supervision provides a way to learn and reuse the corresponding first-order adjustment. Computational amortization is an additional, separate benefit.

W3. Transfer beyond simple simulations, strong baselines, and calibrated inference

“The empirical evidence is insufficient to resolve these concerns.”

Response to W3. The prespecified trained-length continuous grid gives a direct majority comparison: FSP wins 16 of 24 cells against the strongest tested classical method, including all 12 weak-overlap cells. Against cross-fitted AIPW/DML it wins 24/24 (Appendix N.10). We also test mixed real covariates, external outcome benchmarks, larger tables, and all 82 encoded ACIC inputs.

1. Transfer beyond four-stratum inputs. On Adult/Bank tables with 12 continuous and categorical features, nonlinear assignment, interactions, and heterogeneous effects, the matched 1.30M-parameter FSP achieves 0.0449 RMSE under weak overlap and shifts of ±0.60\pm0.60. Matched latent supervision gives 0.0534 and polynomial T-learning 0.0521: reductions of 15.8% and 13.7%, respectively. The paired RMSE-difference interval against T-learning is [−0.0116, −0.0024], conditional on the checkpoints. FSP's mean absolute shift-tracking error is 0.0048 versus 0.0271 for latent supervision (Appendix P.1).

On larger randomized tables, design-aware FSP lowers Criteo visit RMSE by 9.2%–24.6% relative to the tested Hájek AIPW over 16,384–262,144 bootstrap draws, scored against the released-data empirical contrast (Appendix P.3).

Reviewer_sGnUOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a new way to pretrain causal foundational models on synthetic data. Existing models attempt to predict the average treatment effect (ATE) but may miss out how to model the noise. This paper adds an extra loss term during training to force the model to learn how to disentangle the ATE from the noise. The authors use proofs to describe how this additional training loss formally bounds the error. Experiments against comparison systems show the system does about as well as all the other systems but is several orders of magnitude faster than some.

Critical strengths
  • Very interesting modeling approach for trying to model both the ATE and EIF from the data.

  • The theorems provide useful bounds on the performance.

Critical weaknesses
  • I found the paper harder to read than expected. While I appreciate the author's intent to pack in as much as possible, the paper requires a sufficiently close reading to understand the details. The theorem section in particular (as well as the lengthy appendix) are particularly dense. Some of this is potentially unavoidable. I get the sense that the paper has been compressed with some LLM (Claude?) to cram into 9 pages, but some common LLM writing tics are present which distracts from the presentation.

    • One disappointment for me was the related work section is in the appendix. To me, this is a missed opportunity to contextualize the paper with respect to the other works and ground the reader in the topic—especially since causal foundation models are still relatively new. I think moving this to section 2 would help readability and possibly moving some of the less essential theorems to an appendix.
  • I found the results comparison harder to identify. Figure 3 makes it seem like the approach is doing about as well as all the other comparison systems, despite the joint objective. I would have thought that the EIF modeling would have reduced the RMSE or led to bigger gains. The bigger win seems to be speed, which is helpfully highlighted, so this seems to be the more important contribution than the bounded error

Questions for authors
  • Potential explanations for why adding an IFE-based loss doesn't help much are that (1) there's not much model-able noise in the real data you're using that this makes an improvement (2) other approaches to modeling this help just as much without the bound, or (3) the model doesn't actually learn IFEs well so it's not benefiting as much. Any idea which might be happening?
6 comments
AuthorsOpenReview ↗

Thanks for the review and here is the kind response

W1. Readability, theory, and related work

“The theorem section in particular (as well as the lengthy appendix) are particularly dense.”

Response to W1. The revised manuscript places Related Work in Section 2 (p. 3), before the statistical setup. It compares causal PFNs, per-table AIPW/DML, and one-step posterior correction. The abstract now states the empirical trade-off in words, and the captions of Figures 1 and 2 define the notation used in the diagrams. Section 4 defines label ambiguity and minimax prediction risk in the main text and introduces the objects used by Theorems 4.4–4.6 where they first appear. We kept the statements that connect supervision to fixed-mechanism inference in the main text and placed the proofs and expanded diagnostics in the appendix.

W2. Figure 3, point accuracy, and deployment speed

“I would have thought that the EIF modeling would have reduced the RMSE or led to bigger gains.”

“The bigger win seems to be speed”

Response to W2. EIF supervision gives its clearest point-accuracy gain under effect shift. In Figure 3, the matched Raw models differ only in their training label. Adding 0.15 to each stratum effect reverses their RMSE ranking. Raw FSP lowers RMSE from 0.1364 to 0.0624 (54.2%), reduces the sampling defect by 99.0%, and moves the response slope from −0.058 to 0.972, close to the oracle slope of one. At the prior center, Raw FSP also improves on CausalPFN-S (0.0621 vs. 0.0702) and cross-fitted AIPW/DML (0.0664). The matched latent model leads there (0.0176) because shrinkage reduces variance when the effect is close to the prior center. Farther from the center, the squared bias from shrinkage grows and can outweigh that saving (Proposition 4.1, Figure 3, Table 3).

The gains extend beyond the four-stratum example. Summary FSP lowers RMSE by 10–13% against CausalPFN-S across all three Figure 3 mechanisms. In the nonlinear continuous grid, FSP lowers macro RMSE against the S-learner by 7.0% across 24 trained-length cells, winning 16/24 and all 12 weak-overlap cells, with a 14.6% reduction there. On Adult/Bank under weak overlap, FSP has lower point RMSE without a shift (0.0448 vs. 0.0482 for matched latent). At shifts of ±0.60, it lowers RMSE by 15.8% against matched latent supervision and 13.7% against polynomial T-learning. In the original randomized studies, the five-checkpoint Raw FSP ensemble lowers RMSE against the arm difference from 0.0596 to 0.0522 on NSW post-treatment outcomes and from 0.0775 to 0.0628 on turnout (Appendices N.13–N.14). On Criteo visits, the separately trained randomized-design FSP lowers RMSE by 9.2–24.6% versus Hájek AIPW across 16,384–262,144 bootstrap draws. W3 explains the S-learner comparison and extrapolated lengths; Section 5 and Appendices N.10, P and Q report the input-matched and complete-grid results.

Deployment speed. In the single-thread benchmark on 256-row tables with the models loaded, Summary FSP takes 0.138 ms per table, compared with 205.95 ms for the released CausalPFN-S pipeline (1,491×) and 1.605 ms for the S-learner (11.6×). FSP reuses a frozen network, while the released CausalPFN-S pipeline fits a weak learner on each new table. On 16,384-row Criteo tables, randomized-design FSP takes 5.57–5.61 ms versus 8.66 ms for optimized Hájek AIPW in a separate single-thread benchmark (Appendix N.8, Figure 7, and Appendix P.3).

Statistical guarantee. Theorem 4.2 shows that full EIF supervision reduces the sharp minimax risk of predicting the teacher in the finite-stratum model from order n−1n^{-1} for any fixed λ<1\lambda<1 to n−2n^{-2}. Theorem 4.3 then uses the deployment sampling defect to bound the frozen estimator’s bias and its departure from the oracle’s RMSE at a fixed mechanism. When that defect is small, the frozen model can retain the oracle’s first-order sampling behavior. Interval coverage additionally depends on variance estimation. The deployment ATE information limit remains of order n−1n^{-1}.

Q1. Which of the three explanations accounts for the limited EIF gains?

“Any idea which might be happening?”

Response to Q1. All three explanations matter, but in different regimes. (1) Hillstrom offers little measured scope for covariate adjustment. (2) Per-table AIPW and centered ridge can recover comparable information without EIF-supervised pretraining. (3) Matched-label experiments show that EIF supervision does teach a frozen model the oracle sampling response, but this response attenuates outside the training propensity range. Our question is how a frozen table model can learn and reuse an efficient estimator's repeated-sample behavior, and when this improves estimation and inference. Appendix P.8 reports the new diagnostics and protocols.

AuthorsOpenReview ↗

Thanks for the review and here is the kind response(cont'd)

This distinction motivates our theory. A mechanism-level ATE label rewards prediction across synthetic populations and can favor shrinkage toward the training prior. Our full-fluctuation label, TP(Dn)=θ(P)+PnψPT_P(D_n)=\theta(P)+P_n\psi_P, instead teaches how an estimate should change across samples from the same population. Counterintuitively, adding the realized EIF fluctuation can make the label easier to predict from the observed table: in the finite-stratum model, its minimax label-prediction risk is of order n−2n^{-2}, whereas every fixed partial-fluctuation level λ<1\lambda<1 retains an n−1n^{-1} term (Theorem 4.2). The deployment information limit remains n−1n^{-1} (Theorem 4.6); the faster label rate reduces the difficulty of learning the estimation rule during pretraining. The mathematically distinct challenge is transferring that label learnability through network approximation, finite training episodes, optimization, and mechanism shift. Theorem 4.3 makes these costs explicit through the sampling defect and links its control to fixed-mechanism bias, MSE, variance, Gaussian approximation, and—when the variance head is accurate—coverage. Our experiments test where this chain holds and where it breaks.

1. How much is there to learn from the covariates? Outcome predictability and treatment-effect precision are different questions. We used five-fold cross-fitting to compare arm-specific boosted outcome models with training-fold arm means on the same held-out rows, then measured the variance reductions of the corresponding assignment scores using a common marginal assignment fraction.

OutcomeReleased rows usedBrier loss reductionScore variance reduction
Criteo visits131,07232.3%29.1%
Criteo conversions131,07210.3%4.5%
Hillstrom visits64,0002.4%2.4%
Hillstrom conversions64,000−0.3%−0.1%

Why can the two reductions differ? Even in a randomized trial, the two arms can happen to include different proportions of participants with high baseline outcome risk. Covariate adjustment removes part of this chance imbalance. Under i.i.d. randomization with constant treatment probability π\pi, the difference between the influence-function variances of the unadjusted arm difference and the efficient estimator is exactly

=\frac{\operatorname{Var}[(1-\pi)m_1(X)+\pi m_0(X)]}{\pi(1-\pi)},$$ where $m_a(X)$ is the arm-specific outcome regression. If this weighted prognostic mean barely varies across people, even oracle adjustment has little variance to remove. That helps explain Hillstrom, especially conversions. Criteo visits show substantially more predictive and adjustment signal, whereas conversions show that improved outcome prediction need not yield a comparable precision gain. Criteo's outcome-dependent subsampling prevents us from treating these released-data score diagnostics as the original trial's efficiency gain. For visits, adjustment also shifts the score mean from 0.01107 to 0.00845; **variance reduction can be offset by a shift in the estimator's center** when RMSE is measured against the released empirical assignment contrast. **A controlled experiment separates available signal from the frozen model's ability to exploit it.** Using Adult and Bank Marketing covariates, we set $Y=c\,g(X)+A\tau(X)+\varepsilon$ and vary $c\in\{0,1,2\}$ while holding covariates, assignments, residuals, and treatment effects fixed. Because $c\,g(X)$ enters both treatment arms equally, it cancels from their contrast and from the oracle EIF residuals: **the oracle teacher is identical at all three strengths**, even though the unadjusted outcomes become more variable. The table reports $n=1024$ pooled RMSE across 128 paired tables per population and assignment design, and across the two fixed checkpoints per neural objective. | Assignment design | Prognostic strength | Arm difference RMSE | Frozen base FSP RMSE | Matched latent RMSE | | --- | ---: | ---: | ---: | ---: | | Randomized | 0× | **0.0274** | 0.0493 | 0.0425 | | Randomized | 1× | **0.0335** | 0.0726 | 0.0600 | | Randomized | 2× | **0.0477** | 0.0868 | 0.0768 | | Moderate confounding | 0× | **0.0301** | 0.0722 | 0.0646 | | Moderate confounding | 1× | 0.1086 | 0.0700 | **0.0416** | | Moderate confounding | 2× | 0.2166 | **0.0426** | 0.0624 |
AuthorsOpenReview ↗

Thanks for the review and here is the kind response(cont'd)

Under randomization, oracle RMSE stays near 0.0273 as the arm difference grows noisier, but frozen base FSP also deteriorates. Thus available prognostic information does not imply that a finite pretrained model can exploit it; explanation (1) is not sufficient. Under moderate confounding at 2× strength, the same FSP checkpoints instead improve point RMSE by 31.7% over matched latent supervision (0.0426 versus 0.0624) and reduce sampling defect from 2.838 to 1.129 (paired 95% RMSE-difference interval [−0.0250, −0.0145], conditional on fixed checkpoints). At 1×, latent supervision wins. The benefit is therefore not monotone in prognostic strength; outcome structure and assignment mechanism jointly matter.

Can a model trained specifically for randomized designs recover the missed gain? On the same randomized tables at n=1024n=1024 and 2× strength, separate design-aware FSP checkpoints lower RMSE by 22.9% against the arm difference (0.03673 versus 0.04766; paired 95% difference interval [−0.01348, −0.00827]). Relative to matched design-latent checkpoints, they also lower sampling defect from 0.691 to 0.581 (paired difference interval [−0.2124, −0.0164]), although their point-RMSE difference interval includes zero. Known-propensity, four-fold ridge AIPW reaches 0.02897 RMSE, close to the oracle's 0.02729. These comparisons identify both the gain and its limit: design-aware training recovers part of the available adjustment, while a well-fitted per-table estimator still does better. Base versus design FSP changes both the head and the training subset; only the matched design FSP/latent pair isolates the supervision label. In that pair, EIF supervision improves sampling-law fidelity even without an established point-RMSE gain.

2. Can a fitted estimator do as well? Yes, in some settings. AIPW itself uses an EIF score, but fits nuisances and constructs the estimate anew on each deployment table. FSP instead learns a reusable table-to-estimate map during synthetic pretraining. Its contribution is amortizing the efficient sampling response, not claiming that classical estimators lack access to it. We compared Raw FSP with five-fold AIPW on 512 paired tables per cell, crossing two effect levels, two lengths (128 and 512), and three propensity regimes. We varied propensity clipping and also supplied the true propensity as a diagnostic, leaving the outcome regressions fitted.

AIPW comparisonCells with lower FSP point RMSECells with lower FSP sampling defect
Fitted propensity, each clip floor (.025, .05, .10)12/126/12
True propensity, logistic outcomes12/124/12
True propensity, smoothed stratum outcomes12/124/12

Point RMSE favors FSP numerically in all 12 cells for every comparison. Paired 95% table-bootstrap intervals favor it in 12/12 cells at each fitted clip floor and 11/12 for each true-propensity comparator, conditional on the frozen checkpoints. Yet point accuracy and efficient sampling-law fidelity can rank estimators oppositely. For shifted effects at n=512n=512 outside propensity support, FSP has lower RMSE than true-propensity logistic AIPW (0.0537 versus 0.0914), but AIPW has much lower sampling defect (0.462 versus 2.075). Likewise, in the continuous experiment at n=256n=256, cross-fitted DML has slope 1.008 but defect 0.522; FSP has slope 0.819 and defect 0.297; and the S-learner has defect 0.094 (Appendix N.10). A near-unit slope captures the amplitude of oracle fluctuation, not mean offset or other table-level error. Finite-sample shrinkage may improve point risk at one mechanism while failing to reproduce the efficient response needed for inference across nearby mechanisms. This is why our theory controls sampling defect, rather than treating low RMSE or a near-unit slope as a sufficient guarantee of inference.

The real-data comparison identifies a practical boundary. At 131,072 Criteo visit draws, arm-centered FSP achieves RMSE 0.001203 against the held-out empirical assignment contrast, versus 0.001250 for centered ridge and 0.001350 for the arm difference: 10.9% lower RMSE than the latter. Both seed-specific paired intervals comparing FSP with ridge include zero. The adjusted methods use the same outcome-free arm-centering information and two-fold protocol (Appendix P.7). Without that extra centering, the same design FSP checkpoints have RMSE 0.002191 on the follow-up draws. Thus, the improvement cannot all be attributed to EIF supervision, and conventional fitted adjustment remains competitive; the matched-label comparisons below more directly identify the label's effect.

Reviewer_BUcBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper aims to improve causal foundation models for average treatment effect (ATE) estimation. The motivation is that causal foundation models are trained with a certain distribution of ATEs, and if they saw effect strengths centered around zero during pretraining, applying them to a new dataset from a shifted distribution, it will underestimate effects due to a shrinkage-like effect.

The paper changes the training signal from the true effect to the effect a gold-standard statistical method would give if it knew the simulator’s hidden details. While these details are not available at test time, they are available from the prior at pretraining time and the PFN can learn to approximate their result.

The authors prove guarantees in a simplified setting. They test the approach in small simulations and two real RCTs. In simulations it removes the shrinkage under effect shift and beats the released CausalPFN checkpoint by about 10%. However, it is clearly worse than a matched model trained on the true effect when the effect is typical, and it doesn’t consistently beat simple classical methods. On the RCTs it is more accurate than a simple difference in means but it still underestimates effects. Overall, the experiments are careful but small-scale and far simpler than real applications.

I also find the paper very hard to follow despite having published in this field. I believe this is mostly due to the paper's heavy use of LLM-generated text, jargon, undefined nomenclature, etc. For reference, I asked an LLM: "How LLM-generated does the paper read? Is nomenclature introduced before it is used?" The response was as follows (rest of this text box): " How LLM-generated it reads

Heavily, and the authors more or less confirm it. Their AI-use statement lists manuscript drafting and editing, alongside proof drafting, result interpretation and figure generation. Style alone can’t establish who wrote what, but the prose has most of the familiar hallmarks:

Pre-emptive “X, not Y” hedging. Nearly every paragraph anticipates an objection. λ is “an analytical interpolation, not a deployment hyperparameter”; latent-effect means “not a latent-variable model”; the Do-PFN run is “not a like-for-like benchmark”; the dictionary lower bound comes “without asserting a causal-FSP pretraining necessity.” Audit and agent-log vocabulary. The paper speaks of a “scope contract,” “claim-to-evidence map,” “frozen artifact audit” and “locked-final tables.” External results are “emitted only after a completed official checkpoint call,” and commit hashes and a macOS OpenMP workaround appear in the methods text. Slogan-style framing. Takeaways are bolded mid-sentence (“similar coverage concealed different distributional shapes”). Paragraph leads are aphoristic, such as “The label is the intervention,” an unfortunate pun in a causal paper. One result gets three names: phase transition, endpoint transition and singular endpoint. Number density. The abstract alone packs in about a dozen numbers from different experiments. Coined terms that then drift, covered below.

None of this is a scientific flaw in itself, and the formal development in Sections 2–3 is tight. But the style buries the paper’s central trade-off under qualifications, and the caveats that matter most sit in a ~40-page appendix.

Is nomenclature introduced before use?

Inconsistently. Sections 2–3 are careful: the probability conventions, the oracle teacher label, Sₙ and the covering number are all defined before the theorems use them. Appendix N.1 even disambiguates the latent-effect model names. But the main text isn’t self-contained, and some terms are never defined at all:

Method names. “Summary FSP” and “Raw FSP” appear in the abstract, Figure 3 and Section 4, but their architectures are described only in Appendices N.4–N.5. Section 4 defines “Raw latent-effect” as “the Raw-FSP network trained instead on θ(P),” which explains one undefined name with another. Response slope. It is a headline metric in Figure 3 and Section 4 but is defined only in Appendix N.1. There it has two different definitions (through the origin vs. OLS with intercept) for different experiments. Acronyms. “EIF” is used in Figure 1 and the related-work paragraph before “efficient influence function” appears with Eq. (1), and the two are never explicitly linked. OSPC is never expanded, and “MP” can only be inferred from “martingale posteriors” in Appendix A. The “-S” in CausalPFN-S and the “S/RA evaluation” in N.6 are never explained. AIPW, DML, PFN, SCM and CATE are also never spelled out, though they’re standard for this audience. Forward use. R₍ₙ,λ₎ appears in Figure 1, three pages before Theorem 3.2 defines it. “Label ambiguity” runs through the abstract and Section 3 but is tied to E Var(T_λ | Dₙ) only in Appendix D. Theorem statements depend on the appendix. This affects the (J, L) in Theorem 3.5, the laws Q₍λ,σ₎ in Theorem 3.4, the least-favorable path in Theorem 3.6, and the J and m in “M = Jm.” Synonym drift. The defect n·E(θ̂ − T_P)² is called sampling defect, native sampling defect, native defect, teacher defect, full-label defect and FSP defect. The T_P + 0.025 ablation is the biased-label, shifted-label, shifted-teacher and “Biased-teacher” variant. “Prior center” in Figure 3 is “Typical” in Table 3. “Shifted” in Table 3 appears to be the fixed “large-effect” mechanism. Yet “effect shift” also names a random-shift design in N.3 and the semisynthetic sweep in Figure 4c. “Weak overlap” covers at least two different designs, alongside “boundary overlap” and “low overlap.” Symbol overloading. J means panel size, interpolation knots and a random index. d means defect, dictionary dimension and covariate dimension. δ is both the confidence level and the fluctuation in Figure 2, on the same page as Theorem 3.3. “Native” means both uncorrected output and a baseline’s “native form.” Figure 6 uses τ for the ATE, which is θ everywhere else." "

Critical strengths
  • Causal foundation models are very important, and it would be important to make them more reliable.
Critical weaknesses
  • Experiments are very small-scale and I am not convinced that the result will generalize to complex, real-world datasets.
Questions for authors
  • Could you train a sizable model with this approach and evaluate it on more complex and more realistic benchmarks?
8 comments
AuthorsOpenReview ↗

Thanks for the review and here is the kind response

Thanks for the kind review, and we address them point by point.

Q1. Scalability and Generalization to Realistic Applications. Can FSP scale to larger causal foundation models and more realistic datasets, and what evidence supports its effectiveness beyond the simplified simulations and two RCT evaluations?

“The experiments are careful but small-scale and far simpler than real applications.”

“Could you train a sizable model with this approach and evaluate it on more complex and more realistic benchmarks?”

A1

We pretrained two matched pairs of 20M-parameter models on 131,072 synthetic tables per seed, with up to 4,096 rows and 4–16 covariates per table. The simulator covers nonlinear outcomes and both randomized and confounded assignment. Within each pair, FSP and latent-effect supervision share the architecture, initialization, training tables, and optimization schedule. This controls for model size and training budget.

Adult and Bank Marketing contribute 768 semi-synthetic base tables and 3,840 evaluations counting the original and four paired shifts for each table. ACIC adds ten confounding settings with 58 covariates, compressed to 16 components for the frozen models. In the original PC16 comparison in Appendix P.4, boosted AIPW using all encoded covariates leads at longer contexts.

For Criteo and Hillstrom, we separately trained a matched pair of models with randomized-design heads on each seed’s simulated randomized subset (47,299 and 46,966 tables). The head was designed after examining Criteo, and Hillstrom provides a separate check. The releases contain 13,979,592 and 64,000 records, respectively. We compare the frozen models on paired bootstrap tables of up to 262,144 and 32,768 draws against disjoint empirical assignment contrasts. The Criteo maximum is 64 times the maximum training table length.

Benchmark and conditionQuantitative result
Adult/Bank, weak overlap, effect shifts of ±0.60FSP RMSE 0.0449, versus 0.0534 for matched latent supervision and 0.0521 for polynomial T-learning, the strongest classical comparator in these cells. Reductions of 15.8% and 13.7%, respectively.
IHDP, all 100 full-length outcome realizationsFSP RMSE 0.290 versus 0.389 for matched latent supervision: 25.6% lower, paired realization-bootstrap interval [−0.148, −0.055] (Appendix Q.3).
Complete 82-input semisynthesis, n=256n=256, moderate overlap, ±0.60\pm0.60Four-seed FSP RMSE 0.0886 versus 0.0991 for boosted T-learning, the strongest tested fitted comparator in this cell: 10.5% lower, pointwise paired interval [−0.0154, −0.0050] (Appendix Q.4/Figure 24).
Criteo visits, 16,384 rowsRMSE 0.00421 versus 0.00464 for cross-fitted Hájek AIPW and 0.00463 for matched design latent supervision. The arm difference gives 0.00409. Median single-thread evaluation is 5.57–5.61 ms versus 8.66 ms for optimized AIPW in a separate timing run.
Criteo visits, 131,072 rows with outcome-free arm centeringIn a separate follow-up, centered FSP reaches RMSE 0.001203, versus 0.002191 for the same checkpoints without centering and 0.001350 for the arm difference. Both seeds' paired intervals favor centered FSP against the arm difference.

Across Criteo visit contexts from 16,384 to 262,144 rows, the frozen design model's RMSE is 9.2%–24.6% lower than Hájek AIPW's. The strongest comparator depends on the context and endpoint. A3 explains long-table behavior and the centering follow-up; Appendix P.3 gives every endpoint comparison, and Appendix P.7 describes the additional calibration split used for the centering follow-up. See Section 5 and Appendices P and Q.3–Q.4.

Q2. Statistical Trade-offs and Comparison with Existing Methods. What explains the performance reversal between latent-effect supervision and FSP under effect shift? Under which conditions does FSP improve upon classical estimators and existing causal foundation models, and how should its gains in sampling behavior, estimation accuracy, and computational efficiency be understood alongside its losses near the prior center?

“It is clearly worse than a matched model trained on the true effect when the effect is typical, and it doesn't consistently beat simple classical methods.”

“In simulations it removes the shrinkage under effect shift and beats the released CausalPFN checkpoint by about 10%.”

A2.

The expanded Adult/Bank experiments show a weak-overlap advantage with and without effect shift. Our bias–variance analysis also explains the near-center advantage of matched latent supervision and its reversal under a sufficiently large effect shift.

AuthorsOpenReview ↗

Thanks for the review and here is the kind response(cont'd)

Why the RMSE ranking reverses. Under squared loss, latent-effect supervision learns to approximate the posterior mean of the true effect under the training prior. Proposition 4.1 isolates the mechanism. For a noisy effect estimate ZZ with sampling variance τ2=v/n\tau^2=v/n, the prior yields the shrunk estimate κZ\kappa Z for 0<κ<10<\kappa<1. At the prior center this has lower variance than ZZ, by a factor of κ2\kappa^2. At a fixed effect θ\theta away from the center, its squared bias grows as (1−κ)2θ2(1-\kappa)^2\theta^2, while the variance saving stays fixed at (1−κ2)τ2(1-\kappa^2)\tau^2. The full-fluctuation estimate ZZ therefore has lower MSE once the bias cost exceeds the variance saving. This accounts for the near-center advantage of latent supervision and the reversal under a sufficiently large effect shift.

What the EIF label teaches. At a fixed data-generating law, two observed tables share the same population effect but have different covariate mixes and outcome residuals. Latent supervision gives them the same target. FSP labels each table with the population effect plus the mean efficient influence function (EIF) evaluated on that very table. This term is centered across repeated tables and captures their realized variation. The simulator supplies the true nuisance functions for constructing labels during pretraining. At deployment the network uses the observed table and its frozen weights. Theorem 4.3 relates fidelity to the teacher to bias and sampling behavior at a fixed law. Interval coverage also depends on learning the sampling variance accurately.

The paired Adult/Bank intervention isolates this response. Adding AδA\delta changes the true effect and oracle teacher by exactly δ\delta while holding covariates and treatment assignment fixed. The treated outcome regression also increases by δ, so the observed outcome residuals in the EIF stay the same. At shifts of ±0.60, averaging over both overlap regimes, FSP's mean absolute error in tracking that change is 0.0048 versus 0.0271 for matched latent supervision, an 82% reduction. The fitted classical methods translate by the imposed shift by construction. Their comparison therefore tests absolute ATE error in the shifted setting, while the paired shift diagnostic isolates the effect of the two learned objectives.

Adult/Bank settingFSP RMSEMatched latentPolynomial TBoosted AIPW
Weak overlap, no shift0.04480.04820.05210.0717
Weak overlap, ±0.300.04480.04920.05210.0717
Weak overlap, ±0.600.04490.05340.05210.0717

Entries average Adult and Bank, three context sizes, and, for the neural models, two seeds. At ±0.60 under weak overlap, FSP reduces RMSE by 15.8% versus matched latent supervision and 13.7% versus polynomial T-learning, the strongest classical comparator in these cells. Paired 95% intervals for FSP minus comparator are [−0.0115, −0.0055] and [−0.0116, −0.0024], respectively.

Why overlap matters for the classical comparison. Weak overlap leaves fewer observations from one treatment arm in parts of the covariate space. This makes per-table outcome extrapolation harder and amplifies residual noise through the inverse-propensity weights in AIPW. FSP learns an adjustment across synthetic tables, so a good match between simulator and deployment can reduce the error of learning the adjustment from a single table. The oracle EIF itself still reflects the sampling noise imposed by weak overlap. When the outcome model fits the data well, per-table adjustment can have lower RMSE. Polynomial T-learning leads under moderate overlap, and the S-learner has lower RMSE than both FSP variants in the parsimonious four-stratum logistic comparison. Summary FSP lowers teacher defect by 32–53% relative to that S-learner (Section 5 and Table 3).

Additional checkFSPComparatorSetting
RMSE against released CausalPFN-S0.04160.0609Adult/Bank, weak overlap, unshifted, 128 paired tables at 1,024 rows
RMSE against released CausalPFN-S0.06240.0695Same shifted raw simulation tables in Figure 3
Calibrated 95% coverage / mean width95.4% / 0.24393.2% / 0.257Adult/Bank, weak overlap, ±0.60, versus boosted AIPW
AuthorsOpenReview ↗

Thanks for the review and here is the kind response(cont'd)

In the matched raw comparison, latent supervision has lower RMSE at the prior center (0.0176 versus 0.0621), and FSP has lower RMSE under effect shift (0.0624 versus 0.1364). Under shift, FSP also reduces teacher defect by 99.0% and brings the fitted response slope to 0.972, compared with −0.058 under latent supervision (Figure 3 and Table 3). In a separate controlled seven-target sweep at the typical mechanism, the sampling-response slope rises from 0.105 to 0.895 as supervision moves from the latent effect to the full fluctuation (Figure 5 and Appendix N.5). Appendices P.1, P.4 and Q provide the complete overlap, length and input-matched comparisons.

Where the computational gain comes from. FSP learns the adjustment during pretraining and reuses one frozen map. Cross-fitted AIPW fits nuisance models on each table. The evaluated CausalPFN-S pipeline also fits a weak learner at deployment. This explains why inference can be faster even when point RMSE is close. The released CausalPFN-S comparisons measure the pipelines end to end. The matched FSP/latent pairs isolate the effect of supervision. A1 reports the separate timing comparison on Criteo.

Q3. Residual Bias and Reliability on Real Data. Why does FSP retain treatment-effect attenuation on real RCTs, and how should this finding be reconciled with its theoretical guarantees and improved RMSE? What do the current experiments establish about the reliability of the learned estimator?

“On the RCTs it is more accurate than a simple difference in means but it still underestimates effects.”

A3.

On both original randomized trials, the five-checkpoint Raw FSP ensemble lowered RMSE relative to the arm-difference estimator, using the full-trial contrast as the reference. The reduction in sampling variability outweighed residual underestimation. Theorem 4.3 explains why approximation of the centered oracle EIF teacher can leave residual bias at deployment.

In the original trials, lower RMSE coexisted with attenuation. On NSW, the Raw FSP ensemble lowers post-treatment RMSE from 0.0596 to 0.0522 and rerandomized pretreatment-null RMSE from 0.0570 to 0.0479. Its mean post-treatment estimate was 0.0957 against the full-trial contrast of 0.1106 (Figure 4). On the turnout trial, Summary and Raw FSP have RMSE 0.0571 and 0.0628 versus 0.0775 for the arm difference, with signed errors of −0.0077 and −0.0241 against the full-trial contrast (Appendix N.14 and Figure 15). Each endpoint uses 1,500 tables of 256 rows.

Frozen FSP lowers visit RMSE against cross-fitted Hájek AIPW on long Criteo tables. The arm difference has lower point RMSE on this long-context grid, motivating the centering test below. All columns in a row use the same bootstrap tables and the same disjoint empirical assignment contrast. The table reports the five long Criteo visit settings; neural entries average the two seed-specific RMSEs, in units of 10−310^{-3}. Appendix P.3 contains every endpoint, length and arm-difference comparison.

Criteo visit rowsPaired tablesDesign FSPMatched design latentHájek AIPW
16,3843844.214.634.64
32,7683843.203.623.74
65,5362562.813.253.35
131,0721922.583.013.24
262,1441282.222.642.94

At 4,096 Criteo rows, FSP reduces visit RMSE by 3.3–3.6% against the arm difference and both seed-specific paired intervals favor FSP. At 16,384 rows, it reduces RMSE by about 9% against both matched latent supervision and Hájek AIPW and is within 3.1% of the arm difference. The paired 95% intervals against the arm difference include zero for both seeds. In an independent timing run at that length, median single-thread evaluation is 5.57–5.61 ms versus 8.66 ms for a numerically equivalent optimized AIPW implementation, including feature transforms. Across 16,384–262,144 Criteo rows, visit RMSE is 9.2–24.6% lower than Hájek AIPW, with paired 95% intervals favoring FSP for both seeds at every length. On Hillstrom visits at 32,768 draws, both seed-specific paired intervals favor FSP over the arm difference and include zero against Hájek AIPW. Hillstrom's evaluation split has 15,934 distinct records, so longer bootstrap tables reuse records.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Thanks for the review and here is the kind response

A Kind General Response to All Reviewers and Dear AC

We sincerely thank the Area Chair and all three reviewers for their careful reading and constructive suggestions. We made focused revisions, marked in blue, while keeping the main text at ten pages. Detailed replies are posted under Reviewer BUcB, Reviewer sGnU, and Reviewer v4eH.

  1. Scale and realistic transfer

    • Larger matched models, mixed-covariate semisynthesis, randomized studies, IHDP, and a known-effect test retaining all 82 encoded ACIC inputs extend the evaluation. The comparisons identify gains under effect shift and limited deployment data.
    • BUcB–A1–A3, sGnU–W2/Q1, v4eH–W3/Q3, paper Section 5, Appendices P and Q.1–Q.4.
  2. Statistical mechanism and transfer benefit

    • Matched-label experiments explain how supervision changes bias, variance and sampling response. A shared-prognostic specialization gives a finite-sample gain condition involving pretraining error, deployment size, dimension and task mismatch.
    • BUcB–A2/A4, sGnU–Q1, v4eH–W1–W2/Q1–Q2, paper Section 4, Appendices P.8 and Q.5.
  3. Native inference and deployment cost

    • Paired tests compare null rejection, interval width and alternative rejection. Summary FSP achieves shorter intervals and higher power at similar observed null size in the reported regime. Timings specify the comparator, implementation and workload.
    • sGnU–W2/Q1, v4eH–Q4, paper Section 5, Appendices N.8, P.3 and Q.2.
  4. Readability and self-containment

    • Related Work now precedes the setup. Local definitions clarify theorem objects, model variants, metrics and notation, and the revision aligns terminology and figure labels.
    • BUcB–A4–A6, sGnU–W1, paper Sections 2–5.

Thank you again for the time and care invested in these reviews. We welcome further questions and would be happy to discuss any aspect of the revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.