acceptodds
Under review as a conference paper at ICLR 2027

Unfreezing the Prior: Adaptive Prior-Data Fitted Networks

Abstract

Prior-Data Fitted Networks (PFNs) learn to approximate Bayesian prediction from synthetic tasks generated under a training prior, but return only the final predictive distribution, not the posterior over the simulator choices that produced it. We call the ordered set of these choices a *semantic path*. Making the semantic path explicit allows us to replace a pretrained PFN's training prior when predicting for a new population of tasks. We introduce Adaptive-PFN, which separates inference over semantic paths from prediction conditional on them without practically losing training-prior predictive accuracy to a classical PFN while enabling post-training adaptation. Given a population of deployment data histories, we estimate a low-dimensional exponential tilt of the training prior and reweight the inferred distribution over semantic paths without updating network parameters. Adaptation without model checking, however, leaves a critical statistical question unanswered: does the assumed shift family actually explain the deployment population? We address this with a goodness-of-fit test. In controlled experiments with a complex synthetic prior, adaptation reduces mean squared error by 6.22% relative to a direct PFN with approximately the same number of parameters; the test empirically maintains nominal Type-I error and shows high power against prespecified departures from the adaptation model. In synthetic-to-real electricity forecasting, Adaptive-PFN models trained only on simulated data reduce active-hour weighted quantile loss (WQL) by 31.4–45.3% relative to a standard frozen PFN, without retraining. In retail forecasting, Adaptive-PFN also uses a deployment prior estimated independently from population-fitting stores and reduces WQL on open promotion days by 7.5% relative to the direct PFN. Further ablations provide evidence that the proposed method provides benefits beyond simpler output-level calibrations.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
PFNs, prior shift, test-time adaptation, empirical Bayes, probabilistic forecasting

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_XhsuOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces adaptive PFNs, a prior-fitted network model that supports post-training adaptation by separating inference over the semantic paths M from the prediction conditional. The semantic paths are interpretable latents tied to the training prior, representing task-specific parameters. The prior over M can be elicited from historical data, similar to empirical Bayes. The method is evaluated on controlled toy experiments and on two real-world forecasting datasets.

Critical strengths

Interpreting the proposed method as Bayesian updating (Equation 4) offers a very interesting perspective.

Critical weaknesses
  • The experimental evaluation appears limited. (1) It would be valuable to go beyond the linear-regression training prior, for example by using a Gaussian Processes training prior with the kernel type and lengthscale as M. (2) An ablation on the dimensionality of M would also be informative, to show how well the method scales to higher-dimensional semantic paths.

  • Baselines: the set of baselines is quite limited. Although the paper mentions PriorGuide [1] and multi-task Bayesian in-context learning [2], it does not compare against them. For example, M could be implemented as the latent parameter θ\theta in PriorGuide, with the prior over it constructed via empirical Bayes from historical data.

  • Other than above I also have a concern on the broad applicability of semantic path concepts, especially in the case where the semantic paths / latent parameters are not easily defined.

[1] Yang, Yang, et al. "Priorguide: Test-time prior adaptation for simulation-based inference." International Conference on Learning Representations. Vol. 2026. 2026.

[2] Zhu, Qingyang, Eric Karl Oermann, and Kyunghyun Cho. "Multi-Task Bayesian In-Context Learning." arXiv preprint arXiv:2606.20538 (2026).

Questions for authors
  • Would it make sense to extend this approach to a more general setting, such as the SCM-based training priors used by TabPFN/TabICL, i.e., a tabular foundation model rather than a prior-specific PFN or neural process? In particular, can meaningful semantic paths M even be defined for such foundation models? (future works/clarification)

  • The choice of the prior tilt in Equation 2 needs more justification, as its motivation is currently not entirely clear. For example, how does it compare with the power prior [3]? (clarification)

  • Could the target head be made simpler, e.g., a single Gaussian or a GMM with a small number of components as in [4]? Since the conditional prediction given a specific semantic path M should be considerably simpler, would such a head be easier to learn and better behaved? (clarification)

References:

[3] Ibrahim, Joseph G., and Ming-Hui Chen. “Power prior distributions for regression models.” Statistical Science (2000): 46–60.

[4] Chang, Paul E., et al. “Amortized probabilistic conditioning for optimization, simulation and inference.” arXiv preprint arXiv:2410.15320 (2024).

3 comments

Response (1/2)

We thank the reviewer for concrete, actionable suggestions; we implemented the main one exactly as proposed. All changes are shown in red in the updated PDF.

1. Beyond the linear-regression prior: a GP with kernel type and length-scale as M (new §3.4, App. H). We trained Adaptive-PFN on one-dimensional GP regression with M = (kernel family ∈ {RBF, Matérn-3/2, Matérn-1/2}, length-scale). Exact Bayes is computed on a fine grid as a reference. Deployment tilts toward rough, short-length-scale functions. We fit it with the paper's moment estimator from 128 histories, with three training seeds × three deployment repetitions.

  • With 4 observations: adaptation lowers MSE by 7.7–7.8% relative to the unadapted model in every seed (intervals exclude zero) and matches the known tilt.
  • Log loss: its improvement is 95–100% of the improvement exact Bayes obtains from the deployment prior.
  • With 16 observations: the exact-Bayes gain itself is small (0.4% MSE), and adaptation still gains 2.7–2.9%. We note honestly that part of this corrects network error.
  • Accuracy of T1 and T2: T1's kernel-family errors are 20–26% of the prior's, and T2 recovers 90–95% of the exact conditional gain.

2. Dimensionality of M. The semantic path can be large: in the controlled experiment M has 527,898 discrete configurations plus a continuous coordinate. What matters for reweighting is the dimension and concentration of the tilt. We added two pieces of evidence.

  • Tilt-family ablation (App. E.6). We use tilt dimensions 1, 2 and 3 (B; B,K; B,K,U).
    • A too-small family still gives 6.6–7.0%, against 8.1–8.5% for the correct one.
    • Adding an unshifted coordinate leaves both the gain and the ESS unchanged.
  • ESS across all experiments (App. I). We collect the path ESS of every experiment, with tilt dimensions 1–4, in one table.
    • ESS is governed by how concentrated the tilted prior is relative to each posterior, more than by the number of coordinates.
    • The lowest values occur in London, where the fitted tilts narrow the prior from SD 0.25 to 0.02–0.09 (App. F.4).
    • We list every fit that reaches the parameter box (limitation (iii)).

We do not claim scalability to many simultaneously shifted coordinates. App. I states that the method assumes a tilt with few coordinates, and limitation (iii) lists the overlap failures we observe; a systematic sweep over the number of shifted coordinates remains open.

(continued in the next comment)

Response (2/2)

3. Baselines: PriorGuide and Multi-Task Bayesian ICL.

  • The reference these methods would have to match (§3, App. E.6). Like Adaptive-PFN, these methods require dedicated pretraining, here to condition on a prior or on related tasks. Training on the correct deployment prior is what they would need to emulate. A direct PFN retrained from scratch on the deployment prior with the same budget is therefore an informative reference.
    • The frozen Adaptive-PFN recovers 97% [89, 104] of its improvement at history length 8.
    • At length 32 it is more accurate than the retrained network.
  • PriorGuide. It applies the same prior-ratio principle to simulation-based inference with a supplied prior. With M as its latent parameter and a prior estimated from historical data, it corresponds to our density-ratio variant, which we evaluate on Rossmann with a regression-estimated prior (6.4–8.2% vs the direct PFN, three seeds). We state this relation in App. A.
  • Real data with informative histories. On Rossmann, Adaptive-PFN reduces WQL by 4.8–6.7% relative to the direct PFN (three seeds, all intervals exclude zero). An output-level correction given the same population information does worse than no correction there.
  • External baselines. We also add full and readout-only fine-tuning, and a Chronos-2 reference.

4. Applicability when semantic paths are hard to define. We added a Scope paragraph (§1) and guidance on choosing M (App. I). We recommend exposing simulator choices that are:

  • interpretable, so a population change has a stated meaning;
  • plausibly population-dependent;
  • not determined by a single history.

Q1 — TabPFN-style SCM priors. Natural semantic paths there are high-level generator choices: graph density, mechanism family, noise type. Exposing them requires T1 to infer these choices from one dataset, which is a substantive question we leave to future work and now discuss (App. I). We note, however, that in many tabular PFN constructions the synthetic datasets are generated from a directed acyclic graph (DAG), which offers a natural structure for semantic paths: for example, a path that follows the directed edges connecting the nodes of the DAG.

Q2 — Justifying the tilt; comparison with the power prior (§2, App. B.2).

  • Minimum-KL justification. Among all priors whose shift features have given means, the exponential tilt uniquely minimizes KL(p‖p0) (I-projection; Csiszár, 1975). It is therefore the smallest change of the training prior consistent with the shifted moments; we include a short proof.
  • Power prior. The power prior of Ibrahim & Chen (2000) is the special case s(m) = log L(m | D_hist), η = a0.
  • The difference: the power prior fixes the feature through one historical dataset and treats a0 as given. Adaptive-PFN fixes interpretable features of M and estimates η from a population of deployment histories.

Q3 — A simpler target head. A Gaussian or small Gaussian-mixture head, as in ACE, can be substituted without changing the reweighting, and the adapted mixture over paths stays flexible even if each component is simple. We keep the Riemann head because forecasting targets are skewed and the same head is used by the direct PFN (which was the way it was originally constructed in the literature), keeping comparisons matched (App. I).

Also new. We added a corollary giving a sufficient condition for adaptation to improve on the unadapted predictor, and Rossmann with the paper's own moment estimator (4.8–6.7%, three seeds).

We hope the Gaussian-process experiment, which implements your suggestion directly, and the minimum-KL justification of the tilt address your main concerns. If so, we would be grateful if you would consider updating your score. We would also welcome further suggestions during the discussion.

Dear Reviewer,

We thank you again for your constructive comments. As the discussion closes soon, we wanted to make sure our response reached you. We followed your suggestions directly:

  • GP prior with kernel type and length-scale as M, as you proposed (§3.4, App. H). Adaptation lowers MSE by 7.7–7.8% in every seed. It recovers 95–100% of the log-loss gain that exact Bayes obtains from the deployment prior.

  • Dimensionality of M. We added an ablation over the number of tilted coordinates (1–3), in which an extra unshifted coordinate costs nothing. We also report the path ESS for every experiment (App. E.6, App. I).

  • PriorGuide / Multi-Task BICL. These methods condition on a prior, so their target is a network trained on the deployment prior. The frozen Adaptive-PFN recovers 97% of that network's gain (§3).

  • The tilt in Eq. 2. It is the unique KL-minimal change of the training prior, and the power prior is a special case (§2, App. B.2).

  • SCM priors, a simpler head, and paths that are hard to define. These are discussed in App. I and the Scope paragraph (§1).

If these address your concerns, we would be grateful if your assessment reflected them.

Reviewer_wUdfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Adaptive-PFN, a framework that separates inference of the semantic path (a user-selected subset of latent variables) from the posterior conditioned on the semantic path. It uses a semantic posterior network to return the semantic path distribution, conditioned on data D, and a conditional predictive network to predict y, conditioned on x, D, and a semantic path M. The two networks are trained separately.

Critical strengths

I think the motivation is clear, and the approach is principled. The architectural separation of T1 and T2 aligns well with the bayesian decomposition of the posterior inference into "infer M from D" and "infer y from (x, M, D)". It may also be helpful for interpreting distribution shifts in M. I also find the controlled experiments to be well-designed, and adaptive-pfn shows clear improvements over the baselines under various prior shifts. Smaller gains with longer task histories also aligns with what intuition from bayesian inference. The experiments in section 4 show encouraging advantage of adaptive-pfn on real-world data as well.

Critical weaknesses

The main concern I have is about generality. The authors seem to assume that deployment changes only the distribution of the exposed simulator variables, but the conditional data-generating mechanism is preserved. In complex problems, it is not always clear if the data-generating mechanism is sufficiently rich to capture important distribution-shifts as simple tilting of the simulator variables.

The real-world data experiment specifically adapt the distribution of a few coefficients: London experiment essentially changes the distribution of tariff-related coefficients, and Rossman experiment changes the distribution of promotion-related coefficients. While these two examples demonstrate that "simple semantic path tilt" do capture real world distribution shifts, they do not demonstrate the viability of adaptive-pfn on more complex distribution shifts.

Questions for authors
  • For the London experiment: how much does adaptation really help when the household’s own history already includes informative High/Low-tariff observations?
  • Could the networks be trained jointly, so that errors of T1 might be accounted for by the downstream network T2?
  • How would adaptive-pfn work on more complex distribution shifts, e.g. rossman data but in different seasons, or over different years?
3 comments

We thank the reviewer for the positive assessment and for questions that sharpened the paper. All changes are shown in red in the updated PDF.

1. Generality: shifts richer than a tilt of a few coefficients. We agree that Adaptive-PFN assumes deployment changes the distribution of declared simulator variables. That is its intended scope, which we now state explicitly (Scope paragraph, §1; limitations, §5). We added four pieces of evidence on how far the method extends.

  • A prior that is not built around a regression coefficient (new §3.4). We use a Gaussian-process prior whose semantic path is the kernel family and its length-scale. Exact Bayes is available as a reference.
    • With 4 observations, adaptation lowers MSE by 7.7–7.8% in every seed.
    • Its log-loss improvement is 95–100% of the gain exact Bayes obtains from the deployment prior.
  • Mechanism changes outside the declared family (§3, App. E.6). Deployment combines the declared sign shift with a nonlinear (threshold-tree) or heteroscedastic mechanism that no tilt can represent. Adaptation still reduces MSE in all 18 seed–task-set cells, matching the known sign tilt.
  • Wrong tilt family. A tilt family that is too small still gives 6.6–7.0%, against 8.1–8.5% for the correct one. Adding an unshifted coordinate costs nothing.
  • Real shifts outside the declared variables (App. G.7). In a Rossmann window whose targets include Easter, adaptation neither helps nor hurts (−0.1 to 3.3%), and an output-level correction does better. We report this as the method's boundary.

The method therefore degrades gracefully rather than breaking. Where it stops helping is stated, not hidden.

2. Q1 — London when the household's history already contains High/Low-tariff observations. We added exactly this setting, a warm start (§4, App. F.9). For each evaluation event, the history contains the 48 Normal hours plus the target day up to and including the first two hours of its tariff block. The household's own response to the tariff is therefore partly observed, and we forecast the rest of the block. We use ten training seeds: the three of the main analysis and seven more trained identically.

  • Against the direct PFN: adapted Adaptive-PFN reduces active-hour WQL in every seed (7.9–44.7%; intervals exclude zero in eight).
  • Against the unadapted model: it improves in every seed.
  • NLL: it lowers NLL in every seed.

Adaptation therefore keeps helping once the household's history begins to reveal its tariff response. With the 48-hour histories of the main analysis, which contain no tariff hours, Adaptive-PFN serves as prior adaptation, as for a new customer or tariff; we now state this explicitly (§1, §4).

More generally, a better prior helps only as much as the prior still matters once the history is observed. A new table in §5 shows this where the true prior is known: as histories lengthen, the true prior's gain falls (5.8% → 1.8% controlled; 0.097 → 0.013 nats GP), and Adaptive-PFN matches it.

3. Q2 — Could T1 and T2 be trained jointly? They could, but it would undermine the adaptation step (App. I).

  • Joint training through the mixture likelihood would let T2 compensate for T1's errors under the training prior. T2's predictions would then depend on the training-prior posterior.
  • Reweighting requires T2's per-path prediction to stay valid when the path weights change, and under the prior-shift assumption its target P0(y | x, D, m) does not depend on the prior.
  • Separate proper-scoring objectives therefore keep both networks valid under reweighting, and they also allow parallel training.

4. Q3 — Rossmann in other seasons and years. We repeated the full evaluation at two other forecast origins: March 2015 (targets include Easter) and June 2014 (App. G.7, new table). The estimator, settings and metrics are unchanged, with three seeds each.

  • Never harmful: adaptation increases WQL by at most 0.1%.
  • Smaller gains: −0.1 to 3.7% relative to the direct PFN, with intervals that include zero.
  • The Easter window: an output-level correction is more accurate. This is consistent with a seasonal change outside the declared promotion effects, since the simulator represents Easter only as a generic holiday.

We state this in limitation (ii). Adapting to such shifts would require exposing a holiday effect in the semantic path.

Also new. We added the following:

  • three Rossmann seeds;
  • Rossmann with the paper's own moment-matching estimator, giving 4.8–6.7% vs the direct PFN, with all intervals excluding zero;
  • a Chronos-2 reference;
  • prediction cost;
  • a corollary giving a sufficient condition for adaptation to improve on the unadapted predictor.

Thank you again for your supportive review. We believe the added experiments make the paper's scope and evidence considerably clearer; if you agree, we would appreciate it if your assessment reflected this. We are happy to provide further analysis during the discussion.

Dear Reviewer,

Thank you for your supportive review. As the discussion closes soon, here is how the revision answers each of your points:

  • Generality (your main concern). We now test shifts beyond tilting a few regression coefficients. On a Gaussian-process prior whose semantic path is the kernel family and length-scale, adaptation recovers 95–100% of the exact-Bayes gain (§3.4). When deployment adds a nonlinear or heteroscedastic mechanism that no tilt can represent, adaptation still helps in all 18 cells (App. E.6).
  • Q1: London with informative histories. We added exactly this setting, a warm start whose histories include the first tariff hours. Adaptation beats the direct PFN (7.9–44.7%) and the unadapted model in all ten seeds (§4, App. F.9).
  • Q3: other seasons and years. In two further Rossmann windows, adaptation is never harmful. We report the Easter window, a shift outside the declared variables, as the method's boundary (App. G.7).
  • Q2: joint training. Training the networks separately keeps T2 valid when the path weights change, which reweighting requires (App. I).

If this resolves your concerns, we would be grateful if your score reflected it.

I thank the authors for their response. These address my concerns and I maintain my positive score.

Reviewer_VsEcOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Adaptive-PFN. A standard PFN returns only a final predictive distribution. Adaptive-PFN instead uses two networks. T1 infers a posterior over a user-chosen set of simulator variables, called the semantic path MM, from the context. T2 predicts the target given the context and MM. Averaging T2 over T1 recovers the usual PFN prediction.If a deployment population uses these variables at different frequencies than the training prior, T1's samples can be reweighted by Bayes' rule instead of retraining. The new frequencies are estimated from a population of deployment tasks as a low-dimensional exponential tilt of the training prior, using whitened moment matching. A kernel two-sample (MMD) goodness-of-fit test with a refitted bootstrap checks whether such a tilt can explain the deployment data. Theorem 1 bounds the extra log-loss from using an estimated tilt instead of the true one.

Experiments cover three settings. In a controlled sparse linear regression with a shifted sign frequency, MSE is 6.22% lower than a parameter-matched direct PFN. On London electricity, weighted quantile loss (WQL) on tariff hours is 31.4–45.3% lower. On Rossmann sales, WQL on promotion days is 7.48% lower.

Critical strengths
  1. The idea is clean and correct. Exposing the semantic path lets a frozen PFN take a new prior through Bayes' rule, which is a sensible generalisation of label-shift correction. The split costs little: under the training prior, Adaptive-PFN matches a parameter-matched direct PFN (MSE 2.1123 vs 2.1160).

  2. The experiments are run carefully and reported honestly. Baselines are matched in parameters and training budget. Procedures are fixed on development data before evaluation, and the intervals include re-estimating the tilt. Negative results are reported openly, including fine-tuning doing better, tilts hitting the box boundary, and low effective sample sizes.

Critical weaknesses
  1. The evidence shows the method works where it was set up to work, not that it works in general.

    • In the controlled experiment, deployment shifts exactly the variable the method is allowed to shift (s(M)=Bs(M)=B).
    • In both real datasets, the simulator is built around the one effect that changes: the tariff association in London and the promotion association in Rossmann. The paper also states that these two datasets were identified as suitable test beds for the method (AI use statement, p. 10).
    • Rossmann uses a single training seed. Its deployment prior is estimated by store-level regressions outside the method, so the moment-matching estimator is tested on real data only in London.
    • In London, the 48-hour contexts contain only Normal-tariff hours (App. F.1). They carry no information about βH,βL\beta_H, \beta_L, so T1's posterior over these should equal the training prior. The adaptation step therefore reduces to replacing a prior with SD 0.25 by a population estimate with SD 0.05–0.09, and the headline metric is computed only on tariff hours. This is plain empirical-Bayes prior replacement, not evidence for posterior reweighting.
    • The method assumes the user already knows which few variables will shift. The semantic path is defined only as a user-selected subset of simulator variables (Sec. 1), with no guidance on choosing it for richer priors such as GP or TabPFN-style tabular priors. Choosing too few variables breaks the core assumption.
    • Choosing more variables makes the weights collapse, and this already happens with four tilt coordinates in London. For seed 1, the mean path effective sample size (ESS) is 11.8 of 256, with a minimum of 1.01 when 64 paths are used (Tables 5, 10). The fitted tilt also hits the box boundary at −80, with a nearly singular Jacobian.
    • The limitations section is one sentence and does not discuss any of this.
  2. The comparisons are too weak to support the claimed benefit.

    • Prior-conditioning methods. Methods that take prior information as input or infer it from related tasks are cited but never compared against: ACE (Chang et al., 2025), Distribution Transformers (Whittle et al., 2026), and Multi-task Bayesian ICL (Zhu et al., 2026). These need one forward pass, while Adaptive-PFN needs 256–2048 T2 evaluations per prediction.
    • Forecasting models. No standard forecasting model (e.g., Chronos-2, TimesFM) is reported, so the absolute level of the forecasting results is hard to judge.
    • Output-level baseline. The only output-level baseline is weak. In London its NLL is 293–629, against about −0.2 for Adaptive-PFN. In Rossmann it is worse than the uncalibrated direct PFN on all three metrics (Table 12). The claimed benefit over output-level calibration therefore rests on a weak comparator.
    • Fine-tuning. Early-stopped full fine-tuning beats the method: mean WQL is 0.405 at epoch 32 against 0.524 (Table 11). That epoch was chosen by cross-validation on the population-fitting households alone, so it did not need evaluation data.
    • Theorem 1. It bounds the cost of using an estimated tilt instead of the true one, but does not show that adapting improves over the unadapted predictor.
    • Compatibility test. It is applied to real data only once, in London, where it does not reject: p = 1.00, with 0/60 rejections at the fitted tilt. This looks conservative rather than reassuring. The test is not applied to Rossmann.
Questions for authors

See weakness.

5 comments

Response (1/3)

We thank the reviewer for a careful and constructive review. We agree with several of its central points and have revised the paper accordingly; all changes are shown in red in the updated PDF. Below, each point is answered with what changed and where.

1. London as prior replacement, and "the method works where it was set up to work".

London, 48-hour histories. Your reading is correct, and we now state it explicitly (§1, §4).

  • The 48-hour histories contain only Normal-tariff hours, so they carry no information about a household's tariff response. T1's posterior over it equals the training prior, and adaptation replaces that prior with the population estimate.
  • In this setting Adaptive-PFN therefore serves as prior adaptation. That is exactly what a new customer or a new tariff requires, since the household's own data cannot supply it.
  • The population prior is estimated through T1 from other households' informative two-week histories. Applied to the frozen networks, it reduces active-hour WQL by 31.4–45.3% relative to the direct PFN.
  • Seeds: with seven further training seeds (App. F.8), the cold-start reduction is 23.7–46.5% across all ten seeds, significant in nine, and NLL improves in every seed.

London, histories that do carry tariff information (new warm start, §4, App. F.9). To test the method when the history is informative about the tariff response, the history now also contains the target day up to and including the first two hours of its tariff block. We use ten training seeds: the three above and seven more trained identically.

  • Against the direct PFN: adapted Adaptive-PFN reduces WQL in every seed (7.9–44.7%).
  • Against the unadapted model: it improves in every seed.
  • NLL: it lowers NLL in every seed.

So adaptation keeps helping when the household's own history begins to reveal its tariff response.

Same knowledge. We agree that Adaptive-PFN presupposes knowing which simulator variables may shift; label-shift correction likewise presupposes that only class frequencies change. Our experiments therefore ask whether exposing the semantic path uses that knowledge well.

  • Rossmann: each 56-day history contains 19–25 promotion days. Adaptation reduces promotion-day WQL by 4.8–6.7% relative to the direct PFN in every seed.
  • Success is not guaranteed by the setup. The networks never see real data, and the real series differ from the simulator in many respects. The evaluation also shows where the method stops helping, for example in the Rossmann window containing Easter.

When a better prior can help. A better prior helps only as much as the prior still matters once a task's own history is observed. Where the true prior is known, we now compare the gain it yields with the gain Adaptive-PFN achieves (new Table 2, §5):

SettingHistoryReferenceReference gainAdaptive-PFNShare
Controlled (MSE)8 / 32 obs.true tilt5.79% / 1.79%5.77% / 1.81%~100%
GP (log loss)4 / 16 obs.exact Bayes0.097 / 0.013 nats0.093–0.097 / 0.01395–105%
Rossmann (WQL)56 dayssecond prior, estimated from >1 year of sales7.5–8.5%5.5–7.0%73–82%

Where the true prior is known, Adaptive-PFN achieves about the gain that prior offers, and both shrink as histories lengthen.

(continued in the next comment)

Response (2/3)

2. Moment matching tested on real data only in London; Rossmann used one seed.

  • We added two Rossmann training seeds.
  • We now apply the paper's moment-matching estimator to Rossmann, with London's estimator, optimizer and parameter box. Because the Rossmann prior is more concentrated, we use larger banks: 16,384 fitting tasks and 4,096 whitening tasks.
  • Results: it reduces promotion-day WQL by 5.5%, 4.8% and 6.7% relative to the direct PFN, with all three 95% intervals excluding zero. It also lowers NLL in every seed.
  • Recovered prior: the same between-store spread as the store-level regressions (App. G.6).
  • For full disclosure:
    • The development-store gate would have passed for one seed only.
    • In two other forecast windows the gains are smaller and not significant.
    • In the window containing Easter, an output-level correction does better (App. G.7).

3. Shifts outside the declared variables, and choosing M.

  • Controlled departures (§3, App. E.6): deployment combines the sign shift with a threshold-tree or heteroscedastic mechanism that no tilt can represent. Adaptation still improves MSE in all 18 cells.
  • Tilt family size: a family that is too small still yields 6.6–7.0%, against 8.1–8.5% for the correct family. One extra coordinate costs nothing.
  • Kernel prior (new §3.4): a Gaussian-process prior with M = (kernel family, length-scale). Adaptation recovers 95–100% of the exact-Bayes gain.
  • Guidance on choosing M: App. I.
  • Limitations: now a four-point paragraph covering declared variables, out-of-family shifts, overlap and boundary fits, and cost.

4. Weight collapse and boundary fits.

  • ESS: we now report it for every experiment in one table (App. I). It depends on how concentrated the tilted prior is, more than on the number of tilt coordinates.
  • Boundary hits: every one is listed in limitation (iii): London seed 1, the Rossmann Promo2 spread, Rossmann June 2014, and two of nine GP fits.
  • Box sensitivity: widening the Rossmann box to ±160 changes the gains by at most 0.3 points.
  • Boundary fits do not drive the London results: across ten training seeds, four London tilts reach the boundary, and those seeds gain 23.7–46.5% in the cold start, as the others do (App. F.8).
  • London's tilts: the prior they imply is now reported per seed (App. F.4). They narrow the tariff-effect SD from 0.25 to 0.02–0.09, close to household regressions that do not use T1.

6. Theorem 1 does not show that adaptation helps. New Corollary 1 (§2) gives a sufficient condition for R(η̂) < R(0). Adaptation then improves on the unadapted predictor with probability tending to one whenever the true tilt helps the frozen networks.

7. Compatibility test.

  • Scope: we now describe it modestly. A non-rejection shows only that no departure visible in g(D) was detected.
  • Rossmann: we have not validated a g(D) there, so the test is not applied, as limitation (i) states.
  • London: p = 1.00 means the observed statistic is below every bootstrap draw, so the test is conservative there. We call the non-rejection weak evidence (App. F.5). The 60/60 power against the dependence departure shows the test can detect changes visible in g(D), and the controlled experiment establishes its calibration and power.
  • AI statement: reworded to say why these datasets fit the setting. They record interventions whose population-level effects are the object of prior shift.

(continued in the next comment)

Response (3/3)

5. Comparisons.

  • Prior-conditioned methods. Like Adaptive-PFN, these need dedicated pretraining. As a reference for what training on the correct prior reaches, we retrain a direct PFN from scratch on the deployment prior with the same budget. The frozen Adaptive-PFN recovers 97% [89, 104] of its gain at history length 8 and is more accurate at length 32 (§3).
  • Fine-tuning. We state plainly that it can be more accurate when weights can be updated.
    • London: cross-validated full fine-tuning reaches WQL 0.405 against 0.524. Readout-only fine-tuning reaches 0.582–0.604, worse than Adaptive-PFN in two seeds, with lower NLL.
    • Rossmann (new, App. G.8): fine-tuning on the population-fitting stores' earlier sales is more accurate in one seed of the primary window and in June 2014. It is less accurate in March 2015, when its population data include the Christmas season.
    • Adaptive-PFN changes four tilt coordinates and no weights, and needs no stopping rule.
  • Output-level baseline. We agree that the affine calibration is weak in NLL. Readout-only fine-tuning, above, is the stronger output-level comparator. On Rossmann we lead with the comparison against the direct PFN, since the calibration there is worse than no correction.
  • Forecasting foundation models. As suggested, we now report Chronos-2 (120M parameters), given the known tariff and promotion covariates, as a reference for the accuracy attainable on these data rather than as a competing method (§4, App. I).
    • Scores: it reaches active-hour WQL 0.462 on London and 0.0897 on Rossmann.
    • London caveat: its pretraining corpus includes London smart-meter series from the same release, so it is not zero-shot there.
    • Scope: our claim concerns adapting a given PFN without retraining, not state-of-the-art forecasting.
  • Cost. A prediction takes about 41 ms per task on one A100 GPU, against 9 ms for the direct PFN. When batched, Adaptive-PFN takes 81–189 times the direct PFN's per-task time. Both figures are in limitation (iv).

We are grateful for a review that substantially improved the paper. If any concern remains that keeps you from a more positive assessment, we would welcome the chance to address it during the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.