acceptodds
Under review as a conference paper at ICLR 2027

Curves in Context: Zero-Shot Regression of Functional Data

Abstract

Scalar-on-function prediction—mapping an observed curve to a scalar response— is a central task of functional data analysis (FDA). Practice remains dominated by dataset-specific pipelines whose ingredients (basis, smoothing, model family, hyperparameters) must be reselected for every new problem and are tied to a fixed observation grid. We introduce FunPFN, a foundation model for scalar-on-function regression. FunPFN is pretrained once, purely on synthetic episodes drawn from a generative prior over functional tasks, and then predicts on a new dataset in a single forward pass: the training set is supplied as an in-context support set and the model returns a Gaussian predictive distribution for every query curve, with no gradient updates and no hyperparameter tuning. Its encoder combines a Fourier neural operator (FNO) trunk and a support-standardized spectral readout, making predictions discretization-invariant by construction. Across ten simulation scenarios, FunPFN achieves the best average rank among fourteen methods (1.7), significantly ahead of all twelve tuned baselines (Friedman test, Holm post-hoc); the runner-up, TabPFN applied to the discretized curves (4.7), is not separated by this test, yet its normalized excess risk is 1.5 to 4.6 times that of FunPFN in every scenario. On twelve real benchmark tasks, FunPFN ranks second overall while operating over 200 times faster than TabPFN (0.025s vs. 5.983s).

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
functional data analysis, in-context learning, neural operators, prior-fitted networks

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_h63KOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces FunPFN, an in-context model pretrained on synthetic tasks for scalar-on-function regression. It combines an FNO curve encoder, attention pooling, and support-standardized Fourier features with a Transformer that uses labeled support examples to predict a Gaussian response distribution for each query curve. Deployment requires neither gradient updates nor task-specific hyperparameter tuning. Evaluation covers ten synthetic scenarios and twelve prediction tasks from seven public datasets, together with analyses of resolution changes, small-sample performance, and predictive calibration. The main findings are strong simulation rankings, competitive real-data performance, and lower deployment latency than the reported TabPFN configuration.

Critical strengths
  1. The application objective is clear and practically relevant. Reducing repeated representation selection, fitting, and tuning is useful for small-sample functional regression. Using labeled examples as context to produce predictions and uncertainty in one forward pass offers a convenient deployment interface.

  2. Several important qualifications are reported transparently. The authors acknowledge the overlap between simulation mechanisms and the pretraining prior, the absence of significant real-data superiority over TabPFN, and performance saturation as sample size increases. These disclosures help establish the actual scope of the results.

Critical weaknesses
  1. The experiments do not establish the independent value of the functional architecture, and the central contribution is confounded with prior matching. Every simulation target form belongs to a pretraining mechanism family, so the simulation advantage primarily supports amortized inference within the chosen prior. The TabPFN comparison changes the architecture, training prior, and pretraining configuration simultaneously, leaving the source of improvement unresolved. The appendix ablations do not close this gap: the authors explicitly acknowledge that removing the FNO and re-adapting the resulting network does not represent a spectral-score Transformer trained from scratch. Meanwhile, the Fourier readout, its standardization, and mean pooling improve RMSE by only approximately 1.0%, 1.6%, and 0.2%, respectively, after re-adaptation. Without simpler encoders trained under the same prior and comparable budgets, the necessity of the proposed architecture remains unclear. Real-data results do not compensate for this missing evidence: FunPFN's average rank is 4.63, close to Ridge at 4.67 and behind TabPFN at 3.29. Failure to detect a significant difference does not establish equivalence. Moreover, the twelve tasks come from seven datasets, with multiple responses sharing the same curves; sensitivity of the aggregate rankings and significance conclusions to this task dependence is insufficiently assessed.

  2. The claims about function-space applicability and predictive calibration exceed the evidence provided. Discretization invariance is motivated mainly through convergence under grid refinement, rather than finite-grid prediction consistency or error control. The resolution experiment aggregates RMSE after resampling, without directly measuring whether predictive means and variances for the same support/query functions remain consistent across grids. The model is still restricted to shared uniform grids and does not address sparse, irregular, or subject-specific observation times, limiting the present scope of its positioning as a functional foundation model. There is also an implementation ambiguity: the spectral branch requires m≥64m \geq 64, whereas Growth has only nineteen native observations. For uncertainty, pooled coverage can conceal substantial task-specific miscalibration, and matching conditional means and variances with a Gaussian head does not guarantee interval coverage. These issues affect the reliability of the current method; the possibility of future extensions does not resolve them.

Questions for authors
  1. Can the authors compare the FNO encoder with Fourier-score, FPCA-score, or other simple encoders trained from scratch under the same synthetic prior, objective, and comparable pretraining budgets? This comparison is essential to establish the necessity of the proposed architecture, rather than an extension to a new task.

  2. Do the rankings and significance conclusions persist when results are aggregated by the seven source datasets and sensitivity to dependence among responses sharing the same curves is assessed? In simulation, does the relative advantage persist when an entire mechanism family is excluded from pretraining?

  3. How are the nineteen-point Growth curves processed by a spectral branch requiring m≥64m \geq 64? Please specify whether interpolation, padding, adaptive truncation, or another procedure is used, and reconcile this with the statements about avoiding interpolation and grid dependence. Can the authors directly report changes in predictive means and variances for identical support/query functions evaluated on different grids?

  4. Can the authors provide per-task coverage, predictive interval width, and Monte Carlo uncertainty? Does the favorable pooled coverage conceal undercoverage on some tasks offset by overcoverage on others? These results would materially affect the assessment of the current uncertainty-quantification claims.

6 comments
AuthorsOpenReview ↗

Response to Reviewer h63K (1/3)

We thank the reviewer for a demanding review. Its central question led to the most informative experiment of the revision.

h63K-1. The experiments do not establish the independent value of the functional architecture [...]. Can the authors compare the FNO encoder with Fourier-score, FPCA-score, or other simple encoders trained from scratch under the same synthetic prior, objective, and comparable pretraining budgets?

We ran exactly this comparison (new Appendix I). Five alternative encoders replace the FNO encoder, each trained from scratch with everything else fixed: same prior, Gaussian NLL, 12-layer transformer, per-step budgets, schedule and 100k steps; the FNO encoder was retrained under the same budget as control, with two seeds. Geometric-mean RMSE ratio to the control on the twelve real tasks (two values: two seeds):

Encoder (from scratch)ParamsRMSE ratioBetter on
FNO encoder (control)50.2 M1.005 / 0.995–
In-context FPCA, 32 scores0.6 M0.837 / 0.84910 / 10 of 12
In-context FPCA, variance cutoff0.6 M0.868 / 0.8619 / 11 of 12
FNO trunk without spectral convolutions3.0 M0.953 / 0.9979 / 7 of 12
Fixed 128-point grid (TabPFN-style input)0.9 M1.1046 of 12
Fourier coefficients only0.6 M1.3450 of 12

The answer is not the one the submission argued. At equal budget the FNO is not what makes the method accurate on real data: an FPCA encoder computed in context does better, mostly on near-infrared spectra. On the 17 simulated scenarios the picture is balanced. FPCA is better where the response depends on principal-component scores (score-quadratic 0.032–0.043 against 0.063), the FNO on the linear, quadratic and single-index designs, and FPCA fails when curves differ by a random time warp (0.63–0.69 against 0.21–0.22). The FNO is also more stable than FPCA across grids, and it is not faster: per training step it is 4–7 times slower. The Fourier-only arm had twice the validation loss of the others, so we report it as a lower reference only. Finally, the released 300k-step model is better than every 100k-step arm on 14 of 17 scenarios, so the training budget matters more than the encoder. We agree with the reviewer's reading: the contribution is amortized inference under a carefully built functional prior, and the encoder is a modular part of it with measured trade-offs. The method section and the contribution list now say this, and the FNO is presented as our default encoder with the reasons the ablation supports.

h63K-2. The TabPFN comparison changes the architecture, training prior, and pretraining configuration simultaneously.

Two experiments separate these factors. The encoder ablation holds prior and recipe fixed and changes only the architecture; it includes a fixed-grid encoder that gives our transformer a TabPFN-style input, which is 10% worse than the FNO, so our prior does not make any input work. Conversely, TabPFN on 20 FPCA scores keeps TabPFN's prior and recipe and gives it a functional representation. On real data it is the most accurate method, ahead of FunPFN on 8 of 12 tasks; on all 17 simulated scenarios it is behind FunPFN, significantly on 14 (paired Wilcoxon, Bonferroni). An FPCA representation helps both TabPFN and our transformer on real spectra, while our prior gives the advantage on the mechanisms it covers and on nearby ones.

h63K-3. [...] the Fourier readout, its standardization, and mean pooling improve RMSE by only approximately 1.0%, 1.6%, and 0.2%, respectively, after re-adaptation.

We agree, and the from-scratch study confirms the reviewer's reading: the spectral readout is not necessary when a data-adaptive basis is available. We removed the statement that motivated it as necessary, and the lesion study is now labelled as such and placed after the from-scratch ablation.

AuthorsOpenReview ↗

Response to Reviewer h63K (2/3)

h63K-4. FunPFN's average rank is 4.63, close to Ridge at 4.67 and behind TabPFN at 3.29. Failure to detect a significant difference does not establish equivalence.

We agree with both statements. The revised paper no longer calls FunPFN second-best on real data; it states a statistical tie with the most accurate methods, and adds that TabPFN on FPCA scores is more accurate still. The real-data case is now narrower and measured: coverage close to nominal on every task where TabPFN under-covers (h63K-9), a 0.025 s cost per split, about 240 times less than TabPFN, and an advantage concentrated at small training sizes (at 20 curves, the most accurate of FunPFN, TabPFN on FPCA scores and Ridge on half the tasks).

h63K-5. [...] sensitivity of the aggregate rankings and significance conclusions to this task dependence is insufficiently assessed. Do the rankings and significance conclusions persist when results are aggregated by the seven source datasets?

They do not fully persist, and we now report both aggregations (Appendix N). Averaging ranks within each dataset before the Friedman test, TabPFN is first (3.71), Ridge second (4.29) and FunPFN third (4.64). The test still rejects equality (p = 0.001), but the critical difference grows from 5.73 to 7.50, so none of the leading methods separates from the others. The dependence is real: the mean Spearman correlation of method rankings between responses of the same dataset is 0.81 (Tecator) and 0.79 (Cookie), against 0.29 between datasets.

h63K-6. In simulation, does the relative advantage persist when an entire mechanism family is excluded from pretraining?

We tested this in two ways. First, we removed three groups of families from the prior and retrained from scratch (Appendix J). The excluded mechanisms are still learned in context at 1.5–2 times the in-prior error (single-index 0.099 against 0.050; interval interaction 0.092 against 0.050; quadratic 0.024 against 0.016), far below linear methods (about 1.0). The relative advantage, however, does not fully persist: at this 100k-step budget the model then falls behind TabPFN on FPCA scores on two of the three mechanisms (0.099 against 0.058; 0.092 against 0.055), where its in-prior lead was already small. Second, we evaluated the released model on seven mechanisms that no family of the prior generates (SIM1 table, Section 5.1): it keeps the best average rank (1.86 among 17 methods; TabPFN 3.14) and leads TabPFN on FPCA scores on all seven, but the correctly specified FGAM beats it clearly on a non-separable functional additive model (0.033 against 0.198), and linear methods beat it where the Bayes predictor is linear. We report both outcomes and limit our claims accordingly.

h63K-7. [...] without directly measuring whether predictive means and variances for the same support/query functions remain consistent across grids. [...] Can the authors directly report changes in predictive means and variances for identical support/query functions evaluated on different grids?

We now report this (Appendix K). For four tasks we resampled the same curves to 14 grids between 16 and 1,400 points and measured, for every query, the change of the predictive mean (in predictive standard deviations) and the ratio of predictive standard deviations. When support and queries share a grid, predictions are consistent: for 50 points or more, the median mean shift is 0.08 standard deviations and the predictive standard deviation changes by at most 5%; on 16–32 points the mean shifts by about 0.25, as the curves lose information. When only the query grid changes, consistency is task-dependent and can fail: on Tecator the mean moves by at least 0.37 standard deviations on every other grid, and on coarse query grids shifts reach 0.6–2.1 on three of four tasks. A control on exact subsamples, without interpolation, gives the same pattern, so this is a property of the model. We replaced the invariance claim by this measured statement and a deployment rule (queries are resampled to the support grid, at no cost), and state shared uniform grids as a limitation. A preliminary extension to irregular grids is described in Appendix P.

AuthorsOpenReview ↗

Response to Reviewer h63K (3/3)

h63K-8. [...] the spectral branch requires m ≥ 2K, whereas Growth has only nineteen native observations. Please specify whether interpolation, padding, adaptive truncation, or another procedure is used.

Thank you for catching this: the method section stated a requirement that the implementation does not impose. Nothing is interpolated. With m points, each FNO layer uses the ⌊m/2⌋+1 available modes and the readout the ⌊(m−1)/2⌋ available coefficients, the others being set to zero; for Growth, 10 modes and 9 coefficients. We also measured the alternative: RMSE is 4.84 natively (the reported value) and about 4.91 after interpolation to 64, 100 or 200 points. Section 3.2.1 now states this rule.

h63K-9. [...] pooled coverage can conceal substantial task-specific miscalibration [...]. Can the authors provide per-task coverage, predictive interval width, and Monte Carlo uncertainty?

We agree that a Gaussian head guarantees nothing, and we now report the per-task evidence (Appendix M): coverage at four nominal levels, the 90% interval width and the Monte-Carlo standard error over 30 splits for every task. The pooled figure does not hide offsetting errors. At the nominal 90% level coverage ranges from 88.1% (Weather, ±2.5) to 95.0% (Growth, ±0.9), so no task under-covers by more than its Monte-Carlo error. The 50% level varies more (46% to 70%), which indicates that the Gaussian shape is imperfect on some tasks; the paper says so. On the same splits, TabPFN covers 85.2%, Bayesian ridge on FPCA scores 88.8% and a Gaussian process on FPCA scores 71.6% at the 90% level.

We thank the reviewer again; the revision is substantially more honest about what the method does and does not show, and we hope it addresses the concerns about the architecture and the scope of the claims.

Thank you again for suggesting the encoder comparison. It helped us clarify the contribution of the architecture and sharpen our claims. If you feel these revisions have strengthened the paper, would you consider raising your score?

Reviewer_iBtmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors consider scalar-on-function regression using a prior-data fitted network (PFN) approach. There, a transformer, the architecture made popular by large language models, is trained once on a large number of regression datasets simulated from a hand-designed distribution over tasks (the "prior"). The model takes as an input an "in-context" training set together with a query curve and returns a Gaussian predictive distribution for the query's response, with no retraining or tuning per dataset. The authors present an encoder (a Fourier neural operator with pooling and a Fourier-coefficient readout) for curves densely observed on a common uniform grid, whose output feeds the transformer.

Critical strengths

In-context learning with a large pretrained transformer is now widespread across problems in ML. I have not seen this brought to functional data, and I think it is a very interesting approach to functional regression. The model needs no per-dataset tuning, gives predictive uncertainty, and the prior over functional regression tasks is carefully constructed.

Critical weaknesses

The methodology is not new beyond the functional setting: the PFN recipe is taken essentially as is from TabPFN, which was already an instantiation of what was being done in the broader ML community. There are also existing methods for time series and anomaly detection that do something very similar, for example transformers pretrained only on synthetic data that classify or forecast new series in context, without per-dataset training. These should at least be discussed. Alternatives such as adapting an existing model, transfer learning, or fine-tuning on the target data are not discussed either. The model is also limited to evenly spaced grids shared by all curves, so sparse or irregularly sampled functional data are not covered (which isn't disqualifying).

I also find the "zero-shot" terminology misleading. The model conditions on the full labeled training set; what it avoids is gradient updates and tuning. Unlike forecasting/anomaly detection, where the observed series itself informs the target, in regression the response cannot be inferred without labeled examples, so zero-shot regression in the usual sense is not possible. Terms such as "in-context" or "tuning-free" would be more accurate, including in the title.

Another issue is clarity. For example, the features f_j appear with no explanation of how the FNO maps the curve to them; the layers are never written out, and the notation invites confusion with Fourier coefficients. Relatedly, the framing in terms of a prior over function spaces formalizes the method as approximate Bayesian inference, but this adds little: the posterior-predictive interpretation holds only for an ideally trained network and is never checked (the original PFN paper did check this against exact GP posteriors). Distributions over function spaces are also notoriously delicate (e.g., Gaussian measures of different smoothness are mutually singular). Here the prior over regression relationships is a finite list of 27 families, so it puts no mass on relationships outside them, and behavior outside the prior is never tested.

Questions for authors

See weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer iBtm (1/2)

We thank the reviewer for a thoughtful review and for the positive assessment of the direction.

iBtm-1. The methodology is not new beyond the functional setting: the PFN recipe is taken essentially as is from TabPFN.

We agree that the PFN recipe is not ours; the introduction presents it as prior work. Our contribution is a prior over scalar-on-function regression tasks, an in-context protocol for curves, and an empirical study of what carries the performance. The new experiments sharpen this: an encoder ablation trained from scratch shows that the encoder is replaceable, so the value lies in the prior and the amortized inference; the network approximates the exact posterior predictive on a sub-family where it is known (iBtm-7); and leave-families-out runs and seven new mechanisms measure how far the prior generalizes (iBtm-8).

iBtm-2. There are also existing methods for time series and anomaly detection that do something very similar [...]. These should at least be discussed.

We added them to the related work: ForecastPFN (Dooley et al., NeurIPS 2023) and Mamba4Cast (Bhethanabhotla et al., 2024) are pretrained only on synthetic series and forecast new ones without per-dataset training, and Chronos (Ansari et al., TMLR 2024) mixes real and synthetic series. The paragraph also states the difference the reviewer points out: in forecasting the observed series itself informs the target, whereas in scalar-on-function regression the map from curve to response can only be inferred from labelled examples.

iBtm-3. Alternatives such as adapting an existing model, transfer learning, or fine-tuning on the target data are not discussed either.

The related work now discusses them. Fine-tuning or context optimization can adapt a pretrained PFN to a target dataset (e.g., TuneTables, Feuer et al., NeurIPS 2024), at the price of the gradient steps and model selection that in-context inference avoids; with 20–160 training curves, as in our benchmarks, this model-selection step is exactly what is hardest.

iBtm-4. The model is also limited to evenly spaced grids shared by all curves, so sparse or irregularly sampled functional data are not covered.

We agree and state this limitation in the method and discussion. We also measured the grid behaviour directly (our reply to h63K-7): predictions are consistent when support and queries share a grid, less so when only the query grid changes. Appendix P describes a preliminary extension that processes irregular and curve-specific grids with quadrature-weighted non-uniform Fourier layers; in a first simulation it stays far ahead of the linear baselines and of FDNN on irregular and asynchronous designs, but interpolating to a common grid with the same model is still slightly better, so we present it as preliminary and leave a full treatment to future work.

AuthorsOpenReview ↗

Response to Reviewer iBtm (2/2)

iBtm-5. I also find the "zero-shot" terminology misleading. [...] Terms such as "in-context" or "tuning-free" would be more accurate, including in the title.

We agree. The title is now "Curves in Context: Tuning-Free Regression of Functional Data", and "in-context" replaces "zero-shot" throughout.

iBtm-6. [...] the features f_j appear with no explanation of how the FNO maps the curve to them; the layers are never written out, and the notation invites confusion with Fourier coefficients.

Thank you. A new appendix (Appendix A) writes out the FNO trunk: the lifting, the truncated Fourier analysis and synthesis operators, the per-frequency channel mixing, the pointwise affine term and activation, and the residual channel MLP. It defines f_j as the learned feature vector at grid point t_j and states explicitly that it is not a Fourier coefficient.

iBtm-7. [...] the posterior-predictive interpretation holds only for an ideally trained network and is never checked (the original PFN paper did check this against exact GP posteriors).

We now check it (Appendix L). On a sub-family of our prior where the posterior predictive is available in closed form (Gaussian-process curves, a linear functional with a Gaussian-process coefficient function, Gaussian noise), we compared FunPFN's predictive with the exact one for supports of 8 to 512 curves. For smooth curves, the gap between predictive means falls from 0.23 to 0.02 response standard deviations, the ratio of predictive standard deviations goes from 1.54 to 1.00, and the NLL comes within 0.01 nat of the exact posterior's; rough curves show the same convergence (0.44 to 0.03; 1.72 to 1.02). With few curves the network is over-dispersed rather than over-confident, so its 90% coverage stays between 89% and 95%. The results now report this check, and the paper presents the network as an approximation of the posterior predictive, verified where the exact answer is known.

iBtm-8. [...] the prior over regression relationships is a finite list of 27 families, so it puts no mass on relationships outside them, and behavior outside the prior is never tested.

We agree with the formal point: the prior is a mixture over 27 families, not a measure with full support, and the paper now says so. Behaviour outside the prior is now tested in two ways. On seven mechanisms that no family generates, the released model keeps the best average rank (1.86 among 17 methods) and is first on five, but correctly specified models beat it where they apply (FGAM on a non-separable functional additive model; linear methods when the Bayes predictor is linear). When entire groups of families are removed from pretraining, the excluded mechanisms are still learned at 1.5–2 times the in-prior error, and at our ablation budget the model then loses its lead over the strongest baseline on two of three such mechanisms. We report both as the measured scope of the approach.

We thank the reviewer again for comments that made the paper clearer and its claims more precise.

Thank you again for your helpful comments. They encouraged us to add experiments and clarify the limits of our method. If you think these revisions have improved the paper, would you consider raising your score?

Dear Reviewer iBtm,

As the discussion period closes tomorrow, we would like to point briefly to the revisions that answer your main questions, in case they are useful for your final assessment:

  • "Zero-shot" is replaced by "in-context" throughout, including the title.
  • The FNO trunk is written out and fjf_j is defined (Appendix A).
  • The posterior-predictive interpretation is checked against the exact posterior on a closed-form sub-family of the prior (Appendix L).
  • Related work now covers synthetic-prior time-series models and fine-tuning of PFNs, and behaviour outside the prior is tested on seven new mechanisms.

We would be glad to clarify any remaining point before the deadline.

Reviewer_RnW3OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1(−1)
Read review
Paper summary

The paper proposes FunPFN, a zero-shot approach to scalar-on-function regression based on prior-data fitted networks. The model combines an FNO-based encoder with a Transformer and produces Gaussian predictions to capture uncertainty. The authors test FunPFN on ten synthetic scenarios and twelve real-world datasets, comparing it against traditional functional regression methods, deep learning models, and TabPFN. Overall, FunPFN performs well, especially when training data is limited, and is considerably faster at inference than TabPFN.

Critical strengths
  1. The paper addresses a relevant problem in functional data analysis. Extending prior-data fitted networks to functional inputs is a useful direction, especially for small datasets where conventional methods often require repeated fitting and hyperparameter tuning. The synthetic prior covers a wide range of functional relationships, and the overall framework is reasonably well motivated.

  2. The authors examine several aspects of the method, including predictive accuracy, sample efficiency, resolution robustness, and uncertainty estimation. The reported inference speed is a practical advantage, and the appendix provides useful implementation details. The paper also acknowledges some less favorable results, such as TabPFN’s stronger average performance on real-world datasets.

Critical weaknesses
  1. One of my concern is how much the results tell us about generalisation. The authors acknowledge that the simulation scenarios largely fall within the mechanisms covered during pretraining. While this is reasonable for evaluating in-context inference, it leaves open how FunPFN performs on mechanisms not encountered during pretraining. The nonlinear experiments also lack comparisons with dedicated nonlinear functional regression methods. On real data, FunPFN ranks second, but its average rank (4.63) is very close to Ridge (4.67), while TabPFN performs better on seven of the twelve tasks.

  2. The related work gives limited attention to recent developments in adaptive functional representations and nonlinear functional regression. Comparisons with these methods would help clarify the contribution of the proposed architecture. Given that functional representation is an important part of the proposed method, I think these studies deserve more attention. There are a few experimental details I struggled with. In E3, RMSE is averaged across datasets with quite different response scales, and the datasets included change with the training size. I am therefore not sure how much can be concluded from the pooled learning curves. I was also surprised by the ablation results. Removing the spectral readout makes little difference after re-adaptation, despite its role in the proposed architecture. There are also no ablations of the synthetic prior, which seems to be one of the main contributions.

The paper is generally understandable, but I found parts of it rather difficult to read. The frequent use of long sentences, em dashes, and parenthetical explanations makes the paper harder to follow. I had to reread to work out what was already known and what the authors were proposing.

Questions for authors
  1. Please address the weaknesses raised above.

  2. Section 5.2 mentions that spectroscopy guided the prior’s design. Were any E1 real-world datasets used to refine the prior, including the spectroscopy-like generator or family sampling weights, or to select checkpoints? Or was validation based entirely on held-out synthetic tasks?

  3. Could the authors clarify the distance metrics and bandwidth selection used for Nadaraya–Watson and functional k-NN, as well as the tuning procedures for FDNN/FBNN? I would also be interested in how FunPFN compares with simpler alternatives such as FPCA + TabPFN and appropriately specified nonlinear functional regression methods.

  4. Could the authors report per-task or normalised RMSE for E3? Since the datasets have different response scales and the set of evaluated tasks changes with training size, the current pooled results are difficult to interpret.

  5. How is Growth (m=19m=19) handled when the spectral readout requires m≥64m \geq 64? Is interpolation used? Also, does n=800n=800 in SIM2 refer to the support-set size, given that pretraining uses at most S=512S=512?

  6. Since the synthetic prior is a central contribution, could the authors provide prior-level ablations to clarify which design choices matter most? It would also be useful to compare predictive uncertainty with TabPFN in E4, as TabPFN also provides predictive distributions.

6 comments
AuthorsOpenReview ↗

Response to Reviewer RnW3 (1/3)

We thank the reviewer for a detailed review and for questions that each led to a new experiment.

RnW3-1. One of my concern is how much the results tell us about generalisation. The authors acknowledge that the simulation scenarios largely fall within the mechanisms covered during pretraining. While this is reasonable for evaluating in-context inference, it leaves open how FunPFN performs on mechanisms not encountered during pretraining.

We added seven scenarios whose mechanism is absent from the prior, in three groups: latent models invariant to a nuisance (additive and multiplicative EMSC scatter, elastic phase variation), functional additive models ∫F(X(t),t)dt (separable and non-separable), and superpositions of families that the prior contains only separately (Section 4, Appendix C; 30 replications). FunPFN degrades but keeps the best average rank among 17 methods (1.86; TabPFN 3.14, FBNN 5.14, FGAM 8.29; Friedman p = 0.001), its mean excess risk rising from 0.010 within the prior to 0.063. It is first on five of the seven, second on the non-separable FAM, far behind the correctly specified FGAM (0.198 against 0.033), and sixth on additive EMSC, whose Bayes predictor is linear (0.0025 behind PLS). Where the best linear predictor is not already optimal, its risk is 7–29% of the linear floor, while tuned linear pipelines stay at or above it. Separately, leave-families-out pretraining (Appendix J) shows that removed mechanisms are still learned at 1.5–2 times the in-prior error.

RnW3-2. The nonlinear experiments also lack comparisons with dedicated nonlinear functional regression methods.

We added FAM (Müller and Yao, 2008), FGAM (McLean et al., 2014) and AdaFNN (Yao et al., 2021), tuned by inner cross-validation (Appendix D). In simulation, FGAM is the strongest of them: it wins the non-separable FAM, as a correctly specified model should, but its mean rank over the 17 scenarios is 8.06, against 1.76 for FunPFN; FAM and AdaFNN rank around 10. On the twelve real tasks (Appendix N, same 30 splits), FunPFN is more accurate than FGAM on 10 tasks (FGAM leads on DTI and ties on Cookie-sucrose) and than FAM and AdaFNN on 11; in geometric mean FGAM's RMSE is 1.20 times FunPFN's, FAM's 2.40 times and AdaFNN's 2.52 times.

RnW3-3. On real data, FunPFN ranks second, but its average rank (4.63) is very close to Ridge (4.67), while TabPFN performs better on seven of the twelve tasks.

We agree, and the situation is less favourable than the task-level ranks suggest. Ranked by the seven source datasets, to account for responses that share curves, FunPFN is third (4.64), behind TabPFN (3.71) and Ridge (4.29), and TabPFN on FPCA scores is more accurate still. The revised paper drops the second-place claim and states a statistical tie, together with coverage close to nominal on every task and a cost about 240 times lower than TabPFN's.

RnW3-4. The related work gives limited attention to recent developments in adaptive functional representations and nonlinear functional regression.

The related work now covers adaptive representations (AdaFNN) and nonlinear functional regression (functional additive and generalized additive models, index models), and all three methods are baselines. The encoder ablation makes the link explicit: an in-context FPCA encoder, i.e., a data-adaptive representation computed on the support, is more accurate than the FNO on real spectra.

AuthorsOpenReview ↗

Response to Reviewer RnW3 (2/3)

RnW3-5. In E3, RMSE is averaged across datasets with quite different response scales, and the datasets included change with the training size. Could the authors report per-task or normalised RMSE for E3?

The reviewer is right: the pooled curve mixes the effect of the training size with that of which tasks remain. We now report E3 per task, with RMSE normalized by the test-response standard deviation (Appendix N). The per-task view is narrower. At 20 training curves, FunPFN is the most accurate of FunPFN, TabPFN on FPCA scores and Ridge on six of twelve tasks; at 40, on three of the eleven tasks large enough; on the three Tecator responses, the only tasks large enough to go further, TabPFN on FPCA scores leads from 80 curves on. The simulations tell the same story (SIM2): FunPFN is the most accurate method at n = 50 in five of six scenarios, but at n = 800 it remains first on only one, the others being within a factor 1.6 on the linear designs and far behind FGAM on the non-separable functional additive model; the 512-curve context is one reason (our reply to RnW3-12). The text of E3 now states this regime.

RnW3-6. Removing the spectral readout makes little difference after re-adaptation, despite its role in the proposed architecture.

So were we surprised, and it led to the from-scratch encoder study (our reply to h63K-1). It agrees with the reviewer's reading: trained under the same prior and budget, an in-context FPCA encoder is about 15% more accurate than the FNO encoder on real data, on both seeds, so the spectral readout does not drive performance. We removed the claim that it is necessary.

RnW3-7. There are also no ablations of the synthetic prior [...]. Could the authors provide prior-level ablations to clarify which design choices matter most?

We added five prior ablations, each a full pretraining run from scratch (Appendix J). The three leave-families-out runs show that the families matter for the mechanisms they generate: removing the index families doubles the error on the single-index scenario, and removing the quadratic families multiplies it by 1.8 on interval interaction and by 1.5 on the quadratic scenario. Removing the score and additive families changes little, which suggests that other families generate similar responses. Two runs alter the prior's construction: a frozen bank without fresh tasks, and no input-noise augmentation; each changes real-data RMSE by 2% or less. In short, the family composition matters for the mechanisms the model can learn, while construction details matter little at this budget.

RnW3-8. The frequent use of long sentences, em dashes, and parenthetical explanations makes the paper harder to follow.

Thank you; all three reviewers rated the presentation 2, and we take it seriously. We rewrote every part of the paper that changed (abstract, contribution list, encoder section, results paragraphs, discussion and conclusion) with shorter sentences and fewer asides, and we will extend this editorial pass to the whole paper for the final version.

RnW3-9. Were any E1 real-world datasets used to refine the prior, including the spectroscopy-like generator or family sampling weights, or to select checkpoints?

No. No real dataset was used to design the prior, to set family weights, to tune hyperparameters or to select a checkpoint; no comparison of checkpoints was made. The released checkpoint is the last one of a run of 300k optimizer steps, performed in three stages of 100k steps, and development relied on held-out synthetic tasks only. The spectroscopy-like generator follows generic descriptions of absorption spectra (bands on a baseline), not the E1 curves. Section 3.4 now states this.

AuthorsOpenReview ↗

Response to Reviewer RnW3 (3/3)

RnW3-10. Could the authors clarify the distance metrics and bandwidth selection used for Nadaraya–Watson and functional k-NN, as well as the tuning procedures for FDNN/FBNN?

Both use the L² distance between curves. Functional k-NN selects k ∈ {1, 3, 5, 7, 10, 15, 20} and uniform or distance weights by inner 5-fold cross-validation; Nadaraya–Watson uses a Gaussian kernel and selects the bandwidth among 8 values linearly spaced in [0.2, 5.0] by inner 5-fold cross-validation. FDNN and FBNN (Rao and Reimherr, 2023) use one or two functional hidden layers of width 8, selected on a 20% hold-out, Adam (learning rate 5×10⁻³, weight decay 10⁻⁴), at most 600 epochs with early stopping (patience 50); FBNN expands its weights in 7 Fourier functions. Appendix D now gives these details, and those of FAM, FGAM and AdaFNN.

RnW3-11. I would also be interested in how FunPFN compares with simpler alternatives such as FPCA + TabPFN.

We added this baseline (20 FPCA scores fitted on each training split, then TabPFN) to the real data and to all 17 simulation scenarios. On real data it is the most accurate method, ahead of FunPFN on eight of twelve tasks (one of them, DTI, a near tie) and by a wide margin on Tecator (fat: 0.78 against 1.27), while FunPFN is better on Cookie-fat, Gasoline, Growth and Weather. In simulation the order reverses: FunPFN has a lower excess risk in all 17 scenarios, significantly in 14 (paired Wilcoxon over 30 replications, Bonferroni-corrected), including all seven out-of-prior scenarios, where it is 20–47% better. The gap is largest when the response is nonlinear in the scores (interval interaction 0.017 against 0.055). We report both results; together with the encoder ablation, they show that an FPCA representation is very effective on real spectra, and that our prior pays off on the mechanisms it covers and on nearby ones.

RnW3-12. How is Growth (m = 19) handled when the spectral readout requires m ≥ 2K = 64? Also, does n = 800 in SIM2 refer to the support-set size, given that pretraining uses at most S = 512?

No interpolation is used: with m points the readout keeps the ⌊(m−1)/2⌋ available Fourier coefficients and the trunk the ⌊m/2⌋+1 available modes (9 and 10 for Growth), the others being set to zero. Interpolating instead gives a slightly worse RMSE (4.91 against 4.84). In SIM2, n is the support size. Above the 512 curves seen in pretraining, the model is applied to 8 random sub-supports of 512 curves and the Gaussian predictives are combined by moment matching (mean of the means; mean of the variances plus variance of the means). This rule was in the code but missing from the paper, and Section 3.4 now states it; it concerns only n = 800, since every real-data support is smaller (at most 161 curves). It is also one reason why the specialists catch up at large n.

RnW3-13. It would also be useful to compare predictive uncertainty with TabPFN in E4.

We added TabPFN to E4, together with a Gaussian process and Bayesian ridge on FPCA scores, on the same tasks and splits (Appendix M). Coverage separates the methods most clearly: at the nominal 90% level FunPFN covers 91.4%, Bayesian ridge 88.8%, TabPFN 85.2% (from its own quantiles) and the Gaussian process 71.6%. On NLL the comparison is closer than the pooled means suggest: FunPFN has the lowest mean NLL (1.48), but this is driven by one task (Aemet) on which TabPFN and the GP fail; without it FunPFN and TabPFN are close (1.35 and 1.41), and TabPFN's median NLL over splits is lower (1.28 against 1.39). FunPFN is also about 95 times faster than TabPFN on the same GPU (0.017 s against 1.65 s per split).

We thank the reviewer again for questions that made the paper more precise.

Thank you again for your thoughtful comments, which helped us improve the manuscript. If you feel that the revisions have improved the overall quality of the paper, would you consider raising your score?

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank the three reviewers for careful and constructive reviews. They converge on four questions: whether the architecture or the prior carries the method, how the model behaves outside its prior, how reliable its uncertainty is, and how strong the real-data case really is. We ran new experiments for each, all with the checkpoint and code evaluated in the submission, and revised the paper accordingly. Before anything else, we checked that the released checkpoint reproduces the submitted results: it matches the real-data table on all twelve tasks within 0.02 RMSE, and the simulation script reproduces the ten published SIM1 rows to the printed precision.

  1. Encoder ablation trained from scratch (h63K, RnW3; new Appendix I). Six encoders were trained under the same prior, objective, transformer and budget. On real data an in-context FPCA encoder is about 15% more accurate than the FNO encoder on both seeds; on 17 simulated mechanisms no encoder dominates, and FPCA fails when curves differ by phase, where the FNO does not. The encoder is a replaceable component; the method's value lies in the prior and the amortized inference, and the paper now says so.
  2. Behaviour outside the prior (all three reviewers; Section 5.1, Appendices C and J). Seven scenarios whose mechanism is absent from the prior, 30 replications: FunPFN keeps the best average rank (1.86 among 17 methods) and is first on five, while correctly specified models win where they apply (FGAM on a non-separable functional additive model). Leave-families-out pretraining shows excluded mechanisms are still learned at 1.5–2 times the in-prior error.
  3. New baselines (RnW3). FAM, FGAM and AdaFNN, and TabPFN on 20 FPCA scores. The latter is the most accurate method on real data (better than FunPFN on 8 of 12 tasks) but worse than FunPFN in all 17 simulated scenarios, significantly in 14.
  4. Exact posterior check (iBtm; Appendix L). On a Gaussian-process sub-family of the prior, FunPFN's predictive converges to the exact posterior predictive as the support grows.
  5. Calibration per task and against TabPFN (h63K, RnW3; Appendix M). Coverage at the nominal 90% level lies between 88% and 95% on every task; TabPFN covers 85%.
  6. Real-data claims (h63K, RnW3). Ranked by source dataset, FunPFN is third; we removed "second on real data" from the abstract and introduction and state the accuracy case as a statistical tie.
  7. Grids, Growth and context size (h63K, RnW3; Appendix K, Section 3). We report prediction changes for identical functions across grids, state how 19-point curves are processed (no interpolation), and state the 512-curve context and its ensembling rule.
  8. Terminology (iBtm). The title is now "Curves in Context: Tuning-Free Regression of Functional Data", and "in-context" replaces "zero-shot" throughout.

The revised paper therefore makes a narrower claim than the submission, which we believe is the stronger one: a synthetic prior over functional regression tasks, combined with amortized in-context inference, gives a tuning-free regressor that is accurate within and near its prior, has reliable uncertainty, and runs two orders of magnitude faster than TabPFN; it does not dominate on real data, and the encoder is not the source of its accuracy. Appendix letters refer to the revised PDF.

We hope the revised manuscript addresses the reviewers’ concerns and meets the standards for publication at ICLR.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.