- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces FunPFN, an in-context model pretrained on synthetic tasks for scalar-on-function regression. It combines an FNO curve encoder, attention pooling, and support-standardized Fourier features with a Transformer that uses labeled support examples to predict a Gaussian response distribution for each query curve. Deployment requires neither gradient updates nor task-specific hyperparameter tuning. Evaluation covers ten synthetic scenarios and twelve prediction tasks from seven public datasets, together with analyses of resolution changes, small-sample performance, and predictive calibration. The main findings are strong simulation rankings, competitive real-data performance, and lower deployment latency than the reported TabPFN configuration.
Critical strengths
-
The application objective is clear and practically relevant. Reducing repeated representation selection, fitting, and tuning is useful for small-sample functional regression. Using labeled examples as context to produce predictions and uncertainty in one forward pass offers a convenient deployment interface.
-
Several important qualifications are reported transparently. The authors acknowledge the overlap between simulation mechanisms and the pretraining prior, the absence of significant real-data superiority over TabPFN, and performance saturation as sample size increases. These disclosures help establish the actual scope of the results.
Critical weaknesses
-
The experiments do not establish the independent value of the functional architecture, and the central contribution is confounded with prior matching. Every simulation target form belongs to a pretraining mechanism family, so the simulation advantage primarily supports amortized inference within the chosen prior. The TabPFN comparison changes the architecture, training prior, and pretraining configuration simultaneously, leaving the source of improvement unresolved. The appendix ablations do not close this gap: the authors explicitly acknowledge that removing the FNO and re-adapting the resulting network does not represent a spectral-score Transformer trained from scratch. Meanwhile, the Fourier readout, its standardization, and mean pooling improve RMSE by only approximately 1.0%, 1.6%, and 0.2%, respectively, after re-adaptation. Without simpler encoders trained under the same prior and comparable budgets, the necessity of the proposed architecture remains unclear. Real-data results do not compensate for this missing evidence: FunPFN's average rank is 4.63, close to Ridge at 4.67 and behind TabPFN at 3.29. Failure to detect a significant difference does not establish equivalence. Moreover, the twelve tasks come from seven datasets, with multiple responses sharing the same curves; sensitivity of the aggregate rankings and significance conclusions to this task dependence is insufficiently assessed.
-
The claims about function-space applicability and predictive calibration exceed the evidence provided. Discretization invariance is motivated mainly through convergence under grid refinement, rather than finite-grid prediction consistency or error control. The resolution experiment aggregates RMSE after resampling, without directly measuring whether predictive means and variances for the same support/query functions remain consistent across grids. The model is still restricted to shared uniform grids and does not address sparse, irregular, or subject-specific observation times, limiting the present scope of its positioning as a functional foundation model. There is also an implementation ambiguity: the spectral branch requires , whereas Growth has only nineteen native observations. For uncertainty, pooled coverage can conceal substantial task-specific miscalibration, and matching conditional means and variances with a Gaussian head does not guarantee interval coverage. These issues affect the reliability of the current method; the possibility of future extensions does not resolve them.
Questions for authors
-
Can the authors compare the FNO encoder with Fourier-score, FPCA-score, or other simple encoders trained from scratch under the same synthetic prior, objective, and comparable pretraining budgets? This comparison is essential to establish the necessity of the proposed architecture, rather than an extension to a new task.
-
Do the rankings and significance conclusions persist when results are aggregated by the seven source datasets and sensitivity to dependence among responses sharing the same curves is assessed? In simulation, does the relative advantage persist when an entire mechanism family is excluded from pretraining?
-
How are the nineteen-point Growth curves processed by a spectral branch requiring ? Please specify whether interpolation, padding, adaptive truncation, or another procedure is used, and reconcile this with the statements about avoiding interpolation and grid dependence. Can the authors directly report changes in predictive means and variances for identical support/query functions evaluated on different grids?
-
Can the authors provide per-task coverage, predictive interval width, and Monte Carlo uncertainty? Does the favorable pooled coverage conceal undercoverage on some tasks offset by overcoverage on others? These results would materially affect the assessment of the current uncertainty-quantification claims.
6 comments
Response to Reviewer h63K (2/3)
h63K-4. FunPFN's average rank is 4.63, close to Ridge at 4.67 and behind TabPFN at 3.29. Failure to detect a significant difference does not establish equivalence.
We agree with both statements. The revised paper no longer calls FunPFN second-best on real data; it states a statistical tie with the most accurate methods, and adds that TabPFN on FPCA scores is more accurate still. The real-data case is now narrower and measured: coverage close to nominal on every task where TabPFN under-covers (h63K-9), a 0.025 s cost per split, about 240 times less than TabPFN, and an advantage concentrated at small training sizes (at 20 curves, the most accurate of FunPFN, TabPFN on FPCA scores and Ridge on half the tasks).
h63K-5. [...] sensitivity of the aggregate rankings and significance conclusions to this task dependence is insufficiently assessed. Do the rankings and significance conclusions persist when results are aggregated by the seven source datasets?
They do not fully persist, and we now report both aggregations (Appendix N). Averaging ranks within each dataset before the Friedman test, TabPFN is first (3.71), Ridge second (4.29) and FunPFN third (4.64). The test still rejects equality (p = 0.001), but the critical difference grows from 5.73 to 7.50, so none of the leading methods separates from the others. The dependence is real: the mean Spearman correlation of method rankings between responses of the same dataset is 0.81 (Tecator) and 0.79 (Cookie), against 0.29 between datasets.
h63K-6. In simulation, does the relative advantage persist when an entire mechanism family is excluded from pretraining?
We tested this in two ways. First, we removed three groups of families from the prior and retrained from scratch (Appendix J). The excluded mechanisms are still learned in context at 1.5–2 times the in-prior error (single-index 0.099 against 0.050; interval interaction 0.092 against 0.050; quadratic 0.024 against 0.016), far below linear methods (about 1.0). The relative advantage, however, does not fully persist: at this 100k-step budget the model then falls behind TabPFN on FPCA scores on two of the three mechanisms (0.099 against 0.058; 0.092 against 0.055), where its in-prior lead was already small. Second, we evaluated the released model on seven mechanisms that no family of the prior generates (SIM1 table, Section 5.1): it keeps the best average rank (1.86 among 17 methods; TabPFN 3.14) and leads TabPFN on FPCA scores on all seven, but the correctly specified FGAM beats it clearly on a non-separable functional additive model (0.033 against 0.198), and linear methods beat it where the Bayes predictor is linear. We report both outcomes and limit our claims accordingly.
h63K-7. [...] without directly measuring whether predictive means and variances for the same support/query functions remain consistent across grids. [...] Can the authors directly report changes in predictive means and variances for identical support/query functions evaluated on different grids?
We now report this (Appendix K). For four tasks we resampled the same curves to 14 grids between 16 and 1,400 points and measured, for every query, the change of the predictive mean (in predictive standard deviations) and the ratio of predictive standard deviations. When support and queries share a grid, predictions are consistent: for 50 points or more, the median mean shift is 0.08 standard deviations and the predictive standard deviation changes by at most 5%; on 16–32 points the mean shifts by about 0.25, as the curves lose information. When only the query grid changes, consistency is task-dependent and can fail: on Tecator the mean moves by at least 0.37 standard deviations on every other grid, and on coarse query grids shifts reach 0.6–2.1 on three of four tasks. A control on exact subsamples, without interpolation, gives the same pattern, so this is a property of the model. We replaced the invariance claim by this measured statement and a deployment rule (queries are resampled to the support grid, at no cost), and state shared uniform grids as a limitation. A preliminary extension to irregular grids is described in Appendix P.
Response to Reviewer h63K (3/3)
h63K-8. [...] the spectral branch requires m ≥ 2K, whereas Growth has only nineteen native observations. Please specify whether interpolation, padding, adaptive truncation, or another procedure is used.
Thank you for catching this: the method section stated a requirement that the implementation does not impose. Nothing is interpolated. With m points, each FNO layer uses the ⌊m/2⌋+1 available modes and the readout the ⌊(m−1)/2⌋ available coefficients, the others being set to zero; for Growth, 10 modes and 9 coefficients. We also measured the alternative: RMSE is 4.84 natively (the reported value) and about 4.91 after interpolation to 64, 100 or 200 points. Section 3.2.1 now states this rule.
h63K-9. [...] pooled coverage can conceal substantial task-specific miscalibration [...]. Can the authors provide per-task coverage, predictive interval width, and Monte Carlo uncertainty?
We agree that a Gaussian head guarantees nothing, and we now report the per-task evidence (Appendix M): coverage at four nominal levels, the 90% interval width and the Monte-Carlo standard error over 30 splits for every task. The pooled figure does not hide offsetting errors. At the nominal 90% level coverage ranges from 88.1% (Weather, ±2.5) to 95.0% (Growth, ±0.9), so no task under-covers by more than its Monte-Carlo error. The 50% level varies more (46% to 70%), which indicates that the Gaussian shape is imperfect on some tasks; the paper says so. On the same splits, TabPFN covers 85.2%, Bayesian ridge on FPCA scores 88.8% and a Gaussian process on FPCA scores 71.6% at the 90% level.
We thank the reviewer again; the revision is substantially more honest about what the method does and does not show, and we hope it addresses the concerns about the architecture and the scope of the claims.
Thank you again for suggesting the encoder comparison. It helped us clarify the contribution of the architecture and sharpen our claims. If you feel these revisions have strengthened the paper, would you consider raising your score?
Response to Reviewer h63K (1/3)
We thank the reviewer for a demanding review. Its central question led to the most informative experiment of the revision.
We ran exactly this comparison (new Appendix I). Five alternative encoders replace the FNO encoder, each trained from scratch with everything else fixed: same prior, Gaussian NLL, 12-layer transformer, per-step budgets, schedule and 100k steps; the FNO encoder was retrained under the same budget as control, with two seeds. Geometric-mean RMSE ratio to the control on the twelve real tasks (two values: two seeds):
The answer is not the one the submission argued. At equal budget the FNO is not what makes the method accurate on real data: an FPCA encoder computed in context does better, mostly on near-infrared spectra. On the 17 simulated scenarios the picture is balanced. FPCA is better where the response depends on principal-component scores (score-quadratic 0.032–0.043 against 0.063), the FNO on the linear, quadratic and single-index designs, and FPCA fails when curves differ by a random time warp (0.63–0.69 against 0.21–0.22). The FNO is also more stable than FPCA across grids, and it is not faster: per training step it is 4–7 times slower. The Fourier-only arm had twice the validation loss of the others, so we report it as a lower reference only. Finally, the released 300k-step model is better than every 100k-step arm on 14 of 17 scenarios, so the training budget matters more than the encoder. We agree with the reviewer's reading: the contribution is amortized inference under a carefully built functional prior, and the encoder is a modular part of it with measured trade-offs. The method section and the contribution list now say this, and the FNO is presented as our default encoder with the reasons the ablation supports.
Two experiments separate these factors. The encoder ablation holds prior and recipe fixed and changes only the architecture; it includes a fixed-grid encoder that gives our transformer a TabPFN-style input, which is 10% worse than the FNO, so our prior does not make any input work. Conversely, TabPFN on 20 FPCA scores keeps TabPFN's prior and recipe and gives it a functional representation. On real data it is the most accurate method, ahead of FunPFN on 8 of 12 tasks; on all 17 simulated scenarios it is behind FunPFN, significantly on 14 (paired Wilcoxon, Bonferroni). An FPCA representation helps both TabPFN and our transformer on real spectra, while our prior gives the advantage on the mechanisms it covers and on nearby ones.
We agree, and the from-scratch study confirms the reviewer's reading: the spectral readout is not necessary when a data-adaptive basis is available. We removed the statement that motivated it as necessary, and the lesion study is now labelled as such and placed after the from-scratch ablation.