- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
BrainPFN trains a prior-data fitted network for brain–behaviour prediction using synthetic regression tasks defined over real, unlabeled functional-connectivity data. Connectomes are transformed into a feature space, kernel functions generate synthetic targets, and signal calibration controls task difficulty. A transformer then predicts new phenotypes from labeled context examples without task-specific fine-tuning. The paper evaluates transfer across cohorts and context sizes, compares with regression and pretrained models, and studies task-generation choices. Its main practical benefit is in small-context prediction; larger-context results also show the continued competitiveness of ridge regression.
Critical strengths
-
The task generator addresses an important obstacle to pretraining across neuroimaging datasets. Using real connectomes with synthetic targets avoids requiring a common set of behavioral labels while retaining empirical input structure. The kernel family, coefficient distribution, and task difficulty define a concrete reusable learning scheme. This is a substantive methodological contribution independent of whether the prior has a direct biological interpretation.
-
The evaluation addresses the intended small-context use case. Testing across independent cohorts and a range of context sizes assesses whether a frozen model transfers to new prediction problems. The ablations of kernel choice, coefficient sampling, and task difficulty examine consequential parts of the proposed generator. The concentration of benefits at small context sizes gives the contribution a useful, identifiable scope.
Critical weaknesses
-
The spectral explanation does not clearly distinguish the proposed prior from the stated alternative. For a fixed positive-definite kernel matrix , Equation 6 gives the raw context signal covariance proportional to . If the no-RKHS alternative samples coefficients isotropically, its raw signal covariance is instead proportional to , which concentrates relative variance even more strongly in large-eigenvalue directions. Thus the argument about favoring smooth directions does not establish that this alternative produces rougher targets. This calculation concerns the signal before task calibration and does not invalidate Equation 6. However, the actual alternative sampling procedure and subsequent calibration need to be specified clearly enough to explain what the ablation demonstrates about the chosen prior.
-
The claimed exact GP counterpart is not established for the calibrated generator. Section 3.2 standardizes each sampled function using a separate calibration block. Although that block is disjoint, its scale still depends on the sampled function, so this transformation generally changes the original Gaussian function distribution. It defines a valid implicit prior, and the log-loss principle in Equations 1–4 remains applicable. The issue is the stronger statement in Appendix C.2 that the standard GP is an exact analytical counterpart of the training tasks. That correspondence needs justification under the actual calibration, rather than only for an uncalibrated Gaussian construction. The description of a linear regression head in Section 4.2 also leaves unclear which predictive density or posterior summary is represented. These distinctions affect the interpretation of the Bayesian-computation claim and GP comparison.
Questions for authors
-
What property of the coefficient prior explains the no-RKHS ablation? Section 3.2 motivates Equation 6 by favoring high-eigenvalue, smooth directions. For a fixed positive-definite kernel matrix and the raw context signal , Equation 6 gives . If the isotropic alternative instead samples , its signal covariance is , placing relatively more variance in the large-eigenvalue directions. Thus this spectral argument alone does not establish that the alternative produces rougher targets. What distribution is actually used in the no-RKHS ablation, and do both samplers undergo the same function-scale and task-difficulty calibration? Please explain which difference between the resulting function priors the ablation isolates. In particular, distinguish evidence that Equation 6 improves prediction from evidence for the proposed smoothness explanation; the covariance calculation above applies before function-dependent calibration.
-
What posterior quantity does BrainPFN approximate, and is the GP baseline exact for the calibrated generator? Section 3.2 divides each sampled function by a scale estimated on a third subject block. The subjects are disjoint, but the scale still depends on the sampled coefficients. As described, this generally changes the Gaussian function distribution, even at fixed kernel hyperparameters. Appendix C.2 nevertheless identifies the task posterior with a standard GP posterior and its hyperparameter weights with GP marginal likelihoods. Please show how the function-dependent scale is accounted for in this correspondence, or qualify the GP as a related analytical baseline. If the intended equivalence concerns only the posterior mean of the uncalibrated finite kernel expansion, rather than the full predictive distribution, please state that scope and explain whether it survives calibration. Finally, does the linear regression head in Section 4.2 parameterize the predictive density in Equation 4 or output only a point estimate? Please specify the output, implemented training loss, and posterior quantity being approximated so that the Bayesian interpretation can be assessed against the actual generator.
4 comments
2. GP counterpart for the calibrated generator:
We agree. The rescaling factor depends on the sampled function, so the calibrated tasks are not Gaussian and the standard GP is not their exact posterior. The calibrated generator still defines a valid implicit prior, and the log-loss principle of Equations 1 to 4 applies to it, so the Bayesian interpretation of BrainPFN holds with respect to that prior. The revised Appendix C.2 presents the GP as a matched analytical baseline, without the claim of exactness.
Output, loss and target. The linear head returns one point estimate per query subject and is trained with the squared error, which is Equation 4 with a fixed-variance Gaussian. BrainPFN is therefore trained to approximate the posterior predictive mean under the calibrated prior. Section 4.2 and Appendix B.1 will state this.
Does the comparison survive calibration? The exact posterior mean of the calibrated tasks has no closed form and would need Monte Carlo integration for every prediction. We therefore compare with the closest tractable model, a GP that also models the calibration effect: the signal is stronger on the context than on new subjects, by a factor computed from unlabelled features ("GP, training signal model" below). Only the task-to-task variation of this factor is left out. It acts like a spread in the signal-to-noise ratio. GP-BMA, which averages over a range of signal-to-noise ratios, scores at the level of the GP with a fixed ratio ("GP, training hyperprior"), so this approximation is unlikely to change the comparison. BrainPFN leads this GP on every target set (p < 0.05).
(Log-AUC of Pearson r over N = 5 to 800, mean over 5 folds)
| Model | Effect size r | Calibration effect | HCP-YA, all targets | HCP-YA, cognition | HCP-Aging, all targets |
|---|---|---|---|---|---|
| BrainPFN | 0.5 in training | In the training tasks | 0.137 | 0.325 | 0.333 |
| GP-BMA (Appendix C.2) | Averaged over 0.2 to 0.8 | Not modelled | 0.127 | 0.305 | 0.319 |
| GP, training hyperprior | 0.5 | Not modelled | 0.127 | 0.304 | 0.319 |
| GP, training signal model | 0.5 | Modelled | 0.127 | 0.307 | 0.320 |
Thank you for clarifying both the prior comparison and the GP interpretation. Your response addresses both of my concerns. I do not request any additional experiments and I will maintain my score.
Dear reviewer cTnk,
Thank you for acknowledging our response. As all of your concerns have been addressed, would you consider raising your score to a full acceptance?
We thank the reviewer for the careful and rigorous reading. To answer the 2 concerns in short: the reviewer's calculation is correct, so we withdrew the smoothness explanation and revised Sec. 3.2 and 5.4 (the no-RKHS ablation shows that Eq. 6 improves prediction, not why); and the GP is not the exact counterpart of the calibrated generator, so Appendix C.2 now presents it as a matched analytical baseline, which BrainPFN still outperforms.
1. Spectral explanation and RKHS prior
We thank the reviewer for this careful reading. The calculation is correct, and the argument we gave in Section 3.2 was a mistake on our side. The no-RKHS ablation shows that Eq. 6 improves prediction, and it does not support the smoothness explanation, which we fixed. We revised two sections (in green): Sec. 3.2 now states both priors and the covariance each one gives, and Sec. 5.4 now specifies the isotropic sampler, which shares the kernels and the calibration of Eq. 6.
What was wrong in our argument. With f=Kα, Eq. 6 gives Cov(f)=σα2K, and isotropic coefficients α∼N(0,τ2I) give Cov(f)=τ2K2. As the reviewer points out, the isotropic alternative places relatively more variance in the large-eigenvalue directions, so it does not produce rougher targets. The same holds for the RKHS norm: at equal signal variance, the expected norm of the isotropic draws is smaller than or equal to that of Eq. 6, since (∑jλj)2≤rank(K)∑jλj2. We removed the statement that Eq. 6 avoids "arbitrarily rough targets".
What we meant. Eq. 6 samples the coefficients according to the RKHS norm. It is the Gaussian prior p(α)∝exp(−α⊤Kα/2σα2) , where α⊤Kα=∥f∥H2 is the penalty of kernel ridge regression. Writing K=∑jλjvjvj⊤ and f^j=vj⊤f , this norm is ∥f∥H2=∑jf^j2/λj . It penalises the amplitude of f more along directions with smaller eigenvalues, and accordingly the sampled functions have variance σα2λj along vj. Our intent was to state that the coefficients are sampled under the same penalty that kernel ridge applies to them. Our explanation then mixed up two statements. The norm penalises the amplitude of f along the directions with smaller eigenvalues, usually read as the rougher ones. We took this to mean that coefficients drawn without the norm give rough targets. This does not follow: the norm acts on f, while the isotropic prior is flat on α.
What is used in the no-RKHS ablation. The ablation draws α∼N(0,I) and changes nothing else. Both samplers use the same real FC matrices, the same kernel family and bandwidth distribution, and the same calibration: the signal f is standardised with the mean and standard deviation measured on a separate block of subjects, and noise is then added to reach the target correlation r. Because of this standardisation the scales σα and τ drop out. The covariance calculation applies before this calibration, which is the same for both priors. The two priors therefore differ only in how the signal variance is distributed over the eigen-directions of K before calibration: proportional to λj under Eq. 6 and to λj2 under the isotropic prior.
What the ablation isolates. It compares training on functions with covariance K (before calibration) against training on functions with covariance K2, for the same kernel. Before calibration and for a fixed kernel, the Bayes-optimal in-context predictor under Eq. 6 is kernel ridge with that same kernel: for noise of variance σ2, it keeps a fraction λj/(λj+σ2/σα2) of the label component along vj. Under the isotropic prior this fraction becomes λj2/(λj2+σ2/τ2), which corresponds to a kernel ridge regressor with K2 as the kernel. After calibration the tasks are no longer exactly Gaussian (App. C.2), so these two filters describe the priors up to the calibration shift and scale. The ablation therefore removes the agreement between the prior over targets and the RKHS norm of the kernel that generates them.
What the ablation shows. It shows evidence that Eq. 6 improves prediction: removing it gives the largest drop of all ablations (Sec. 5.4). It does not show evidence for the smoothness explanation, which was incorrect. The revised Sec. 5.4 reports this result and presents the following explanation as a hypothesis: isotropic targets concentrate on a few leading directions of K, so a model trained on them learns to treat the weaker directions as noise, while real behavioural scores may also depend on those directions.