acceptodds
Under review as a conference paper at ICLR 2027

SeriesPFN: Unifying Time-Series Classification and Extrinsic Regression through End-to-End In-Context Learning

Abstract

Time-series classification (TSC) and extrinsic regression (TSER) predict a label or a continuous value from an entire time series, respectively. Unlike conventional approaches that require training a separate model for each dataset, pretrained foundation models such as Prior-Data Fitted Networks (PFNs) enable prediction on unseen tasks through in-context learning. They condition on a task's labeled training examples to predict targets for new series without updating their parameters. Two-stage PFNs achieve strong performance across both tasks, but rely on separate pipelines per task type and incur substantial inference cost. End-to-end time-series PFNs instead predict directly from raw series, but existing methods address classification only and trail the strongest approaches in accuracy. We introduce SeriesPFN, the first end-to-end PFN trained jointly for TSC and TSER. We develop a unified prior that generates classification and regression tasks through a shared construction, with process parameters varying between classes for classification and continuously with the target for regression. With a single checkpoint, SeriesPFN achieves the best mean accuracy rank on the UCR112 and UEA26 classification benchmarks and the second-best mean RMSE rank on the TSER62 regression benchmark. Runtime comparisons place SeriesPFN on the performance-efficiency frontier across all three benchmarks.

Primary area:
learning on time series and dynamical systems
Keywords:
time-series classification, time-series extrinsic regression, in-context learning, Prior-Data Fitted Networks, foundation models, synthetic data, multivariate time series

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_zU1WOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3(+1)
Soundness
3(+1)
Read review
Paper summary

The paper introduces SeriesPFN, an end-to-end prior-data fitted network (PFN) that performs time-series classification and extrinsic regression through in-context learning. It supports univariate and multivariate time series and predicts targets for new examples by conditioning on labeled context examples, without updating its parameters. Its main contributions are (1) a unified synthetic task prior for classification and regression and (2) a shared encoder and in-context learning architecture with task-specific target embeddings and prediction heads.

The method is validated primarily through benchmark comparisons and ablation studies. The reported results are strong, although a clearer explanation of the rationale behind some design choices would help readers understand how the authors arrived at the final method and when those choices are expected to generalize.

Critical strengths

SeriesPFN achieves strong reported performance on both classification and extrinsic regression using a single jointly trained checkpoint: the best mean accuracy rank on UCR112 and UEA26, and the second-best mean RMSE rank on TSER62. These results support the practical value of the unified approach.

Critical weaknesses

Limited explanatory insight into the unified approach. The ablations support the method's effectiveness, but leave open which target-relevant temporal characteristics are shared between classification and regression and how joint training exploits them. A concrete real-data example would help explain the observed transfer benefit. Similarly, the relationship between the two-endpoint regression prior and the diversity of real regression tasks remains insufficiently illustrated. These gaps limit the insight readers can gain beyond the reported performance improvements.

Questions for authors
  1. Class-label embeddings and class capacity (clarification). Appendix B.4 describes assigning task labels to learned class slots, and Table 5 specifies a maximum of 64 slots. How should readers reconcile this capacity limit with Section 4.3's description of the retrieval decoder as non-parametric in the number of classes? Could the same checkpoint support more than 64 classes, or would that require modifying and retraining the model?

  2. Number of [CLS] tokens (design justification). Section 4.2 explains that the four tokens gather information and are pooled and concatenated into a 512-dimensional series representation. What motivated choosing four tokens rather than one or another small number? Was this choice based on an architectural requirement or an empirical comparison, and how sensitive is performance to the token count when representation size and computational cost are accounted for?

  3. Regression prior expressivity (clarification). Section 3.4 generates regression tasks by linearly interpolating between two endpoint parameter settings. How restrictive is the two-endpoint linear parameterization, and what evidence suggests that it provides sufficient diversity for real regression tasks?

  4. Motivation for joint training (clarification). Section 5.3 suggests that classification benefits from representations learned through regression, and the single-task ablations provide some supporting evidence. Could the authors provide a concrete illustrative example from real data showing a temporal characteristic useful for both classification and regression, and explain how learning it through regression could benefit classification? Even a small case study would help readers understand the motivation for joint training beyond the aggregate performance results.

  5. TiCT preprocessing (clarification). Could the authors confirm whether TiCT's input time series were resampled to length 512 before inference? The local TiCT release enables length normalization with a target length of 512 and applies scipy.signal.resample(data, max_len, axis=2) when the input length differs, for arrays shaped (samples, channels, time). Appendix C.1 already discusses differences in the train/test split protocol; confirming the preprocessing would help rule out an additional source of the reported performance discrepancy.

Suggestion for future work. A sensitivity study of the 50/50 classification/regression and source-dependent/combiner-dependent task mixtures would help assess how robust the results are to these sampling choices.

5 comments
AuthorsOpenReview ↗

Response to zU1W (1/2): Architecture and regression prior

Thank you for the review and the detailed questions. We appreciate your recognition of the "strong reported performance" on both tasks and the "practical value of the unified approach".

In brief: we give a real-data example of a cue shared by classification and regression, and we explain how the two-endpoint prior relates to real regression tasks, including a limitation that the revision now documents. We respond to each question below.

Q1.

Could the same checkpoint support more than 64 classes, or would that require modifying and retraining the model?

The current checkpoint supports at most 64 classes; supporting more would require larger label embeddings and training with that capacity. "Non-parametric in the number of classes" describes only the retrieval decoder: its learned projections do not depend on the class count (Appendix B.4). The 64-class limit comes from the separate learned label embeddings (Appendix B.1 and Table 5). TabPFN-3, Section 2.2 makes the same distinction.

Q2.

What motivated choosing four tokens rather than one or another small number? [...] how sensitive is performance to the token count when representation size and computational cost are accounted for?

Four 128-dimensional summary tokens concatenate directly into the 512-dimensional ICL representation while keeping the encoder width at 128 (Appendix B.3). This follows the four-token design in TabICL, Section 3.3, TabICLv2, Appendix A.2, TimEE, Appendix B.1, and TabPFN-3, Appendix C, which explicitly specifies the same 4 × 128 = 512 arrangement. It was an architectural choice that we adopted from these models. We did not compare token counts, so we cannot say how sensitive performance is to this choice. With direct concatenation, changing the count also changes the ICL width; keeping that width fixed would require a compensating projection or a different encoder width.

Q3.

How restrictive is the two-endpoint linear parameterization, and what evidence suggests that it provides sufficient diversity for real regression tasks?

The interpolation is between two settings of the generating process, not between two time series. The interpolated parameters then pass through the combiners, temporal filters, and nonlinear activations of the graph, so a straight path in parameter space does not make the series depend linearly on the target (Section 3.4; Appendices A.5 and A.6). For example, a bilinear combiner multiplies two weighted inputs. If both sets of weights are interpolated linearly, its output already depends quadratically on the target.

There is still a restriction: within one task, the parameters follow a single straight path. Sources, graphs, and endpoints differ between tasks.

The evidence that this construction transfers to real tasks is the TSER62 result: SeriesPFN is trained only on these synthetic tasks and ranks second among all compared methods. The gap to the first, TS2TabPFN, comes mostly from the 14 datasets that the other compared methods predict most accurately. There, SeriesPFN's RMSE is 120% higher, against 9% on the other 48 (Appendix D.8). In the examples we examined, the published target definitions involve averages, totals, or maxima over the observation period. We hypothesize that such tasks are rare in our prior, because its targets set the parameters of the process that generates a series and are not computed from the series. We have not yet tested this hypothesis.

We have also not trained with curved or multi-anchor paths, so we cannot say whether they would reduce the gap. They remain future work, as the conclusion notes.

AuthorsOpenReview ↗

Response to zU1W (2/2): Shared features and inference

Q4.

[...] a concrete illustrative example from real data showing a temporal characteristic useful for both classification and regression [...]

An illustrative example is beat timing. BIDMC32HR predicts continuous heart rate from PPG and ECG signals. If we grouped its targets into rate intervals, the same beat timing would support classification. Learning to resolve the continuous rate then also provides the information needed to distinguish the groups. Our prior is built in the same way: a regression target varies continuously the same generating parameters that a class label varies discretely.

This example illustrates the motivation; we did not run it as an experiment. The single-task ablations give indirect support. In the joint model, about half of the training tasks are classification, so a classification-only model trained for the same 100k steps sees roughly twice as many classification tasks. It is still 0.39 accuracy points worse on UCR112 (p = 0.003). The revision adds a classification-only model trained for 50k steps, which sees about as many classification tasks as the joint model. It is 2.64 points worse on UCR112 and 4.48 on UEA26 (both p < 0.001; Appendix E.1). This model also trains for half as long, so some of this difference may come from the shorter training.

In both comparisons the joint model is better at classification. This is what we would expect if learning from regression tasks also helps classification, as in the heart-rate example above. We have not measured which features the model shares between the two tasks, so the paper states this as a hypothesis (Section 5.3).

Q5.

Could the authors confirm whether TiCT's input time series were resampled to length 512 before inference?

Yes. TiCT inputs are resampled to length 512 when needed using scipy.signal.resample. Our wrapper uses (samples, time) arrays, so resampling is along axis 1, the same temporal axis as axis 2 in the released (samples, channels, time) layout.

Suggestion for future work.

A sensitivity study of the 50/50 classification/regression and source-dependent/combiner-dependent task mixtures would help assess how robust the results are to these sampling choices.

Appendix E.1 tests the single-task and source-only/combiner-only endpoints. Studying intermediate mixture weights remains future work.


Thank you again for the detailed questions, which helped us explain the reasoning behind the design more clearly.

Reviewer_zU1WOpenReview ↗

Follow-up on the author response

Thank you for the detailed response and manuscript revisions. The explanations of the 64-class capacity, the four-token architectural choice, and TiCT’s preprocessing address my clarification questions. The additional single-task ablations and per-dataset analysis of the regression gap also provide useful context.

The beat-timing example is helpful: it makes the motivation for joint training more concrete by illustrating how the same temporal characteristic could support both classification and regression. I could not find this example in the revised manuscript. Could the authors include it in the paper so that readers can benefit from this explanation? It would be useful to preserve the distinction between this illustrative motivation and experimental evidence identifying which features the model actually shares.

The expressivity of the regression prior and the mechanism of transfer remain open questions, but I appreciate that the authors acknowledge these limitations and distinguish hypotheses from tested conclusions. Overall, the response strengthens my assessment, and I maintain my recommendation of weak acceptance.

Reviewer_HPNROpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SeriesPFN, an end-to-end Prior-Data Fitted Network for both time-series classification and extrinsic regression. SeriesPFN is trained on a unified synthetic prior in which target-dependent process parameters vary across classes for classification tasks and continuously with the target value for regression tasks. The method combines this prior with an architecture consisting of temporal, channel, and sample attention, followed by in-context prediction without per-dataset weight updates. Experiments show that SeriesPFN achieves competitive results while remaining competitive in inference efficiency.

Critical strengths
  1. The paper employs a single end-to-end PFN that unifies time-series classification and extrinsic regression. Rather than simply "adding a regression head to a classification model," the authors start from the essence of PFNs, recognizing that the key to a PFN lies in the task-distribution prior used during training, and accordingly construct a unified supervised task prior that spans both classification and regression.
  2. The architecture is intuitive and well-designed. SeriesPFN explicitly decomposes interactions along three structural axes: temporal, channel, and sample axis. Such a tri-axial encoder design is reasonable for time-series in-context prediction.
  3. The experiments include univariate classification, multivariate classification, and extrinsic regression. These three benchmarks cover the main scenarios and show the effectiveness of SeriesPFN.
Critical weaknesses
  1. Although the authors use a single checkpoint that supports both TSC and TSER, on TSER62, SeriesPFN's mean RMSE rank is 3.81, compared with 1.61 for TS2TabPFN, which is a substantial gap. The unified model, to some extent, trades task coverage for a partial loss in regression performance. While the paper acknowledges this gap in its conclusion, the analysis of this gap in the main text is not sufficiently deep. If regression is intended to be an important part of the contribution, the current evidence reads more as a proof of feasibility than as a comprehensive advance over prior methods.
  2. The training budget for the ablation studies is inconsistent with that of the final model. The ablations are trained for only 100k steps, whereas the main checkpoint is trained for 500k steps. As a result, many conclusions regarding the importance of specific prior components may not directly carry over to the final model, which weakens the strength of the ablation evidence.
  3. SeriesPFN’s performance depends heavily on how well the synthetic prior matches the distribution of real tasks. The synthetic prior in this paper contains many design choices in the dynamics graph that appear to be largely empirical, including the graph-size distribution, the Erdős–Rényi connectivity parameterization, the set of seven combiner types, the set of eight activation functions, and various parameter ranges. While the paper includes several ablations, it does not provide a systematic analysis of these dynamics-graph design choices. As a result, it remains unclear how robust the final performance is to these specific prior configurations.
  4. Ablation results are statistically significant for UCR112 but largely insignificant for UEA26 and TSER62. In particular, the removal of components such as Combiner-only target dependence, Independent settings, Uniform regression targets, No statistics embedding, and No channel attention does not significantly degrade performance on UEA26/TSER62. The authors should provide a more critical discussion regarding the actual utility of these components and address potential redundancy in the model design.
Questions for authors

See Weaknesses.

4 comments
AuthorsOpenReview ↗

Response to HPNR (1/2): Regression gap and ablation budget

Thank you for the thoughtful review. We appreciate your assessment that the paper "starts from the essence of PFNs" by building a unified task prior, and that the architecture is "intuitive and well-designed".

In brief: the revision adds two analyses of the TSER62 gap. A new single-task ablation complements the one in the submission, and neither finds that the gap comes from sharing the model with classification. A per-dataset analysis shows where the gap is concentrated. We respond to each weakness below.

W1.

on TSER62, SeriesPFN's mean RMSE rank is 3.81, compared with 1.61 for TS2TabPFN, which is a substantial gap. [...] the analysis of this gap in the main text is not sufficiently deep.

We agree that the gap to TS2TabPFN on TSER62 is substantial, and that the submission did not analyse it in enough depth. We do not claim to improve on TS2TabPFN in regression accuracy. Our claim is that one model, trained with one prior, stays competitive with the leading methods on both tasks at a much lower inference cost: on TSER62, its inference is 85.8x faster than TS2TabPFN's. It is also the first end-to-end time-series PFN that handles regression.

The unified model, to some extent, trades task coverage for a partial loss in regression performance.

We tested whether sharing the model with classification costs regression performance, and found no evidence that it does: in two comparisons, a regression-only model is not significantly better than the joint model.

In the joint model, about half of the training tasks are regression, so a regression-only model trained for the same 100k steps sees roughly twice as many regression tasks. It is not significantly different from the joint model on TSER62 (-0.9% geometric-mean RMSE, p = 0.40). The revision adds a regression-only model trained for 50k steps, which sees about as many regression tasks as the joint model but trains for half as long. It is again not significantly different (+0.5%, p = 0.40; Table 1 and Appendix E.1). Therefore, whether we match the number of training steps or the number of regression tasks, we do not find evidence that the gap comes from sharing the model.

We also analysed the gap per dataset (Section 5.2 and Appendix D.8). Most of the gap comes from the 14 datasets that the other compared methods predict most accurately:

TSER62 datasetsSeriesPFN geometric-mean RMSE above TS2TabPFN
14 with most predictable targets120%
Other 489%
All 6228%

In the examples we examined, the published target definitions involve averages, totals, or maxima over the observation period. We hypothesize that such tasks are rare in our prior, because its targets set the parameters of the process that generates a series and are not computed from the series. TS2TabPFN, in contrast, receives explicit summary statistics as tsfresh features. We have not yet tested this hypothesis. The conclusion now names adding such tasks as the most immediate direction for future work.

W2.

The ablations are trained for only 100k steps, whereas the main checkpoint is trained for 500k steps. [...]

Yes, we cannot be certain that the ablation results hold at 500k, and the paper notes this (Appendix E.1). We ran the ablations at 100k steps to make three seeds per ablation feasible. The eleven ablations and the control require 36 runs, which at 500k steps would take about 3,000 GPU-hours. However, the 100k model already achieves UCR112 accuracy within 1.38 points of the final model.

Training ablations on a smaller budget is common in PFN research, for example in TabPFN (Appendix B.4), TimEE (1.6M training tasks against 7.2M) and TempoPFN (500k iterations against 3M).

AuthorsOpenReview ↗

Response to HPNR (2/2): Prior sensitivity and component effects

W3.

[...] it remains unclear how robust the final performance is to these specific prior configurations.

These are hand-designed choices, and we agree that the paper does not provide a systematic sensitivity analysis of them. Each setting would need its own training runs with three seeds, so we limited the ablations to removing whole components. Two things limit how much the result depends on the specific settings.

First, the general construction follows the structural causal model prior of TabPFN, extended to time series in CauKer, and its combiners and activations are generic function families. Graph size, connectivity, and parameter values are sampled per task from deliberately broad ranges to cover diverse generating processes. We therefore expect their coverage to matter more than their exact endpoints.

Second, development used a separate 36-dataset meta-validation subset; the remaining 164 datasets were held out until the model was frozen (Appendix D.1). The ablations that remove or replace parts of the graph structure are repeated on these held-out datasets alone, and their losses there are about as large as on the full benchmarks. This suggests that the choices generalize beyond the datasets they were developed on.

W4.

Ablation results are statistically significant for UCR112 but largely insignificant for UEA26 and TSER62. [...]

Indeed, not every component is significant on every benchmark. SeriesPFN is a single checkpoint intended to cover the task types of all three benchmarks, so we consider a component useful when it has a significant effect on any one of them. These effects matter in practice; for example, removing channel attention alone moves the 100k model from second to fifth on UEA26 when methods are ordered by mean accuracy rank.

UEA26 and TSER62 also have far fewer datasets than UCR112, making modest effects harder to detect after Holm correction. Even where the differences are not statistically significant, removing or replacing the components still reduces performance, consistently matching the direction of the significant results. The overall pattern is therefore consistent with a benefit across benchmarks, even where individual comparisons do not reach significance.


Thank you again for the detailed feedback on the regression results and prior design.

I thank the authors for their response and explanations. However, my concerns remain.

First, the authors emphasize that their contribution lies in a "one model for two tasks" framework that "stays competitive." However, the evidence provided shows that "on TSER62, SeriesPFN's mean RMSE rank is 3.81 (compared with 1.61 for TS2TabPFN)". This is a substantial gap.

Second, about "The ablations are trained for only 100k steps, whereas the main checkpoint is trained for 500k steps". However, the authors explicitly stated in the final part of Appendix E that these short-run ablations “they do not establish that the effect sizes are unchanged at the 500k-step budget of the evaluated SeriesPFN checkpoint. A nonsignificant comparison should not be read as evidence of equivalence.” This seems to contradict their rebuttal claim attributing the limited ablation scope to "GPU-hours," as it suggests the ablation results may not be reliable.

Furthermore, the authors acknowledged the use of “hand-designed choices,” relying on intuition rather than systematic analysis. While I understand that analyzing these design choices may be challenging, doing so without any justification is unconvincing and fails to provide effective guidance for future research.

Finally, regarding statistical significance, I specifically question the results in Tables 29-30. The authors' own experiments indicate that the effects of certain components are not statistically significant. The authors argue that “we consider a component useful when it has a significant effect on any one of them,” but I find this rationale insufficient. Such inconsistency suggests that these components lack robustness, failing to demonstrate consistent effectiveness even within the authors' own test data.

Based on the reasons above, I do not believe this paper currently meets the acceptance standards for ICLR. Therefore, I maintain my score.

Thank you for clarifying your concerns.

This is a substantial gap.

We agree that the regression accuracy gap to TS2TabPFN is substantial. SeriesPFN's inference is also 85.8 times faster on TSER62, so the comparison involves a practical tradeoff between accuracy and inference cost. Our claim of competitiveness reflects both.

This seems to contradict their rebuttal claim attributing the limited ablation scope to "GPU-hours," as it suggests the ablation results may not be reliable.

The compute explanation is consistent with the appendix caveat. Computational cost explains why we used 100k steps. Comparisons against a control trained for the same budget provide evidence about component effects at 100k. The appendix separately states that we have not established whether these findings carry over to the 500k-step model. That limits extrapolation to longer training; it does not invalidate the comparisons at 100k.

Doing so without any justification is unconvincing.

The construction follows earlier SCM-based priors, using generic function families and broad parameter ranges to cover diverse generating processes. Our rationale is that coverage matters more than the exact range endpoints. The component ablations and held-out comparisons provide empirical evidence for the resulting design. To clarify what further evidence you are requesting, could you give a concrete example of the sensitivity analysis you have in mind and what you would expect it to establish?

Such inconsistency suggests that these components lack robustness.

The observed changes favour the complete model even where they are nonsignificant. Smaller panels provide less power to detect modest improvements, especially after multiple-testing correction. Consequently, improvements in the same direction can produce different significance outcomes across benchmarks. Here, the measured effects remain consistent in direction; what differs is whether each comparison reaches the significance threshold.

The authors argue that “we consider a component useful when it has a significant effect on any one of them,” but I find this rationale insufficient.

For redundancy, the question is whether a component can be removed without losing performance on the tasks the shared model supports. A significant degradation on one benchmark provides evidence against removing it as redundant, even when the differences elsewhere are not statistically significant.

Reviewer_VrnzOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces SeriesPFN, a single model that handles both time-series classification and regression using the same checkpoint. SeriesPFN takes the time series and their corresponding labels as input examples and predicts new ones without dataset-specific training. The authors build the training prior using synthetic tasks, where Gaussian-process samples are passed through randomly generated graphs of nonlinear transformations. The time series are split into patches, and attention is applied both within each series and between different channels and series. On top of this, it uses a TabPFN-style in-context learning stack. Classification and regression are handled by separate output heads, a retrieval-based decoder for classification and a Riemann distribution head for regression.

Critical strengths
  1. A single model handles both classification and extrinsic regression, and it is close to the best methods while being much faster (about 20 s vs 1,700 s per TSER dataset compared with TS2TabPFN).
  2. The prior is relatively straightforward, with one generative system for each task and only one group of parameters changing depending on the target. The evaluation is also carefully done. Some of the ablations are also useful. For example, removing the shared template or the dynamics graph leads to a large drop in performance, even on univariate data. These aspects make the reported results more convincing, although I do not fully agree with how the results are presented.
Critical weaknesses

The paper puts a lot of emphasis on combining TSC and TSER in one model, but I do not see much benefit from training on both tasks. Classification-only training loses less than half a point on UCR112, while regression-only training makes no difference on TSER62. So the results do not really show that the two tasks help each other.

Also, most of the components are already based on previous work. The ablations show that removing the shared template or random graph leads to large performance drops. But these are not specific to the main idea of this paper. The changes made for classification and regression seem to have a much smaller effect. I agree that having one checkpoint for both tasks is useful, but I am still not sure how much the unified design itself adds beyond that.

Questions for authors
  1. What are the main results if the preprocessing ensemble is removed? It would also be fairer to use a similar ensemble for TS2TabPFN and MASHT.
  2. What is the performance of SeriesPFN on the meta-test datasets only?
  3. Were other ways of generating regression tasks tested, such as a curved path or targets that depend on the signal level? This might reduce the gap to TS2TabPFN.
  4. How sensitive are the results to the training seed? The ablations use 100k training steps, and I would like to know if they also hold for the final 500k-step model.
4 comments
AuthorsOpenReview ↗

Response to Vrnz (1/2): Joint training and contribution

Thank you for the careful review. We are glad the evaluation made you "trust the reported numbers", and that you see SeriesPFN as "close to the best methods while being much faster".

In brief: the ablations, including a new single-task comparison added in the revision, show a small classification benefit on UCR112 and no detectable regression cost, rather than substantial gains on both tasks. We have revised the introduction and Section 5.3 to reflect this better. Our contribution is different: earlier end-to-end PFNs handle classification only and trail the leading methods, while SeriesPFN is competitive with them on classification and regression from one checkpoint, at 10x to 370x lower inference cost.

classification-only training loses less than half a point on UCR112, and regression-only training makes no difference on TSER62, so the two tasks do not seem to help each other.

We test whether sharing one model between the two tasks helps or costs either of them. It costs neither, and it gives classification a small benefit: in two comparisons, the classification-only model is significantly worse than the joint model on UCR112, and the regression-only model is not significantly different on TSER62.

A single-task model can be compared with the joint model in two ways. In the joint model, about half of the training tasks are classification and half are regression. If both models train for the same number of steps, the single-task model sees roughly twice as many tasks of its type. If the single-task model trains for half as many steps, it sees about as many tasks of its type as the joint model, but it also trains for half as long. Neither comparison is exact, so we report both. The submission reported the first at 100k steps. The revision adds the second, with two single-task models trained for 50k steps (Table 1 and Appendix E.1).

Single-task model, compared with the joint model (100k steps)Classification-only: UCR112 accuracyClassification-only: UEA26 accuracyRegression-only: TSER62 RMSE
Trained for 100k steps (same steps as the joint model)-0.39 points (p = 0.003)-1.88 points (p = 0.30)-0.9% (p = 0.40)
Trained for 50k steps (same tasks of its type as the joint model)-2.64 points (p < 0.001)-4.48 points (p < 0.001)+0.5% (p = 0.40)

Each cell is the single-task model minus the joint model. A negative accuracy difference or a positive RMSE difference means the single-task model is worse.

Some of the classification difference at 50k steps may come from the shorter training. But even at 100k steps, with twice as many classification tasks as the joint model, the classification-only model is still worse, which supports a small benefit of joint training for classification.

Most components come from earlier work [...]

We agree that the architecture builds on earlier PFNs, and that the prior uses known building blocks, which the paper credits. What is new is the task construction built from them: one generative process whose parameters the target changes, by class for classification and continuously for regression. Earlier end-to-end time-series PFNs generate only classification tasks, so without this construction the model could not be trained for regression at all.

The target-related choices, which are the main new part, matter much less.

Yes, these choices have smaller effects than removing the dynamics graph, which is expected: that ablation removes a large part of the generator, while the target ablations only change where the target enters it. However, their effects are still significant; for instance, when the target only changes the source, the model loses 1.37 accuracy points on UCR112 (p < 0.001) and 3.05 on UEA26 (p = 0.003), and RMSE on TSER62 rises by 8.1% (p < 0.001; Table 1). For reference, this change alone moves the 100k model from second to ninth on UEA26 when methods are ordered by mean accuracy rank.

[...] it is not clear what we learn from the unified design.

We learn two things. First, a classification prior can also generate useful regression tasks: when classes differ in continuous parameters of the generator, interpolating between two classes gives a continuous target. Second, one model can be trained on both task types with no detectable cost on either. This was not a given, since the two could have competed for the model's capacity. In addition, the transfer we detect runs one way: regression tasks help classification, while we detect no benefit of classification tasks for regression.

AuthorsOpenReview ↗

Response to Vrnz (2/2): Design interpretation and Q1-Q4

We have not measured which representations the two tasks share, so we offer a hypothesis for the direction of this transfer (Section 5.3). A regression task asks the model to resolve small changes in the generator's parameters, while a classification task only asks it to separate distinct settings. Sensitivity to small changes may help tell classes apart, while separating distinct settings may teach little about the points between them.

Q1.

What are the main results without the preprocessing ensemble?

Appendix D.3 reports the results with and without temporal preprocessing:

MetricOriginal-series ensemblePreprocessing ensemble
UCR accuracy (%)87.6889.30
UEA accuracy (%)75.6275.74
TSER relative RMSE0.4800.485

The preprocessing ensemble mainly helps on UCR. UEA is almost unchanged, and TSER is slightly worse. We chose it on meta-validation before looking at meta-test results, and the UCR gain carries over to meta-test (+1.55 points).

[...] it would also help to give TS2TabPFN and MASHT a similar ensemble.

Both already average eight predictions in our evaluation, using the ensembling their authors designed. We kept each method's own recipe so that we compare against them as published. Our temporal preprocessing was chosen for a model that reads the raw series, and it is not obvious how to combine it with their feature extraction, so we did not attempt it. We agree it would be an interesting comparison.

Q2.

How does SeriesPFN rank against all methods on the meta-test datasets alone?

On meta-test alone, SeriesPFN ranks second on all three benchmarks. The mean ranks are:

Meta-test benchmarkFirst-placed methodSeriesPFN
UCR94TS2TabPFN: 4.594.95
UEA20HIVE-COTE 2.0: 4.425.50
TSER50TS2TabPFN: 1.563.76

Appendix D.1 of the submission reported SeriesPFN's ranks on meta-test, and the revision adds the first-placed method on each benchmark. The meta-validation/meta-test split is specific to our study. We held these datasets out while developing SeriesPFN, but the other methods were developed without this split, so the same datasets may have been available to them during development. The held-out comparison is therefore, if anything, conservative for SeriesPFN. For this reason we report it alongside the standard full-benchmark comparison.

Q3.

Did you try other ways of generating regression tasks, for example a curved path or targets linked to the signal level? [...]

Thank you for both suggestions. We have not yet trained with either, so we cannot say whether they would close the gap. The only alternative we tested is drawing regression targets uniformly, which increases geometric-mean RMSE by 6.0% at 100k steps (p = 0.017; Appendix E.1).

A per-dataset analysis we added in the revision (Appendix D.8) does point in the direction of your second suggestion. Most of the gap to TS2TabPFN comes from 14 datasets whose targets are easy to predict: on each, at least one of the nine methods other than SeriesPFN and TS2TabPFN reaches an RMSE below 30% of the standard deviation of the targets. On these, SeriesPFN's RMSE is 120% higher than TS2TabPFN's, against 9% on the other 48. In the examples we examined, the published target definitions involve averages, totals, or maxima over the observation period. We hypothesize that such tasks are rare in our prior, because its targets set the parameters of the process that generates a series and are not computed from the series. Targets linked to the signal level, as you suggest, are one way to add such tasks, and the conclusion now names this as the most immediate direction for future work.

Q4.

How much do the results change with a different training seed? And do the 100k-step ablation results still hold for the final 500k-step model?

Across the three control seeds at 100k steps, accuracy varies by 0.31 points on UCR112 and 1.38 on UEA26. We cannot measure this at 500k, where the model has a single training run.

We cannot be certain that the ablation results hold at 500k, and the paper notes this (Appendix E.1). We ran the ablations at 100k steps so that each could be trained with three seeds: the eleven ablations and the control are 36 runs, which at 500k steps would take about 3,000 GPU-hours. Training ablations on a smaller budget is common in PFN research, for example in TabPFN (Appendix B.4), TimEE (1.6M training tasks against 7.2M) and TempoPFN (500k iterations against 3M).


Thanks again for the careful reading and the suggestions for extending the regression prior.

Thank you for the detailed response and the revision. However, I still have the same concern. At 100k steps, the difference from joint training is quite small (0.39 points on UCR112), and there is no improvement on regression. I also noticed that the abstract, introduction and conclusion still put a lot of emphasis on the best rank on UCR112 and UEA26. Given these results, I find the lower training cost a stronger point of the paper.

I keep my score.

Thank you for following up.

At 100k steps, joint training gives only a small classification benefit (0.39 points on UCR112) and no effect on regression, and the 50k comparison is confounded by the shorter training.

We agree with this assessment, which matches the interpretation in Section 5.3: a small classification benefit and no detectable regression cost. That section also notes that part of the difference in the 50k comparison may come from shorter training.

Yet the abstract, introduction and conclusion still claim the best rank on UCR112 and UEA26. I would suggest framing the main claim as "competitive at much lower cost," which the results support well.

Following your suggestion, the abstract, introduction and conclusion no longer state benchmark ranks. They now emphasize that a single checkpoint lies on the performance-efficiency frontier across all three benchmarks. The conclusion summarizes the results as competitive predictive performance at low inference cost.

Section 5.2 retains the ranks alongside their statistical significance. Neither SeriesPFN's classification lead on the full benchmarks (Holm-adjusted p = 0.74 on UCR112 and 0.44 on UEA26) nor its deficit on the held-out meta-test datasets (p = 0.95 on UCR94 and 1.00 on UEA20) is significant. Appendix D.1 now includes these meta-test tests.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Revision summary

We thank all three reviewers for their constructive feedback. Reviewers highlighted the unified supervised task prior spanning classification and regression (HPNR), the "intuitive and well-designed" architecture (HPNR), and performance "close to the best methods while being much faster" (Vrnz). They also recognized the careful evaluation, including standard protocols, corrected tests, and a development split fixed in advance (Vrnz), and the "practical value of the unified approach" (zU1W).

We have uploaded a revised manuscript with the following main additions and clarifications:

  1. Joint-training ablations (HPNR, Vrnz, zU1W). We added classification-only and regression-only models trained for 50k steps, complementing the existing 100k-step comparisons by approximately matching the number of tasks of each type seen during joint training. We clarify the distinction between these two training budgets and update the Holm correction to include the added comparisons. Results appear in Section 5.3 and Appendix E.1, particularly Tables 28–30.

  2. TSER62 performance-gap analysis (HPNR, Vrnz, zU1W). We expanded Section 5.2 to quantify where the regression gap is concentrated and added a detailed per-dataset analysis in Appendix D.8. The analysis groups datasets using the performance of the nine compared methods other than SeriesPFN and TS2TabPFN, examines illustrative published target definitions, and discusses a hypothesis about prior coverage. The corresponding future-work direction is described in Section 6.

  3. Clarified held-out comparisons (Vrnz). Appendix D.1, Table 10 now identifies the best-ranked comparator on each meta-test partition. The accompanying discussion clarifies that these partitions were held out during SeriesPFN development, while the other methods were developed without our study-specific split, and explains why we report both held-out and full-benchmark comparisons.

AuthorsOpenReview ↗

Revision update

We have uploaded a further revision with two changes:

  • Performance claims (Vrnz). The abstract, introduction, and conclusion now emphasize the single checkpoint's position on the performance-efficiency frontier instead of benchmark ranks. Section 5.2 and Appendix D.1 retain the ranks and clarify their statistical significance, including newly added tests on the meta-test partitions.
  • Shared-feature example (zU1W). Section 5.3 now includes the beat-timing example alongside the joint-training hypothesis, illustrating how the same temporal characteristic could support classification and regression.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.