acceptodds
Under review as a conference paper at ICLR 2027

FROM 1D TO 4D: A UNIFIED SEQUENCE PREDICTION FRAMEWORK WITH THE KUN-V2 ARCHITECTURE

Abstract

Sequence prediction is central to modern deep learning. In recent general-purpose time-series models, sequence completion and reconstruction, predicting the hid- den part of a sequence from its visible part, have become the common basis of forecasting, imputation and anomaly detection. These models, however, are gener- ally hard to extend to higher dimensionalities. Neural operators such as FNO and U-NO can be extended, but at a comparatively high complexity. To address the or- der in which subspaces are processed and the efficiency on high-dimensional data, we propose KUN-V2, a unified encoder–decoder framework that processes inputs of arbitrary spatial dimensionality via two ingredients: an auto-factorization that splits each spatial axis into balanced chunks, and a compact spec language that selects in what order and grouping the chunks are reduced. We show that the time complexity is linear in the product of all input dimensions and invariant under any reordering or regrouping of the chunk schedule. We validate the framework in-distribution on a single task, masked-half sequence completion, across all four targeted dimensionalities and, on synthetic data, a fifth with a single shared code- base: only the input shape, spec string, and chunk multiplicities change between settings. The evaluation covers 21 datasets, comprising synthetic and real data at N = 1 . . . 4 and the two synthetic families at N = 5. Against eleven learned baselines, KUN-V2 places in the top three on 17 and first on 11, and on a synthetic cost ladder its measured training time and peak memory stay within the linear-in- element-count bound. KUN-V2 thus reduces the cost of moving to a new dimen- sionality from architectural re-design to a configuration change. Code is available at https://github.com/jccxfgj-beep/kun-v2/tree/main.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
deep learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_D4Z3OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposed KUN-V2, a UNet like archtichure for sequence prediction on any number of dimensions. The authors claims that the model’s compute grows linearly with the number of cells in the input, and the reshaping/reordering never loses information. The model is evaluated in a few benchmarks and top on some of them.

Critical strengths
  1. the model effort is justified and it's good to have a framework for dimension-agnostic sequence prediction
  2. code is released
Critical weaknesses
  1. The theory are trival, eg Thm 2 and Cor 3. They are just used to inflate the paper
  2. The baselines looks undertrained or not trained properly. Mamba scores 1.08 on 1d sine, worse than predicting all zeros (0.95)
  3. From appendix E.2, most of the models are Linear. So essentially they are just a big matrix multiplication. This mean all of the introduced arch eg chunking, kernels, encoders decoders, are just a waste of time. Also this points to that the linear baseline is not trained properly.
  4. The authors runs a lot of ablations and selected the best based on test score, as explained in Appendix G. This is wrong and they should use validation score, especially for these small dataset.
Questions for authors
  1. did you select the model based on test score?
  2. did you properly train baselines?
10 comments
AuthorsOpenReview ↗

## Reponse to Reviewer 3 (D4Z3)

We thank the reviewer for the comments. Following them, we reran every learned model, our Mamba baseline (now called Mamba Flatten) and Linear included, under one training protocol with the same data splits and three seeds. We also enlarged the datasets, for example 1D sine from 64 to 1024 samples, and tuned the widths of Mamba Flatten, Transf. Flatten, FNO and U-NO on validation error. Every configuration is now chosen on the mean validation error.

With these changes, the baselines train properly. Mamba Flatten reaches 0.11 on 1D sine, against 10.1 for the Zero predictor, and Linear reaches ≤ 0.01 on the 1D–3D sine rows. Kun-v2 is first on 15 of the 22 rows and in the top three on 18. Below we also explain why we keep Theorem 2 and Corollary 3 and when the linear kernel is chosen.

AuthorsOpenReview ↗

### 1. "The theory is trivial (e.g. Theorem 2 and Corollary 3); it only decorates the paper."

We agree that Theorem 2 and Corollary 3 are clear. To the best of our knowledge this isn't yet declared in the literature, thus the novelty can be confirmed. Theorem 2 states that the reshape and permutation induced by a spec is a bijection, and Corollary 3 states that, for any valid spec, the encoder reduces every axis to size 1 and the decoder restores it. We still regard these results as a contribution of the paper, for the following reason.

Patching in high-dimensional space, that is, cutting a field of arbitrary rank into chunks and putting it back together, has not been treated adequately in prior work. Several works show the difficulties that patch operations cause in space. For Mamba, for example, the order in which the patches are scanned has to be designed anew to keep spatial continuity, as in the four-direction cross-scan of VMamba [1] and the zigzag scan of ZigMa [2]. Even among our baselines, the 3D and 4D models (Earthformer, VideoMamba and SwiFT) each need their own patch or window code written for their rank, and none of this code carries over to another rank.

Theorem 2 and Corollary 3 guarantee that any string accepted by the spec parser yields a well-typed, information-preserving encoder–decoder pair at any rank. The patching can therefore be changed freely, without shape checks for each spec or each rank. We keep both results in the main text, with their proofs in the appendix. The main theoretical result remains Theorem 1, the O(E) cost bound, which holds independently of the rank and of the number of chunk layers; the cost experiments (Table 1, Fig. 3 and Fig. 6) test this result.

Fig. 6 in App. H.2 tests the bound up to rank 8 by measuring the GPU time per element of one training step from rank 3 to 8, while E grows 32768 times. For Kun-v2 with one block at d = 32, this time stays between 0.69 and 0.92 ns with the fully joint spec and between 2.7 and 3.2 ns with the axis by axis spec, so the bound holds for both specs and the spec changes only the constant. That of FNO at d = 8 grows 1.5 times from rank 3 to 7 and that of U-NO 3.4 times, since their Fourier transforms make them grow faster than Kun-v2 (see the table in §0.4).

AuthorsOpenReview ↗

### 2. "Baselines look under-trained: Mamba scores 1.08 on 1D sine, worse than all-zero (0.95)."

We thank the reviewer for pointing this out; the observation is correct for the submitted version. The main cause was the small dataset, since the submitted 1D sine row had only N_sample = 64 samples. In addition, Mamba Flatten ran at a single small width under a different schedule.

The appendix experiment on dataset size (Fig. 4) shows that Mamba Flatten improves steadily as the data grow. This experiment trains nine models on 1D sine with pool sizes from N_sample = 64 to 1024, using three seeds and one recipe for all models (at most 500 epochs, early stopping after 50 epochs without improvement, constant learning rate 5×10⁻⁴). Its results are given in the table of §0.1, as test masked MSE ×10 averaged over the three seeds.

In that table the Zero predictor scores 10.1. At the submitted size of N_sample = 64, Mamba Flatten and Transf. Flatten are far from fitting the field, with 7.59 and 5.27. At N_sample = 1024 both are below 0.2, and in Table 2 Mamba Flatten reaches 0.11 ± 0.08 and Transf. Flatten 0.05 ± 0.06. This shows that the problem in the previous version came mainly from having too few samples to fit the field, and not from a fault in the implementation of the baselines.

Reviewer_JngYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes KUN-V2, a dimension-agnostic encoder-decoder model that uses auto-factorisation and a specification language to handle data with different spatial dimensions, without requiring the architecture to be redesigned for each dimension. The authors evaluate the model on 21 dataset-task combinations and compare its performance with 14 baseline models. KUN-V2 ranks among the top three learned models on 17 tasks and achieves the best results on 11 of them. The paper also presents a theoretical analysis of computational complexity, supported by experiments demonstrating favourable computational scaling. Overall, the work aims to provide a flexible and computationally efficient framework for multidimensional learning.

Critical strengths
  1. The idea is simple and useful. One code works for all dimensions, and it uses little memory. At 5D it needs only about 686 MB, while FNO and U-NO need 16 to 24 GB.

  2. The tests are wide, and the authors are honest. They share their code and settings, and they say clearly where their model loses.

Critical weaknesses
  1. Unfair comparison. On the sine and wavelet rows, KUN-V2 reports its best cell from a search of up to 257 settings (spec, width, depth, kernel). The baselines get one fixed setting. Epoch budgets also differ (500 vs 200 on wavelet). It is not clear whether the best cell was picked on validation or on test.

  2. Weak evidence. Most real datasets have only about 100 samples. Many gaps are smaller than the seed spread (for example Burgers: 0.111 vs 0.116, std about 0.04). Some cells have one seed. FNO or U-NO wins on several real datasets.

Questions for authors
  1. Was the best setting chosen using validation data or test data? Please show results with one fixed setting per dimension, against other models tuned with the same effort.

  2. Please add confidence intervals, finish the missing runs, and explain how you chose the spec for real datasets.

  3. Can you add an N-D U-Net and a patch-based Transformer, and test on larger data sizes?

10 comments
AuthorsOpenReview ↗

## Response to Reviewer 2 (JngY)

We thank the reviewer for the comments and suggestions. Following them, we reran every learned model under one training protocol with the same data splits, and we now choose every configuration on the mean validation error over three seeds. The spec sweep (203 specs, at most 260 cells per dataset, Fig. 7 in App. I) is kept as an ablation only. We increased most datasets to about 1000 samples, using all available samples on ETT, ERA5 and KITTI, and every cell now reports the mean and standard deviation over three seeds. We also added the requested baselines, among them an ND U-Net (nnU-Net and U-Net-4D) and patch-based Transformers (PatchTST, SwinIR, Earthformer and SwiFT), as well as a ranking without any per-row choice (Table 14).

In the resulting comparison against 18 learned baselines, Kun-v2 is first on 15 of the 22 datasets and in the top three on 18. With one fixed configuration per model, Kun-v2 and FNO are each in the top three on 15 rows. We give the details below.

AuthorsOpenReview ↗

### 1. "Unfair comparison: KUN searched up to 257 settings, baselines one; 500 vs 200 epochs; selection on val or test?"

We have fixed all three issues in the revision, as described in §0.2.

  • The spec sweep (203 specs, at most 260 cells per dataset). The sweep is now an ablation only, and nothing in Table 2 is selected from it. We use it only to study how the knobs affect the error. It showed, for example, that on high-dimensional datasets, reducing a subset of the axes first lowers the error by a larger margin than on low-dimensional ones.

  • Training settings. Every learned model trains under the same settings, which Table 5 lists:

    SettingValue
    OptimizerAdamW (β₁ = 0.9, β₂ = 0.999, ε = 10⁻⁸)
    Learning rateconstant 5×10⁻⁴
    Weight decay1×10⁻⁴
    Gradient clippingglobal ℓ₂ norm 1.0
    Batch size16
    Training lengthat most 500 epochs, early stopping with patience 100 on the validation error, at most 60 min per run
    Lossmasked MSE on the hidden half
    Datathe same split for every model (0.7 / 0.1 / 0.2 train / validation / test, redrawn per seed), z-scored on the training fold only
    Seeds2026, 2016, 2006
    Reported valuetest masked MSE at the epoch with the lowest validation error
  • Selection. All selection uses the mean validation error over the three seeds, and the test error is only reported.

    • Kun-v2. The cell of Table 2 is the best of the 4 fixed candidates of Table 9. On the 2D–5D rows, a three-block, d = 128 spec of Table 12 replaces it only if that spec has a lower validation error, which happens on 9 rows (marked §).
    • In-house baselines. FNO, U-NO, Transf. Flatten and Mamba Flatten each choose between 2 widths (Table 10).
    • Third-party models. Each third-party model uses its official configuration, with the input and output layers and the size tuned empirically, keeping the setting with the lowest validation error.
    • Size of the option sets. Per row, Kun-v2 has at most 10 settings: the 4 candidates plus the 3 non-default specs × 2 kernels at three blocks. On the 1D rows it has only the 4 candidates, and on 5D wavelet four of the six spec cells run out of memory. The baselines together have 10 in-house settings plus the third-party models of the row's rank (12 in all), so the two sides are comparable. App. F.2 lists the full configuration of every model.
  • Fixed-configuration ranking (Table 14). To remove every per-row choice, Table 14 uses one fixed setting per model on all 22 datasets. In this ranking, the best fixed Kun-v2 is in the top three as often as FNO, on 15 rows each, and FNO has two more first places.

AuthorsOpenReview ↗

### 2. "Insufficient evidence:  100 samples on real data, gaps smaller than seed spread, single-seed cells, FNO/U-NO better on several real rows"

  • Data size. Most rows now use about 1000 samples, which removes most of the imbalance between rows. The sine, Burgers, Lorenz-96, Navier–Stokes, GeoLife and Moving MNIST rows use N_sample = 1024. The wavelet rows use every distinct member at 1D and 2D (300 and 600 samples) and about a thousand samples at 3D–5D. Where the source has fewer samples, every available one is used: 272 for ETT, 1029 for ERA5 and 363 for KITTI. No real row uses N_sample = 100 any more.

  • Seeds. Every cell now has three seeds, so no single-seed cell remains. Every cell also prints its standard deviation, except the two reference columns of Table 9, which show the mean only.

  • Where Kun-v2 loses. We agree with the reviewer, and the revision says so. On the 12 real rows, Kun-v2 is first on 5 and in the top three on 8. Two of the 5 first places are ties on the Burgers rows, shared with other models; the outright wins are 2D ETTh1, ERA5 and KITTI. On several real rows, other models are clearly better: FNO on Lorenz-96 (0.31 against 1.07), Earthformer on Navier–Stokes (0.21 against 0.48) and on GeoLife (4.30 against 4.83), PredRNN on Moving MNIST (2.91 against 5.28). Linear is also better on the two 1D ETT rows.

    Kun-v2 is not meant to be the most accurate model on every dataset. Our aim is a model whose accuracy is adequate on data of any rank and whose cost is much lower. For example, on 5D sine (Table 15), averaged over eight three-block Kun-v2 models (two kernels times four specs), Kun-v2 has a test MSE of 0.161 (raw MSE), takes 2.63 s per training epoch and needs 7.13 GB of peak GPU memory, not counting the training data. FNO, averaged over d = 8 and d = 32, has 0.241, 12.3 s and 15.8 GB, so Kun-v2 is 1.5 times more accurate, 4.7 times faster and needs 2.2 times less memory. Against U-NO, averaged in the same way, the factors are 3.2, 10 and 2.8. Such a model lets us check new ideas quickly and build new prototypes on data of any rank, which is what the follow-up work needs.

Reviewer_MemgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Kun-v2, a unified encoder–decoder framework for sequence prediction on inputs of any spatial dimensionality. Architecture: a U-Net-style encoder–decoder with skip connections. Auto-factorization splits each spatial axis into balanced chunks; a spec string sets the order and grouping in which the encoder reduces them, each with a small linear or MLP kernel. Moving to a new dimensionality changes the input shape, the spec and the chunk counts, without redesigning the architecture. Claimed contributions: (1) the spec language with auto-factorization; (2) an O(E) arithmetic-cost bound that holds for every valid spec under the theorem's assumptions of fixed-width kernels and chunk sizes of at least two (Theorem 1), and invertibility of the reshape and permutation (Theorem 2); (3) an empirical study on one task, masked-half completion, over 21 datasets: synthetic and real data at ranks 1 to 4 and two synthetic datasets at rank 5. Validation: against eleven learned baselines and three trivial predictors, the paper reports top three on 17 of 21 rows and first on 11 (Table 2); on the cost ladder, parameters grow 4.5×, time 2.0× and memory 31× as the element count grows 4096×; and the leading spec family changes between the two synthetic families.

Critical strengths

The same architecture handles the evaluated 1D to 5D inputs, and its arithmetic cost has a linear bound under stated assumptions. The architecture is retargeted through its input shape, axis names, spec and chunk counts, while the benchmark also varies capacity settings. With a fixed number of kernel layers, fixed channel and interior widths, and consumed group volumes of at least two, Theorem 1 bounds the arithmetic cost by O(E) for every valid spec. The measured cost agrees: over a 4096× growth in E, memory grows 31× and ends at 686 MB, while the two spectral operators reach 24.2 and 16.6 GB (Fig. 3). This supports the cost half of the paper's central claim, that moving to a new dimensionality is a configuration change.

Critical weaknesses
  1. I do not trust the rankings in Table 2, because the comparison protocol is not controlled. Ten of the 21 rows use S = 100 samples, the epoch budget, early stopping and learning-rate schedule were not shared across cells, and baseline budgets and widths differ. Most cells average three seeds, and a few marked cells use fewer. The paper reports seed spreads but does not analyze rank stability. Unequal tuning effort may therefore contribute to the "top three on 17, first on 11" result. This matters because the ranking table carries the main cross-model claim.

  2. The paper does not show that choosing a spec improves accuracy, although the spec is the main addition to its predecessor. Kun-v2 directly extends Kernel-U-Net, which already used custom-kernel chunk reductions for 1D multivariate forecasting, and the linear bound recovers the predecessor's one-dimensional result. The theorems hold for every valid spec under their assumptions, so they do not rank schedules. The only spec comparison covers the sine and wavelet data, where no family leads at every rank, and its bars keep up to five best configurations with width, depth and kernel free and stop the 5D wavelet cells at very different epochs, so they do not isolate the schedule. No comparable real-data sweep is reported, and the discussion describes optimal spec selection as manual. Thirteen of the 21 Kun-v2 cells are also affine, which makes the comparison with linear controls important for attributing the gains.

  3. The evidence of practical benefit is thin: one task, small grids, and small gains over persistence on several real rows. All results come from one task, masked-half completion. The synthetic grids have non-temporal axes of 8 or 16 cells, and several real-data fields are pooled to small grids, such as Moving MNIST from 64 × 64 to 8 × 8 frames. On KITTI, Kun-v2 is worse than repeating the last visible slice, and its gains over that predictor on ERA5 and two ETT rows are modest; the paper itself says these rows measure persistence about as much as forecasting skill. The abstract states the single-task scope, so the concern is significance beyond the tested setting rather than an overclaim.

  4. Several tables and figures are hard to read or report results ambiguously, which makes the evidence hard to check. The ranking table has 21 rows and 15 model columns. It is printed in small type, and the algorithm boxes and cost-plot legends are also small at normal page size. Table 1 labels italic entries as out of memory, although the run description includes guarded, unattempted and unrecorded cases. The radar caption does not say that it summarizes only the synthetic rows, so it can be read as averaging all rows at each rank. This matters because the main claims rest on these tables and figures.

Questions for authors

Clarifications (each could change my assessment):

  1. What tuning budget did Kun-v2 and each baseline get per row, and on which split were Kun-v2's spec, depth and width chosen? Comparable tuning budgets and configuration selection using validation data alone would reduce my concern about the rankings; selection using test errors would leave that concern unresolved even with equal budgets.

  2. In the existing per-seed results, how often is Kun-v2 first and how often is it in the top three, using the same seed for each model comparison? Stable ranks would reduce my concern about seed sensitivity, while leaving the tuning-budget concern open.

  3. Were alternative specs evaluated on the real-data rows, or compared on the synthetic rows with width, depth, kernel and training length fixed, and how were the reported specs selected? Evidence that the choice of spec matters under such controls would answer my second concern; without it, I would credit the cost bound as the main benefit.

  4. Were any of the existing linear-control runs matched to Kun-v2's parameter count on the affine rows? If their errors are comparable, I would give less credit to the spec interface for those gains.

10 comments
AuthorsOpenReview ↗

Reponse to Reviewer 1 (Memg)

We thank the reviewer for the detailed comments and for the concrete controls they asked for. Following these comments, we reran every model under one training protocol, with the same data splits and three seeds, and we now choose every configuration on the mean validation error. We also added three kinds of controls. Table 14 ranks the models without any per-row choice and on each seed alone. Table 12 compares specs at fixed width and depth, on real as well as synthetic rows. For the affine Kun-v2 cells, we compare against a dense linear map and against other specs of the same function class. Finally, we revised the fonts, the number format and the table layout.

The results are as follows. Kun-v2 is first on 15 of the 22 datasets and in the top three on 18. With one fixed configuration per model, it is in the top three as often as FNO, on 15 rows each, and these counts move by at most three rows between seeds. At fixed width and depth, the spec alone changes the error by up to more than two orders of magnitude on the 4D and 5D sine rows, and on the real rows where the time-first spec wins it lowers the error of the default spec by 2–25 %. We give the details below.

AuthorsOpenReview ↗

### W1 / Q1–Q2. "The ranking in Table 2 is not controlled" — tuning budget, split used for selection, seed stability

Protocol. We agree that the submitted version mixed training budgets. In the revision, every learned model on every row is trained under one protocol, which Table 5 in App. F.1 lists in full. All models use AdamW with a constant learning rate of 5×10⁻⁴, a weight decay of 10⁻⁴, gradient clipping at norm 1.0 and a batch size of 16. Each run trains for at most 500 epochs, stops early when the validation error has not improved for 100 epochs, and is limited to 60 minutes. All models on a row see the same 0.7 / 0.1 / 0.2 train / validation / test split, which is redrawn for each of the three seeds (2026, 2016 and 2006). The data are z-scored with statistics of the training fold only, and the reported value is the test masked MSE at the epoch with the lowest validation error.

Data size. No row uses N_sample = 100 any more. Most rows now use about 1000 samples, and the sources with fewer samples use every available one (details in §0.1).

Selection. Every configuration is chosen on the mean validation error over the three seeds, and the test error is only reported. Kun-v2 chooses among its four candidates of Table 9 and, on the eligible 2D–5D rows, among the three-block spec cells of Table 12. FNO, U-NO, Transf. Flatten and Mamba Flatten each choose between two widths (Table 10). Linear, MLP and the third-party models use one configuration each, namely the setting with the lowest validation error found by empirical tuning. The table below lists the options of every model.

ModelConfigurations compared per rowChosen on
Kun-v24 fixed candidates (1 / 3 blocks × MLP / linear kernel, d = 128, default spec)*mean val. MSE, 3 seeds
FNO, U-NO2 widths (d = 8, 32)mean val. MSE, 3 seeds
Transf. Flatten2 widths (d = 128, 256)mean val. MSE, 3 seeds
Mamba Flatten2 widths (d = 128, 512)mean val. MSE, 3 seeds
Linear, MLP1 (tuned empirically, keeping the setting with the lowest validation error)val. MSE, empirical tuning
The other 12 baselines: TimeMachine, Earthformer, PredRNN, SimVP, SwiFT, Mamba-4D, nnU-Net / U-Net-4D, DeepONet, SwinIR, PatchTST, VideoMamba, ConvLSTM1 (official configuration; I/O layers and size tuned empirically, keeping the setting with the lowest validation error)val. MSE, empirical tuning

* On the 2D–5D rows, Kun-v2 can also choose one of the spec cells of Table 12. These cells use the three-block, d = 128 Kun-v2 with the three non-default specs and both kernels, and they are chosen by the same mean validation error.

Size of the option sets. The two sides have option sets of comparable size. Kun-v2 chooses among at most 10 settings per row, namely the 4 candidates of Table 9 and the 6 spec cells of Table 12. It is compared with 18 learned baselines, plus the Zero, Last and Copy floors. Together the baselines offer 22 settings: two widths each for FNO, U-NO, Transf. Flatten and Mamba Flatten, and one configuration for each of the other 14 models.

Size tuning. We fixed the candidate widths of every model beforehand, in one size-tuning round on 4d_sine and 4d_wavelet (Table 8). This round compared six Kun-v2 variants, FNO and U-NO at widths from 8 to 48, and Mamba Flatten and Transf. Flatten at d ∈ {128, 256, 384, 512}, and it also selected on validation error only.

How the Kun-v2 cell is chosen. On every row, the Kun-v2 cell of Table 2 is the candidate of Table 9 with the lowest mean validation error. On the 2D–5D rows, a three-block, d = 128 spec of Table 12 replaces this candidate only if the spec has a lower mean validation error. This happens on 9 of the 22 datasets, which are marked § in Table 2: Lorenz-96, 2D ETTh1, 2D ETTh2, Navier–Stokes, Moving MNIST, 4D sine, 5D sine, ERA5 and KITTI (App. G). To show how the ranking looks without this choice, we added a ranking with fixed configurations, which we describe next. It takes the per-row choice away from every model, Kun-v2 included.

Ranking without any per-row choice (new Table 14). In the new Table 14, every column is one fixed configuration that is used unchanged on every row. The twelve columns are the four Kun-v2 candidates (1 or 3 blocks with the MLP or linear kernel, d = 128) and two widths each of FNO (d = 8, 32), U-NO (d = 8, 32), Transf. Flatten (d = 128, 256) and Mamba Flatten (d = 128, 512). Kun-v2 uses only the default spec here, so no spec cell of Table 12 enters this table. Each row is ranked over these twelve columns only, so no model can choose per row. The other models of Tables 2 and 11, namely Linear, MLP and the 12 third-party models, are not part of this ranking; this is why its counts differ from those of Table 3. Over the 22 datasets, with the mean of three seeds, the counts are:

Fixed configuration1sttop 31st, synthetic (10)1st, real (12)top 3, real (12)
FNO, d = 321115566
Kun-v2, 3 blocks, linear915816
Kun-v2, 1 block, linear814536
Kun-v2, 3 blocks, MLP510324
Kun-v2, 1 block, MLP58323
FNO, d = 859324
U-NO (d = 8 / 32)3 / 67 / 111 / 42 / 23 / 6
Transf. Flatten (d = 128 / 256)5 / 48 / 73 / 32 / 15 / 3
Mamba Flatten (d = 128 / 512)2 / 35 / 81 / 01 / 34 / 5

Without any per-row choice, FNO at d = 32 has slightly more first places than the best fixed Kun-v2, 11 against 9 of the 22 datasets, and the two tie on top-three places with 15 each. The fixed Kun-v2 leads on the synthetic rows, where it is first on 8 of 10 against 5 for FNO. FNO leads on the real rows, where it is first on 6 of 12; the fixed Kun-v2 is first there once with three blocks and three times with one block. On the real rows, both are in the top three equally often, on 6 rows each.

We report this result in App. G as it is. It shows that part of Kun-v2's lead in Table 2 comes from the choice per row among the 4 candidates and the spec cells. Under the same fixed budget, Kun-v2 is in the top three about as often as the strongest baseline, FNO, on 15 rows each. Of the two, only Kun-v2 also runs on the 5D wavelet row, where FNO and U-NO run out of memory.

Comments on the paper

10 comments
AuthorsOpenReview ↗

Response to Reviewers — Kun-v2 ("From 1D to 4D")

We thank the reviewers for their comments. Following these comments, we added datasets and baseline models and reran all experiments of the main tables. Against 18 learned baselines, Kun-v2 is now first on 15 of the 22 datasets and in the top three on 18. Kun-v2 is also cheap: on the 5D sine data, averaged over eight three-block Kun-v2 models (Table 15), Kun-v2 has a test MSE of 0.161 (raw MSE), needs 2.63 s per training epoch and 7.13 GB of peak GPU memory (training data not counted), against 0.241, 12.3 s and 15.8 GB for FNO averaged over its two widths. Kun-v2 is thus 1.5 times more accurate, 4.7 times faster and needs 2.2 times less memory. Against U-NO, averaged over its two widths, Kun-v2 is 3.2 times more accurate, 10 times faster and needs 2.8 times less memory. With Kun-v2, moving to a new dimensionality therefore requires only a change of configuration instead of a new architecture, while the accuracy stays competitive and the computational cost stays low.

The concerns of the three reviewers overlap in many places. We therefore first summarise the revision under five headings, which Section 0 describes in detail, and then answer each reviewer point by point.

  • Data. We chose the dataset sizes again so that every row has enough samples, whereas many rows of the submission had only 100. Most rows now use about 1000 samples, and real sources with fewer samples use every sample they have. The wavelet rows use every distinct member at 1D and 2D (300 and 600 samples) and about a thousand samples at 3D–5D. Details are in §0.1.
  • Fairness. All learned models are now trained under one training protocol on the same data splits, and every configuration is chosen on validation error. Kun-v2 has four basic architectures, four baselines have two settings each, and the other 14 baselines use one configuration each, the one with the lowest validation error after empirical tuning. Kun-v2 is additionally tested with 3 or 4 spec combinations. Details are in §0.2.
  • Results. The main results are in Table 2, and the further baselines are in Table 11. On 9 rows, the Kun-v2 cell of Table 2 comes from the spec comparison of Table 12; these cells are marked §. On 8 of these 9 rows, the better spec reduces the time axis first. Details are in §0.3.
  • Ranking and cost. Details are in §0.4.
    • Overall ranking: Table 3 counts how often each model is first or in the top three. Kun-v2 is in the top three on 18 of the 22 datasets, whereas FNO, the next model, is in the top three on 13.
    • Rank robustness: Table 14 repeats the ranking with one fixed configuration per model, so that no model can choose per row; in this setting Kun-v2 and FNO are in the top three equally often, on 15 rows each.
    • Computational complexity: a new cost experiment from rank 3 to 8 (Fig. 6) shows that the GPU time per element of Kun-v2 stays flat, so its cost grows linearly with E, while that of FNO and U-NO rises with log E.
  • Readability. We enlarged the fonts, and every table now uses one number format (units of 10⁻¹ with three digits). Every cell is the mean over three seeds and, apart from the reference columns of Table 9, also shows its standard deviation. Details are in §0.5.
AuthorsOpenReview ↗

## 0. Summary of changes in the revision

0.1 Data

Dataset sizes. We chose the number of samples again for every row. Most rows now use about 1000 samples, and rows whose source has fewer samples use all of them. The sizes are as follows:

  • All five sine rows (1D–5D), 1D and 2D Burgers, Lorenz-96, Navier–Stokes, GeoLife and Moving MNIST use 1024 samples each.
  • ETTh1 and ETTh2 at 1D and 2D use every available window, which gives 272 samples each. ERA5 uses all of its 1029 samples and KITTI all of its 363.
  • The wavelet pools are set by the synthesis, which must avoid duplicate windows. They contain all distinct members at 1D and 2D (300 and 600 samples) and 1026, 1008 and 990 samples at 3D, 4D and 5D.

Darcy replaced by Lorenz-96. We removed the 2D Darcy row and use Lorenz-96 instead. Darcy flow is a steady-state flow through a porous medium, so it has no time axis: both axes of our Darcy row are spatial, taken from the 64 × 64 FNO release and pooled to 64 × 8. Masking one half of this field and completing it is therefore spatial in-filling rather than prediction along time, which is outside the scope of this paper. Lorenz-96, in contrast, is a chaotic system observed as a time series, with one time axis and one variable axis.

The table below gives both rows under the common protocol, as test masked MSE ×10 with the mean ± std over three seeds. Kun-v2 uses three blocks, d = 128 and the MLP kernel; FNO uses d = 32, the width it selects on validation.

RowKun-v2, default spec v³t³Kun-v2, spec t³v³FNO, d = 32
2D Darcy (dropped)3.01 ± 0.142.39 ± 0.102.59 ± 0.13
2D Lorenz-96 (Table 2)1.37 ± 0.091.07 ± 0.130.31 ± 0.03

On both rows, the spec t³v³, which reduces the leading axis first, has the lower validation error. On Darcy this spec gives Kun-v2 a lower error than FNO, while on Lorenz-96 FNO remains about 3.5 times better.

Effect of the dataset size. We chose the sample count of 1024 empirically. An appendix experiment (Fig. 4) trains the models on 1D sine with pool sizes from N_sample = 64 to 1024, using three seeds and one recipe for all models (at most 500 epochs, early stopping after 50 epochs without improvement, constant learning rate 5×10⁻⁴), and it shows that at 1024 samples the models can fit the dataset. The table below gives Mamba Flatten, Transf. Flatten and Kun-v2 as test masked MSE ×10, averaged over the three seeds; the Zero predictor scores 10.1, and 0.01 stands for ≤ 0.01.

N_sample641282565121024
Mamba Flatten7.593.801.750.300.15
Transf. Flatten5.275.280.950.180.14
Kun-v2 (linear kernel)0.010.010.010.010.01
Kun-v2 (MLP kernel)2.651.860.120.020.02

In the submission, Mamba Flatten scored 1.08 in raw MSE on 1D sine, which is worse than the 0.95 of the Zero predictor. That run used only N_sample = 64 samples, and the table shows that the poor result came mainly from this lack of data. The revision corrects it.

AuthorsOpenReview ↗

### 0.2 Fairness

  1. One training protocol. Every learned model is trained under the same protocol (App. F.1). Each run trains for at most 500 epochs and stops early when the validation error has not improved for 100 epochs. Each run is also limited to 60 minutes, and the learning rate is a constant 5×10⁻⁴. All models on a row share the same 0.7 / 0.1 / 0.2 train / validation / test split, which is redrawn for each seed (2026, 2016 and 2006).

  2. Selection on validation error only. Every configuration is chosen on the mean validation error over three seeds, and the test error plays no part in any choice. The Kun-v2 cell of Table 2 is the best of four fixed candidates, namely 1 or 3 blocks with the MLP or the linear kernel, all at d = 128 (Table 9). FNO, U-NO, Transf. Flatten and Mamba Flatten each choose between two widths (Table 10). The spec comparison of Table 12 can also enter Table 2 by the same rule: when a spec has a lower mean validation error than the Kun-v2 candidate chosen from Table 9, Table 2 reports that spec instead. This happens on 9 rows, which are marked §.

    Every spec cell of Table 12 now uses the three-block, d = 128 model, so every one can be chosen in this way. The one-block, d = 32 model of the submission, whose 5D sine memory of 686 MB the reviewers cite, is no longer used in Table 12.

  3. The spec sweep no longer selects anything. It covers 203 specs on the two synthetic families (the sine field with 128 to 1,024 samples of 64 × 8^(N−1) elements and the wavelet bank with 60 to 900 samples of 64 × 16^(N−1) elements, at ranks N = 2 to 5), 1,260 cells in all and at most 260 per dataset (the submission counted 257 on the 5D sine field), and Fig. 7 is drawn from it. We keep it in App. I only to show how each knob (width, depth, kernel and spec family) changes the error. No cell of Table 2 comes from it.

  4. Three seeds in every cell. Every cell of Table 2 (main results), Table 9 (Kun-v2 candidates), Table 11 (further baselines) and Table 12 (spec comparison) is now the mean over three seeds, with the standard deviation printed next to it. The only exception is the two reference columns of Table 9 (Kun-v2 with one block at d = 32, and the Linear baseline), which are also means over three seeds but are printed without the standard deviation. All values are the masked MSE in units of 10⁻¹ (MSE × 10), shown with three digits.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.