- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposed KUN-V2, a UNet like archtichure for sequence prediction on any number of dimensions. The authors claims that the model’s compute grows linearly with the number of cells in the input, and the reshaping/reordering never loses information. The model is evaluated in a few benchmarks and top on some of them.
Critical strengths
- the model effort is justified and it's good to have a framework for dimension-agnostic sequence prediction
- code is released
Critical weaknesses
- The theory are trival, eg Thm 2 and Cor 3. They are just used to inflate the paper
- The baselines looks undertrained or not trained properly. Mamba scores 1.08 on 1d sine, worse than predicting all zeros (0.95)
- From appendix E.2, most of the models are Linear. So essentially they are just a big matrix multiplication. This mean all of the introduced arch eg chunking, kernels, encoders decoders, are just a waste of time. Also this points to that the linear baseline is not trained properly.
- The authors runs a lot of ablations and selected the best based on test score, as explained in Appendix G. This is wrong and they should use validation score, especially for these small dataset.
Questions for authors
- did you select the model based on test score?
- did you properly train baselines?
10 comments
### 1. "The theory is trivial (e.g. Theorem 2 and Corollary 3); it only decorates the paper."
We agree that Theorem 2 and Corollary 3 are clear. To the best of our knowledge this isn't yet declared in the literature, thus the novelty can be confirmed. Theorem 2 states that the reshape and permutation induced by a spec is a bijection, and Corollary 3 states that, for any valid spec, the encoder reduces every axis to size 1 and the decoder restores it. We still regard these results as a contribution of the paper, for the following reason.
Patching in high-dimensional space, that is, cutting a field of arbitrary rank into chunks and putting it back together, has not been treated adequately in prior work. Several works show the difficulties that patch operations cause in space. For Mamba, for example, the order in which the patches are scanned has to be designed anew to keep spatial continuity, as in the four-direction cross-scan of VMamba [1] and the zigzag scan of ZigMa [2]. Even among our baselines, the 3D and 4D models (Earthformer, VideoMamba and SwiFT) each need their own patch or window code written for their rank, and none of this code carries over to another rank.
Theorem 2 and Corollary 3 guarantee that any string accepted by the spec parser yields a well-typed, information-preserving encoder–decoder pair at any rank. The patching can therefore be changed freely, without shape checks for each spec or each rank. We keep both results in the main text, with their proofs in the appendix. The main theoretical result remains Theorem 1, the O(E) cost bound, which holds independently of the rank and of the number of chunk layers; the cost experiments (Table 1, Fig. 3 and Fig. 6) test this result.
Fig. 6 in App. H.2 tests the bound up to rank 8 by measuring the GPU time per element of one training step from rank 3 to 8, while E grows 32768 times. For Kun-v2 with one block at d = 32, this time stays between 0.69 and 0.92 ns with the fully joint spec and between 2.7 and 3.2 ns with the axis by axis spec, so the bound holds for both specs and the spec changes only the constant. That of FNO at d = 8 grows 1.5 times from rank 3 to 7 and that of U-NO 3.4 times, since their Fourier transforms make them grow faster than Kun-v2 (see the table in §0.4).
### 2. "Baselines look under-trained: Mamba scores 1.08 on 1D sine, worse than all-zero (0.95)."
We thank the reviewer for pointing this out; the observation is correct for the submitted version. The main cause was the small dataset, since the submitted 1D sine row had only N_sample = 64 samples. In addition, Mamba Flatten ran at a single small width under a different schedule.
The appendix experiment on dataset size (Fig. 4) shows that Mamba Flatten improves steadily as the data grow. This experiment trains nine models on 1D sine with pool sizes from N_sample = 64 to 1024, using three seeds and one recipe for all models (at most 500 epochs, early stopping after 50 epochs without improvement, constant learning rate 5×10⁻⁴). Its results are given in the table of §0.1, as test masked MSE ×10 averaged over the three seeds.
In that table the Zero predictor scores 10.1. At the submitted size of N_sample = 64, Mamba Flatten and Transf. Flatten are far from fitting the field, with 7.59 and 5.27. At N_sample = 1024 both are below 0.2, and in Table 2 Mamba Flatten reaches 0.11 ± 0.08 and Transf. Flatten 0.05 ± 0.06. This shows that the problem in the previous version came mainly from having too few samples to fit the field, and not from a fault in the implementation of the baselines.
## Reponse to Reviewer 3 (D4Z3)
We thank the reviewer for the comments. Following them, we reran every learned model, our Mamba baseline (now called Mamba Flatten) and Linear included, under one training protocol with the same data splits and three seeds. We also enlarged the datasets, for example 1D sine from 64 to 1024 samples, and tuned the widths of Mamba Flatten, Transf. Flatten, FNO and U-NO on validation error. Every configuration is now chosen on the mean validation error.
With these changes, the baselines train properly. Mamba Flatten reaches 0.11 on 1D sine, against 10.1 for the Zero predictor, and Linear reaches ≤ 0.01 on the 1D–3D sine rows. Kun-v2 is first on 15 of the 22 rows and in the top three on 18. Below we also explain why we keep Theorem 2 and Corollary 3 and when the linear kernel is chosen.