- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Non-Uniform ViT (NUViT), a transformer architecture for Fourier-space data. Instead of tokenizing Fourier images with Cartesian square patches, NUViT evaluates the Fourier transform on a non-uniform polar grid via NUFFT, represents values as amplitude plus sine/cosine phase channels, uses random Fourier feature positional encoding and channel embeddings, and then feeds tokens into a standard transformer encoder.
The paper evaluates NUViT on Fourier-domain natural image classification, synthetic 2D crystal lattice classification, MRI severity classification, cryo-EM image classification, and cryo-EM 3D reconstruction by replacing the cryoDRGN encoder. NUViT often outperforms MLP and ViT baselines, with especially large gains in some cryo-EM classification settings.
Critical strengths
-
The architectural motivation is compelling: standard Cartesian patches are a poor fit for many Fourier-domain signals, and polar/frequency-aware tokenization is a natural inductive bias.
-
The paper tests the idea across several domains rather than only one benchmark. The cryo-EM classification and amortization results are particularly interesting and suggest that the approach may be useful beyond toy tasks.
Critical weaknesses
-
Experimental control is not yet strong enough. The paper needs repeated runs, confidence intervals, and matched-budget baselines. NUViT has similar parameter count to ViT but higher per-epoch time and memory, and the current evidence does not fully separate the benefit of polar tokenization from extra preprocessing, representation choices, hyperparameter tuning, or compute.
-
The baseline set is incomplete for the strength of the claims. A simple log-polar-resampled Fourier image with a standard ViT, a radial-frequency-binned ViT, stronger tuned ViT baselines, and more domain-specific Fourier/cryo-EM baselines would make the result much more convincing. FIT is omitted from cryo-EM due to cost, but the paper still makes broad Fourier-architecture claims.
Questions for authors
-
Are all reported classification and reconstruction numbers averaged over multiple random seeds? If not, how large is run-to-run variation?
-
How does NUViT compare to a standard ViT run on a precomputed log-polar or polar-resampled Fourier image without NUFFT inside the model?
-
Are the baselines matched for token count, input resolution, training time, and hyperparameter tuning budget?
-
How sensitive is performance to R, N_r, N_phi, patch size, NUFFT oversampling, amplitude versus log-amplitude, and channel positional encoding?
-
Can the MRI claim be tested on raw k-space data rather than reconstructed images transformed back to Fourier space?