acceptodds
Under review as a conference paper at ICLR 2027

CANDI: Continuous Amortized Implicit Neural Representations for Neuroimaging Data

Abstract

Neural activity unfolds as a continuous spatiotemporal field, yet most neuroimaging models operate on discrete sampling geometries imposed by acquisition hardware, such as voxel grids and fixed electrode montages. Implicit neural representations (INRs) provide a natural framework for modeling continuous signals, but conventional INRs are fitted independently to each instance, requiring costly test-time optimization and producing representations that do not transfer across subjects. We introduce CANDI (Continuous Amortized Neuroimaging Decoding via INR), a common continuous-field representation framework that can incorporate modality-specific observation geometry and inductive biases to amortize continuous INR inference across subjects. CANDI uses a selective state-space backbone to encode long spatiotemporal contexts in linear time and introduces spherical harmonic positional encodings for scalp-based imaging motivated by the differential operators underlying spherical spline interpolation. These encodings capture the intrinsic geometry and smoothness of continuous fields observed on two-dimensional scalp manifolds. CANDI outperforms current amortized INR baselines on 4D fMRI spatiotemporal reconstruction and outperforms classical spherical splines on EEG unseen-electrode spatial querying on hold-out sets of up to 87.5% unseen electrodes, while producing representations and reconstructions that retain strong downstream-task utility.

Primary area:
applications to neuroscience & cognitive science
Keywords:
generalizable implicit neural representation, state-space model, neuroimaging

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_bC99OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces CANDI, a generalizable INR framework that combines a bidirectional Mamba encoder with a coordinate-conditioned decoder adapted explicitly from LAGINR. The encoder takes patchified observation tokens interleaved with learnable tokens and outputs instant-specific latent representations. Two modality-specific contributions are introduced. For 4D fMRI, Sequential Locality Bias (SLB, Eq. 4) adds a quadratic penalty on the sequence distance between a query's patch and each latent in the decoder cross-attention, promoting latents close to the query's data token in the serialized sequence. For EEG, spherical harmonic positional encodings (SHPE, Eq. 5) and a field bending-energy penalty (Eq. 7) regularize the surface smoothness of the learned model. The authors quantitatively evaluate generalization to electrode locations withheld during training for EEG, reconstruction quality for fMRI, and downstream utility for both modalities. Note that my expertise is in INRs and computational imaging, but not specifically in fMRI or EEG.

Critical strengths
  • The EEG evaluation protocol meaningfully tests spatial generalization by holding out a fixed 40–87.5% of sensor positions from all supervision, directly testing whether the decoder represents a continuous field rather than a fixed sampling pattern.

  • The gains under EEG sparsity are promising. In Table 1, CANDI improves over ZUNA and SSI on all 3 metrics on both datasets across 3 different scales. It also degrades more gracefully than SSI and ZUNA as test-time context shrinks.

  • The paper offers a principled geometric motivation for EEG inductive biases. The connection between spherical harmonics, the Laplace-Beltrami operator, and spline smoothness is clearly explained and developed, motivating SHPE and the bending loss.

  • CANDI improves reconstruction quality and scales better in memory than the full-attention baselines. At matched patch sizes, CANDI outperforms other baselines across different volume sizes. The proposed method can also use finer patches in settings where LAGINR, TransINR, and Linformer, as implemented, run out of memory.

Critical weaknesses

Major weaknesses:

  • The current ablations do not fully isolate the benefits of the proposed changes over existing GINR components.

    • In Table 5, replacing SHPE while keeping the bending loss results in training collapse, but removing both still produces a trainable model. This suggests an interaction between the encoding, the bending loss, and optimization. Because replacement encoding and its tuning are not specified, the failure is difficult to interpret as positive evidence for the advantage of SHPE. Another potentially useful ablation would be to replace Mamba with a Transformer encoder while keeping all the other remaining components.
    • Table 6 only compares SLB against no bias. It does not show whether the gain depends on Mamba's serialization or whether a spatial-distance bias, or the same sequence-distance bias with a different encoder, would help equally.
  • The fMRI task is apparently autoencoding rather than full reconstruction, and the arbitrary-resolution claim is untested. The intended application utility of the fMRI experiments is not entirely clear.

    • In computational imaging, “fMRI reconstruction” usually means recovering images from raw (for example, k-space) measurements through a forward model. Here, the inputs are already-reconstructed HCP volumes in MNI 2mm space, and the model encodes a volume and decodes that same volume with no forward model involved. Calling this “4D fMRI spatiotemporal reconstruction” (Line 24–25) or “4D volume reconstruction” (Line 428; Table 4) is therefore misleading; “autoencoding” or “representation fitting” would be more accurate.
    • Separately, Figure 1(e) describes “fMRI full-volume reconstruction at arbitrary sizes” (243→64324^3 \to 64^3), which suggests encoding at one resolution and decoding at another. However, Appendix A.2.2 resizes each frame by trilinear interpolation to the resolution required by each experiment, so each column of Table 4 appears to be a separately trained model whose input and output share one resolution. If so, decoding at a resolution other than the input’s is never evaluated.

Minor weaknesses:

  • Statistical reporting of quantitative results is either unclear or missing. Table 1 neither defines whether ±\pm denotes standard deviation nor whether variability is computed over subjects, windows, layouts, or runs. The remaining results (Tables 2-6: fMRI reconstruction, both downstream experiments, and both ablations) do not report any variability metrics.

  • Implementation details for EEG baselines are missing. The paper describes CANDI's training in detail but does not comparably specify the ZUNA checkpoint/version, its inference settings, whether it is pretrained, retrained, or fine-tuned. Similarly, the SSI spline order, number of Legendre terms, and numerical interpolation are not reported. These choices matter for interpreting Table 1, particularly because SSI outperforms ZUNA in several settings, and would help assess whether configuration differences affect the comparison results.

  • Section 3.2 attributes the cross-attention decoder to LAGINR, so the coordinate-to-latent cross-attention itself is not new and the contribution here is the SLB term added to its attention softmax (Eq. 4). However, Figure 2 places the contribution star on the entire “Sequential Locality-Biased Cross-Attention” block. Please mark only SLB as the contribution in Figure 2 and its caption.

  • In Eq. 4, cc in the expression for bc,lb_{c,l} seems to refer to the query's data token index, which overloads cc as the query coordinate. Please define ll and cc explicitly and report how α\alpha is selected.

  • Figure 2 and Section 3.2 present spatiotemporally decoupled positional encodings as one of the paper's contributions. However, separately encoding spatial and temporal coordinates with different frequency ranges has precedent in dynamic neural fields. For example, D-NeRF (Pumarola et al., 2021) [1] uses (L=10) for spatial coordinates and (L=4) for time. Please cite relevant prior works and clarify what is novel about CANDI’s use of this design.

  • A relevant and recent EEG baseline that supports querying unseen coordinates is missing. Lines 142-144 acknowledge that Liu et al. (2026a) and ZUNA can both query new positions, but the experiments compare only against SSI and ZUNA. The argument for excluding fixed-montage methods in Lines 357-360 does not explain this omission.

  • Some numerical results need clarification:

    • In Table 4 at volume size 483×248^3 \times 2, Linformer and TransINR (both with patch size 43×24^3\times 2) report exactly identical values of 5.20×10−45.20\times10^{-4} and 0.21470.2147. I'd like to confirm if this is a typo or the actual result for both.
    • The full model in Table 5 (first row with NMSE 0.3683, SNR 4.34, and PCC 0.8190) matches no row of Table 1 and the ablation's scale and experimental setup are also not specified.
  • Under identical normalization, SNR\mathrm{SNR} essentially equals −10⋅log⁡10(NMSE)-10\cdot \log_{10}(\mathrm{NMSE}), so having both metrics is a bit redundant (though perhaps convenient for readers).

  • Writing issues:

    • Please make sure you are using \citep() appropriately for parenthetical citations. For example, it should be "(Gu et al., 2022)" on Line 160, "(Perrin et al., 1989)" on Line 168, "(Wahba, 1981)" on Line 171, etc. In general, if you would not say the citation out loud when reading the sentence, \citep is appropriate.
    • Typos include "low-passnfiltering" in Lines 301-302, "metrics(Table 5)" in Lines 464-465, and "oto facilitate" in Line 813.
    • Lines 466-467 read as if removing both SHPE and the bending loss collapses training, but Table 5 shows that configuration trains while removing SHPE alone does not. Please clarify.

[1] Pumarola, Albert, et al. D-NeRF: Neural Radiance Fields for Dynamic Scenes. CVPR 2021.

Questions for authors
  • Algorithm 1 computes a single loss ℓ(x^S,xS)\ell(\hat x_S, x_S) over all seen electrodes S=C∪TS = C \cup T while Section 3.4 and Eq. 8 weight the target and reconstruction terms separately with wreconw_{\text{recon}}. Can the authors clarify this inconsistency and update Algorithm 1 to match Eq. 8?

  • Can the authors provide implementation details for the EEG baselines (SSI and ZUNA) and fMRI baselines (LAGINR, Linformer, TransINR, and SCENT)? Can the authors also clarify the experimental settings of the two ablations (Tables 5 and 6)?

  • Can the authors report uncertainty estimates for all experiments (main experiments, downstream experiments, and ablations) and clearly identify the source of variability (e.g. seeds, layouts, test subjects, runs)?

  • For Table 5, can the authors also ablate the Mamba encoder (e.g., replace it with a different Transformer encoder)? For Table 6, can the authors compare SLB with a spatial distance bias and with the same bias applied to a different encoder?

  • For the Table 4 setting, can the authors report encoder and decoder costs separately, full-volume inference latency, and the attention implementation used for the Transformer baselines (e.g., whether a memory-efficient kernel like FlashAttention was used)?

  • Mamba is order-sensitive, but EEG positions form an unordered set and the fMRI scan order is unspecified. How are tokens ordered in each case, how sensitive are the results to that order, and does it affect the interpretation of SLB?

  • Can the authors clarify the fMRI resolution setup in Table 4: at which volume and patch size was each model trained, and is it evaluated at that same size (representation) or at a different one (upsampling or downsampling)? If cross-resolution decoding is supported, can the authors evaluate it (for example, encode 24324^3 and decode 48348^3) against interpolation baselines such as trilinear, cubic, or sinc upsampling?

7 comments
AuthorsOpenReview ↗

Extended Ablations (EEG)

1. Extended ablations to isolate CANDI mechanisms

a) Strengthening the EEG ablations (Table 5)

For Table 5, can the authors also ablate the Mamba encoder (e.g., replace it with a different Transformer encoder)? ... In Table 5, replacing SHPE while keeping the bending loss results in training collapse, but removing both still produces a trainable model. This suggests an interaction between the encoding, the bending loss, and optimization. Because replacement encoding and its tuning are not specified, the failure is difficult to interpret as positive evidence for the advantage of SHPE. Another potentially useful ablation would be to replace Mamba with a Transformer encoder while keeping all the other remaining components.

Clarifying the positional encoding ablation. The alternative encoding in Table 5 is the standard Fourier positional encoding as formulated in Eq. 3, which is also used for all fMRI experiments. When retaining the bending-energy regularizer without SHPE, this configuration exhibits training instability characterized by a sharp increase in the smoothing loss early in training, which the model does not recover from. We tested multiple Fourier bandwidths (from [128, 32, 8] to [2, 1, 0.5]), peak learning rates (from 5×10−45\times10^{-4} to 1×10−51\times10^{-5}), and smoothing loss weights (from 0.01s0.01s to 0.00010.0001), but did not obtain stable training. We have not conducted rigorous interpretability tests on exactly why this interaction occurs, but one possible explanation is that SHPE is naturally aligned with the spherical geometry underlying the bending energy regularizer, whereas standard Fourier features do not explicitly encode this geometry. We agree that this result reflects an interaction between encoding and regularization, rather than independently establishing SHPE's superiority, and have clarified this interpretation in the manuscript.

Additional Transformer ablation. Following the reviewer's suggestion, we replace Mamba with a Transformer encoder while retaining SHPE, bending regularization, the frequency curriculum, and the coordinate-query decoder. We use the same optimization settings and training budget without independently tuning the Transformer. The geometric components give similar NMSE and PCC with Mamba (0.3683, 0.8190) and Transformer (0.3684, 0.8136), supporting their usefulness across encoder types. For experimental details, see Section 4.4 and Table 5 of the revised manuscript.

Rebuttal Table 1: Ablation of EEG-specific components on SEED-V unseen-electrode reconstruction. DNT denotes that it does not train, with PCC approximately 0.00.

EncoderSHPEBending LossCurriculumRecon LossNMSE ↓SNR ↑PCC ↑
Mamba✓✓✓✓0.36834.340.8190
Mamba✗✓✓✓DNTDNTDNT
Mamba✗✗✓✓1.2141−0.840.6565
Mamba✓✗✓✓0.42633.700.7549
Mamba✓✓✗✓DNTDNTDNT
Mamba✓✓✓✗0.42363.730.7938
Transformer✓✓✓✓0.36844.340.8136
Transformer✗✗✓✓0.94400.250.6001

Removing SHPE together with the bending loss worsens both encoders, while Fourier PE with bending loss does not train stably in the tested settings. See Section 4.4 and Table 5 for the full ablation.

AuthorsOpenReview ↗

Extended Ablations (SLB)

b) Strengthening the SLB ablations (Table 6)

For Table 6, can the authors compare SLB with a spatial distance bias and with the same bias applied to a different encoder? ... Table 6 only compares SLB against no bias. It does not show whether the gain depends on Mamba's serialization or whether a spatial-distance bias, or the same sequence-distance bias with a different encoder, would help equally.

We thank the reviewer for these suggestions that improve the isolation of SLB and its dependence pn architectural properties. To investigate whether SLB's improvement depends on the locality formulation, encoder architecture, or token serialization, we conduct additional ablations using the same setting as Table 6 at 256 learnable tokens. For details, see Appendix E.1.1 (Table 28) and E.1.2 (Table 29).

Spatial bias and encoder comparison. We compare Mamba + SLB against Mamba with a spatial-distance bias and Transformer + SLB to investigate the roles of the locality formulation and encoder architecture. Under the tested configurations, Mamba + SLB yields lower MSE than Mamba + spatial bias (1.6384 vs. 1.9927, both ×10−4\times10^{-4}), while SLB does not improve the Transformer (2.7248 vs. 2.5194, both ×10−4\times10^{-4}). We restrict this conclusion to the evaluated bias formulation; see Appendix E.1.1, Table 28. Values in (a) are mean ± reported SEM.

(a) Bias and encoder ablations

EncoderBiasMSE ↓ (×10⁻⁴)
MambaNone1.8561 ± 0.0401
MambaSLB1.6384 ± 0.0358
MambaSpatial1.9927 ± 0.0554
TransformerNone2.5194 ± 0.0816
TransformerSLB2.7248 ± 0.0812

Mamba is order-sensitive, but EEG positions form an unordered set and the fMRI scan order is unspecified. How are tokens ordered in each case, how sensitive are the results to that order, and does it affect the interpretation of SLB?

As explained in Section 3.2, SLB is used only for fMRI. For EEG, each sequence token is a temporal patch containing the entire electrode set; Mamba scans temporal patches, not electrodes individually (Appendix B.1.1). For fMRI, the default patch order is HWDT, expanding the leftmost dimension fastest to the rightmost slowest (Appendix E.1.2).

Serialization-order comparison. We evaluate SLB and no bias under HWDT, WDHT, and DHWT orderings. SLB improves MSE by 9.55–10.99% across these three single-seed comparisons; this is not a permutation-invariance claim. See Appendix E.1.2, Table 29. (b) uses a single seed.

(b) Serialization-order ablations

OrderingSLBNo bias% gain
HWDT1.70 × 10⁻⁴1.91 × 10⁻⁴10.99%
WDHT1.80 × 10⁻⁴1.99 × 10⁻⁴9.55%
DHWT1.73 × 10⁻⁴1.93 × 10⁻⁴10.36%

These comparisons support an SLB benefit for the tested Mamba configurations, without establishing that all spatial-distance biases are inferior. Further setup and interpretation are in Appendix E.1.1–E.1.2, Tables 28–29.

Repeated-seed evaluation. We repeat the SLB versus no-bias comparison using three matched seeds at 64, 128, and 256 learnable tokens. We report the mean paired MSE improvement and its 95% confidence interval below; individual seeds and the complete statistical analysis are in Appendix C.6, Table 26.

Rebuttal Table 2: SLB ablation across three matched training seeds. MSE (×10⁻⁴) is reported as mean ± SD. Improvements denote paired-seed relative MSE reductions (mean ± SD), with 95% confidence intervals. See Appendix C.6, Table 26 for individual seed results and detailed statistical analysis.

TokensNo SLB MSESLB MSE% Improvement95% CI
642.018 ± 0.0681.935 ± 0.1104.12% ± 4.42%[−6.87%, 15.10%]
1281.951 ± 0.0211.775 ± 0.0529.02% ± 3.24%[0.98%, 17.06%]
2561.906 ± 0.0521.702 ± 0.06110.65% ± 2.80%[3.69%, 17.61%]

SLB improves MSE for all nine matched-seed comparisons, with 95% confidence intervals for mean improvement excluding zero at 128 and 256 tokens, but not at 64 tokens (Appendix C.6, Table 26).

AuthorsOpenReview ↗

Clarifying Scope

2. Clarifying the fMRI reconstruction task and experimental scope

The fMRI task is apparently autoencoding rather than full reconstruction, and the arbitrary-resolution claim is untested. The intended application utility of the fMRI experiments is not entirely clear.

Definition of reconstruction. We thank the reviewer for highlighting this ambiguity. We use "reconstruction" in the image-domain sense established in the INR literature [1, 2], where a learned implicit representation reproduces an observed signal. This differs from the computational-imaging definition involving recovery from raw measurements through an acquisition forward model. We agree that our fMRI task is a form of autoencoding: CANDI operates on preprocessed HCP volumes, not raw k-space measurements, and does not address the MRI acquisition inverse problem. We have added an explicit definition of our use of "reconstruction" in Section 1 [068–070]:

"Throughout this paper we use 'reconstruction' in the image-domain sense common in INR and GINR literature (Lee et al., 2023; Sitzmann et al., 2020; Chen and Wang, 2022), referring to reproducing an observed signal through a learned implicit representation, rather than recovering images from raw acquisition measurements."

Experimental objectives and scope. Our fMRI experiments evaluate whether CANDI can learn compact representations that faithfully preserve high-dimensional spatiotemporal signals. Same-resolution reconstruction error measures signal preservation through the encoder-decoder architecture and allows us to assess scalability across volume resolutions and latent-token budgets. We complement this with downstream sex classification and age regression to evaluate whether the learned representations retain information useful beyond voxel-level reconstruction. We prioritize these tasks over interpolation because established methods provide strong baselines on regularly sampled fMRI grids [3], while the slow BOLD response limits the physiological information recoverable through naive temporal interpolation [4, 5]. More nuanced forms of temporal interpolation like slice-timing correction are outside of the scope of this paper. Thus, reconstruction serves as a representation-learning benchmark rather than an MRI acquisition inverse problem; we do not claim arbitrary-resolution upsampling/downsampling or temporal extrapolation for fMRI.

Can the authors clarify the fMRI resolution setup in Table 4: at which volume and patch size was each model trained, and is it evaluated at that same size (representation) or at a different one (upsampling or downsampling)? ... Separately, Figure 1(e) describes "fMRI full-volume reconstruction at arbitrary sizes", which suggests encoding at one resolution and decoding at another.

Manuscript clarifications. We make several changes to the manuscript to make the claims from the experimental objectives and scope above more explicit and avoid confusion. We acknowledge that the original Figure 1(e) could suggest cross-resolution reconstruction for fMRI. We have corrected this figure by explicitly noting that the input and output dimensions of reconstruction are the same, by resizing the physical squares representing input and output to be the same, and removing the "arbitrary size" language. We have also revised Section 4.3 [411-413] to specify that Table 4 evaluates separately trained models at the same input and output resolution, not upsampling or downsampling; no cross-resolution comparison is claimed.

[1] Lee et al. (2023). "Locality-Aware Generalizable Implicit Neural Representation." NeurIPS 2023.

[2] Chen and Wang (2022). "Transformers as Meta-Learners for Implicit Neural Representations." ECCV 2022.

[3] Lehmann, T. M., Gönner, C., and Spitzer, K. (1999). "Survey: Interpolation methods in medical image processing." IEEE Transactions on Medical Imaging, 18(11), 1049–1075.

[4] Boynton, G. M., Engel, S. A., Glover, G. H., and Heeger, D. J. (1996). "Linear systems analysis of functional magnetic resonance imaging in human V1." Journal of Neuroscience, 16(13), 4207–4221.

[5] Logothetis, N. K. (2008). "What we can do and what we cannot do with fMRI." Nature, 453(7197), 869–878.

Reviewer_6GY8OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CANDI, a framework for amortized implicit neural representations (INR) of neuroimaging signals. The model employs a bidirectional Mamba encoder and interleaved learnable tokens to extract sample-specific latent representations, subsequently using a coordinate-conditioned decoder to predict signal values, thereby avoiding the need for per-sample optimization of the INR. For fMRI, the authors propose Sequential Locality Bias (SLB), which leverages the relative distance between query locations and latent tokens within the sequence to guide cross-attention. For EEG, the authors introduce Spherical Harmonic Positional Encoding (SHPE), spherical bending energy regularization, and frequency-based curriculum training to accommodate the spatial geometry and smoothness of scalp potentials. Experiments cover two EEG datasets and the HCP fMRI dataset. EEG experiments utilize both subject-disjoint splits and target electrodes held out entirely during training to validate spatial interpolation capabilities under sparse observations; fMRI experiments evaluate reconstruction quality and computational scalability across varying resolutions and patch sizes. Furthermore, the paper assesses the downstream utility of the reconstructed signals and latent representations through EEG motor task classification and fMRI-based age and gender prediction.

Critical strengths

1.CANDI demonstrates rigorous EEG evaluation and distinct advantages under sparse conditions. Target electrodes are entirely excluded during training and evaluated on unseen subjects; CANDI outperforms SSI and ZUNA in highly sparse settings across both datasets, strongly validating its spatial query capabilities.

2.It balances computational scalability with representation utility. CANDI successfully completes training and maintains high reconstruction accuracy even in settings where multiple Transformer baselines suffer from out-of-memory errors, and reconstruction-based pre-training significantly enhances downstream prediction performance.

Critical weaknesses

1.The manuscript contains several internal inconsistencies that hinder an understanding of the methodology and its reproducibility: the main text states that the spherical harmonic order increases layer by layer, yet Table 7 reports [3, 2, 1]; the number of EEG training epochs is listed as 300 in Table 7, whereas the total across curriculum stages in Table 8 is 360; and the reported number of HCP training samples differs between the main text and the appendix. Furthermore, Algorithm 1 fails to explain how the loss is calculated when the target set is empty, and key parameters and the scanning order for SLB lack clear specification. These issues make it difficult for readers to determine the actual configuration used to generate the reported results.

2.The evidence for downstream utility remains limited. The EEG classification experiments use a run-level split without subject-level exclusivity (Appendix A.1.1). Although this setting assesses whether reconstructed signals preserve task-relevant information, it does not establish cross-subject generalization. In Table 3, CANDI improves over SSI by only 0.84 and 1.00 percentage points at the 4× and 8× downsampling scales, respectively, and performs slightly worse at 2×. Given that the classifier is trained with a single random seed (Table 10), the robustness of these modest gains remains unclear. The downstream fMRI results likewise rely on a single seed. Moreover, the absence of a controlled comparison using the same encoder backbone with different pretraining objectives makes it difficult to distinguish the benefits of continuous coordinate decoding from those of reconstruction pretraining more generally.

3.The manuscript contains several inconsistencies in its references and mathematical notation. Gu & Dao (2024a, 2024b) share the same title and arXiv identifier, while Tang et al. (2023a, 2023b) have identical bibliographic details. LAGINR and TransINR are also listed separately under their preprint and conference versions, with inconsistent author-name ordering for LAGINR. These entries should be consolidated where appropriate, and author metadata and acronym capitalization should be standardized. The notation also requires correction: several sets indexed from 0 to n contain n+1 elements despite being described as having n; the latent-set expression on page 5, line 227 uses i instead of j as the running index; c denotes both a coordinate vector and a sequence index in Equation (4); and L denotes both sequence length and maximum spherical-harmonic degree. These issues reduce the precision and clarity of the technical presentation.

4.The manuscript would benefit from careful proofreading to correct typographical errors, grammatical mistakes, and ambiguous wording. Examples include “low-passnfiltering” (p. 6, line 302), “setting oto facilitate reproducibility” (p. 16, lines 812–813), and “The dataset comprises of” (p. 7, line 349), which should read “low-pass filtering,” “settings to facilitate reproducibility,” and “The dataset comprises,” respectively. The phrase “a code supplemental” (p. 10, line 521) should also be replaced with “supplementary code.” In addition, “its sequence position” (p. 5, lines 242–243) should explicitly refer to the corresponding input patch, and the definition of DNT in Table 5 should specify whether training fails through divergence, non-convergence, or another failure mode.

Questions for authors

1.What positional encoding replaces SHPE when it is removed? Were the frequency range, initialization, learning rate, and smoothness regularization strength independently tuned for the alternative encoding? Does training still fail after these adjustments? Please also specify the downsampling scale, context proportion, and how results were aggregated across electrode layouts in Table 5.

2.Which ZUNA checkpoint was used, and was it fine-tuned? How were signal scaling, EEG referencing, and inference settings handled? For SSI, what smoothing parameters and spherical-harmonic expansion settings were used, and were they selected using the validation set? These details are necessary to assess the fairness of the comparisons.

3.Does “±” in Table 1 denote standard deviation, standard error, or a confidence interval? Is the statistical unit a subject, a signal segment, or an electrode layout? Can the main gains in Tables 2, 3, and 6 be validated through repeated training runs or subject-level paired statistical analyses? In particular, is the approximately one-percentage-point improvement in downstream EEG classification robust?

4.With the same Mamba encoder, training data, and computational budget, would pretraining with a conventional discrete reconstruction decoder achieve downstream performance comparable to CANDI? If such experiments are available, they would help isolate the gains attributable to continuous-field modeling.

5.Please clarify the order in which spherical-harmonic degrees are introduced into the decoder and whether EEG training lasts 300 or 360 epochs. Please also explain how the target loss in Equation (8) is computed when Algorithm 1 samples C=S, leaving the target set empty, and specify the scan order and α setting used for SLB.

4 comments
AuthorsOpenReview ↗

Internal inconsistencies, notation, and proofreading

1. Internal inconsistencies, notation, and proofreading

a) Configuration inconsistencies

"the main text states that the spherical harmonic order increases layer by layer, yet Table 7 reports [3, 2, 1]"

We thank the reviewer for catching this. The decoder applies degrees in ascending order (layer 1 uses degree 1, the final layer uses degree 3), as stated in Section 3.2. Table 7 listed the schedule in the order it is stored in our configuration file, which is reversed relative to layer order; the same applied to the Fourier bandwidth schedules. All hierarchical schedules in Table 7 and Section 4.4 are now written in layer order (ℓmax=[1,2,3]\ell_{\text{max}} = [1, 2, 3], σℓ,t=[64,256,1024]\sigma_{\ell,t} = [64, 256, 1024], σℓ,s=σℓ,t=[32,128]\sigma_{\ell,s} = \sigma_{\ell,t} = [32, 128] for fMRI), and the Table 7 caption states that schedules are listed from the first (coarsest) to the last (finest) decoder layer.

"the number of EEG training epochs is listed as 300 in Table 7, whereas the total across curriculum stages in Table 8 is 360"

EEG training lasts 300 epochs. The per-stage schedule in Table 8 is [40, 40, 40, 40, 80, 30, 30], which sums to 300 (4×40+80+2×304\times40 + 80 + 2\times30) and matches Table 7.

"the reported number of HCP training samples differs between the main text and the appendix"

The correct count is 5,178 training volumes (863 training subjects × 6 chunks), as reported in Appendix A.2.1. The main text (Section 4.1) has been corrected to match.

"Algorithm 1 fails to explain how the loss is calculated when the target set is empty"

We thank the reviewer for pointing out the ambiguities in the algorithm's loss calculation. Algorithm 1 has been changed to compute each term of Eq. 8 explicitly and shows that Ltarget=0\mathcal{L}_{\text{target}} = 0 when T=∅T = \emptyset (i.e., when C=SC = S), so the loss reduces to the weighted reconstruction and smoothness terms. Section 3.3 states this case in the text.

"key parameters and the scanning order for SLB lack clear specification"

SLB uses α=5.0\alpha = 5.0, selected by sweeping [0.5,20][0.5, 20]; this is stated in Section 3.2 and listed in Table 7. The default fMRI scan order is HWDT (leftmost dimension varies fastest). Appendix E.1.2 (Table 29) evaluates two alternative orders (WDHT, DHWT), and SLB improves MSE by 9.55–10.99% under all three.

b) References and notation

"Gu & Dao (2024a, 2024b) share the same title and arXiv identifier, while Tang et al. (2023a, 2023b) have identical bibliographic details. LAGINR and TransINR are also listed separately under their preprint and conference versions"

We have consolidated duplicate entries to a single version of each work (conference versions where available), corrected the LAGINR author ordering, and standardized acronym capitalization in reference titles.

"c denotes both a coordinate vector and a sequence index in Equation (4)"

Eq. 4 now uses kck_c for the sequence index of the data patch pcp_c containing the query coordinate cc, which remains the coordinate vector.

"several sets indexed from 0 to n contain n+1 elements despite being described as having n; the latent-set expression on page 5, line 227 uses i instead of j as the running index ... and L denotes both sequence length and maximum spherical-harmonic degree"

All sets in Sections 3.1–3.2 are now indexed from 1 to nn, the latent set uses jj as its running index, and the maximum spherical-harmonic degree is denoted ℓmax⁡\ell_{\max} (Eq. 5, Section 3.2, Table 7), leaving LL for sequence lengths.

c) Proofreading

"low-passnfiltering" ... "setting oto facilitate reproducibility" ... "The dataset comprises of" ... "a code supplemental"

All four are corrected ("low-pass filtering," "settings to facilitate reproducibility," "The dataset contains," "supplementary code").

"'its sequence position' (p. 5, lines 242–243) should explicitly refer to the corresponding input patch"

Section 3.2 now refers to the sequence index kck_c of the input patch pcp_c containing the query.

"the definition of DNT in Table 5 should specify whether training fails through divergence, non-convergence, or another failure mode"

DNT denotes divergence for the no SPHE only setting, and convergence to the mean temporal signal for the no curriculum setting: for the former, the smoothing loss spikes within the first 40–50 epochs and does not recover, and for the latter there is no spike but SNR and PCC never improve; PCC on unseen electrodes falls to approximately zero fpr both. Section 4.4 now states this.

AuthorsOpenReview ↗

Statistical Significance

2. Statistical analysis and robustness

"Does '±' in Table 1 denote standard deviation, standard error, or a confidence interval? Is the statistical unit a subject, a signal segment, or an electrode layout?"

"±" in Table 1 is the standard error of the mean over 4-second signal segments, following Liu et al. (2026b); the caption now states this. Because segments are not independent, Appendix C.1 (Table 16) adds paired subject-level 95% bootstrap confidence intervals on NMSE improvement, averaging the four layouts within each subject. On PhysioNet, five of six comparisons exclude zero (the exception is SSI at 2×). On SEED-V, all six improvements are positive, but with only three test subjects we treat them as descriptive.

"Can the main gains in Tables 2, 3, and 6 be validated through repeated training runs or subject-level paired statistical analyses?"

We add a new Appendix C with paired statistical tests for each table:

  1. Table 2 (fMRI downstream), Appendix C.3, Tables 19–21: Paired DeLong tests show both CANDI configurations achieve significantly higher sex-classification AUC than SwiFT and Omni-fMRI after Holm correction. Accuracy differences are not significant (McNemar). For age regression, the CANDI linear probe significantly outperforms Omni-fMRI (pHolm=0.012p_{\text{Holm}} = 0.012), but not SwiFT.
  2. Table 3 (EEG downstream), Appendix C.4, Table 23: Subject-level paired bootstrap over 81 subjects.
  3. Table 6 (SLB ablation), Appendix C.6, Table 26: Three matched training seeds. SLB lowers MSE in all nine paired comparisons, with 95% confidence intervals on the mean improvement excluding zero at 128 and 256 tokens (9.02% and 10.65%), but not at 64 tokens.
  4. Table 4 and Table 5: Subject-level SEM for Table 4 is in Appendix C.2.1 (Table 18), with three-seed reproducibility for representative configurations in Appendix C.2 (Table 17). Subject-level SEM for Table 5 is in Appendix C.5 (Table 25).

"In particular, is the approximately one-percentage-point improvement in downstream EEG classification robust?"

No, the paired subject-level analysis (Appendix C.4, Table 23) shows no statistically significant difference between CANDI and SSI at any scale (+1.00 and +0.93 points at 4× and 8×, with confidence intervals including zero). CANDI significantly outperforms the low-resolution input at every scale (+6.90, +10.38, and +9.90 points). We have revised Section 4.2 accordingly: CANDI preserves downstream class-relevant structure at a level comparable to SSI, we do not claim that it does any better at classification.

"Given that the classifier is trained with a single random seed (Table 10), the robustness of these modest gains remains unclear. The downstream fMRI results likewise rely on a single seed."

We agree with the reviewer. The paired tests above quantify variability across test subjects for fixed trained models, not across training seeds, and we state this limitation in Appendix C.4.

"The EEG classification experiments use a run-level split without subject-level exclusivity (Appendix A.1.1). Although this setting assesses whether reconstructed signals preserve task-relevant information, it does not establish cross-subject generalization."

We agree with this characterization. The experiment tests whether reconstructions retain task-relevant information, not cross-subject generalization. We use a run-level split because current classifiers do not give reliable results on subject-disjoint PhysioNet-MM/MI classification, and Appendices A.1.1 and B.1.4 state that these results should be interpreted accordingly.

AuthorsOpenReview ↗

Additional Ablations and Baseline Training Details

3. Additional ablations and clarification of ablation setting

a) Isolating continuous coordinate decoding

"the absence of a controlled comparison using the same encoder backbone with different pretraining objectives makes it difficult to distinguish the benefits of continuous coordinate decoding from those of reconstruction pretraining more generally" ... "With the same Mamba encoder, training data, and computational budget, would pretraining with a conventional discrete reconstruction decoder achieve downstream performance comparable to CANDI?"

Following the reviewer's suggestion, we train CANDI-Patch (Appendix E.3), which keeps the Mamba encoder, learnable-token bottleneck, SLB, training data, and reconstruction objective, but replaces the voxel-wise INR decoder with a patch-wise decoder. A FiLM-modulated MLP predicts all voxels in a patch jointly from one patch-level query.

Rebuttal Table 1: Comparison of CANDI and a discrete patch-decoding baseline on downstream HCP tasks (Appendix E.3, Table 31).

ModelSex Acc ↑Sex AUC ↑Age MSE ↓Age MAE ↓
CANDI (scratch)0.86240.911515.60973.3498
CANDI (linear probe)0.88070.98998.83722.4966
CANDI (pre-train + fine-tuned)0.94500.983710.27692.6934
CANDI-Patch (linear probe)0.90930.99119.06812.5271
CANDI-Patch (pre-train + fine-tuned)0.85900.954914.16933.0140

Paired tests on the same 109 subjects (Appendix E.3, Table 32) show no significant difference between the two decoders under linear probing. After fine-tuning, coordinate-wise CANDI achieves significantly higher sex AUC (pHolm=0.0004p_{\text{Holm}} = 0.0004); its accuracy and age-MAE advantages are not significant (pHolm=0.062p_{\text{Holm}} = 0.062 and 0.1560.156). Linear probing mostly gives better accuracy as well. Reconstruction pretraining clearly helps over training from scratch. Continuous coordinate decoding imposes no downstream penalty relative to discrete decoding, but we do not claim that it provides a consistent downstream gain; its main advantages are coordinate queryability and the unseen-electrode setting for EEG.

b) EEG ablation setting (Table 5)

"What positional encoding replaces SHPE when it is removed? Were the frequency range, initialization, learning rate, and smoothness regularization strength independently tuned for the alternative encoding? Does training still fail after these adjustments?"

SHPE is replaced with the standard Fourier positional encoding of Eq. 3, the same encoding used for all fMRI experiments. With the bending loss retained, we swept Fourier bandwidth schedules from [8, 32, 128] to [0.5, 1, 2], peak learning rates from 5×10−45\times10^{-4} to 1×10−51\times10^{-5}, and smoothing weights from 0.01 to 0.0001. Training still diverged in every setting (Section 4.4). We interpret this as an interaction between the encoding and the bending regularizer rather than as independent evidence for SHPE. The comparison with the bending loss removed isolates the encoding directly: SHPE reaches 0.4263 NMSE versus 1.2141 for Fourier PE.

"Please also specify the downsampling scale, context proportion, and how results were aggregated across electrode layouts in Table 5."

Table 5 uses SEED-V at 4× downsampling with pc=1.0p_c = 1.0, evaluated on a single layout (layout 2, Appendix B.2, Table 15), so no aggregation across layouts is performed. We also add a Transformer-encoder variant: with SHPE, bending loss, and curriculum, it matches the Mamba model (0.3684 vs. 0.3683 NMSE).

4. Baseline training details for EEG and fMRI

"Which ZUNA checkpoint was used, and was it fine-tuned? How were signal scaling, EEG referencing, and inference settings handled?"

We use the released Zyphra/ZUNA1.1 checkpoint (zuna package v1.1.6), frozen and not fine-tuned on either dataset (Appendix B.1.3, Table 10). Inference uses 50 diffusion steps and guidance scale 1.0, with signals resampled to ZUNA's internal 256 Hz and back. ZUNA's additional high-pass filter is disabled. Signals use dataset-specific scaling that is inverted after inference. ZUNA's native pipeline normalizes channels using global statistics that include unseen electrodes, which leaks held-out information. We therefore initialize each unseen channel with the signal of its nearest observed electrode before inference.

"For SSI, what smoothing parameters and spherical-harmonic expansion settings were used, and were they selected using the validation set?"

SSI uses spline order m=4m = 4, regularization λ=10−5\lambda = 10^{-5}, and 50 Legendre terms on unit-sphere-projected electrode coordinates. These settings are fixed across datasets, scales, and layouts and were not tuned on the validation set (Appendix B.1.3, Table 10).

fMRI baseline architectures and optimization settings are reported in Appendix B.1.2, Table 9.

Reviewer_6GY8OpenReview ↗

Response to authors

I thank the authors for their helpful clarifications and additional analyses, which have addressed several of my concerns. I encourage the authors to further strengthen the evaluation of cross-subject generalization in downstream EEG classification, provide clearer evidence of classification improvements over SSI, and better establish the benefits of continuous decoding for downstream tasks. Overall, I appreciate the authors' efforts and maintain my original positive rating.

Reviewer_tYpcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CANDI, a framework for integrating EEG and fMRI data. It employs bidirectional Mamba encoders to capture signal context, passing latent variables to a cross-attention-based Implicit Neural Representation (INR) decoder that supports continuous coordinate queries. The fMRI branch leverages a Sequential Locality Bias (SLB) to exploit priors regarding temporal/sequential proximity, while the EEG branch incorporates Spherical Harmonic Positional Encoding (SHPE) alongside a regularization term based on the bending energy of the scalp potential field. The authors propose an "unseen-electrode" evaluation protocol for EEG, where a subset of electrodes is excluded from supervision during training and used exclusively for testing—covering approximately 40% to 87.5% of the total electrodes. Furthermore, the framework is evaluated on fMRI reconstruction (using the HCP dataset), computational/storage scalability, and downstream tasks such as gender classification and age regression. Under an 8× sparse sampling configuration, CANDI achieves an EEG Normalized Mean Squared Error (NMSE) of 0.5767 on the PhysioNet dataset—outperforming SSI (1.4379) and ZUNA (0.8952)—and 0.4955 on the SEED-V dataset (compared to 1.6821 and 0.6924 for the baselines). For fMRI, the model achieves lower MSE and HFEN (High-Frequency Error Norm) across various volumetric scales, whereas some Transformer-based INR baselines encounter "out-of-memory" (OOM) errors when handling longer sequence lengths.

Critical strengths
  1. The evaluation of spatial generalization to new EEG electrodes is sound and demonstrates significant improvement. By completely masking the supervision signal for a subset of electrodes during training—as defined in Algorithm 1—the method avoids the common pitfall of weak validation, where models are merely tested on the same montage used for training. Across two datasets, the method consistently outperforms SSI and ZUNA in terms of NMSE, SNR, and PCC, with strict subject-wise separation. This serves as the most direct evidence for the "continuous geometric representation" claimed by the authors.

  2. Comprehensive cross-scenario benchmarking and mechanistic ablation studies. The large-scale reconstruction experiments on the HCP dataset include comparisons with LAGINR, TransINR, Linformer, and SCENT. Additionally, Appendix C.1 provides measurements of VRAM usage and token throughput on an A100 40GB GPU. The use of SLB (Structured Latent Bottleneck) yields relative MSE improvements of approximately 2.42%, 3.05%, and 12.76% when using 64, 128, and 256 latent tokens, respectively. This is far more compelling than a mere theoretical assertion regarding the linear complexity of the Mamba architecture.

Critical weaknesses
  1. The strong claim regarding a "cross-modal continuous field" extends beyond what the current experiments can directly substantiate. While the EEG experiments do involve coordinates not encountered during training, the core fMRI results primarily focus on reconstructing two adjacent time frames within the spatial grid used during training—rather than testing arbitrary time points, novel scanning geometries, or cross-device sampling. Furthermore, each modality employs its own distinct training strategy and positional encoding. A more accurate conclusion at this stage is that a unified Implicit Neural Representation (INR) framework can function across the specific tasks of both modalities; it does not necessarily imply the validation of a shared cross-modal representation or a generalized continuous spatiotemporal extrapolation capability.

  2. Claims regarding computational resources, robustness, and downstream performance gains should be interpreted with greater caution. The advantage of using finer patches for fMRI is linked to the encoding capabilities of the Mamba architecture; however, the query cost of the cross-attention decoder does not scale linearly across all dimensions. In fact, the high-resolution reconstruction results indicate that the proposed method—utilizing 23×22^3 \times 2 patches—encounters OOM errors when applied to a 643×264^3 \times 2 grid. Additionally, the fMRI experiments rely on a single training seed and 16 GPUs. Regarding the EEG downstream tasks, the proposed method does not consistently outperform SSI at low sparsity levels—for instance, at a 2×2\times ratio, the performance is 0.51540.5154 versus 0.52090.5209. Consequently, a more comprehensive analysis of computational costs and performance variance is required.

Questions for authors
  1. Can fMRI data be accurately interpolated across continuous time coordinates, voxel grids, or spatial resolutions not encountered during training? Could you provide "held-out-coordinate" test results—constrained by a specific query budget—to substantiate the claim of "continuous 4D" ​​localization?

  2. Please report the end-to-end throughput, peak GPU memory usage, and inference latency for the complete encoder-decoder architecture, ensuring these metrics are evaluated under consistent conditions (i.e., matching latent variable counts, reconstruction accuracy, and GPU resource budgets). Additionally, explain the conditions under which the method encounters OOM errors, specifically at the finest patch resolution of 643×264^3 \times 2.

  3. For the primary EEG and fMRI experiments, could you provide confidence intervals based on multiple random seeds or independent data splits? Furthermore, please include controlled comparisons—such as replacing components with Mamba or SHPE, or incorporating smoothing—to isolate the effects of specific design choices. Regarding the downstream age/gender prediction tasks using fMRI data, is there evidence of generalization to independent, held-out cohorts?

5 comments
AuthorsOpenReview ↗

Scoping Correct Claims

1. Scoping the "cross-modal" and "arbitrary-resolution" claims

"The strong claim regarding a 'cross-modal continuous field' extends beyond what the current experiments can directly substantiate." ... "A more accurate conclusion at this stage is that a unified Implicit Neural Representation (INR) framework can function across the specific tasks of both modalities..."

We agree with the reviewer's characterization and have adopted it. CANDI does not claim a shared cross-modal representation or generalized spatiotemporal extrapolation for fMRI. Our contribution is a common amortized INR formulation that accommodates modality-specific observation geometry, instantiated as independently trained EEG and fMRI models. The revised manuscript states this explicitly:

  • Introduction (line 059): "We instantiate and train CANDI separately for EEG and fMRI, sharing the amortized encoder-decoder formulation."
  • Introduction (lines 065–067): the primary fMRI objective is scalable reconstruction as representation pretraining, not continuous-field interpolation.
  • Section 4.3 (lines 411–413): "the reconstruction task expects the same output volume size as the input; it does not query off-grid coordinates."
  • Conclusion (lines 539–540): multimodal fusion and cross-dataset transfer are listed as future work.

"the core fMRI results primarily focus on reconstructing two adjacent time frames within the spatial grid used during training—rather than testing arbitrary time points, novel scanning geometries, or cross-device sampling"

We have redesigned Figure 1(e). The previous "arbitrary sizes" wording is removed, input and output volumes are drawn at the same size with the same shape s3×ts^3 \times t, and the panel notes that each volume size is trained independently. We also corrected Figure 1(c), where two electrodes on the right were miscolored.

Our focus on same-resolution reconstruction is deliberate. fMRI is densely and regularly sampled, so classical spatial interpolation (cubic, B-spline, sinc) is a strong baseline [1, 2]. The slow BOLD hemodynamic response also limits the neural information recoverable by interpolating intermediate time points [3, 4]. Temporal and cross-resolution interpolation for fMRI are outside the scope of this paper.

Additional fMRI held-out-coordinate experiment.

"Can fMRI data be accurately interpolated across continuous time coordinates, voxel grids, or spatial resolutions not encountered during training? Could you provide 'held-out-coordinate' test results—constrained by a specific query budget—to substantiate the claim of 'continuous 4D' localization?"

To address the spatial part of this question directly, we add a held-out-patch experiment analogous to our EEG protocol (Appendix E.2). We partition 363×236^3\times2 volumes into 43×24^3\times2 patches and globally reserve a fixed, randomly selected 10% of brain-masked patches, shared across all subjects. These patches are never supervised during training. At evaluation, CANDI and biharmonic inpainting receive identical observed context (20–100% of the remaining patches), and error is computed only on all of the voxels of the held-out set.

Rebuttal Table 1: Globally held-out fMRI patch reconstruction (Appendix E.2, Table 30). Values are subject-level mean ± SD.

ContextCANDI MSE ↓ (×10⁻³)Biharmonic MSE ↓ (×10⁻³)CANDI HFEN ↓ (×10⁻¹)Biharmonic HFEN ↓ (×10⁻¹)
20%2.799 ± 0.5504.679 ± 1.0002.875 ± 0.3004.562 ± 0.220
40%2.731 ± 0.5104.070 ± 0.8902.838 ± 0.3004.201 ± 0.190
60%2.717 ± 0.5103.394 ± 0.7302.824 ± 0.3003.672 ± 0.180
80%2.713 ± 0.5102.671 ± 0.5702.824 ± 0.3003.123 ± 0.170
100%2.698 ± 0.5002.566 ± 0.5402.813 ± 0.3003.097 ± 0.170

CANDI achieves lower HFEN at every context level and lower MSE at 20–60% context. Biharmonic inpainting is slightly better in MSE when 80–100% of the context is available. As with EEG, CANDI degrades more gracefully than the classical interpolant as context shrinks. This experiment tests spatial generalization to unsupervised regions on the training grid; we do not extend it to claims of temporal or cross-resolution interpolation.

[1] Lehmann, T. M., Gönner, C., and Spitzer, K. (1999). "Survey: Interpolation methods in medical image processing." IEEE Transactions on Medical Imaging, 18(11), 1049–1075.

[2] Lehmann, T. M., Gönner, C., and Spitzer, K. (2001). "Addendum: B-spline interpolation in medical image processing." IEEE Transactions on Medical Imaging, 20(7), 660–665.

[3] Boynton, G. M., Engel, S. A., Glover, G. H., and Heeger, D. J. (1996). "Linear systems analysis of functional magnetic resonance imaging in human V1." Journal of Neuroscience, 16(13), 4207–4221.

[4] Logothetis, N. K. (2008). "What we can do and what we cannot do with fMRI." Nature, 453(7197), 869–878.

AuthorsOpenReview ↗

Compute Analysis

2. Computational costs

"Please report the end-to-end throughput, peak GPU memory usage, and inference latency for the complete encoder-decoder architecture, ensuring these metrics are evaluated under consistent conditions (i.e., matching latent variable counts, reconstruction accuracy, and GPU resource budgets)."

We add two benchmarks (Appendix D). All use batch size 2, bf16 mixed precision, and a single 40 GB A100.

  • Controlled token-length scaling (Appendix D.1, Figures 4–5): decoder configuration, query budget (0.4% of the grid), and learnable-token count (128) are fixed while input tokens vary through patch size. LAGINR and TransINR memory grows rapidly and exceeds 40 GB. CANDI and Linformer remain feasible, and Linformer achieves higher throughput at long sequences. The latency growth of LAGINR and TransINR comes from their encoders; decoder latency is roughly constant at a fixed query and latent count.
  • Table 4 settings (Appendix D.2, Table 27): latent-token counts are matched across models.

Rebuttal Table 2: Representative Table 4 compute settings (Appendix D.2, Table 27): end-to-end training throughput and peak memory, chunked full-volume inference latency, and separate encoder and decoder latency. Reconstruction values are taken from Table 4, not remeasured.

ModelLℓL_\ellTraining samples/sTraining peak MBFull vol. (ms)Encoder (ms)Decoder (ms)MSE / HFEN (Table 4)
B1: 48³×2, patch 4³×2 (1,728 input tokens)
CANDI19218.59787869.613.3255.893.30×10⁻⁴ / 0.1344
LAGINR19215.091117891.735.5755.755.18×10⁻⁴ / 0.2163
Linformer19219.29839174.118.2855.515.14×10⁻⁴ / 0.2154
B2: 48³×2, patch 2³×2 (13,824 input tokens)
CANDI15365.5624554324.875.07249.361.76×10⁻⁴ / 0.0686
Linformer15366.6328000351.0101.91248.795.31×10⁻⁴ / 0.2214

At matched latent counts and GPU budget, CANDI uses the least training memory and has the lowest inference latency in both settings, while also achieving the lowest error. Linformer trains slightly faster.

"the query cost of the cross-attention decoder does not scale linearly across all dimensions" ... "explain the conditions under which the method encounters OOM errors, specifically at the finest patch resolution of 643×264^3\times2"

We agree. Mamba makes the encoder linear in input length, but decoder cross-attention costs O(QLℓd)O(QL_\ell d) for QQ queries and LℓL_\ell latent tokens (Appendix D.2), and both grow with volume size. At 643×264^3\times2 with 23×22^3\times2 patches, the input has 32,768 tokens and the fixed Ld:Lℓ=9:1L_d : L_\ell = 9:1 ratio gives about 3,641 latent tokens. That is 2.4× the 48348^3 setting on both QQ and LℓL_\ell, and the 48348^3 setting already peaks at 24.6 GB, so the decoder term exceeds the 40 GB per-GPU budget. With 43×24^3\times2 patches (4,096 tokens), CANDI trains at 64364^3 and outperforms the efficient baselines (Table 4).

"the fMRI experiments rely on a single training seed and 16 GPUs"

The 16 GPUs are used for data-parallel training throughput. All compute benchmarks above are measured on a single 40 GB GPU, so the memory comparisons reflect per-device feasibility. For the attention baselines, LAGINR, Linformer, and TransINR use manually implemented attention (not FlashAttention), so their OOM results characterize the evaluated implementations rather than architectural lower bounds (Appendix B.1.2). Seed variability is addressed in Section 3.

AuthorsOpenReview ↗

Statistical Analysis and Reproducibility

3. Statistical analysis and reproducibility

"For the primary EEG and fMRI experiments, could you provide confidence intervals based on multiple random seeds or independent data splits?"

fMRI reconstruction (Table 4). We retrain a representative subset with three independent seeds (Appendix C.2, Table 17). We chose two baselines: LAGINR, the strongest when memory allows, and Linformer, the strongest memory-feasible efficient baseline. We used two volume sizes where CANDI's advantage differs, each at the finest patch size the model can train. Subject-level SEM for every cell of Table 4 is in Appendix C.2.1, Table 18.

Rebuttal Table 3: Three-seed fMRI reconstruction reproducibility (Appendix C.2, Table 17). Mean ± SD across seeds; MSE in ×10⁻⁴.

VolumeModel (patch)MSE ↓HFEN ↓
24³×2CANDI (2³×2)0.6056 ± 0.07310.0280 ± 0.0010
24³×2LAGINR (2³×2)1.5587 ± 0.19370.0708 ± 0.0086
24³×2Linformer (2³×2)1.7604 ± 0.04780.0799 ± 0.0021
48³×2CANDI (2³×2)2.1937 ± 0.38960.0847 ± 0.0148
48³×2LAGINR (4³×2)*4.8887 ± 0.60000.2046 ± 0.0206
48³×2Linformer (2³×2)5.2733 ± 0.03180.2204 ± 0.0009

*LAGINR runs out of memory at 2³×2 for 48³.

CANDI achieves the lowest MSE and HFEN for every seed and in the mean. Seed variability is larger for CANDI at 48348^3, but its worst seed still outperforms both baselines' best seeds.

EEG interpolation (Table 1). We add paired subject-level analyses against SSI and ZUNA (Appendix C.1, Table 16). For each subject, we average the NMSE difference across the four matched electrode layouts and bootstrap over subjects (10,000 resamples). Retraining separate models per layout and seed is costly, so we report subject-level rather than seed-level uncertainty and acknowledge that seed variability remains unmeasured for EEG.

Rebuttal Table 4: Paired subject-level NMSE improvements of CANDI over SSI and ZUNA (Appendix C.1, Table 16). Positive values favor CANDI; intervals are 95% percentile bootstrap CIs over subjects.

DatasetScaleBaselineNΔ^\hat{\Delta} [95% CI]
PhysioNet2×SSI120.054 [−0.012, 0.169]
ZUNA120.173 [0.135, 0.215]
4×SSI120.167 [0.053, 0.330]
ZUNA120.228 [0.118, 0.392]
8×SSI120.952 [0.392, 1.828]
ZUNA120.347 [0.183, 0.600]
SEED-V2×SSI30.055 [0.036, 0.083]
ZUNA30.580 [0.480, 0.687]
4×SSI30.142 [0.085, 0.200]
ZUNA30.135 [0.121, 0.152]
8×SSI31.183 [0.849, 1.621]
ZUNA30.197 [0.175, 0.216]

On PhysioNet, five of six intervals exclude zero; the exception is SSI at 2×, where CANDI's advantage is not statistically established. On SEED-V, all six improvements are positive, but with only three test subjects we treat these intervals as descriptive rather than confirmatory.

"Regarding the EEG downstream tasks, the proposed method does not consistently outperform SSI at low sparsity levels—for instance, at a 2× ratio, the performance is 0.5154 versus 0.5209."

We agree. A paired subject-level analysis over 81 subjects (Appendix C.4, Table 23) finds no significant accuracy difference between CANDI and SSI at any scale: −0.38 points at 2×, with 95% CI [−1.44, +0.68]. CANDI significantly outperforms the low-resolution input at every scale (+6.90, +10.38, and +9.90 points). We have revised Section 4.2 to claim only that CANDI preserves class-relevant structure at a level comparable to SSI.

fMRI downstream (Table 2). Paired tests on 109 held-out subjects (Appendix C.3, Tables 19–21) show that CANDI's sex-classification AUC is significantly higher than both SwiFT and Omni-fMRI after Holm correction (DeLong), but accuracy differences are not significant (McNemar). For age regression, the CANDI linear probe significantly outperforms Omni-fMRI (pHolm=0.012p_{\text{Holm}} = 0.012) but not SwiFT. These tests capture test-subject variability for fixed trained models, not seed variability.

Comments on the paper

1 comment

Dear Reviewers and AC,

Thank you all for your time and thoughtful feedback. We have done our best to address every concern raised during the review process and updated the manuscript accordingly, with the changes highlighted in blue.

Crucially, the reviewers' feedback helped us sharpen and strengthen the presentation of our core contribution. Specifically, our revisions focus on better positioning our work within the existing literature, adding further empirical validation, and expanding on key technical details. The comments reference this manuscript for more detailed explanations of any responses or experiments. The main changes are summarized below:

  • Scoped claims. We thank the reviewers for pointing out the confusion in exactly how our paper is scoped. CANDI is a common amortized INR formulation, trained separately for EEG and fMRI; it is not a cross-modal fusion model. For fMRI, we evaluate same-resolution reconstruction as representation pretraining, not cross-resolution or temporal querying. We redesigned Figure 1(e), defined "reconstruction" in its image-domain INR sense, and revised the Introduction, Section 4.3, and Conclusion to match. We also add a held-out-patch fMRI experiment (Appendix E.2) testing generalization to spatial regions never supervised during training, however we do not claim that this generalizes to arbitrary coordinate querying for fMRI or any utility for neuroscience tasks.

  • New ablations isolating our contributions. For EEG, a Transformer-encoder swap shows that the gains come from SHPE and the bending loss rather than the encoder (Table 5). We also specify the Fourier-PE replacement and the tuning sweep behind its training collapse. For SLB, we compare against a spatial-distance bias, apply SLB to a Transformer encoder, test alternative scan orders (Appendix E.1), and repeat the SLB ablation over three seeds (Appendix C.6). A discrete patch-decoding variant isolates the effect of continuous coordinate decoding on downstream tasks (Appendix E.3).

  • Statistical analysis for every main result. Appendix C adds paired subject-level tests and seed reproducibility for every main table:

    • EEG reconstruction: bootstrap confidence intervals (C.1)
    • fMRI reconstruction: three seeds (C.2)
    • fMRI downstream: McNemar, DeLong, and Wilcoxon significance tests (C.3)
    • EEG downstream: subject-level paired analysis (C.4)
    • Ablations: uncertainty for both (C.5–C.6)

    Where the tests do not support a claim, most notably EEG downstream classification against SSI, we revised the main text to match.

  • Complete baseline details. Appendix B now reports:

    • fMRI baselines (LAGINR, Linformer, TransINR, SCENT): architecture, optimization, and attention implementation (Table 9)
    • EEG baselines: SSI parameters, plus the ZUNA checkpoint, inference settings, and held-out-channel handling (Table 10)
    • Downstream models: configurations for the classifiers and for SwiFT and Omni-fMRI (Tables 11–14)
  • Compute measured at the Table 4 settings. Beyond the controlled token-length scaling study (Appendix D.1), we benchmark representative Table 4 configurations at matched latent-token counts (Appendix D.2, Table 27). We report end-to-end training throughput, peak memory, full-volume inference latency, and separate encoder and decoder latency.

We believe these revisions address the reviewers' feedback and clarify both the contributions and empirical strengths of CANDI. We appreciate your time and consideration, and we really appreciate the insightful reviews that have allowed us to elevate the level of our work.

Best, Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.