acceptodds
Under review as a conference paper at ICLR 2027

HiPPO Functional Memory: Coordinate-Queryable Attention for Irregular Time Series

Abstract

Irregular time series are observed at nonuniform coordinates, yet predictions may be required at different times. We introduce HiPPO Functional Memory (HiPPO-FM), a functional-memory attention operator that compiles irregular observations into a fixed-rank, coordinate-addressable state for direct evaluation at arbitrary query times. HiPPO-derived functional coordinates and matched kernel geometry couple memory write and read, while content interaction can precede compression without changing the functional query space. We characterize the resulting finite-dimensional query space and its compression limits, and demonstrate strong continuous-coordinate reconstruction, competitive irregular representation learning, and real-data extrapolation. Because the encoded state is reusable across query sets, HiPPO-FM replaces observation-indexed retrieval with direct queries to a compact function-valued memory.

Primary area:
learning on time series and dynamical systems
Keywords:
irregular time series, continuous-time modeling, functional memory, HiPPO, efficient attention, interdomain Gaussian processes, time series forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_dxs7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed HiPPO-FM to map irregular sequential data to fixed rank and a resuable coordinate-addressable state. It applies self-attention before writing observations into kernel-smoothed Legendre coordinates, then it uses a matched Gram solve for coordinate evaluation. The experiments showed improved performance on continuous coordinate reconstructions.

Critical strengths
  1. The experiments covers different task types and reports results across multiple seeds.
  2. The RKHS analysis clearly discussed exact functional values and learned normalized memory, and the stated proofs is sound.
Critical weaknesses
  1. The below paper [1] is very relevant but not cited. Its HiPPO–Legendre interdomain covariances and predictive mean correspond to the continuous counterparts of the kernel geometry and readout in Eq. (5)–(9) of this submission. The authors should justify the novelty.

[1] Chen, Wenlong, Naoki Kiyohara, Harrison Zhu, Jacob Curran-Sebastian, Samir Bhatt, and Yingzhen Li. "Recurrent memory for online interdomain gaussian processes." Advances in Neural Information Processing Systems 38 (2026): 134836-134873.

  1. There is no ablation study to proof that whether the gains come from self-attention, the Legendre features, or the matched Gram readout.

  2. The event prediction experiments didn't define the conditional intensity or density, likelihood computation, prediction heads, or training objective.

Questions for authors

Please see Weaknesses section and consider each point as a question.

1 comment

Response to Reviewer dxs7

We thank the reviewer for identifying the missing attribution to OHSVGP (Chen et al., 2025) and raising questions about novelty, ablations, and reproducibility. We revised the manuscript to clarify the mathematical foundations, architecture, and experimental protocol.

Q1. Missing citation of Chen et al. (2025) and methodological novelty.

We added citations and discussion in the Introduction, Section 2, Section 3.2, and Appendix B.1.

Mathematical connection. Both methods use HiPPO–Legendre functionals and kernel-induced interdomain geometry:

HiPPO-FMInterdomain GP / OHSVGP
Ψθ(X)\Psi_\theta(X)KfuK_{fu}
GθG_\thetaKuuK_{uu}
Ψθ(R)(Gθ+ϵI)−1CP\Psi_\theta(R)(G_\theta+\epsilon I)^{-1}C_PKfuKuu−1μuK_{fu}K_{uu}^{-1}\mu_u

The HiPPO–Legendre functionals and inverse-Gram readout follow constructions established in OHSVGP. HiPPO-FM incorporates this geometry into a deterministic, end-to-end trained memory architecture. Its encoder applies self-attention over the observed support, writes content-updated values into a fixed-rank state, and evaluates the retained coefficients at arbitrary temporal coordinates. The memory stores learned neural values, whereas OHSVGP maintains a variational GP posterior. Interdomain Attention updates basis coefficients through SSM recurrence; HiPPO-FM forms its state through a single write after observed-support interaction. ContiFormer evaluates observation-dependent representations at requested times, whereas HiPPO-FM reuses the same encoded state across coordinate queries.

Q2. Ablation evidence.

We added five-seed ablations on Spiral reconstruction (Section 4.5, Table 5; Appendix D).

VariantOverall RMSEExtrap. RMSESeeds worse (k/5k/5)
Full0.54 ± 0.140.63 ± 0.20--
w/o encoder FM; MHA retained9.88 ± 0.5913.79 ± 0.775/5
Cosine basis0.61 ± 0.100.66 ± 0.154/5
No kernel smoothing0.49 ± 0.060.50 ± 0.053/5
No Gram solve0.63 ± 0.070.64 ± 0.074/5

RMSE: mean ± sample SD, in units of 10−210^{-2}; lower is better.

Removing encoder FM while retaining MHA, layer normalization, and FFN increases RMSE substantially (paired tt-test p=3.5×10−6p=3.5\times10^{-6}; exact Wilcoxon p=0.0625p=0.0625).

  • Encoder FM is the most critical component for this task, as removing it causes substantial performance degradation.
  • HiPPO coordinates show a benefit over alternative basis.
  • Kernel geometry exhibits component-dependent effects: removing kernel smoothing lowers mean RMSE, but degrades 3/5 seeds, whereas removing the Gram solve increases RMSE in 4/5 seeds. This suggests that kernel smoothing and Gram-based correction play distinct roles, with their effects depending on the task and optimization trajectory.

Q3. Event-prediction protocol.

We expanded Section 4.3 and Appendix A.4 with the model architecture, conditional intensity, likelihood, prediction rules, and training objective.

Architecture. HiPPO-FM replaces only the history encoder in the ContiFormer reference implementation; event embedding, prediction head, likelihood, and loss are retained. Each marked event (tj,kj)(t_j,k_j) is embedded, processed over support {tj}\{t_j\}, and passed through the reference single-layer LSTM to obtain hjh_j.

Conditional intensity and likelihood:

\operatorname{softplus}_{\beta_k} (w_k^\top h_j+b_k+\alpha_k(t-t_j)), \quad \lambda=\sum_k\lambda_k,$$ where $\operatorname{softplus}_{\beta}(x)=\beta^{-1}\log(1+e^{\beta x})$. $$\begin{aligned} \log\mathcal L &=\sum_j\log\lambda_{k_{j+1}}(t_{j+1}\mid\mathcal H_{t_j}) -\sum_j\int_{t_j}^{t_{j+1}}\lambda(s\mid\mathcal H_{t_j})\,ds,\\ p_j(\tau) &=\lambda(t_j+\tau)\exp\left(-\int_0^\tau\lambda(t_j+s)\,ds\right),\\ \hat\tau_{j+1} &=\int_0^{t_{\max}}\tau p_j(\tau)\,d\tau,\\ \hat k_{j+1} &=\arg\max_k\int_0^{t_{\max}} \frac{\lambda_k(t_j+\tau)}{\lambda(t_j+\tau)}p_j(\tau)\,d\tau. \end{aligned}$$ The intensity integral uses a uniform grid with step 0.01; predictions use the trapezoidal rule. **Training objective:** $$\mathcal{J}=-\log\mathcal{L}+\gamma^{\mathrm{mark}}\operatorname{CE}(\hat{k},k)+(\gamma^{\mathrm{time}})^{-1}\sum(\hat{\tau}(j+1)-\tau(j+1))^2$$ Here, CE uses label smoothing of 0.1, and the sum is over events. The reference configuration uses $t_{\max}=5$, $\gamma^{\mathrm{mark}}=1$, and $\gamma^{\mathrm{time}}=100$.
Reviewer_zstpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces HiPPO-FM, an operator for irregular time series that transforms irregularly sampled observations into a fixed-rank bank of functional coefficients. These coefficients form a compact memory which can be queried at arbitrary temporal coordinates. Because this transformation creates a fixed-size representation from the original irregular observations, it facilitates training models on irregular data (i.e., the complexity of handling the irregular data is moved into the operator). The authors incorporate the operator into a self-attention network and explore its use through experiments on reconstruction, classification, extrapolation, and event prediction.

Critical strengths

Modelling irregular time-series data is an important and open problem. Designing an operator that transforms irregular measurements into a fixed-rank representation for downstream models to build on is an interesting and useful direction. The ability to query the same representation at arbitrary times without reprocessing the history is particularly appealing. The approach also has a principled functional interpretation. The authors characterize what the memory can represent and provide an error decomposition that separates limitations of the functional representation from errors in the learned memory. This provides a useful theoretical basis for understanding the proposed operator, which I found quite compelling.

Critical weaknesses

I find the paper hard to follow and not well presented. The underlying innovation and approach are interesting, but they could be presented much better, both in the write-up and in the overview figure. For example, the authors might show a system or environment that produces measurements at irregular intervals: What is the problem with current approaches? They struggle with missing data and need interpolation. How do they handle these observations, and what makes querying them at new time points difficult or expensive? What does HiPPO-FM do? It takes irregular data and, through a series of transformations, turns it into a compact, queryable functional memory that can predict values at arbitrary times without reprocessing the history. Showing this process visually, and explaining what each transformation does and why it is needed, would make the approach and its contribution much easier to understand. It took me quite a few reads to understand figure 1.

I also find the evaluation somewhat weak or confusing in demonstrating what the proposed approach is useful for. The central motivation is to turn irregular observations into a compact memory that can be queried repeatedly at arbitrary times. However, the classification experiments bypass this readout ("UEA classification pools the observed-support representation and bypasses the final P->R functional readout"), while the reconstruction and extrapolation experiments mainly measure prediction accuracy. These are useful results, but they do not clearly demonstrate the benefit of the proposed interface. For example, an experiment that queries the same history at different resolutions and compares accuracy, latency, and memory usage against existing approaches would make the practical value much clearer. Even better, taking an existing known benchmark or use case that is known to be prone to missing data and struggle with missigness (eg disease classification). This is a big problem for example in medical data interpretation where we for example have electronic health records (EHRs) at irregular time intervals. Or any other concrete problem would make this much more tangible.

Lastly, I was also not really sure how this work relates to prior work on modelling irregular time series. The related work section is rather short and merely lists a few recent efforts. The conclusion/discussion section does not provide a comparison with prior work, which makes it hard to judge this work within the context of the broader research field.

Questions for authors
  1. Please clarify and compare how HiPPO-FM relates to prior work on modelling irregular time series.
  2. Could you design a better experiment showing the superiority of your approach compared to prior methods? For example on EHR modelling we often work with irregular data, and showing that integrating your operator into a SoTA approach and increases performance would strongly improve the paper.
  3. Why does UAE classification bypass the functional readout?
3 comments

Response to Reviewer zstp

We sincerely thank you for your careful reading and constructive suggestions regarding presentation, experiments, and prior work. We revised Figure 1, expanded Related Work, and added repeated-query and real-world extrapolation experiments.

For reproducibility, our anonymized repository provides the implementation and reproduction instructions:

Anonymous code repository: https://github.com/kumayazamana-byte/anon-ml-repro-7f3c


Q1. Please clarify and compare how HiPPO-FM relates to prior work on modelling irregular time series.

Response.

We reorganized Section 2 into four themes: irregular time-series modeling, decoding at arbitrary inputs, compact sequence memory, and interdomain/functional memory.

MethodMemoryMechanismQuery interface
OHSVGPProbabilistic GPSparse inducing variablesGP posterior
Interdomain AttentionBasis coefficientsSSM recurrenceQuery feature map
ContiFormerObservation-dependent representationsContinuous attentionContinuous-time queries
HiPPO-FMDeterministic neural valuesPre-write attention and functional writeCoordinate readout

HiPPO-FM shares the HiPPO–Legendre functionals and inverse-Gram geometry established in interdomain GP constructions. Its contribution is integrating this geometry into an end-to-end trained deterministic memory: self-attention processes the observed support, content-updated values enter a fixed-rank state, and retained coefficients are evaluated at arbitrary coordinates. Unlike OHSVGP, it does not maintain a variational GP posterior; unlike Interdomain Attention, it does not use recurrent SSM updates; and unlike ContiFormer, it reuses one encoded state across query sets.


Q2. Could you design a better experiment showing the superiority of your approach compared to prior methods? For example, on EHR modeling we often work with irregular data, and showing that integrating your operator into a SoTA approach increases performance would strongly improve the paper.

Response.

We added a redesigned Figure 1, repeated-query evaluation, and real-world extrapolation experiments.

(1) Architecture (Figure 1). Four panels illustrate: (a) the overall architecture; (b) pre-write content interaction; (c) functional write and coordinate readout; and (d) reuse of one retained state for dense, coarse, and extrapolated queries.

(2) Repeated queries (Appendix A.3, Figure 4). For a fixed history, encoding once and reusing the state across 1,000 query grids reduces total latency from 9,756.26 ms to 1,605.35 ms (6.08×). Changing query order, grouping, or chunking changes predictions by at most 2.611×10−52.611\times10^{-5}.

(3) Real-world EHR extrapolation (Section 4.4, Appendices C.4 and D.5). On PhysioNet Challenge 2012, we use 8,000 patient records and 37 time-varying variables, observing the first 24 hours and predicting the withheld measurements up to 48 hours.

ModelHidden MSE ↓s/epoch ↓Peak memory (GiB) ↓Parameters
CRU0.006465 ± 0.00040945.84 ± 12.134.12128,654
HiPPO-FM0.005099 ± 0.0001002.01 ± 0.140.19655,978

HiPPO-FM reduces hidden-region MSE by 21.1%, epoch time by 22.8×, and peak memory by approximately 21×. CRU uses its native masked Gaussian NLL, whereas HiPPO-FM uses deterministic coordinate decoding with an MSE objective. Thus, this comparison concerns point-prediction accuracy and implementation efficiency.

Additionally, for event prediction, HiPPO-FM replaces only the history encoder in the ContiFormer reference implementation, retaining the event embedding, prediction head, likelihood, and training objective (Section 4.3, Appendix C.3). This provides an additional evaluation of the proposed encoder within an existing prediction architecture.


Q3. Why does UEA classification bypass the functional readout?

Response.

UEA is a sequence-level task and uses last-valid pooling over the content-conditioned encoder representation:

\left(\operatorname{Pool}(H_{\mathrm{enc},P})\right).$$ The output FM head serves coordinate-valued tasks such as Spiral reconstruction and PhysioNet extrapolation. Classification uses last-valid pooling, whereas next-event prediction applies the reference conditional-intensity head to the encoded representation at each event. Both bypass the output FM head, as explained in Section 3.3. Thus, UEA evaluates the content-conditioned encoder and its internal FM operator, while reconstruction, extrapolation, and repeated-query experiments evaluate including coordinate querying.

Thank you very much for your detailed response. I appreciate the clarifications and additional experiments. However, I unfortunately still have the impression that the paper is very hard to follow and understand. I think it could strongly benefit from a clearer narrative and better writing, and a more high level figure. While I do appreciate the new figure in the revised version, I still think the approach can be both, introduced more clearly and more clearly distinguished from related work and other methods. For these reasons, I keep my score unchanged.

Follow-up Clarification to Reviewer zstp

Thank you for your continued feedback. We understand that our previous response emphasized additional experiments but did not sufficiently clarify the conceptual narrative. We would therefore like to explain the method from the underlying problem to its architectural design.

The central idea of HiPPO-FM is to encode an irregular history once into a compact, coordinate-addressable function, and then evaluate that function at arbitrary requested times without revisiting the observations.

(1) Why existing approaches face a query-dependent computation path.

Let PP denote the irregular observation coordinates and RR the requested query coordinates. Existing methods generally use two representations. Continuous-time models (Neural ODE/CDE and CRU) evolve a latent state to requested times through numerical integration. Observation-based attention methods (mTAN and ContiFormer) retain observation-indexed representations and compute query-dependent interactions with them. Both approaches therefore connect new queries to the historical representation through additional computation.

(2) What HiPPO-FM changes.

Instead of retaining the history as observation tokens, HiPPO-FM writes it into a fixed-rank functional state:

\widehat O(R)=\Psi_\theta(R)B_P.$$ Here, $B_P\in\mathbb R^{M\times d}$ contains the retained coefficients, and $\Psi_\theta(R)$ evaluates them at requested coordinates. The same state can therefore serve dense, coarse, and extrapolated query sets without re-encoding the history. **(3) Why self-attention precedes functional writing.** This ordering follows directly from the write operator. Its aggregation weights depend on temporal coordinates, not on pairwise observation content. Therefore, content-dependent interaction must occur before compression. Masked self-attention first determines which observations inform each other; the functional features then determine where the updated values are stored in time. As described in Section 3.3, these are complementary roles rather than two interchangeable attention mechanisms. **(4) How it differs from related work.** HiPPO-FM shares the HiPPO–Legendre functionals and inverse-Gram geometry established in interdomain GP work, including OHSVGP. However, OHSVGP uses these constructions for online sparse variational GP inference, whereas HiPPO-FM stores deterministic neural values in an end-to-end trained attention architecture. ContiFormer constructs continuous-time queries, keys, and values over observation-dependent representations, whereas HiPPO-FM retains a fixed-rank function that can be evaluated directly at new coordinates. Interdomain Attention maintains basis coefficients through SSM recurrence, whereas HiPPO-FM forms its state through a functional write after observed-support interaction. **Figure 1 follows this same narrative:** (a) the complete encoder and task-specific readouts; (b) content interaction before compression; (c) the matched functional write and coordinate read; and (d) reuse of one state across query sets. The corresponding explanations are provided in the Introduction, Section 2, Sections 3.2–3.5, and Appendix A. We appreciate your feedback and agree that the distinction between content interaction and functional memory is central to understanding the contribution. We hope this more direct explanation makes the motivation, architecture, and relationship to prior work clearer.
Reviewer_LzzuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose a functional memory framework, HiPPO-FM, for irregular time-series modeling. By exploiting pre-write self-attention, the proposed method encodes irregular observations into a fixed-rank functional state, enabling direct evaluation at arbitrary temporal coordinates. Experiments evaluate the proposed method across diverse tasks, including reconstruction, classification, and extrapolation.

Critical strengths
  1. The authors propose a unified framework for encoding irregular observations into a functional state that can be evaluated at arbitrary temporal coordinates. This is a useful approach for handling different temporal query requirements.

  2. The proposed method achieves competitive results in diverse tasks for irregular time series.

Critical weaknesses
  1. The distinction between HiPPO-FM and existing methods is not sufficiently clear. The limitations addressed by the proposed method and its methodological advances should be clarified.

  2. The evaluation lacks comprehensive ablation studies, comparisons with recent methods, and sufficient analysis of computational efficiency and memory capacity. In addition, the performance improvements are not consistently demonstrated across datasets and evaluation metrics, making it difficult to assess the robustness and significance of the proposed method.

Questions for authors
  1. What are the key methodological differences between HiPPO-FM and existing approaches, and what limitations does it address?

  2. Could the authors provide ablation studies to clarify the individual contributions of pre-write self-attention, HiPPO-based coordinates, and kernel geometry?

  3. How does the memory rank affect predictive performance and memory efficiency?

  4. How do computational costs and memory usage scale with the number of observations and query coordinates?

  5. Could the authors provide comparisons with more recent competitive methods for irregular time-series modeling?

  6. Are the performance improvements over competitive baselines statistically significant? Could the authors further discuss the relatively small performance gains and the effects of increasing observation-drop ratios on UEA datasets, as well as provide additional discussion of the results in Table 3?

3 comments

Response to Reviewer Lzzu

We sincerely thank you for your careful review and constructive questions regarding novelty, ablations, and computational efficiency. We conducted additional experiments and revised the manuscript accordingly.

Q1. Methodological differences.

Clarification added to the Introduction and Sections 2–3.

HiPPO-FM decouples history encoding from coordinate evaluation. Existing methods either integrate latent trajectories at requested times (Neural ODE/CDE, CRU) or revisit observation-indexed representations through query-dependent attention (mTAN, ContiFormer).

HiPPO-FM instead applies observed-support self-attention once, writes content-updated values into a fixed-rank functional state using kernel-smoothed HiPPO–Legendre features and a regularized Gram solve, and evaluates the retained state at arbitrary coordinates. Unlike OHSVGP, it stores deterministic neural values rather than a variational GP posterior.

Encoding complexity O(N2d+Nd2)O(N^2d+Nd^2), query complexity O(NRd2)O(N_Rd^2), and persistent state O(Md)O(Md).


Q2. Component ablations.

We added five-seed paired ablations (Section 4.5, Table 5).

VariantOverall RMSEExtrap. RMSESeeds worse
Full0.54 ± 0.140.63 ± 0.20--
w/o encoder FM; MHA retained9.88 ± 0.5913.79 ± 0.775/5
Cosine basis0.61 ± 0.100.66 ± 0.154/5
No Gram solve0.63 ± 0.070.64 ± 0.074/5
No kernel0.49 ± 0.060.50 ± 0.053/5

RMSE: mean ± sample SD, in units of 10−210^{-2}.

Removing encoder FM while retaining MHA, layer normalization, and FFN increases RMSE substantially (paired tt-test p=3.5×10−6p=3.5\times10^{-6}).

  • Encoder FM is the most critical component.
  • HiPPO coordinates show a modest, non-significant benefit over cosine basis (p=0.24p=0.24).
  • Kernel geometry has task-dependent effects: removing kernel smoothing slightly lowers mean RMSE but degrades 3/5 seeds (p=0.56p=0.56); removing the Gram solve increases mean RMSE (p=0.10p=0.10).

Q3. Memory-rank sensitivity.

We evaluated M∈{8,16,32,64,128}M\in\{8,16,32,64,128\} over five seeds (Appendix E, Figure 6).

MMOverall RMSEExtrap. RMSE
80.00565 ± 0.001550.00593 ± 0.00179
160.00536 ± 0.001370.00625 ± 0.00197
320.00552 ± 0.001980.00641 ± 0.00286
640.00566 ± 0.001040.00634 ± 0.00111
1280.00645 ± 0.001910.00734 ± 0.00245

The relationship is non-monotonic: M=16M=16 achieves the lowest overall RMSE, while M=8M=8 performs best on extrapolation. Larger rank does not necessarily improve accuracy, as learned coefficients also depend on optimization dynamics. We retain M=16M=16 as the default configuration.


Q4. Observation/query scaling.

Efficiency experiments (Section 4.5).

Observation scaling (N=32→1024N=32\rightarrow1024):

  • Persistent state: HiPPO-FM grows from 82.7 to 102.6 KB, versus 328.5 KB to 10.5 MB for the token cache, yielding 102.4× compression at N=1024N=1024.
  • Encoding latency: 8.8–8.9 ms for HiPPO-FM versus 5.8–6.1 ms for token-based encoding.
  • Readout latency: At N=1024N=1024, 1.63 ms versus 0.85 ms.

Query reuse: Encoding once and reusing the state across J=1000J=1000 query grids reduces total latency from 9,756.26 to 1,605.35 ms, achieving a 6.08× reduction (Appendix A.3, Figure 4). Query ordering, grouping, and chunking change predictions by at most 2.611×10−52.611\times10^{-5}.

Q5. Competitive methods.

Our benchmarks cover recurrent, continuous-time, state-space, and attention baselines. Under a nominal 30-s RTX 5090 budget (Section 4.5, Figure 3):

ModelTest RMSE (10−210^{-2})Iterations
HiPPO-FM0.483 ± 0.0286507 ± 241
Transformer1.015 ± 0.05318469 ± 233
Neural ODE8.924 ± 1.127231 ± 21
Latent ODE9.415 ± 1.317183 ± 17
Neural CDE16.890 ± 7.69887 ± 0
ContiFormer24.841 ± 6.5629 ± 0

HiPPO-FM achieves 2.1× lower RMSE than all baseline model under the same wall-clock budget, despite fewer iterations.

Q6. Statistical significance and UEA.

HiPPO-FM obtains the highest UEA aggregate accuracy (0.7306) and best average rank (1.00).

DropBaselineΔACC (pp)Win/Tie/LossRaw ppHolm pp
30%S5-0.0111/0/90.8691.00
30%ContiFormer+2.3912/0/80.2161.00
50%S5+0.5711/0/90.8121.00
50%ContiFormer+1.309/0/110.4551.00
70%S5+0.4612/0/80.7561.00
70%ContiFormer+0.489/0/110.9701.00

We report the best aggregate ranking without claiming statistically significant superiority. Under heavier dropping, HiPPO-FM improves over S5 by 0.57 and 0.46 percentage points at 50% and 70%, respectively.

For event prediction (Table 4), HiPPO-FM achieves the highest event-type accuracy on all five multi-type datasets and the highest log-likelihood on four of six.

Thank you for the thoughtful response and the additional experiments. Although your response addresses several of my initial concerns, I still have some questions about the contributions of individual components and the scope of the baseline comparisons. Thus, I keep my original score.

Follow-up Clarification to Reviewer Lzzu

Thank you for clarifying your remaining concerns. We would like to address the two points more specifically.

(1) Individual component contributions.

We believe there may be an ambiguity in the interpretation of our ablation. Appendix E, Table 12 explicitly includes w/o encoder FM (MHA only). This variant removes only the encoder FM operator while retaining observed-support self-attention, layer normalization, and the feed-forward network. It is therefore a controlled MHA-only encoder, not an experiment that removes MHA.

Comparing this variant with the full MHA-to-FM encoder directly isolates the incremental contribution of functional memory beyond standard self-attention: overall RMSE decreases from 9.889.88 to 0.540.54 (×10−2\times10^{-2}), with degradation in all five seeds for the MHA-only variant (paired tt-test p=3.5×10−6p=3.5\times10^{-6}; exact Wilcoxon p=0.0625p=0.0625).

The remaining ablations independently replace the HiPPO basis with cosine modes, remove the Gram solve, or remove kernel smoothing while retaining the functional-memory structure. Thus, our Spiral and UEA study already examine the functional bottleneck and its principal geometric components under a paired protocol. We do not interpret these results as proving that every geometric choice independently improves accuracy; rather, they identify the functional bottleneck as the dominant component.

(2) Scope of baseline comparisons.

Our baseline selection follows the task-matched benchmark suites and protocols established by ContiFormer where applicable, with additional representative methods. The comparison is deliberately organized across four task families rather than a single leaderboard:

  • Spiral reconstruction: Transformer, Neural ODE, Latent ODE, Neural CDE, and ContiFormer, covering standard attention and continuous-time dynamics.
  • UEA classification: ten competing methods across recurrent missing-data models (GRU-D, GRU-Δt), continuous-time models (ODE-RNN, Neural CDE), time-aware attention (mTAN, ContiFormer), state-space models (S5), Transformers (TST), CADN, and graph/patch modeling (t-PatchGNN), evaluated on 20 datasets at three observation-drop ratios.
  • Event prediction: classical and neural point-process methods, including Hawkes, RMTPP, NeuralHP, SAHP, THP, NSTKA, and sequence-modeling baselines.
  • PhysioNet extrapolation: CRU, using its released preprocessing protocol and a matched 100-epoch budget.

We select baselines based on methodological relevance, task compatibility, and reproducibility, rather than publication date alone. In particular, methods designed for specialized forecasting or incompatible output interfaces are not necessarily valid direct comparisons for arbitrary-coordinate reconstruction and extrapolation.

We acknowledge that the direct PhysioNet comparison is narrower and limit our claim accordingly. Overall, our evaluation covers multiple established and competitive model families across distinct irregular-time-series tasks, rather than claiming superiority over every recent model.

For reproducibility, our anonymized repository provides the implementation and reproduction instructions:

Anonymous code repository: https://github.com/kumayazamana-byte/anon-ml-repro-7f3c

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.