acceptodds
Under review as a conference paper at ICLR 2027

Learning Continuous Fields with Hierarchical Latent Modeling and Sparse Context Encoding

Abstract

Reconstructing continuous physical fields from sparse sensor observations is a fundamental challenge in scientific machine learning, with applications in climate, atmosphere, and ocean sciences. Implicit neural representations (INRs) provide a powerful coordinate-based framework, but existing sparse-to-dense INR approaches struggle to represent the multiscale spatiotemporal variability of physical fields. We present , an INR architecture built on two key innovations: that uses sparse observations together with sparse coordinates with a latent cross-attention mechanism to compress high-dimensional fields into a latent embedding, enabling accurate reconstruction; learns diverse spatial structures through a hierarchy of blocks built on rational basis functions. We evaluated in configurations that span four geophysical datasets, four reconstruction tasks with fixed or variable sensors and locations, three sparsity levels, and temporal extrapolation. achieves the best average rank across all experiments. Ablation studies confirm the contribution of each architectural component.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
scientific machine learning, geoscientific data analysis, sparse to dense reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_c4CjOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes HiLS, an INR-based framework for sparse-to-dense continuous physical-field reconstruction. HiLS combines Sparse Context Encoding (SCE), which jointly encodes sparse observation values and locations into compact latent representations, with Hierarchical Latent Modeling (HLM), which retains and fuses representations from multiple Rational Function blocks. The method is evaluated across multiple scientific datasets, sampling configurations, sparsity levels, and temporal extrapolation settings.

Critical strengths

strengths:

  1. Comprehensive baseline evaluation. The paper compares against a broad range of INR baselines, although the baselines can be dated and omits several most relevant methods.

  2. Comprehensive dataset x sampling-ratio evaluation. The experiments span four datasets, four sensor/sampling tasks, and multiple sparsity levels, providing broad coverage of different sparse-observation settings.

Critical weaknesses

weaknesses:

  1. Critical prior work is missing, which fundamentally undermines the paper's novelty claims and baseline evaluation.

The paper explicitly claims that, “unlike other INRs, the content information is encoded together with the coordinates,” and later identifies “Context Input, Spatial Context Encoding, Hierarchical Modeling, and use of Rational function layers” as architectural distinctions absent from its selected INR baselines.

These claims omit highly relevant conditioned neural field / INR. Most notably, SCENT [1] already models continuous scientific fields conditioned on observations and uses learned queries/cross-attention for reconstruction at arbitrary locations from sparse observations. OmniField [2] further demonstrates observation-conditioned continuous fields under sparse and irregular multimodal inputs. Thus, the general premise that incorporating observation content into continuous neural fields is a distinguishing feature of HiLS is not supported.

This omission of these work is nontrivial to the soundness of the manuscript. The novelty claims need to be fundamentally rewritten relative to these works, and SCENT in particular represents a highly relevant missing baseline that should be experimentally compared. The reported breadth of 64 settings cannot compensate for omission of the closest comparison class under fair settings such as iso-param and iso-compute.

  1. After stripping away established components, the paper does not clearly identify and isolate what architectural design is actually new.

SCE combines observation conditioning [1], prior Gabor coordinate encoding, and Perceiver/Q-Former-style learned-query cross-attention/pooling [5,6]. In Eqs. 163–167, (W_A) can be equivalently interpreted as a bank of learned query embeddings attending to the coordinate features (G(K)). HLM similarly builds on mid-layer feature reuse, a well-established principle in architectures such as U-Net, DenseNet [3] and FPN [4]. Rational/KAT [7,8] components are also adopted from prior work.

The current leave-one-component-out ablations remove large bundles (-SCE, -Hierarchical, -Rational) rather than isolating the non-standard part of each design. The ablation study should instead first clearly identify the actual architectural choice or recipes that are new to current state of research, then compare that choice against its closest best-performing alternative.

More fundamentally, demonstrating that a large combination of established techniques works well is, in my opinion, not sufficient to establish a useful contribution to the community. Without isolating which design choice is responsible for the gain, the resulting method remains difficult to understand and unnecessarily difficult to extend. Researchers cannot know which parts are essential and which can be safely replaced or omitted.

  1. The -Hierarchical ablation is not sufficiently defined to establish what HLM contributes.

The paper does not clearly specify the architectural difference between HLM and its “sequential” counterpart, nor is the sequential implementation provided. This makes it impossible to determine what is actually held fixed in the comparison, considering HLM itself is a heavily-involved with many coupled design: retaining intermediate Z^{(r)}, constructing level-specific reconstruction representations V^{(r)}, and fusing them across levels.

Replacing HLM with an unspecified “sequential” model removes several coupled components at once rather than constituting a clean ablation of an isolatable design choice. As a result, the reported -Hierarchical degradation cannot be attributed specifically to intermediate-feature reuse, level-specific reconstruction, cross-level fusion, or their interaction.

Figures 5–6 inherit the same ambiguity. They visualize scalar “block-wise representations” and intermediate reconstructions, although the intermediate latent/block features are high-dimensional, without clearly explaining how these quantities are decoded for either architecture. Consequently, neither the ablation nor these figures cleanly establish that HLM preserves complementary or multiscale spatial structures.

  1. Unclear Rational Function formulation and effect.

The Rational Function component appears to follow the KAT-style formulation of Yang and Wang [7], and the released code directly includes the rational_kat_cu implementation [8]. Appendix A.1 discusses the underlying rational function (\phi(x)), but does not provide the full definition of the Rational Function block (R_r) actually used in HLM. The main method section also does not properly reference the KAT formulation. Proper citations should be provided for the implemented Rational Function, and the complete definition of (R_r) should be included in the appendix.

The claimed benefit of Rational Functions is also difficult to establish from Figure 7. The figure provides only qualitative signed-error maps without corresponding per-example errors, and the visual differences are not consistently in favor of the rational variant. In particular, in the first, second, and last rows, the MLP residuals appear comparable to or in some regions milder than those of the rational model.

[1] Park et al., SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data via Scalable Conditioned Neural Fields, ICML 2025.

[2] Valencia et al., OmniField: Conditioned Neural Fields for Robust Multimodal Spatiotemporal Learning, ICLR 2026.

[3] Huang et al., Densely Connected Convolutional Networks, CVPR 2017.

[4] Lin et al., Feature Pyramid Networks for Object Detection, CVPR 2017.

[5] Jaegle et al., Perceiver: General Perception with Iterative Attention, ICML 2021.

[6] Li et al., BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models, ICML 2023.

[7] Yang and Wang, Kolmogorov-Arnold Transformer, ICLR 2025.

[8] Yang, rational_kat_cu: CUDA/Triton Rational Function for Kolmogorov-Arnold Transformer (KAT), GitHub repository.

Questions for authors

See weakness for the need to rewrite overclaimed novelty, present a cleanly isolated and validated technical contribution beyond a bundle of known techniques, and improve literature attribution and technical method clarity.

Reviewer_UJU3OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This is a very interesting paper on spatio-temporal modeling of physical phenomena to derive a neural representations of physical fields under sparse observations. I enjoyed reading the paper and the analysis across a broad spectra of datasets with extensive baseline comparisons. The authors demonstrate that their algorithms achieve superior performance an array of baselines.

Critical strengths

a. Good problem formulation b. Extensive evaluations across different datasets with strong improvements over baselines c. Extensive comparisons with baselines across different types of spatiotemporal evaluation constraints d. I also appreciate the qualitative results since it provides a clearer interpretable picture of sudden transitions, extreme fluctuations, local hotspots

Critical weaknesses

a. Theoretical modeling of the framework is largely not present b. The authors need to do a better job explain what is fundamental about their proposed algorithm since it is an end-to-end pipeline composing well established mathematical components. c. There are specific climate datasets which are very good for "extreme events" analysis. This paper would benefit extending their analyses to such datasets to showcase their ability to learn extreme and sudden variations in space and time which are very hard for conventional models.

Questions for authors

This is a strong paper with a solid problem formulation and extensive evaluation.

The authors can do a better job of explaining the three weaknesses outlined above:

(i) What is fundamental about the algorithm proposed in this paper:

The paper starts with a great summary when it highlights four limitations of current implicit neural representation models.

However, the paper continues the argument to state that they are the first to handle all four limitations. I think that line of writing is not helpful. I would love to see a finer grained explanation of how their proposed algorithm helps in addressing each point.

What makes your technique better at handling physical field construction beyond interpolation? How do you exploit global structural correlations? Why is single dominant directional learning a bad thing if the direction is able to adapt across space and time? How does HLM really capture periodic functions?

(ii) Extreme events and sudden local variations

All the datasets chosen in the evaluations are standard climate datasets which is very good. There are specific scenarios where the data shows extreme events and sudden variations in space/time. The way the evaluation is presented is really focused on the average case improvement which I think does not highlight the value of their carefully chosen sparse encoding.

The authors have two options: a. Take their qualitative analysis and then expand upon demonstrating how well they capture these extreme event modeling and local variations in their models. The answer is partly there in the analysis but not well highlighted in my opinion b. Extend the analysis to specialized extreme event data analysis.

(iii) Many spatiotemporal modeling papers come with some notions of theoretical guarantees. In this regard, this paper is weak but the strength is in the extensive empirical evaluation. I dont know if this is fixable.

Reviewer_nqR6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes HiLS, a hierarchical architecture for reconstructing continuous geophysical fields from sparse, irregular observations. HiLS combines Sparse Context Encoding (SCE) to compress sparse coordinates and content via latent cross-attention, Hierarchical Latent Modeling (HLM) using Rational Function blocks (Safe Pade Activation Units) to retain multi-depth representations, and a Target Field Reconstruction Module (TFRM) to decode query points conditioned on space, time, and context latents. Evaluations are conducted across 4 geophysical datasets and multiple sampling configurations.

Critical strengths
  • The effective rank analysis (Figure 4) clearly illustrates that sequential deep latent blocks suffer from representation collapse (reff≈1r_{\text{eff}} \approx 1), whereas multi-level hierarchical preservation maintains diverse latent directions (reff≈2.2−4.1r_{\text{eff}} \approx 2.2 - 4.1).

  • Table 3 isolates the utility of rational basis functions versus standard MLPs, showing consistent gains when capturing complex, oscillatory geophysical signals.

  • Benchmarking across 64 settings (including extrapolation and variable sensor distributions) is thorough and well-documented.

Critical weaknesses
  • The manuscript frames the inclusion of sparse observation content (SCE) and spatiotemporal target conditioning (TFRM) as novel paradigms in implicit representation learning. This ignores the established literature on Conditioned Neural Fields (CNFs)—notably SCENT (Park et al., 2025) and OmniField (Valencia et al., 2026)—as well as Neural Processes. Ingesting (x,v(x))(x, v(x)) via cross-attention and decoding at (xq,tq)(x_q, t_q) is the standard formulation of CNFs. Presenting this as a novel breakthrough by contrasting solely against coordinate-only INRs misrepresents the state of the art.

  • While the paper compares extensively against 9 classical, coordinate-only INRs (SIREN, WIRE, MFN, FINER, etc.) and evaluates a few general operator models in the appendix, it completely omits dedicated spatiotemporal Conditioned Neural Fields (such as SCENT or OmniField). Because these models also condition on sparse context values and coordinates, comparing only against coordinate-only INRs creates an uninformative baseline disparity.

  • Once the context-encoding and query-decoding pipelines are recognized as standard CNF design patterns, the core contribution reduces to inserting rational basis blocks (Safe PAU) into a multi-scale skip hierarchy. While empirically effective, this is an incremental architectural refinement rather than a foundational shift in scientific field modeling.

Questions for authors
  • How does HiLS perform when directly compared against modern Conditioned Neural Fields (e.g., SCENT, OmniField) on identical splits, rather than older coordinate-only INRs (SIREN, WIRE, MFN)?

  • If the SCE cross-attention encoder is swapped for a standard CNF transformer encoder while keeping the HLM rational blocks intact, how much of the performance advantage is preserved?

  • Given that the problem formulation, T1–T4 task designs, and CESM2/MUR datasets directly follow the benchmark established in MMGN (Luo et al., 2024), why is the paper framed as contrasting against a generic limitation of "all existing INRs" rather than properly contextualizing the shift from auto-decoded INRs to conditioned neural fields?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.